Cisco Tetration and AnyConnect NVM: Endpoint Visibility for Cloud Workload Protection Policy

In my last blog, I discussed what questions we could get answered when streaming AnyConnect NVM (Network Visibility Module) to Cisco Tetration:  Cisco Tetration Endpoint Visibility with AnyConnect NVM

If interested in learning more on how Tetration will help answer the questions, please continue.  I have an in-depth 15-minute demo of the solution here: 

If you would rather not spend 15 mins and jump to a section that is of interest please proceed to the following links:

1. AnyConnect NVM and Tetration Endpoint Profile:
2. Tetration Inventory Filters leveraging LDAP Annotations:
3. AnyConnect NVM and Tetration Flow Search:
4. AnyConnect NVM and Tetration Zero-Trust Policy Creation and Enforcement:
5. AnyConnect NVM and Tetration Policy Analysis and Compliance:

Hope you enjoy, and most of all learn something new! Please reach out to me with any questions or comments. Thanks!

Hard Candy Shell – Soft, Gooey Center

Mmmm… crunchy outside, gooey inside - sounds good, right? Well, if you’re talking about a piece of candy, sure it does, but if you’re talking about your IT info-sec posture, it’s not hard to argue it’s just going to make you sick.

If you look at the headlines over the past few years, there is nothing more eye-catching than the millions of customer records that have been compromised due to a known vulnerability. Apache Struts got a lot of press recently and perfectly illustrate this type of activity.

Anatomy of a Compromise

What’s fascinating to know about this is the way the data loss happened. Typically, the valuable data rarely lives on the systems that get compromised due to this type of vulnerability, but rather on disparate systems that may or may not be associated with the system that is vulnerable. The way this works, especially in the example of the well-publicized Apache Struts compromise, is somewhat like this: there is a bug in some code (Struts) that allows a remote attacker to send a specifically designed message to the server and that allows direct access with escalated privileges. The great thing about this from the attacker’s point of view is that it looks like just regular connections from a port/protocol perspective, so many tools won’t see the difference. Of course, the connection characteristics are different, but this type of differential is challenging to get visibility into.

Once the attacker who now has admin rights to the compromised box can start probing the internal network to see if there are high-value systems that they could get information from. It’s not difficult once someone has admin control to load some scanning programs and start digging for well known ports like possibly mySQL or SQL Server or something like that. If there are no digital walls inside an infrastructure, its not hard to see how quickly one edge compromise can lead to an amplified risk to all other elements in the rest of the infrastructure. This is like breaking open the candy to get to the good, gooey center…

Think about it… when you go to a hotel, you would prefer a hotel that has keys for every door, or just one key to get into the elevator? If the one-key-per-door approach didn’t exist, all you would have to do is get in and you could do anything you wan where ever you want. That is what you’d call a “bad thing”.

Make All the Things “More Crunchy™”

So… we can see how one small crack in the outer shell can allow someone to get to all the gooey goodness, so how can we get around this?

Enter Cisco Tetration (and in my head, we cue the “EnterSandman” music).

Edge or perimeter defense is where most people spend almost all of their time securing workloads. That’s good… but as you can see from the gooey center, it’s not enough. The trade off is that if you provide perimeter-like defense for every server and service inside an infrastructure, you will be doing a massive amount of “unnatural things”, such as policy-based routing to ping-pong traffic to firewalls and things like that… but that can increase complexity in a significant fashion and therefore increasing fragility, which is never good. It is possible to do this, but it’s not an easy thing to do, and being able to do it with confidence takes a significant amount of visibility and knowledge, and would also require a significant amount of automation to execute the rules. If all of that can be done, you will also still have to deal with hardware limitations in asics/tcam tables for rules storage and execution. Basically, this is putting a really large amount of strain on a set of boxes that are not really designed for this level of granularity, so that would mean a company’s investment in the size and number of boxes to satisfy this need would be extremely unwieldy.

Why Tetration

What Tetration is built to do is to decrease risk and increase efficiency of building a defense in depth strategy through these high-level intents:

• Visibility
• By utilizing sensors on servers, virtual machines and in the network, Tetration can increase visibility down to being able to gather meta data for every packet for every flow inside a DC, in any cloud deployment, as well as data on system integrity and software vulnerabilities

• Understanding
• Tetration uses the data from all of the sensors and applies machine learning algorithms to build application dependency mappings, which provides a policy list that can be applied to create the walls between the application tiers and even between the workloads within the tiers
• This data can be used internally through native protection (see next bullet) and can be exported manually or programmatically for use in other protection mechanisms (i.e. fw, acl, aci contracts/filters, etc.)

• Protection
• Systems instrumented with the Tetration software sensors can do native policy enforcement based on the application dependencies that have been discovered
• Tetration does software inventory checks against a robust CVE database which allows identification protection mechanisms against vulnerable software. In the example above, we could write a rule that says “anything with the apache struts vulnerability cannot talk to production or internet, but can talk to our patching servers”
• Tetration watches process behavior in the kernel and will report and record process anomaly events, such as privilege escalation and shellcode execution (two of which were used to crack into the candy shell above), as well as other kernel events, such as socket creation, side channel attack, file access, binary changes, and others.

Same Candy, More Crunch

With the tools that Tetration is able to bring to bear, we have the ability to deliver unprecedented understanding of what is going on in the infrastructure as well as new tools to protect the security zones inside the perimeter, providing protection against unwanted lateral movement as well as mechanisms to see vulnerabilities and kernel process abnormalities in ways not seen before.

In other words, with Tetration, it’s the same candy with WAY more crunch.

Give me MORE!

If you'd like to learn a little more about all the good stuff Tetration can help you with, please go check out our lightboard series on Tetration!


Thanks!

Cisco Tetration Endpoint Visibility with AnyConnect NVM

Customers often ask me that they want to achieve better visibility around the following questions they have in day to day operations:

• Who (user context) is accessing applications?
• What AD groups are accessing which applications?
• What users and AD groups are communicating laterally in the campus?
• What device type accessed the application?
• What processes are running on these devices?
• What is the zero-trust policy for the enterprise and connecting to applications?
• Performance troubleshooting with user context-aware telemetry?

And while some tools can get you some of this data, there is nothing as comprehensive as Tetration to give you pervasive flow visibility with context. So to make Tetration even more impactful to your network and security operations, Tetration is introducing endpoint visibility using Anyconnect Client with the Network Visibility Module (https://bit.ly/2zMPqHH). That's right if you are an Anyconnect client you can enable the NVM module with no additional agent.  

The open architecture of Tetration allows us to collect telemetry from Tetration software sensors, OOB Sensors, Hardware sensors, Netflow(v9), IPFIX, AWS VPC Flow Logs, Netscaler/F5/AVI IPFIX flow data and of course now Anyconnect NVM.  

So, what will Tetration do with this new telemetry from Anyconnect NVM? 

Tetration discovers user and group zero-trust segmentation policy to control access to the application workloads and lateral movement in the campus!  Get to see who is accessing what, what URLs were accessed, search by user-specific flows, and much more!  

Now what? Where do you deploy the end-user and discovered segmentation policy? Tetration publishes the zero-trust policy discovered to a Kafka message bus for any vendor to consume, normalize and enforce within the campus infrastructure. Think about rolling out Cisco DNA with a zero-trust policy day 1.  How about having ISE ingest dynamically discovered policy from Tetration and enforcing with Trustsec? A consistent zero-trust policy across the campus, branch, WAN and wherever your application workloads exist. Don't forget Tetration will also natively enforce policy across your application workloads via the software sensor manipulating iptables and windows advanced firewall in the data center or public cloud.  A consistent zero-trust policy across the enterprise is the outcome. 

If we look at the big picture in a zero-knowledge environment, you can dynamically discover and write an intent-based hierarchical zero-trust policy for the enterprise. A policy such as:

• Allow access to "FinApp" only to users in group "Finance."
• Allow access to "HIPPA-Apps" only to users in group "Medical."
• Allow access to "ProductionApps" only to users in group "employees."
• Allow access to "DevelopmentApps" only to users in group "developers."
• Allow access to "ATM-Apps" only to devices in group "ATMs."

Hierarchically stack the intent defined above with segmentation policy dynamically discovered for each unique application, and you have an enterprise zero trust policy with cloud workload protection.

To summarize with Anyconnect NVM added as telemetry source here are new business outcomes that are achievable:

• Long-term data retention for user flows, and process forensics
• Ability to capture and tie each user and their flow(s) back to what server/application it accessed
• Better contextual information including access to the server process, clusters, and application - from the endpoint/user perspective
• Application dependencies, policy recommendation/testing, and server workload protection policy with user/group context

IT says, “It can’t be done!” Cisco Tetration says, “Hold My Beer...”

Just received my daily email update from Talos.

"Today, Microsoft has released its monthly set of security advisories for vulnerabilities that have been identified and addressed in various products. This month's advisory release addresses 74 new vulnerabilities, with 14 of them rated critical, and 59 of them rated important. These vulnerabilities impact Internet Explorer, Edge, Exchange, Scripting Engine, Windows Shell and more."

How can Tetration help me get visibility into what workloads are still vulnerable and unpatched? How can Tetration help me quickly make sure these workloads are unable to communicate until patched?  

Wow, that's a LOT! How in the world can I get visibility to which workloads are vulnerable and unpatched? How can I quickly make sure that workloads with critical issues are limited in their ability to communicate? How can I do all of this in a short time across thousands of servers or virtual machines? Cisco Tetration… that's how.

Application owners need some amount of autonomy to make application-level changes quickly, while security and network teams need to control the global aspects of application interconnectivity and shared services. Cisco Tetration flattens intent in a deterministic order, prioritizing intent of higher-authority users over the intent of application owners. Therefore, Tetration allows a global rollout of security rules to identify vulnerable workloads and quickly quarantine them. 

Tetration supports any mixture of blacklist/whitelist security models for different applications, let application owners define very fine-grained policies to secure their applications while simultaneously allowing the security teams to enforce their guidelines and best practices on wide sets of applications. 

Here is an example of defining an inventory filter that dynamically catalogs all workloads that match on any of the MSFT critical CVEs listed in the Talos email.  Filters are saved inventory searches that can be used when defining policies, config intents, etc. You can view existing filters by clicking on them, which lists all endpoints that match on a time-series basis.  

Tetration identifies all installed packages across all instrumented workloads and compares them with a CVE database to identify vulnerabilities. Tetration also tracks process behavior for malicious patterns such as side channel communication, shell code execution, privilege escalation and others, and records and alerts on that behavior. Process hashes are computed and compared with fingerprints registered with VirusTotal. As a bonus, Tetration also tracks a workloads open & listening ports and identifies whether or not a flow has been observed on a particular socket. Tetration will also tell you what user-id owns the process that opened the socket and what command line argument was used to launch the process.

Now it is possible to take that inventory filter created above and roll out a global rule that prevents anyone from communicating with the vulnerable endpoints except the patching server. 

Jason Gmitter and Loy Evan