Endpoint security tools are now more capable than ever but with that power often comes complexity. Their complicated nature means that they can be difficult to tune and maintain (not unlike network security solutions), which is why instrumentation is so crucial to achieving and maintaining security effectiveness on endpoints.
We often hear customers say that although their endpoint security tool performed great during the POC, they’re not sure if they’re getting the same value after a few months or quarters have passed. They want to be able to validate that they’re blocking and alerting and that the rest of their infrastructure, like their SIEM, is properly integrated and correlating as expected in perpetuity.
But it doesn’t stop there. Another common issue customers have is comparing multiple endpoint security tools to determine which ones work best for their unique environments and use cases despite biased marketing testimonials and reviews. They want to know if product A is better than product B – not just what product has a better default configuration – and they want to know if product C is better than product D – not just which product had more experienced SEs running the POC.
The simple fact is, it’s really challenging to figure out the best product for your environment. This is where Security Instrumentation comes in. Security Instrumentation solutions like the Verodin Security Instrumentation Platform (SIP) help you make apples-to-apples comparisons of your endpoint security tools during POCs and maintain that value over time post-POC until it’s time to retire and replace your endpoint security tool. In both production and throughout the life of the endpoint security tool, instrumentation helps ensure your security tools are effective and you are receiving the most value.
Of course, Verodin SIP doesn’t just apply to endpoint security. SIP is used to instrument a wide variety of security tools across endpoint, email, network, and cloud as well as the management stack with tools like SIEMs and log managers. Verodin has published multiple blog posts on the value of using SIP to instrument these technologies. You can read them all here.
Verodin SIP empowers you with a platform to safely and automatically measure the impact of real test behaviors within your production environment. The end result? You better understand how your endpoint tools and other security tools will respond to behavioral-based security tests so you can not only measure the value of your security tools but manage, improve, and communicate their effectiveness in an empirical and data-driven approach that business leaders can then leverage for better-informed decision-making that includes cyber metrics.
Let’s get right to an example.
Consider the following scenario: you want to find out which endpoint security tool is best for you but you don’t necessarily want the one with the best marketing and stats or the most impressive SEs. You want the one that is going to perform the best based on your particular use cases in your production environment and with your security team.
Verodin SIP allows you to safely leverage test behaviors to validate how the endpoint security tools and the security tools they are integrated with will respond. You can also measure, report, and improve the effectiveness of your endpoint tools against MITRE ATT&CK and related models. If you are interested in how Verodin SIP helps with MITRE ATT&CK, check out this Verodin blog: “MITRE ATT&CK - Don’t Just Measure and Report...Improve.”
You may want to conduct non-destructive and destructive validation. You may also like to run tests based on varied privileges. Some examples include:
- Turning off the endpoint firewall, DLP, and or anti-malware tools
- Modifying the host file
- Renaming files
- Running CLI, PowerShell or BASH commands, and scripts
- Exfiltrating data
- Safely detonating malware with the Verodin SIP Protected Theatre
By conducting these tests, you can determine if the endpoint security tool(s):
- Block and alert
- Block and not alert
- Not block and alert
- Not block and not alert
And further, you’ll be able to see what that non-destructive or destructive testing all looks like from the perspective of an endpoint security manager, SIEM, log manager, etc. For example, you’ll see if the event on the endpoint triggered an alert that made it to the endpoint manager. Then you’ll see if that alert made it from the endpoint manager to the SIEM for example. Finally, you’ll see if the SIEM generated an alert based on a correlated event and triggered a human response and process.
For the sake of simplicity in this example, let’s keep SIEM out of the equation and illustrate how to use a single attacker behavior to test the basic security endpoint capabilities across multiple endpoint security tools using a test behavior that should be familiar to most security professionals: Mimikatz.
You can search for Mimikatz behavioral tests within Verodin SIP. As displayed in the Verodin SIP screenshot below, we have a Mimikatz test behavior pulled from the library. This particular behavior attempts to dump the local OS hashes. In this example, we’re going after hashes – no cached credentials, golden tickets, or other actions that Mimikatz can be used for.
In the Verodin SIP Screenshot above, you can see that we have the Mimikatz attack spread across five different groups. Verodin doesn’t care what the endpoint security tool is and doesn’t need to integrate with the tool, so add as many endpoint security tools as you would like to validate. There is no need to run Verodin Actors on every endpoint in your environment, either. Just pick the endpoint security tools and versions of those tools you want to validate, install them on a few non-production VMs or a dedicated piece of hardware and begin your tests.
In this example, I’m running Windows 7 on five systems, each with a different endpoint security product enabled or disabled. They are as follows:
- Microsoft Defender enabled
- Microsoft Defender disabled
- Malwarebytes enabled
- Cylance enabled
- Symantec enabled
As illustrated below, we’re getting ready to run the actual Mimikatz tests. There will be a separate test launched against each Verodin Actor to validate each security tool independently.
The screenshot below illustrates the Verodin SIP findings from the Mimikatz tests against the various endpoint security tools. There are no false positives and there are no assumptions. The results are evidence-based and binary – the tool worked or it didn’t under the test conditions.
Note that these results are not intended to suggest that product A is better than product B. These tests simply illustrate product effectiveness, against Mimikatz, under the specific test conditions of our lab and the configurations of each product. Some products were intentionally deployed with default or non-optimized configurations in order to illustrate how differing results may appear within Verodin SIP. In fact, most of these products can be configured to effectively prevent and alert on Mimikatz. But, it is precisely this ability to help determine where products can be optimized, based on evidence and not assumptions, that is one of the core benefits of a security instrumentation platform.
Based on the testing in the Verodin lab of the below endpoint security tools against Mimikatz, here are the results:
- Microsoft defender enabled – Mimikatz is blocked and creates events
- Microsoft defender disabled – Mimikatz is not blocked and there are no events
- Malwarebytes enabled – Mimikatz is not blocked and there are no events
- Cylance enabled – Mimikatz is not blocked and there are events
- Symantec enabled – Mimikatz is blocked and there are no events
Finally, let’s take a look at what Mimikatz looked like from the perspective of the OS. This is done by clicking on the “View CLI Log” button pictured on the right side in the screenshot above.
When Mimikatz is blocked, what we find in the CLI Log is displayed below. Because Mimikatz didn’t execute, there isn’t much to look at because the endpoint security tool did its job.
But when Mimikatz is allowed, the CLI Log display looks like the image below. Because Mimikatz was allowed to execute, there is much more to look at, including the targeted hashes. I truncated it due to the high amount of data in this output but those of you familiar with Mimikatz will recognize it very quickly.
The Verodin SIP validation process can be run in an ad-hoc approach, like we did in this example, or in an automated way. Automation is extremely valuable once you get your security tools to a known good state. For example, maybe your endpoint security tools have now been configured to block and alert on Mimikatz and the logs are successfully making it to the endpoint security manager, then to the SIEM, then they are correlated and generate an alert at the SIEM for an analyst to respond. Verodin SIP can automate tests to validate that these controls continue to work as needed and should they drift from a known good state, you’ll be alerted to the issue and equipped with the knowledge to remedy it. And once you apply a fix, you can revalidate to ensure the correction worked as desired.
Security instrumentation for endpoint security with Verodin SIP allows you to pick the best solution, optimize and validate the configuration, and ensure it stays optimized for the life of the product. Request a demo here.