Code42 @ CSO50 2022

The Code42 Security Operations team is thrilled to be attending CSO50 2022 (September 19-21) to accept an award for their project “how to train your incident handling robots”.

how to train your incident handling robots

Code42 is leading the industry in Insider Risk Management (IRM) because we understand that IRM requires an effective Insider Risk Management program as well as an excellent IRM tool, like Code42 Incydr or Incydr Gov. A dedicated program is necessary because IRM investigations are sensitive, and the result can have a serious impact on the career progression of the person involved.

In recognition of the sensitivity of IRM investigations, the Code42 Insider Risk Management team developed an investigation style called Empathetic Investigations™. Read more from Chrysa Freeman in How to Approach Insider Risk Incidents with Empathy on the Code42 product blog.

Good artists borrow, great artists steal.

Pablo Picasso

Just like Picasso, the Code42 Security Operations team learned one way to be great by stealing the concept of Empathetic Investigations™ from our IRM program. Traditional models of Security Operations investigations tend to be a cold and alienating process for people being interrogated about security events. Often context or “evidence” is withheld, leading to individuals answering inquiries in a bubble and feeling they are receiving “guilty until proven innocent” treatment.

In addition to the angst investigations can cause for the person being investigated, it is also a large investment in security team effort to facilitate direct communication with individuals for a wide variety of alerts. It leads to fatigue, stale alerts (e.g. awaiting user responses), and inconsistent perception of the security team due to always working with individuals with different communication styles.

Using the power of chat robots, our SecOps team is building automated workflows that facilitate investigations through direct messages to users from their friendly Slack Security Robot.

So, how are we training our incident handling robots?

Use friendly language and give context for the activity

  • Friendly language invites collaboration and demonstrates partnership with you and the security team.
  • Example:
    “You have triggered a security alert. Please confirm this activity” —> “Hey! It’s your friendly SecOps robot. At 8AM today, your account signed in from a new location in the world. Does this look familiar to you?”

Empower users by enabling them to classify activity

  • Responses to Empathetic Investigations™ in Slack are included in calculating whether an alert is benign or should be elevated
  • Users know their systems and environments best; this invites them to inform the Security team when unusual activity occurs

Follow up and be clear on next steps

  • Follow up with education to provide in-context information about preferred behaviors when behavior was unintended
  • Clearly communicate next steps to the user
  • Thank respondents for their commitment to protecting themselves and others in your organization

Applied Empathetic Investigations™

The goals of applying Empathetic Investigations™ to security operations are to:

  • Improve security culture and inspire folks to see security team positively
  • Improve efficiency of security staff through reduced flow of manual follow-up tasks
  • Develop user awareness of when they traverse security boundaries and educate on preferred behaviors

Shortly after deploying an early version of these incident handling robots, they demonstrated value by educating an admin that an AWS workflow was applying an access rule they had never noticed before. It’s been a joy to see how this improvement to the security investigation process educates individuals about potentially risky actions that automated workflows perform without their knowledge!

We are grateful to have been selected as a 2022 CSO50 Award Recipient, and we look forward to seeing you next week at MGM National Harbor in Maryland!

Echoes of Log4Shell

Originally composed February 18, 2022

When CVE-2021–44228 (Log4Shell) began in December of 2021, the Code42 SecOps team moved quickly to respond by monitoring for the latest news, developing strategies to test our systems for vulnerabilities, and, of course, establishing alerts with the developing indicators of compromise (IoCs).

After the fervor and frenzy of December waned and we returned to something resembling a usual workday, we addressed some alert fatigue and sprawl by keeping valuable alerts and disabling the others.

We made the necessary changes to remove the disabled alerts and went about our days doing Purple Team Stuff.

Until today.

Today, an alert named “Log4j Stage 2” appeared in our queue. My first thought was, “of course.” After all, Friday before a three-day weekend is statistically proven to be the day where there is at least a 75% chance of a major security incident.

My second thought was “wait, what?”

Those of you who recall the Log4Shell exploit may remember that the “Stage 2” action of the attack (downloading the malicious class) would only be detected on internal systems if they were being used to host the malicious payload.

Attack Scenario Credit: Luciano Avendano

If this is a true positive, it may indicate that one of our hosts has been compromised and is now being used to deliver malicious Java classes to an external party…. yikes! Big yikes!

Before I pushed the big red button, however, I had to do due diligence. This was the first time this alert had ever been triggered, and it was well worth validating the findings.

One of the greatest skills I have learned in my years handling incidents is the power of taking a big, deep breath to release the sense of urgency and tension upon discovery. That deep breath gives my rational brain space to correlate and draw conclusions about the cold hard facts: the data itself.

Let’s take a look at the events that triggered the alert:

[source_ip] — — [17/Feb/2022:17:42:20 -0600] “GET HTTP/1.1” 302 138 “-” “Mozilla/9.0 AppleWebKit/202110.05 (KHTML, like Gecko) Chrome/20.21.1005.122 Safari/211.05” “-” — -

[source_ip] — — [17/Feb/2022:17:36:28 -0600] “GET HTTP/1.1” 302 138 “-” “Mozilla/9.0 AppleWebKit/202110.05 (KHTML, like Gecko) Chrome/20.21.1005.122 Safari/211.05” “-” — -

What stood out to me, and what may stand out to you, is the fact that this HTTP GET request is not attempting to download the expected file type (.class). Rather, it is attempting to download a file with a .7z (7zip) extension. Clearly something wasn’t well defined there.

Now, we all know that the name of the file and its type are not always in sync… for the sake of this exercise I will assume the external party was intentionally seeking to download a 7zip file.

The next thing you will notice is that the server returned an HTTP 302 (“Found”) response. I was curious where it redirected, so I followed the URL and it redirects to a 404. Oh, and by the way, the Code42 404 is delightful and you should totally check it out.

Anyway, clearly this was some errant, random web request that happened to match our alert definition. Now that I had validated what happened and ruled out system compromise, I was curious exactly why this event triggered the alert.

As soon as I saw the query definition in our SIEM, the answer was clear. The alert was configured to search for:

  • HTTP GET Requests
  • That contain the string “class” in the raw text.

And that was it! Wow those are some broad search terms. You can see that the events above were caught by the string “class” being present in the HTTP request. Perhaps this alert definition would have been more effective if it were looking for the string “.class”!

Mystery solved! Well, kind of. Remember how I mentioned that our team went through alert cleanup as we closed the book on our Log4Shell response?

Well, we had.

With my very own eyes I confirmed that this specific alert was no longer present in our server configuration, but somehow it had still generated an incident in our SOAR platform, Palo Alto XSOAR.

A bit of digging later, and it turns out we’ve got a project on our hands implementing the D part of CRUD (Create Read Update Delete) in our Ansible playbook. But that’s a blog post for another day.

Cheers, and stay safe out there!

Follow me on Medium for more (well, mostly the same in a different interface) blog content!

Sysmon — Your next favorite (free) tool!

Are you a curious blue teamer itching for more Windows events? Or potentially you’re a threat hunter searching for mischievous activity on your network. Either way, if you’re tasked with the responsibility of keeping your organization safe, Sysmon might be your next favorite resource. 

Sysmon is a powerful monitoring tool that runs as a Windows service and enhances visibility across the Windows platform. Sysmon extends the capabilities of the Windows EventLog by collecting 25 new event IDs which are logged to Microsoft-Windows-Sysmon/Operational. While sysmon doesn’t provide out of the box analytics, it does provide detailed insight into process creation, network connections, scheduled tasks, file events and much more. Sysmon’s visibility makes it an instant win for security analysts, threat hunters and incident response teams.

The latest version of Sysmon is available for download on Microsoft’s Sysmon Documentation Page. 

Sysmon is simple to install from the command line or with your favorite endpoint management tool. Typical installations should include an XML configuration file that manages the event collection rule definitions for each event ID. Two highly recommended configuration files are:

You can’t go wrong with either configuration — however, you will want to modify the configuration file to filter out noisy events specific to your environment. Once you have the Sysmon.exe executable and the XML configuration file, you are ready to install Sysmon! Run the following command with administrator rights. A system reboot is not required to complete the install.

sysmon.exe -i -accepteula <config file>

Sysmon will immediately begin writing events to the Windows EventLog, which you can review with Windows Event Viewer, or ingest into a log analytics platform, such as Splunk, ELK or Sumo Logic. Now that Sysmon is installed and logging events to your platform of choice — it’s time to discuss a few security use cases!

Event ID 1: Process Creation

One of my favorite Sysmon use cases is to monitor process creation. This is easily achieved with event ID 1 — Process Creation! This event provides rich context about the created process, parent process, command line and working directory. It also includes filename, hash and GUID data. With this data you can monitor for:

  • New Scheduled Tasks — indicating potential persistence in your environment. 
  • Chrome.exe launching a shell or script (cmd.exe, powershell.exe, etc.)
  • AdFind.exe (Active Directory enumeration utility) used for reconnaissance.

These potential use cases are just scratching the surface of process creation events that might spark the interest of an inquisitive blue teamer. Along with process creation events, you might also be interested in driver load events or Sysmon Event ID 6. 

Event ID 6: Driver Loaded

A driver is a piece of code that allows the operating system to interact with a hardware component of that computer. Software installed on a computer may load one or more drivers to complete their defined task. Starting in 2017, Microsoft required that all loaded drivers must be signed by a legitimate digital certificate. However, even with these new security controls in place, older device drivers may be vulnerable to privilege escalation and utilized by malware to complete their objective. Does your organization monitor new drivers? With Sysmon event ID 6, you can easily monitor the signature, hash and file location of all loaded drivers in your environment.

Code42Filter.sys Driver Load Event
Code42Filter.sys Driver Load Event

Now armed with Sysmon, 25 new Windows Events, and a couple interesting use cases — you are ready to improve your organization’s security! If you have any questions or would like to chat about Sysmon, feel free to contact me on Twitter @JeremyThimmesch!