Thoughts on Running a Cost Effective SIEM

A simple reality of leading a threat detection program is that your data collection requirements grow faster than your SOC budget. Especially these days with cloud and AI workloads generating tons of telemetry and creating new attack surfaces, it makes sense to reassess your security monitoring strategy and decide how you prioritize what data gets ingested and what is being alerted on.

Those two choices are what threat detection is all about. At a high level it's all about applying detection logic to data. The questions you will need to answer as a SOC leader are:

• What detections do I need and where do I run them
• What data do I collect and where do I store it

There are many ways of approaching this topic, each with its advantages and tradeoffs. Leading a mature threat detection function is knowing what those are and navigating them to meet the two main objectives SOCs have, detection and response.

A recent thought piece about this topic comes from Gartner. They've released a paper on SIEM cost reduction in which Abstract was noted for its data pipeline capabilities. In their document, Gartner outlined four strategies for balancing visibility and cost efficiency:

• Distributed detection. Move detections to dedicated threat detection and response tools like EDR and ITDR, which saves us from having to ingest the data into the SIEM to run detections there.
• Detection engineering. Apart from the technical details, the goal is to make sure SOC teams don't ingest data without a good reason for it, like a detection, dashboard or report.
• Telemetry pipelines. Per Gartner the goal is to use pipeline technology to remove duplicate or unnecessary data, dropping the volume by 20 to 50%.
• Multitier storage. Gartner suggests using storage tiering to keep readily accessible data in hot storage, while offloading rarely accessed, contextual data to a cost effective alternative like a data lake.

This is a good approach and while not being groundbreaking it delivers solid advice.

In this article I dive deeper into the first two, detection engineering and moving detections left, specifically to discuss how we can plan for our threat detection targets and why running detections in-stream can be the ultimate time and cost saving approach. Storage tiering deserves its own piece and I'm leaving it alone here.

Understanding threat detection needs of an organization

Gartner tells us to plan ahead, and it makes sense. If you know what you want to detect, you know what data you need to support those detections, both running them and investigating the alerts they generate.

So detections inform what data needs to be onboarded. But what informs which detections an organization needs?

It can be experience. You can sit down with your team and discuss what you think your SOC should be able to detect. There's nothing wrong with that approach, but I don't believe it's optimal. First of all, we are all biased:

Secondly, people in security tend to hold strong opinions. Imagine working with a client and they ask you, "why should we detect that?" or "surely we need to collect firewall logs, right?" An approach based solely on experience can be dismissed if the experience of a more senior stakeholder differs. I've seen many such cases:

Lastly, your team's experience may simply not apply to the organization you're in. If you moved from a Linux shop with a heavy on-prem and GCP presence to a company that's mostly a Microsoft shop with the full M365 E5 suite and a strong Azure footprint, your experience may not be applicable.

None of that even accounts for MSSP environments. While you can rely on experience to plan detection needs for an in-house SOC, you can't do it at scale, nor do you want to. Otherwise you turn every engagement into a bespoke one, and that's the last thing you want as an MSSP.

So if relying on experience isn't optimal, what can we do to predict which detections an organization needs, or which threats are likely to apply to it? 

I believe the answer is threat modeling.

Threat modeling and threat detection targets 

What if, instead of trying to detect everything, which we can't do, we focused on our ability to detect the most likely attack scenarios? If we could do that, we'd focus our efforts on a narrow set of detections that require less data. We'd be not only more cost efficient, but also more effective at detecting realistic attack scenarios. In other words, our SOCs would be cheaper and better at detecting threats.

The threat model I have in mind when discussing detection needs for an organization is always a mix of the likely threat scenarios based on known threat actor capabilities combined with the internal context of an organization, such as industry, geography, types of systems, history of past incidents and the list of crown jewels.

In most cases I find it makes sense to start small and keep updating the threat model as our understanding of an organization improves. This is a moving target and the ideal, maybe even aspirational, situation is to run a detection engineering program based on a continuously updating threat model of an organization. That way we make sure that our detection capabilities reflect both our internal context and the broader threat landscape.

A simple example would be consulting MITRE's group listings to see which threat actors target organizations like yours and what techniques they use. List the techniques across every actor you're concerned about and you'll notice some of them repeat. Those repeats are your detection targets. Start by onboarding the data and writing the detections needed to cover them, while keeping an eye on your SOC's capacity, because every new detection adds to your analysts' workload. 

We will be releasing separate blog posts about how to run this type of threat modeling, watch this space.‍

Moving detections left

Does it make sense to run certain detections not in the SIEM, but in the downstream security tools?

I for sure think so, and Gartner agrees. Dedicated detection and response tools are typically more efficient at detecting threats out of the box, live closer to the monitored systems which translates to a better MTTD, and carry built-in automation capabilities allowing them to act on alerts as soon as they fire.

The downside of moving detections out of the SIEM is that analysts will have to work in multiple security tools, and some would argue that the very purpose of a SIEM is to centralize security monitoring efforts of an organization. Having worked as an analyst I absolutely agree that managing alert status in a central console, and then jumping to an EDR or CDR to actually investigate, gets tiring quickly.

The reality is that moving detections out of the SIEM is by far the more budget friendly approach, but at the same time it goes against what the SIEM was supposed to be, the central threat detection platform for an organization.

What if we moved detections even further left and ran them in-stream? Gartner does not mention that in their article on SIEM cost reduction, but it solves many of the weaknesses of moving detections outside of a SIEM and into dedicated threat detection tools.

Pushing detections into downstream detection and response tool consoles scatters the analyst workflow. In-stream detections don't. The logic runs in the pipeline, the alert still lands in one place. You decentralize where detection happens without decentralizing where analysts work.

Additionally, you detect on data you'd never pay to store. DNS, proxy, firewall logs, raw endpoint telemetry and similar sources have real security value, but due to their verbosity are often kept outside of a SIEM. The pipeline is the perfect place to run detections against this type of verbose data in a budget friendly manner.

And finally, contrary to threat detections tools like EDRs, pipeline detections are typically stored as code and can be version controlled, diffed, and easily tested which is more than you can say for anything running natively inside a vendor's product. Distributing detections doesn't have to mean losing sight of them, as long as those detections can be inspected.

Obviously, not every detection can be run in-stream, same as not every data source can fit into a SIEM. The correct approach is to make use of the flexibility allowed by modern technology and place detections where they best serve their purpose.

The road to a cost effective SIEM

Running a cost effective security monitoring program requires both process and technology. Gartner suggests using multitier storage and telemetry pipelines, having a mature detection engineering function and moving detections left to dedicated threat detection tools as some of the strategies for achieving a cost effective SIEM.

Those are all good recommendations, but here's where I'd put the emphasis differently.

A cost effective SIEM starts with a security monitoring strategy that deliberately aims to cover specific threat detection scenarios. There are many ways of arriving at an understanding of which threats matter for a specific organization. I find that just being able to articulate your data collection and detection engineering choices puts a SOC in a good place on the maturity spectrum.

That strategy then needs technology to land. Flexibility in where detections run, how data is transformed and routed, and what enrichment gets applied is no longer a nice to have. Data onboarding demands are only going up, and good strategy alone won't absorb that.

‍