Saturday, October 25, 2008

Security Event Correlation: Looking Back, Part 3

I'm back with another look at security event correlation. This time it's a June 2008 review of SIEM technology by Greg Shipley titled SIEM tools come up short. The majority of the article talk about non-correlation issues, but I found this section relevant to my ongoing analysis:

"Correlation" has long been the buzzword used around event reduction, and all of the products we tested contained a correlation engine of some sort. The engines vary in complexity, but they all allow for basic comparisons: if the engine sees A and also sees B or C, then it will go do X. Otherwise, file the event away in storage and move onto the next. We'd love to see someone attack the event reduction challenge with something creative like Bayesian filtering, but for now correlation-based event reduction appears to be the de facto standard...

Ok, that sounds like "correlation" to me. Let's see an example.

For example, one of the use cases we tackled was the monitoring of login attempts from foreign countries. We wanted to keep a particularly close watch on successful logins from countries in which we don't normally have employees in. To do this, there are a few things that had to be in place: We had to have authentication logs from the majority of systems that would receive external logins (IPsec and SSL VPN concentrators, Web sites, any externally exposed *NIX systems); we had to have the ability to extract usernames and IP addresses from these logs; and, we had to have the ability to map an IP address to a country code. Not rocket science to do without a SIEM, but not entirely trivial, either.

That doesn't quite seem to match. This use case says "if any system to which a user could log in registers a login from a foreign country, generate an alert." This is simply putting login records from a variety of sources in one place so that a generic policy ("watch for foreign logins") can be applied, after which an alert is generated. Do you really need a SIEM for that?

Here's a thought experiment for those who think "prevention" is the answer: why aren't foreign logins automatically blocked? "If you can detect it, why can't you prevent it?" The key word in the example is "usually," meaning "we don't know our enterprise or business well enough to define normality, so we can't identify exceptions which indicate incidents. We can't block the activity, but we'd like to know when it happens, i.e., drop the P, put back the D between I and S.

Back to correlation -- I think a real correlation case would be "if you see a successful login, followed by access to a sensitive file, followed by the exfiltration of that file, fire an alert." Hold on, this is where it gets interesting.

There are three contact points here, assuming the foreign login is by an unauthorized party:

  1. Access via stolen credentials: If it's not the user, the credentials were stolen. However, you didn't stop it, because you don't know the credentials are stolen.

  2. Access to a sensitive file: How did you know it was sensitive? Because the intruder is impersonating a user whose status is assumed to permit access, you don't stop it.

  3. Exfiltration of the file: If this account (under legitimate or illegitimate control) shouldn't be removing this file, why is that allowed to happen? The answer is that you don't know beforehand that it's sensitive, and there is no real control at the file level for preventing its removal.

If you knew enough to identify that this activity is bad, at each contact point you should have stopped it. If you're not stopping it, why? It's probably because you don't know any of these contact points are bad. You don't know the credentials are stolen (yet). The impersonated user probably has legitimate access to a file, so you're not going to block that. Legitimate users also probably can move files via authorized channels (such as would be the case via this "login"), so you don't block that.

In other words, if you're not smart enough to handle this, why would correlation via a SIEM be any smarter?

Cue my Hawke vs the Machine post from almost two years ago:

Archangel: They haven't built a machine yet that could replace a good pilot, Hawke.

Hawke: Let's hope so.

Back to Greg's case. It turns out that generic policy application against disparate devices appears to be the "win" here:

Q1 Labs' QRadar had all of the functionality to do this, and we were able to build a multi-staged rule that essentially said, "If you see a successful login event from any devices whose IP address does not originate from one of the following countries, generate an alert". Because of the normalization and categorization that occurs as events flow into the SIEM, it's possible to specify "successful login event" without getting into the nuances of Linux, Windows, IIS, VPN concentrators. This is the convenience that SIEM can offer. (emphasis added)

Is that worth the money?

Finally, I'm a little more suspicious about the following:

Most modern SIEM products also ship with at least a minimum set of bundled correlated rules, too. For example, when we brought a new Snort IDS box online, there was a deluge of alerts, the majority of them considered false-positives. Because of useful reduction logic, there was only one alert out of 6,000 that actually appeared on our console across all of the products tested. That alert was based on a predefined correlation rule that looked for a combination of "attack" activity and a successful set of logins within a set period of time.

It's more likely the SIEMs considered the "deluge" events to be of lower priority, so they never appeared on screen. Think of the myriad of ICMP, UPnP, and other alerts generated by any stock IDS ruleset being "tuned down" as "informational" so they don't make the front of the dashboard. If I knew this SIEM test correlated vulnerability data with IDS attack indications, the useful reduction logic would make more sense. I can't be sure but you can guess which way I'm leaning.


Anonymous said...

Hi Richard,

How about three very simple scenarios that show that a SIEM might be useful?

Scenario 1:

* Your IDS pickups a buffer overflow attack to a Windows machine (at this point, you're not sure if the exploit was successful or not)
* Your Windows machine forwards a Windows event that indicates an admin account was added
* Your certainty goes up to 99% that the machine has been compromised

Scenario 2:

* A lot of failed logins followed by a successful login, all from the same IP.
* The above is definitely worth investigating if it's past a certain threshold. An admin looking through the logs might not notice this if there are other events between the logs. Of course, you might be able to detect this using something like swatch, but can swatch understand the un-normalized logins from 100s of different devices?

Scenario 3:

* A low and slow brute-force password attack on a key server.
* There's a good chance that you wouldn't notice it unless a SIEM started elevating it's importance and you could take action before the attacker guessed the right-password.

There are other useful scenarios. I think the problem isn't so much that the SIEM isn't worthwhile, it's just that you should understand the rules that it's applying and perhaps write your own to get more our of it.


Anonymous said...

I am sorry, but these "scenarios" feel like they are coming right off the marketing glossies, written by someone who SOLD, but never USED a SIEM....

Anonymous said...

80% of the SIEM were purchased for compliance reasons. ;)
Probably the other 20% were purchased to help SOC/IDS/IPS. And the point is this: if a company cannot manage to figure out an IDS, how does it think that it can figure out a SIEM?
A SIEM without network and asset modeling and categorization, and w/o vulnerability data is going to produce even more false positives than a misconfigured IDS.

I am wondering when McAfee is going to buy Arcsight. ;)

Anonymous said...

Anonymous @3:04 PM

The first scenarios are all very basic scenarios that any SIEM can handle.

Anonymous @4:07 PM

Practically every SIEM nowadays integrates vulnerability data and allows for asset categorization (depending on business value)

I'm not saying that SIEMs are the solution to everything, I'm just interested in how you deal with the above scenarios without a SIEM across all the different devices, etc.

One of the key benefits of the SIEM is that although the above correlation is very simple (and you should be able to do some more sophisticated correlations), you can apply them regardless of the device (eg. syslog, Windows events, Checkpoint firewall, whatever IPS you have, etc.)

PS. I'm currently in the evaluation stage but have run some PoCs

Anonymous said...

It reminds me when one one my (former)colleagues complained about the fact that the SIEM was showing a way too many "red lights". The consultant in charge w the SIEM had a very blunt answer: "You need to put some intelligence into this devices". The SIEM does not know if an SQL XP Command is comming from a DBA computer or from DMZ. A human needs to feed this information into the SIEM.

Richard is right, he is not saying that a SIEM is bad, but rather it is just not correctly understood. I can see this at my managers, who do not know what is a Win Null Session, but play all day long with terms such as: "deep" and "end-to-end" correlation.

Anonymous said...

I always find it amusing when SIEM vendors talk about their GUI rule builders that allow non-technical people to write rules.

IMO Correlation outside of a few thousand very large companies that ca afford armies of people that understand in depth the attack surface and can tune and maintain the rule sets is a pipe dream.

SIEM is very good in the compliance case, good for analytics and resonable for automating a few hundred basic conditions -- like 50 failed logins from a single IP but beyond that the complexity of correlation far exceeds the benefit.

g said...

It's probably worth differentiating a SIEM from the concept of "correlation." Nobody would argue that an organization with hundreds or thousands of event sources needs some place to dump them all, normalize them, and start to deal with them. As "Anonymous@10:55" said, there's value in simply being able to apply basic rules across a variety of normalized log types. In this sense, I think a SIEM is a necessary component of a SOC... Most of these SIEMs will of course feature correlation, the value of which remains to be proven. We shouldn't rule SIEMs as guilty by association merely because they have features of dubious value.