Saturday, October 30, 2010

What Do You Investigate First?

A colleague of mine who runs another Fortune 10 CIRT asked the following question:

Let's say for example, there is a cesspool of internal suspicious activity from netflow, log and host data. You have a limited number of resources who must have some criteria they use to grab the worst stuff first. What criteria would you use to prioritize your investigation activities?

There are two ways to approach this problem, but they will likely converge at some point anyway:

  1. Focus on the assets.

  2. Focus on the threats.


Focus on the assets means identify the most critical assets in your organization. You pay the most attention to them regardless of who you think is causing suspicious or malicious activity that may or may not affect those assets. In other words, whether you believe a mindless malware sample or an advanced threat may be affecting those critical assets, you still devote resources to collection and analysis of activity involving those assets.

Focus on the threats means identifying the most worrisome threats to your organization. You pay the most attention to them regardless of what assets they may target. In other words, whether you see these threats conduct reconnaissance or enterprise-wide exploitation, you still devote resources to collection and analysis of activity involving those threats.

I say these two approaches are likely to converge, because at some point you will see your most critical assets targeted by your most worrisome threats. In fact, you are likely to determine that a threat is the most worrisome precisely because it spends the most time and effort trying to access your critical assets.

I think operationalizing both approaches is tough, because many don't really know what is most important, or how to identify the most worrisome threats. Identifying critical assets is probably easier. If you identify critical assets, and then identify dedicated threats to those assets, you're probably in a position to now deal with both approaches.

You probably notice I've mentioned two of the three components of the risk equation -- assets and threats. I did not mention vulnerabilities yet. Yes, you could decide what assets have the most vulnerabilities and focus on suspicious and malicious activity affecting those assets. A lot of shops do this because it is probably the easiest approach, since identifying and categorizing vulnerabilities is probably easier than doing the same for assets and threats. However, you might waste a lot of time chasing assets which aren't as important as others. Still, this is another approach if you find that you can't make any progress on the asset- or threat-centric approaches.

11 comments:

Marc Ruef said...

Hello,

This kind of dilemma seems to be one step heavier if you ask the same question within a vulnerability assessment or penetration test. How do you priorize your countermeasures: Severe findings first or valuable assets first?

As an external analyst you usually lack the information about assets. You just know the severity/risk of the determined flaws. I think it is the job of the client to compare your issue-related risk into an asset-oriented priority.

But as you say: Most customers are not capable of doing this -- And fixing bugs becomes a more or less focussed task ;(

Regards,

Marc

--
Personal http://www.computec.ch/mruef/ | Work http://www.scip.ch | Twitter http://twitter.com/mruef

escalante blogger said...

That's a very good question. But for me, I think I go for the assets first.

Keydet89 said...

Great post...concise and to the point.

However, the response to this question, even in spite of the reasoning presented, depends heavily on your perspective. Richard, you and your friend have a maturity of thought in this area brought on by who you are, and your position. As a responder, I see compromised organizations with no visibility into their networks that have no idea what constitutes their "critical assets", where they are, and feel that the latest bit of spyware or fake security malware constitute their primary threat.

I like the question not so much because of the reasoning you presented with it, but more so because it serves as an excellent litmus test for responders who answer the phone or go on-site to assist a compromised organization. Time and again, I have started off my response by saying, "Where are your critical assets?", only to be greeted by a room full of blank stares, from the day-shift DBA, all the way up folks with "C" at the beginning of their title.

Fibonacci said...

Focusing on the assets is the way to go because in many cases you don't exactly know where the threats are coming from.

Hackers aren't that big threat as employees who has their own motivations like revenge, money, spying, working for competitors, not to mention fired ex-employees. An outsider hacker even if he has access to the network might not be even driven by financial motivations. It's also possible he don't care about the company at all.

The best thing to do is monitor all irregularities and do regular audits what your employees do including system, network administrators.

But I think what most companies care for even more than information or assets is finance. They keep a close watch on anyone who has access to their cash flows. Even if the company is owned like hell the money shouldn't get skimmed.

So instead of thinking that the threats are coming from poor hackers you should be pointing fingers to people like this:

http://www.geek.com/articles/apple/heads-roll-as-apple-manager-pleads-not-guilty-to-accepting-bribes-20100817/

Anonymous said...

Another perspective is vertical specific. For banking/finance, how much money is flying out the door? IT rarely has the context to understand this - it normally takes fraud specialists to recognise the issue, identify potential suspect IT conduits then involve the incident response team.

There is nothing like a dollar amount to help everyone know what they focus on.

Another factor is time. The prospect of financial recovery is frequently determined by channel and time - with time the biggest factor. I won't go into details but its fair to say that certain methods of money transfer have a certain 'half life' when it comes to investigations and recovery.

Cheers
C

Anonymous said...

"here is a cesspool of internal suspicious activity from netflow, log and host data."

I think in order to have "suspicious activity", you must already have some sense of the threats and vulnerabilities...so now focus on the assets. Perhaps asset value influence the level of monitoring but not the level of investigation? What is the value of a printer? What if it's on the top floor where all the execs sit? What is the value of a workstation?

Joe said...

Assets. Too many assumptions can be made about threats and you could waste a lot of investigation time in the wrong direction. You'd need serious threat intel and then again, it's intel, not as reliable as the facts you can obtain from your assets and monitoring.

cw said...

Knowing where the assets are is critical, but even then it can be a tough battle. This is tangential to your article, but related. Take for example a hypothetical situation involving an institutional banking function, executed by an employee on a Windows XP system, which generates an IDS alert corresponding to the Torpig trojan. A serious concern, but is it a false positive? An indication of a different threat? Little public intel exists to dig deep. Now imagine the same box after having been retrieved by the IT Security office, and finding many types of malware on the system, including malware installed weeks ago. You know what data passed through the box, and credentials to any affected systems have been changed, yet you struggle to determine exactly what happened *on* the box. There is no quick and easy cross-reference to what commodity or advanced threat is using which particular malware binaries, so the IT Security function has a tough job trying to put the whole thing back together with their limited resources. Did I mention they are in a widely distributed .edu environment with limited intel? Time to break out IDA Pro and debuggers perhaps - VirusTotal gives little but generic malware dropper warnings, and ThreatExpert and other online sandboxes give little insight. Did I mention that the IT Security staff is bogged down with many other incidents? In the meanwhile, the compromises roll in. And that's just what they can detect. What else is out there? Perhaps the most critical assets should be monitored much more heavily - a tough call in a decentralized environment with limited organizational governance.

dre said...

In a mature organization, certainly focusing on threats is going to be a win.

However, it might not be a win if you can't identify the threats or respond to them fast enough!

The primary focus should be on balancing incidents with staffing and staffing with tools. You need to hire a certain number of people to handle incident data, and you need to arm them with the appropriate tools and infrastructure to do their jobs.

Ideally, this would be very cross-organizational i.e. via information sharing alliances.

Thus, this is primarily a human resources issue -- governance of infrastructure, social, individual, and instructional capital.

So the flaw with the Fortune 10 CIRT's question is that he or she is assuming "a limited number of resources". The reason why he or she is having issues prioritizing investigations is because the organization is under-funded and under-staffed.

Anonymous said...

"Let's say for example, there is a cesspool of internal suspicious activity from netflow, log and host data. You have a limited number of resources who must have some criteria they use to grab the worst stuff first. What criteria would you use to prioritize your investigation activities?"

This responder needs context and guidance of what is most important to focus on first. His senior leadership should be providing clues in a general sense. Information security/assurance is about mitigating risk to an organization. It is the job of senior leadership to:

1) define what is important and of value to the organization so it is known what to protect
2) identify what the various risks are to that which is valuable


Risk = Threats x Vulnerabilities x Value of Asset

I use the terms value and asset to refer to what is of importance to the company, whether that is public reputation, intellectual property, PII, etc. Realistically as an organization matures in it's defense and response capabilities, the focus will probably go in this order on a continuum:

1) Focus on vulnerabilities
2) Focus on assets
3) Focus on threats

In each stage of maturation the organization is forced to look at itself, how it operates, what it's policies and procedures are, how much visibility it has into the infrastructure, etc. This is the real on-the-ground effect of "risk management", not economic models. Questions are asked at each stage like:

1) How are we managing vulnerabilities?
2) What are our assets and where are they? How do we instrument visibility into the infrastructure at various tiers of the NSM model so we know what is going on?
3) Who is mostly likely to go after what is important to the company and how do we develop intelligence on tools, tactics, and procedures of these threat-actors?

This is how I helped bring operational change as a network security analyst in a Fortune 500 company who got tired of evaluating and responding to alerts/events/incidences with no context. I asked questions and managed upwards.

-cjd

Michael Cloppert said...

I'm getting to this blog post a bit late, but wanted to offer my own perspective for posterity.

In short, we prioritize our response based on the phase of the intrusion that is detected, latter to earlier. That is to say, in this case, our approach would be to classify the suspicious activity by which phase of the intrusion it represents - recon, weaponization, delivery, exploit, install, c2, exfil/actions on objectives. The data indicative of the latter phases must be investigated first, as it represents immediate and possibly ongoing impact to the security of the organization's data. Endpoints engaged in this activity require immediate triage, the classic IR process, etc. As resources are available to investigate for possible successful intrusions linked to earlier-phase indicators, they are so allocated.

This approach is broadly threat-centric, but can be further prioritized based on knowledge of the affected endpoints or targeted technologies.