Benefits of Short Term Incident Containment

One of the regulars in the #snort-gui IRC channel of irc.freenode.net asked me the following question via email. This is an excerpt, and my response follows:

"I am very interested to hear your insight on the topic of 'incident containment' via TCP resets... I am concerned about whether or not incident containment should even be used. From a purely technical standpoint it seems like 'Sure, it's better than just leaving the connection live. It's helping to interfere, after-all.'

But when I think about it in a real-world application, it seems like many malicious hackers will notice TCP resets as a clear sign they have been spotted. It seems like this understanding on their part will cause them to attempt to shoot in again even if only for the brief seconds required to 'rm -rf /'.

The alternative, no TCP resets, it seems the intruder will most likely think their presence is yet unknown and they may be content with their backdoor...

I guess the overall idea is in some ways it seems safer to -not- implement 'incident containment' via TCP resets. If their access is limited to a lower level user access or something then sure TCP resets may be great in preventing them from escalating their privs further, but if they have gained root/admin access it seems very dangerous to try such an ad-hoc method of temporary containment.

Are TCP resets effective enough to prevent last-ditch efforts at wiping a machine?"

This is a good question. I will frame my answer using the concepts in The Tao of Network Security Monitoring. Let's start by describing a scenario. You're performing network security monitoring, and your sensor generates alert data indicating the compromise of your Web server. You check your session data and realize the intruder has connected to a back door installed as part of the initial exploit. Reviewing your full content data, you see the intruder has root level access on the target. (If the full content data were encrypted, preventing useful review of the back door communications, I would assume root level access anyway.)

The question at this point is "what now?" Your level of situational awareness is good, since you know the intrusion attempt succeeded and the intruder has control of the target. (If you were not following NSM principles and were not collecting alert, session, and full content data using something like Sguil, you'd still be at the alert review process wondering what the alert meant!) Now you have to decide what you value more: information about the threat or controlling the threat.

If you're running a Honeynet, the answer is clear: you value information about the threat. You will not take actions to deny further access to the intruder. What if the target is a normal production system? The decision to limit intruder further access depends on the knowledge the victim already possesses about the intruder.

In this scenario, we saw the method of entry and its effects. If we value learning the intruder's ultimate goal (vandalism, espionage, theft, etc.), we should not interfere with his back door. This is very risky, especially if we see credit card databases transferred to Russia while we sit and watch the intruder.

If we value recovering the security of the target, we should definitely cut off the intruder's access. In the case I described, I would immediately implement Short Term Incident Containment (STIC) to deny further intruder access (more on how shortly).

In a different scenario, we might stumble upon an indicator that a victim is compromised, without knowledge of the initial point of entry. For example, a review of our statistical NSM data could show an increase in ICMP traffic. Closer review of full content data shows the ICMP traffic does not meet normal standards and is indicative of an ICMP-based back door. Learning of an intrusion in progress is more common than seeing an intrusion from start to finish, in my incident response consulting experience.

In a case where we do not know how the intruder gained access, or how many targets he has compromised, we would probably value intelligence gathering over containment. In other words, we might let the intruder maintain access so as to determine his modus operandi and what other systems he has compromised. Once we have a sense of the scope of the compromise, we might then institute STIC.

The original question mentioned 'TCP resets.' These are a way for a sensor to spoof reset traffic to fool the source and destination hosts into thinking they each want to end an active connection. This technique is offered as a way for a sensor sitting off-line to interrupt an intrusion attempt. I've used this technique since 1998 and I can report it is not foolproof for a variety of reasons. TCP resets should be used as a last-ditch STIC method.

The best way to limit an intruder's manueverability is to use an access control device. This is best done with a true firewall, although stateful router ACLs can be applied as well. Sometimes a target will be cut off from the outside world after an attack, and then reconnected with special access control measures taken to limit an intruder's manueverability. This is called "fishbowling" a target.

As for whether an intruder will try to quickly destroy a target to eliminate evidence: I never consider this when making the deny/allow decision. You've got to decide whether you value information about the threat or controlling the threat.

I follow the rule that it is unwise to 'hack back' or otherwise touch intruders, since I don't want to tip off intruders that I've seen their activities. (Some consider deterrence a valid goal of NSM operations, and there is evidence that the Air Force had the best results deterring intruders due to its vigilance. I don't think the same applies to the commercial world because .gov is more willing to pursue and prosecute than .com or .edu.) I consider shutting down remote access and thereby alerting the intruder a necessary step when the victim values controlling the threat.

Any time an intruder has root level access, the situation should be immediately considered extremely poor. The only mitigating factor is the level of knowledge you have of the intruder's activity on the target. If your NSM operation has complete awareness of the intruder's actions, you may recover without completely rebuilding the target. If you have any doubt as to what actions the intruder may have taken on the target, I recommend a complete rebuild from trusted sources.

I talk more about STIC, the decision to watch intruders, and related issues in my book. Thank you for the question, and I look forward to comments and other queries.