Problems with Signature and Protocol Anomaly Detection Methods

I often have to describe what differentiates network security monitoring from traditional intrusion detection. Now that "intrusion prevention" is all the rage (when was preventing intrusions not popular?), I have to think in terms of blocking traffic that is potentially suspicious or malicious. Recently while performing network security monitoring, I received the following Snort alert, as reported by Sguil. I had never seen an alert like this before. Here is a text-based representation of what Sguil displayed in its GUI:


Count:1 Event#3.57183 2005-03-31 23:24:14
FTP invalid MODE
1.30.163.130 -> 2.35.23.101
IPVer=4 hlen=5 tos=0 dlen=48 ID=42411 flags=2
offset=0 ttl=117 chksum=0
Protocol: 6 sport=9542 -> dport=21

Seq=3804195312 Ack=735521810 Off=5 Res=0
Flags=***AP*** Win=63842 urp=23116 chksum=0
Payload:
4D 4F 44 45 20 5A 0D 0A MODE Z..


That looks odd. What is "MODE Z"? Here is the Snort rule that produced this alert:

alert tcp $EXTERNAL_NET any -> $HOME_NET 21
(msg:"FTP invalid MODE"; flow:to_server,established;
content:"MODE"; nocase; pcre:"/^MODE\s+[^ABSC]{1}/msi";
classtype:protocol-command-decode; sid:1623; rev:6;)

I am no PCRE expert, but I can see that "MODE Z" will trigger this rule. The Snort rule documentation doesn't shed any real light on the subject. A Google search result reveals that "MODE Z" enables dynamic data compression via the zlib library.

If we were using a Web-based alert browser, this would be the end of the line. There would be no other data to review other than the packet that tripped the rule and the rule itself. How did the Web server respond? Did it respond at all? What FTP server is running? These and other questions remain unanswered when you use an IDS whose chief purpose in life is to generate alert data.

Fortunately, we're using Sguil. We recognize there's more to life than alert data; there's session, full content, and statistical data. We can generate a transcript of their entire conversation between the FTP client and server if we collected the full content data between both parties. Here is that transcript:

Sensor Name: bej-sensor-fl
Timestamp: 2005-03-31 23:24:14
Connection ID: .bej-sensor-fl_57183
Src IP: 1.30.163.130 (mail.example.com)
Dst IP: 2.35.23.101 (Unknown)
Src Port: 9542
Dst Port: 21
OS Fingerprint: 1.30.163.130:9542 - Windows XP Pro SP1, 2000 SP3
OS Fingerprint: -> 2.35.23.101:21 (distance 11, link: ethernet/modem)

DST: 220 Serv-U FTP Server v5.1 for WinSock ready...
DST:
SRC: USER example
SRC:
DST: 331 User name okay, need password.
DST:
SRC: PASS example
SRC:
DST: 230 User logged in, proceed.
DST:
SRC: SYST
SRC:
DST: 215 UNIX Type: L8
DST:
SRC: FEAT
SRC:
DST: 211-Extension supported
DST:
DST: AUTH TLS
DST: SSCN
DST: PBSZ
DST: PROT
DST: CCC
DST: CLNT
DST: MDTM
DST: MDTM YYYYMMDDHHMMSS[+-TZ];filename
DST: SIZE
DST: SITE PSWD;EXEC;SET;INDEX;ZONE;CHMOD;MSG
DST: REST STREAM
DST: XCRC filename;start;end
DST: MODE Z
DST: 211 End
DST:
SRC: CLNT SmartFTP 1.1.984
SRC:
DST: 200 Noted.
DST:
SRC: PWD
SRC:
DST: 257 "/" is current directory.
DST:
SRC: MODE Z
SRC:
DST: 200 MODE Z ok.
DST:
SRC: PASV
SRC:
DST: 227 Entering Passive Mode (2,35,23,101,9,24)
DST:
SRC: LIST -aLT
SRC:
DST: 150 Opening ASCII mode data connection for /bin/ls.
DST:
DST: 226 Transfer complete.
DST:
SRC: TYPE I
SRC:
DST: 200 Type set to I.
DST:
SRC: PASV
SRC:
DST: 227 Entering Passive Mode (2,35,23,101,9,25)
DST:
SRC: STOR test.txt.asc
SRC:
DST: 150 Opening BINARY mode data connection for test.txt.asc.
DST:
DST: 226 Transfer complete.
SRC: QUIT
SRC:
DST: 221 Goodbye!
DST:

Based on the transcript, we see the Serv-U FTP Server v5.1 for WinSock offer MODE Z, and the client decides to use it before transferring several files. There is nothing suspicious or malicious about this exchange. It is completely within business norms.

This example highlights the use of a signature to implement protocol anomaly detection. The signature provided a "white list" of acceptable MODEs. These MODEs were believed to represent normal use of the FTP protocol. Deviations from these norms were thought to be anomalies that indicate intrusion.

This is a fine approach to detecting intrusions, but it highlights a problem that I constantly debate with security developers. I contend that the Internet and its protocols are too dynamic to code into highly reliable systems. Application developers will forever be ahead of security developers. Application developers will always be crafting new protocols or bending and breaking old protocols. Security developers will constantly try to keep redefining what is "normal" and "abnormal" to enforce access control, detect intrusions, and so on.

The consequences of firing an alert for the MODE Z case are minimal when an IDS is involved. If the IDS were replaced by an inline, so-called "IPS," it may have dropped the traffic. The layer 7 firewall -- sorry, "IPS" -- might believe MODE Z is an attack, thereby denying legitimate business functions. Sure, you can "tune it," but there's another application developer out there creating another twist on what the security developer considers normal. So, in addition to the never-ending battle with intruders, security staff face a never-ending battle with developers!

What is my answer? Block what you can, as smartly as you can, and then keep track of everything else via a network audit system. Content-neutral systems store session and full content data to provide network audit. If you ever forget the network audit piece of the IDS/"IPS" puzzle, and concentrate on alert data, you will lose.

Comments

Anonymous said…
Nice example and analysis. I am constantly happified (eek!) with your writing and presentation styles.

To me, it seems somewhay obvious that if you code only what is good, you will never be able to keep up and create that reliable product. It is just not possible...and if you did try to keep up, I bet that would introduce a lot of mistakes where you are including truly malicious traffic as being "white" noise. I certainly would rather err on the side of caution and certainty. In your example, it took some time to research, but you were able to explain away an alert and probably feel that much more in tune with the heartbeat of your network traffic. :)
-LonerVamp
Anonymous said…
Wow, this is a really nice networking related blog. Maybe you'd like to trade links with me? http://www.roothash.com. If nothing else, you might be interested in joining the topsites list there.
Anonymous said…
I see perfectly well that that pcre expression matches "MODE Z":
"/^MODE\s+[^ABSC]{1}/msi" translates to:
match anything starting with "MODE" followed by at least one space and the next character is one NOT A, B, S or C. And the matching should be performed case insensitive (i), multiline (m) and period metacharacter matches all characters (s)

"MODE Z" starts with "MODE", has one space and follows with a character "Z", which is not A, B, S or C. So the rule matches.
Boy, I sure blew my explanation of what was happening with that PCRE. I just changed the original blog text to reflect what I was thinking, not what I ended up saying!
Anonymous said…
About the MODE Z attack, maybe this was directed to the FileZilla flaw disclosed at the end of March.
http://xforce.iss.net/xforce/xfdb/19778
Hello,

Thanks for the link to the X-Force advisory. Remember that the alert in question was not specifically looking for MODE Z. It was looking for MODEs that did not fit one of several values predetermined to be "normal." That was the point of this analysis. It's difficult for security developers to know what is normal since application developers are always creating new programs and protocols.

Popular posts from this blog

Zeek in Action Videos

New Book! The Best of TaoSecurity Blog, Volume 4

MITRE ATT&CK Tactics Are Not Tactics