Nothing to See Here
Recently I noticed a posting at the SANS Internet Storm Center titled Deformed TCP Options - Got Packets? The story featured packets like the following:
In English, 129.250.128.21 is sending SYN ACK packets to the obfuscated IP addresses. 129.250.128.21 is sending SYN ACK packets because it is replying to SYN packets from the obfuscated IP. I knew right away what was happening. Some people called it "backscatter." When I wrote my first paper on the subject in 1999 I used the exceptionally exciting term "third party effects," thereby consigning my research to some vague corner of the Internet. In brief, 129.250.128.21 was being SYN flooded; it is a victim, not a perpetrator.
The ISC handlers wrote:
Generally the source ports are 80, 6667, 6666, and 443. However there are also a number of packets with random source ports between 1024 and about 1300. Ports 1024-1300 are also used as the target port. Looking at the Dshield data for ports 6666 and 6667 there was a small peak for these ports as source ports on the 27/28 Feb.
So we are seeing multiple hosts sending SYN,ACK with truncated option to targets (sometimes both scan the same IP) from known ports. The response is typically a RST, or RST,ACK. The window size is also often changed in the response.
What it looks like is a mapping/fingerprinting exercise using the target stacks' response to bad options. But at this stage the end goal is unclear.
I thought this was not right, and I told the handlers as much. They didn't believe me, so I asked them if they thought it might be a good idea to step out of the cave of packet conspiracies and ask the owner of 129.250.128.21 what was happening. (This was the strategy I followed in my original paper.) They decided not to pursue this non-technical investigative method, so I emailed hostmaster [at] ameri [dot] ca based on the resolution for the IP: compton.ameri.ca.
Since then I've been exchanging emails with Brad Dreisbach. He wrote:
i have been getting tcp syn attacked for about 3 weeks now. i have re-installed the OS on the host just to be safe, but im fairly sure my systems are secure. i have also taken measures with my upstream, whom i also work for, to migitate the attack. some stuff is still getting through but at this point im just waiting for the attackers to give up...
There you go -- Brad is being attacked. 129.250.128.21 is the victim. 129.250.128.21 is not doing some kind of reconnaissance.
When I asked Brad if he had captured any traffic related to his ongoing denial of service attack, he replied:
this pcap file doesnt appear to be a tcp syn attack, but some other form of tcp attack. whoever is doing this is changing their stategy though. the attack has changed a few times over its duration. they even attacked me via ipv6 for a few hours.
Here's part of the traffic he sent:
This looks like an ACK flood to port 80 TCP. The victim will reply with RST packets (not RST ACK) if it can.
I think ISC got thrown off the trail because SANS has a history of seeing global conspiracies in weird packet patterns. This dates back almost ten years and is documented in my first book. The unfortunate result of this attitude is hundreds, if not thousands, of other security analysts have "learned" to see these patterns as malicious too. ISC also spent too much time concentrating on "broken TCP options" on the backscatter traffic. They did not accept my observation that victims under DoS attack tend to respond slowly, weirdly, and eventually not at all when they are flooded.
If you decide to post a nasty reply here because you love ISC and SANS, please keep in mind you're only seeing the public side of the story. ISC and I exchanged very polite emails, but I am not going to disclose the additional justifications of the ISC evil packet theory here because those emails were private.
I think ISC does a great service to the security community, but I would like to see SANS officially abandon its adherence to this misguided view of the world. I am not trying to pick a fight with SANS. All I want is for security analysts to know this pattern is often seen as "malicious" by many when it is utterly benign -- at least as far as the backscatter recipient is concerned. Remember the site sending the SYN ACK, RST ACK, or RST traffic is responding to a SYN flood (to an open or closed port, respectively) or to an ACK flood.
The moral of the story is that it is not a good idea to assume the world is out to get you when a stray packet arrives at your doorstep. A second lesson is that it sometimes make sense to investigate issues by doing human analysis rather than peering at the world through the eyes of Tcpdump/Ethereal/Wireshark. A third lesson is that it makes little sense to interpret traffic as being malicious if the probablility of the "scanner" learning anything remotely useful is extremely low.
Update: One of my friends noted that the Virginia Tech DShield has 129.250.128.21 listed as one of their "Top 10 Most Wanted" "attackers". This demonstrates the trouble with automated reporting systems.
Update 2: Be sure to read the exciting conclusion here.
07:11:45.781421 IP (tos 0x0, ttl 113, id 9433, offset 0, flags [DF],
proto: TCP (6), length: 48)
129.250.128.21.1256 > www.xxx.yyy.zzz.1229: S, cksum 0x5ed4 (correct),
2627126762:2627126762(0) ack 257795 6091 win 1460
0x0000: 4500 0030 24d9 4000 7106 4944 81fa 8015 E..0$.@.q.ID....
0x0010: wwxx XXYY 04e8 04cd 9c96 c5ea 99a8 7cfb ...{..........|.
0x0020: 7012 05b4 5ed4 0000 0204 05b4 0102 0403 p...^...........
07:11:51.517325 IP (tos 0x0, ttl 113, id 21659, offset 0, flags [DF],
proto: TCP (6), length: 48)
129.250.128.21.1252 > www.xxx.yyy.zzz.1070: S, cksum 0xa40c (correct),
1381904945:1381904945(0) ack 2301854615 win 1460
0x0000: 4500 0030 549b 4000 7106 764c 81fa 8015 E..0T.@.q.vL....
0x0010: wwxx XXYY 04e4 042e 525e 3231 8933 8397 ........R^21.3..
0x0020: 7012 05b4 a40c 0000 0204 05b4 0102 0403 p...............
In English, 129.250.128.21 is sending SYN ACK packets to the obfuscated IP addresses. 129.250.128.21 is sending SYN ACK packets because it is replying to SYN packets from the obfuscated IP. I knew right away what was happening. Some people called it "backscatter." When I wrote my first paper on the subject in 1999 I used the exceptionally exciting term "third party effects," thereby consigning my research to some vague corner of the Internet. In brief, 129.250.128.21 was being SYN flooded; it is a victim, not a perpetrator.
The ISC handlers wrote:
Generally the source ports are 80, 6667, 6666, and 443. However there are also a number of packets with random source ports between 1024 and about 1300. Ports 1024-1300 are also used as the target port. Looking at the Dshield data for ports 6666 and 6667 there was a small peak for these ports as source ports on the 27/28 Feb.
So we are seeing multiple hosts sending SYN,ACK with truncated option to targets (sometimes both scan the same IP) from known ports. The response is typically a RST, or RST,ACK. The window size is also often changed in the response.
What it looks like is a mapping/fingerprinting exercise using the target stacks' response to bad options. But at this stage the end goal is unclear.
I thought this was not right, and I told the handlers as much. They didn't believe me, so I asked them if they thought it might be a good idea to step out of the cave of packet conspiracies and ask the owner of 129.250.128.21 what was happening. (This was the strategy I followed in my original paper.) They decided not to pursue this non-technical investigative method, so I emailed hostmaster [at] ameri [dot] ca based on the resolution for the IP: compton.ameri.ca.
Since then I've been exchanging emails with Brad Dreisbach. He wrote:
i have been getting tcp syn attacked for about 3 weeks now. i have re-installed the OS on the host just to be safe, but im fairly sure my systems are secure. i have also taken measures with my upstream, whom i also work for, to migitate the attack. some stuff is still getting through but at this point im just waiting for the attackers to give up...
There you go -- Brad is being attacked. 129.250.128.21 is the victim. 129.250.128.21 is not doing some kind of reconnaissance.
When I asked Brad if he had captured any traffic related to his ongoing denial of service attack, he replied:
this pcap file doesnt appear to be a tcp syn attack, but some other form of tcp attack. whoever is doing this is changing their stategy though. the attack has changed a few times over its duration. they even attacked me via ipv6 for a few hours.
Here's part of the traffic he sent:
2007-02-22 17:40:20.844503 IP (tos 0x0, ttl 119, id 5698, offset 0,
flags [DF], proto: TCP (6), length: 40)
68.102.133.89.3072 > 129.250.128.21.80: .,
cksum 0xd7b5 (correct), 0:0(0) ack 0 win 0
0x0000: 4500 0028 1642 4000 7706 21bf 4466 8559
0x0010: 81fa 8015 0c00 0050 0000 0000 0000 0000
0x0020: 5010 0000 d7b5 0000
2007-02-22 17:40:20.855549 IP (tos 0x0, ttl 120, id 48455, offset 0,
flags [DF], proto: TCP (6), length: 40)
72.207.227.118.3072 > 129.250.128.21.80: .,
cksum 0x752f (correct), 0:0(0) ack 0 win 0
0x0000: 4500 0028 bd47 4000 7806 1733 48cf e376
0x0010: 81fa 8015 0c00 0050 0000 0000 0000 0000
0x0020: 5010 0000 752f 0000
2007-02-22 17:40:20.948933 IP (tos 0x0, ttl 118, id 3193, offset 0,
flags [DF], proto: TCP (6), length: 40)
71.237.133.30.3072 > 129.250.128.21.80: .,
cksum 0xd469 (correct), 0:0(0) ack 0 win 0
0x0000: 4500 0028 0c79 4000 7606 293c 47ed 851e
0x0010: 81fa 8015 0c00 0050 0000 0000 0000 0000
0x0020: 5010 0000 d469 0000
This looks like an ACK flood to port 80 TCP. The victim will reply with RST packets (not RST ACK) if it can.
I think ISC got thrown off the trail because SANS has a history of seeing global conspiracies in weird packet patterns. This dates back almost ten years and is documented in my first book. The unfortunate result of this attitude is hundreds, if not thousands, of other security analysts have "learned" to see these patterns as malicious too. ISC also spent too much time concentrating on "broken TCP options" on the backscatter traffic. They did not accept my observation that victims under DoS attack tend to respond slowly, weirdly, and eventually not at all when they are flooded.
If you decide to post a nasty reply here because you love ISC and SANS, please keep in mind you're only seeing the public side of the story. ISC and I exchanged very polite emails, but I am not going to disclose the additional justifications of the ISC evil packet theory here because those emails were private.
I think ISC does a great service to the security community, but I would like to see SANS officially abandon its adherence to this misguided view of the world. I am not trying to pick a fight with SANS. All I want is for security analysts to know this pattern is often seen as "malicious" by many when it is utterly benign -- at least as far as the backscatter recipient is concerned. Remember the site sending the SYN ACK, RST ACK, or RST traffic is responding to a SYN flood (to an open or closed port, respectively) or to an ACK flood.
The moral of the story is that it is not a good idea to assume the world is out to get you when a stray packet arrives at your doorstep. A second lesson is that it sometimes make sense to investigate issues by doing human analysis rather than peering at the world through the eyes of Tcpdump/Ethereal/Wireshark. A third lesson is that it makes little sense to interpret traffic as being malicious if the probablility of the "scanner" learning anything remotely useful is extremely low.
Update: One of my friends noted that the Virginia Tech DShield has 129.250.128.21 listed as one of their "Top 10 Most Wanted" "attackers". This demonstrates the trouble with automated reporting systems.
Update 2: Be sure to read the exciting conclusion here.
Comments
1) juno-z (tcp syn) and bang.c (tcp reflection) both use invalid or wrong tcp options... seems to be common among dos/ddos tools - probably intended to break the tools for script kiddies. PoC authors tend to do this sort of thing for all kinds of vulnerability disclosures
2) ACK storms make me think of TCP session hijacking
your analysis was great. SANS is not on my approved security information resource list. ISC and DShield clearly need to take reflection-type attacks into account or their services will remain mostly worthless and certainly unscientific.
I've found in my speaking to security and law enforcement as well as my own experiences in corporate IT/security that conspiracies are great fiction and make for dramatic stories. But typically, crime and attacks (analog or digital) are really not that conpiracy-laden or intricate.
However, I'm not seeing (direct, if any) correlation between the packets you got from Brad and those that have been posted on the ISC site.
Specifically, from Brad you got this:
68.102.133.89.3072 > 129.250.128.21.80
72.207.227.118.3072 > 129.250.128.21.80
71.237.133.30.3072 > 129.250.128.21.80
This is clearly a Syn DoS (if there is enough traffic to be a DoS attack).
However, notice the source ports coming from Brad's IP addresses in ISC's post:
129.250.128.21.1256 > www.xxx.yyy.zzz.1229
129.250.128.21.1251 > www.xxx.yyy.zzz.1070
In other words, for this traffic to happen, Brad's server would need services listening on ports 1251 and 1256. Have you confirmed with him that this really happened? Otherwise, I would expect RST packets coming from his machine.
Do you have any explanation for this?
As far as what SANS posted, the SYN ACKs from source ports 1256 and 1251 do imply that those ports were listening and replied. That sounds odd to me too. However, stranger things have happened to boxes under various forms of DoS attack.
However, the other thing sounds really strange to me. I don't know that a box under a DoS attack (as Brad's would be, according to your theory) would mess up the TCP flags (put S/A instead of R), and with all that calculate a proper checksum for the packet.
There is one more difference that you didn't comment and the packet capture from Brad doesn't show that.
Packets on ISC's web all had extra (and sometimes incorrect) TCP options. The ACK flood coming to Brad's machine that you posted has no TCP options. It would be interesting to see, from the same capture, how Brad's server replied back and if it set those TCP options.
I know this all can be attributed to a server under a DoS stress, but there are a lot of factors here it looks.
My point here is that there is *way* too much guessing in both your post and ISC's, without any conclusive points (from both sides).
We don't know if Brad has a firewall (why didn't you ask him), we don't know what services he was running at the time (why didn't you ask him - services on ports 1251 and 1256 are not common so he should be aware of them), we don't know how his system (did you ask him what OS it is?) would behave under a DoS attack ...
I'm not questioning that there was an ACK flood against Brad's box - that's clear from the posted pcap files (so don't bother with the C&C traffic).
But, in my opinion, there is nothing connecting that with the SYN/ACK packets posted on ISC's web site.
Unless you get packet dumps from Brad, showing that his machine really sent those packets, your only argument seems to be that "you've seen machines do weird things under DoS attacks". While I'm not saying that's not possible, it is something that I'd like to see more proof about.
Below is a small snippet of the traffic:
-----------
Feb 26 16:59:16 xx.xx.xx.xx (xx.xx.xx.xx:6667) :ESP|846305!njhvef@xx.xx.xx.xx PRIVMSG ##r0x## :nzm
(tcp.plg) »» Done with ack flood to IP: 129.250.128.21. Sent: 19186 packet(s) @ 2KB/sec (1MB).
Feb 26 16:59:16 xx.xx.xx.xx:6667 :ESP|846305!njhvef@xx.xx.xx.xx PRIVMSG ##r0x## :nzm (tcp.plg) »» Done
with ack flood to IP: 129.250.128.21. Sent: 19186 packet(s) @ 2KB/sec (1MB).
Feb 26 16:59:23 xx.xx.xx.xx:6667 :ESP|187844!guwcpbmq@xx.xx.xx.xx PRIVMSG ##r0x## :nzm (tcp.plg) »»
Done with ack flood to IP: 129.250.128.21. Sent: 49633 packet(s) @ 7KB/sec (2MB).
Feb 26 16:59:24 xx.xx.xx.xx (xx.xx.xx.xx:6667) :ESP|187844!guwcpbmq@xx.xx.xx.xx PRIVMSG ##r0x## :nzm
(tcp.plg) »» Done with ack flood to IP: 129.250.128.21. Sent: 49633 packet(s) @ 7KB/sec (2MB).
Feb 26 16:59:52 xx.xx.xx.xx:6667 :PRT|113722!owfxzrp@xx.xx.xx.xx.rev.xxximus.pt PRIVMSG ##r0x## :nzm
(tcp.plg) »» Done with ack flood to IP: 129.250.128.21. Sent: 47952 packet(s) @ 7KB/sec (2MB).
Feb 26 16:59:52 xx.xx.xx.xx (xx.xx.xx.xx:6667) :PRT|113722!owfxzrp@xx.xx.xx.xx.rev.xxximus.pt PRIVMSG
##r0x## :nzm (tcp.plg) »» Done with ack flood to IP: 129.250.128.21. Sent: 47952 packet(s) @ 7KB/sec (2MB).
Feb 26 17:00:12 xx.xx.xx.xx:6667 :SWE|925314!htniygb@cxx.xx.xx.xx.xxxxband.comhem.se PRIVMSG ##r0x##
:nzm (tcp.plg) »» Done with ack flood to IP: 129.250.128.21. Sent: 2151325 packet(s) @ 315KB/sec (123MB).
Anonymous, I'm afraid your demands for full knowledge of the entire incident will not be met!
I consider this case closed. If you want more details try asking ISC. Too bad they don't feel the need to say anything more about this event despite all the evidence I dug up here.
great post.
-dave
It is my honest belief that the ISC personnel are attempting to completely understand what is going on. If Richard's hypothesis is correct, then under what operating systems and what exact conditions will cause the response seen in the packets.
In the presentation, 3 different IPs were shown responding in the exact same way. SOOOOO. Under Richard's assertion, all three systems were under a Denial of Service and the responses were malformed by the victim system. That is all well and good and believable. And I guarantee the ISC understands that.
But then no one asks the scientific question of.... why does it occur? What OS and/or tcp/ip stack responds in this odd way? Is the inappropriate response caused solely as the result of the odd packets being sent the victim site or is it more of a case of the system overload that causes this type of packet? Is there a security weakness, or even a reliability issue in that particular set of OS and tcp/ip stack that may need to be addressed ? Could this type of response occur under other conditions as in not a Denial of Service attack, but actual overload of the system with legit traffic due to improper sizing of the system or not tuning appropriately?
It seems all the ISC is trying to do is determine why the identical bad TCP options from multiple systems.