SQL Injection Challenge and Time-Based Security

Thanks to this Tweet by @ryancbarnett, I learned of the lessons learned of the Level II component of the ModSecurity SQL Injection Challenge.

As stated on the challenge site, the goal is "To successful execute SQLi against the scanning vendor demo websites and to try and evade the OWASP ModSecurity CRS." The contestants need to identify a SQL injection vector within one of four demo websites, then enumerate certain information from the target.

As also stated on the challenge page, "Winners of this level will be anyone who is able to enumerate the data listed above for each demo app without triggering an Inbound ModSecurity Alert. If ModSecurity sees any inbound attacks or outbound application defects/info leakages, it will prepend a warning banner to the top of the page."

This is interesting, but what caught my attention is the time-based security metrics describing the results of Level II of the challenge. I'll reproduce the relevant section here:

Hacking Resistance (Time-to-Hack)

Many people wrongly assume that installing a Web Application Firewall will make their sites "Hack Proof." Sadly, this is not reality. The real goal of using a web application firewall should be to gain visibility and to make your web applications more difficult to hack meaning that it should take attackers significantly more time to hack a vulnerable web site with a WAF in front in blocking mode vs. if the WAF was not present at all.

The idea is to substantially increase the "Time-to-Hack" metric associated with compromising a site in order allow for operational security to identify the threat and take appropriate actions...

With this in mind, we analyzed how long it took for each Level II winner to develop a working evasion for the CRS v2.2.0. We are basing this off of the correlated IP address in the logs that was tied to the final evasion payloads submitted to the ModSecurity team. We also saw that many Level II winners actually tested their payloads using the CRS Demo page so we had to correlate test payloads there as well.

Avg. # of Requests to find an evasion: 433
Avg. Duration (Time to find an evasion): 72 hrs
Shortest # of Requests to find an evasion: 118
Shortest Duration (Time to find an evasion): 10 hrs

This data shows that having active monitoring and response capabilities of ongoing web attacks is paramount as it may only a matter of hours before a determined attacker finds a way through your defenses.

I [Ed: Ryan, not Richard] realize that there are a multitude of variables and conditions involved where people can say that these numbers are off (either too high or too low) depending on your defenses and attacker skill level. Keep in mind that this metric was obtained from the ModSecurity WAF using mainly a negative security model ruleset. The point of presenting this data, however, is to have some form of metric available for active web application monitoring and defense discussions related to exploitation timelines.

What a great use of empirical data to make a point about security! Like Ryan says, you can argue about the rating of the intruder (does 10 hours really reflect a skilled intruder?) or the defenses (is ModSecurity really sufficient?). I'd answer that they those aspects of the challenge are sound enough to use as benchmarks for a certain portion of the threat community and state-of-the-practice for defenses.

Ten hours, then, represents the window of time between when an intruder would first start trying to compromise the Web app, and when he succeeded. That means the IR team has no more than 10 hours to detect the activity and take action to close the window of vulnerability. That's a tall order, but we have a metric now based on more than hand-waving that we can use to start a discussion of capabilities.

On a related note, this is the sort of activity that a red team could undertake to simulate threat action and identify IR team effectiveness.

Tweet