Wednesday, August 02, 2006

Analog Penetration Testing

While watching the evening news I saw the story Investigation: U.S. borders perilously porous -- Federal investigators easily pass border checks using fake identification. On Wednesday the Government Accountability Office (yes, they changed their name) will release a report on an analog penetration test performed against the US border. What do I mean by that?

[GAO] agents successfully entered the United States using fictitious driver's licenses and other bogus documentation through nine land ports of entry on the northern and southern borders. CBP [Customs and Border Protection] officers never questioned the authenticity of the counterfeit documents presented at any of the nine crossings.

On three occasions -- in California, Texas, and Arizona -- agents crossed the border on foot. At two of these locations -- Texas and Arizona -- CBP allowed the agents entry into the United States without asking for or inspecting any identification documents.


This excerpt is from a draft report (.pdf) which will be delivered by GAO to the US Senate on Wednesday. Initial reports indicate lawmakers are really upset by these findings, because the situation has not improved since the last test in 2003.

What does this tell me? Apparently, decision-makers listen when findings are presented in a simple manner. If CBP fails to prevent people with forged IDs from entering the country, then it's clear they are not fulfilling their mandate. Simulating threat activity and discovering that attacks succeed 100% of the time is a damning critique of one's security measures. When presented in this manner, it's easy to see what works and what doesn't.

This is why I advocate penetration tests as a means to assess security. If it takes me five minutes to gain access to information you expect to keep private, that's a clear indication your organization has serious security problems. It's performance-based security measurement. Just how well do your people, products, and processes handle a real event?

This sort of thinking is second nature to anyone with military, law enforcement, or fire fighting backgrounds. (I'm sure there are others -- feel free to name them as comments.) These organizations assess their capability to perform their missions by exercising. Sure, you should take inventories, theorize, and so on, but the proof lies in how well you can execute in a near-real-world environment. (Executing in the real world is obviously the best test, but you don't want to put people's lives on the line unnecessarily.)

Do you want to know how well your airport screeners detect weapons in luggage? Don't measure your training budget, the education level of the personnel, or the number of steps in their checklist. Run fake weapons through X-ray machines and see who catches them.

How well is border security inspecting IDs? Don't count increases in the numbers of agents, measure their salaries, or inspect their guidebooks. Send agents across the border with fake IDs and see if CBP stops them.

How well does your enterprise protect sensitive information from unauthorized access? Don't pretend to assess threats, assign fake risk values, and count the number of packets blocked by your firewall. Hire a pen tester to steal your information.

Repeat the process in 6 months and see if it's more difficult. If yes, your security has improved. If no, your security has degraded. It's really as simple as that. Be careful to ensure the second pen tester is as skilled as, or superior to, the first pen tester.

5 comments:

Anonymous said...

I saw the same story on another nightly news show - the worst part was that when presented with 3 drivers licenses, a bartender picked out the fake one - from West Virginia.

LonerVamp said...

Excellent post! This is one of those examples that really digs home for non-technical people on the use of pen-testing, especially with a few real world examples.

"Don't measure your training budget, the education level of the personnel, or the number of steps in their checklist. Run fake weapons through X-ray machines and see who catches them.

"How well is border security inspecting IDs? Don't count increases in the numbers of agents, measure their salaries, or inspect their guidebooks. Send agents across the border with fake IDs and see if CBP stops them."

All I can say is, Boom! I just want to say "yes, yes, yes!" and show this to every IT buddy I have around.

Is there where a difference in "security assessment" and "penetration test" come into play?

Bastian Schwittay said...

I would think that during an assessment, people know that you are actually assessing and examining their security measures, perhaps you might even work together with them. During a pen-test, ideally only a few people actually know that it is going to occur, often only management and not the ones responsible for security measures.
Right?

LonerVamp said...

I've typically heard "red team" or even "black bag" as terms for a secretive pen-test where even the admins of the systems didn't know it was going on, thus testing their own processes and not just the technology.

Anonymous said...

Yes, there should be one to five trusted agents when performing a pen-test who know what is going on. Unfortunately, this rarely happens because of what I like to call, Capitol Hill Syndrome, or CHS. People can't keep their mouths shut and cause skewed results that affect the accuracy of the testing/mission. Or, they don't want to get embarrassed and step up security to unrealistic levels that hinder operational productivity. Once the pen testers are gone, it's back to SNAFU.

John Collins