TSA Lessons for Security Analysts
In the past I've run several security teams, such as the Air Force CERT's detection crew and the MSSP division of a publicly traded company. In those positions I was always interested in assessing the performance of my security analysts. The CNN article TSA tester slips mock bomb past airport security contains several lessons which apply to this domain.Jason, a covert tester for the Transportation Security Administration, has been probing airport weaknesses for five years, beginning with big mock bombs before switching to ever smaller devices as the TSA adapts to evolving terrorist threats...
Even before the September 11, 2001, terror attacks, government agencies deployed "red teams" such as this one to look for holes in airport security...
But instead of running from tests, the agency has embraced the idea that testing has a value that goes beyond measuring the performance of individual screeners.
Tests, the TSA says, can show systemwide security vulnerabilities...
[S]creeners who fail to detect contraband are "pulled off the line" and retrained before being allowed back.
The test CNN witnessed was conducted by the TSA's Office of Inspection, which the agency calls the most sophisticated of its covert tests. But there are others.
For starters, every TSA X-ray machine has a Threat Image Projection system, which digitally inserts images of guns, knives and bombs into the X-rays of luggage, to keep screeners alert...
If screeners observe a suspicious object, they can check with the simple click of a computer mouse. If they detect a threat object, the computer congratulates them. Successes and failures are recorded for use in a screener's performance evaluation and are factors in determining pay.
Some 69,929 threat image tests are conducted on an average day, or more than 25 million tests per year. An array of other tests also are conducted to assess screeners, including the red team ones.
I've described elsewhere why I support red teams. I certainly recognize that one of my Three Wise Men savages red teams, but I've never seen anything else -- short of an actual incident -- make a dent in the attitudes of management. Furthermore, red teaming, as a real-life test, tends to discover and link vulnerabilities in ways not anticipated by some vulnerability assessors (blue teams) and general security architects. There's no ground truth like saying "I accomplished the mission using this method" when someone is claiming their network is "secure."
I also like the method to test analysts by inserting false images. Fighting analyst boredom is a big problem in some operational teams.
Comments
I was a TSA screener over the holidays in 2002, so my knowledge is pretty dated. But I can tell you that the TIPS system (Threat Image Projection Sys) was pretty limited at the time and not really as good a training aid as an alertness monitor. Essentially, most of the TIPS images were so obvious that you'd have to be totally zoned out to miss them. And yes, we would get multiple TIPS images during a shift.
Just recently I saw a traveler preparing to go through a checkpoint, just before going through the scanner, he placed his cell phone and bluetooth in the tray on the xray belt, walked through the scanner, picked his bluetooth up plaed it on his ear while he picked up his shoes and carry on tray. I was not close enough to hear but seriously doubt he missed much of his call.
The length of time people are allowed to observe, describe over cell phone calls and interact with loved ones or friends up to the screener area, any effort for security would be quickly accounted for and defeated.