Tuesday, July 06, 2010

Ponemon Institute Misses the Mark

Today the Ponemon Institute announced results of a survey they conducted titled Growing Risk of Advanced Threats: Study of IT Practitioners in the United States. Unfortunately, this survey looks like it is mainly the blind asking the blind to describe a threat neither really understands. For example, the survey states:

While the definition of what constitutes an advanced threat still varies within the industry, for purposes of this research we have defined an advanced threat as a methodology employed to evade an organization’s present technical and process countermeasures which relies on a variety of attack techniques as opposed to one specific type.

The predominant majority of these threats are represented by unknown, zero-day attacks, but there are increasingly many instances where known attacks are being re-engineered and repackaged to extend their usefulness.

If this survey stuck with this definition, and didn't mention Advanced Persistent Threat, then I could possibly live with it. Unfortunately they veer off into the land of speculation and confusion with questions and answers like the following:

Q1d. What other terms are used to describe an advanced threat? Please select all that apply.

  • Advanced persistent threat (50%)

  • Emerging threat (41%)

  • Spear-phishing (38%)

  • SQL Injection (31%)

  • Cyber warfare (25%)

  • Continuous attack (21%)

  • Cyber terrorism (21%)

  • Denial of service attack (19%)

Please. No. Make it stop. It's bad enough to pollute the APT term with the "advanced threat" definition Ponemon manufactured, but now it includes SQL injection and DoS? And the statement "the predominant majority of these threats are represented by unknown, zero-day attacks" in no way describes how APT acts. They can elevate to research, weaponize, and use zero-day, but that does not define them.

The ultimate shame is seeing SearchSecurity.com fall for this with their article More firms targeted by advanced persistent threats, study finds:

Advanced persistent threats (APTs), which are carried out by organized cybercriminal groups, may be a growing trend as a new survey finds an increase in advanced threats over the last 12 months.

No. APT is not cybercrime.

While there might be some interesting survey data in the Ponemon results, please don't think for a second it has anything to do with APT whatsoever.


Anonymous said...

Don't take it personally, there are lots of really bad writers out there who don't realize it when they totally use the wrong terms, or simply don't understand enough to use terms correctly.

webjedi said...

We had this same debate at DC3 over terminology - APT... what is it? What defines advanced, what defines persistent, and of course, what defines the threat.

Problem here is, like the lawmakers, it's usually that "blind leading the blind" scenario. I had to handle and OMB data call or two recently for my agency. Oh My God... seriously... it was a laugh. Then you can get into... what the heck are they planning on doing with all the data they expect to get from tools that are CyberScope "ready" - there's this massive effort to roll all these (as yet to be defined) metric up to OMB... do they have the skills and talent to decipher and act, honestly, upon all of this.

Then Pokemon, I mean Ponemon, muddies the water by a poorly designed/worded survey survey. Well of course people get confused and perpetuate this misunderstanding.

When will it end?