MSSPs: What Really Matters
Bamm Visscher pointed me to this Security Incite post about the new NWC article Managed Security Service Providers by Joanne VanAuken. Ms. VanAuken managed to get five MSSPs -- BT Global Services, Cybertrust, Internet Security Systems, LURHQ and SecureWorks -- to answer. The others?
VeriSign Managed Security Services initially accepted and did an outstanding job completing its RFI, but backed out because of the risk of exposing too much confidential data by publishing its RFI responses online (a requirement to participate). Equant also initially accepted but could not complete our RFI in time due to its rebranding to Orange Business Services. MCI, which has partnered with Verizon Business, declined. Accenture, AT&T Networking Outsourcing Services, Capgemini, Computer Sciences Corp. (CSC), Connetic, EDS, Getronics, IBM Global Services, Perimeter Internetworking, Science Applications International Corp (SAIC), Solutionary, Sprint, Symantec, TruSecure, Unisys and VigilantMinds didn't respond to our invitation.
Symantec didn't respond? And why wasn't Counterpane invited?
Here are a few fun excerpts from the story.
The million dollar question is, Which will be the safest choice? Investigate the financial health of your MSSP. If a provider meets your SLA (service-level agreement) and technical requirements but its business road map and financial health are questionable, walk away.
"The safest choice?" What about caring if the MSSP can deliver?
[MSSP] vendor relationships mean MSSPs often receive advance notice of worm and viruses outbreaks...
Vendor to MSSP: "Hey, a worm outbreak is coming. Watch out!"
MSSP to Vendor: "Ok, thanks."
This doesn't happen. If anything, the MSSP might be lucky to see worm activity and then notify its customers.
All vendors indicated a focus on preventing, rather than detecting, intrusions... In the rare event that an MSSP does suspect a compromise, all vendors said that escalations are immediately executed while potential damage is contained through every countermeasure available until both the MSSP and client agree on whether a compromise actually occurred.
Of course "prevention" is king. Unfortunately, focus on "prevention" often drowns out considerations for detection. As a result, compromise is seldom "rare."
After evaluating five RFI proposals, we gave our Editor's Choice to Internet Security Systems because of it wealth of service offerings, outstanding SLA agreements, highly skilled personnel and bundled pricing options.
"Highly skilled" may indeed be true, but how does NWC know this?
My overall assessment of this "review" is that was a large amount of work for NWC to write the RFI (.doc) and for the vendors to respond. In the end, the process identified many plausibly necessary conditions for running a good MSSP, but none are sufficient to assess if the MSSP is any good at its core task: helping customers resist, detect, and respond to intrusions. (I no longer say "prevent" intrusions. "Resist" seems more accurate when prevention eventually fails.)
Take a look at the Report Card to see how the MSSPs were rated. 25% is assigned to "Service Levels." 20% is "Price." 10% is "Environmental," like physical security and facility features. That's 55% of the score based on non-technical metrics. The remaining areas, "Support" at 25% and "Security Practices" at 20%, dance around the core issue. Items like "Technical Expertise/experience," at 10%, appears to have been measured by "certifications." Throw that out the window. A portal at 7%? Again, irrelevant. Aside from 4% for equipment compatibility and 4% for research organization partnerships, the remaining metrics are all based on plans, policies, and other paperwork.
This is similar to the football analogy I offered last month. Think about the difference in focus. From the review:
[Two MSSPs] earned an edge by describing their power, air conditioning and fire-prevention setups in granular detail, including such items as contracts with diesel fuel suppliers, audits of power performance, the target temperature for SOC (security operations center) rooms, and the time (in seconds) allowed after a smoke alarm sounds for employees to leave the room before the fire-suppression system kicks in.
Great -- will these MSSPs notice if I use stolen credentials to access a monitored customer resource, install a back door, and remove proprietary information using a stateless covert channel?
It's easy for me to criticize, you might say. How would I have done such a review? Well, I have done them. Sometimes I review MSSPs, and sometimes I review in-house teams. My company's NSM Assessment and Evaluation is a two-phase process that works for in-house or outsourced security teams. I use a self-modified version of the NSA-IAM for the Assessment and the NSA-IEM for the Evaluation. The NWC review would be similar to the Assessment, since it is non-technical and hands-off. The Evaluation is technical and hands-on, and tests how well all the MSSP people, products, and processes actually perform.
I've also trained some of the analysts at VeriSign (ex-Guardent), and I've met or spoken several times with the folks at LURHQ and Verizon (ex-NetSec). I also know people at many of the other MSSPs.
If you want to know more, email me: richard [at] taosecurity [dot] com.
As far as Mike Rothman's views go, he says this:
Yes, MSS is a Commodity. So what... There seem to be very little outward differentiation and the biggest decision point is probably how long it takes you to wade through all of the pricing and packaging options. But this is good news, in that the business is mature enough to have tight operational processes - which is pretty much what you want to see in an outsourcing situation.
Ugh, that's the problem. If there's no outward differentiation, then what's being measured may be irrelevant to the core problem. As Bamm Visscher asks, "Is your MSSP just a 'Security Device Management Provider'?"
VeriSign Managed Security Services initially accepted and did an outstanding job completing its RFI, but backed out because of the risk of exposing too much confidential data by publishing its RFI responses online (a requirement to participate). Equant also initially accepted but could not complete our RFI in time due to its rebranding to Orange Business Services. MCI, which has partnered with Verizon Business, declined. Accenture, AT&T Networking Outsourcing Services, Capgemini, Computer Sciences Corp. (CSC), Connetic, EDS, Getronics, IBM Global Services, Perimeter Internetworking, Science Applications International Corp (SAIC), Solutionary, Sprint, Symantec, TruSecure, Unisys and VigilantMinds didn't respond to our invitation.
Symantec didn't respond? And why wasn't Counterpane invited?
Here are a few fun excerpts from the story.
The million dollar question is, Which will be the safest choice? Investigate the financial health of your MSSP. If a provider meets your SLA (service-level agreement) and technical requirements but its business road map and financial health are questionable, walk away.
"The safest choice?" What about caring if the MSSP can deliver?
[MSSP] vendor relationships mean MSSPs often receive advance notice of worm and viruses outbreaks...
Vendor to MSSP: "Hey, a worm outbreak is coming. Watch out!"
MSSP to Vendor: "Ok, thanks."
This doesn't happen. If anything, the MSSP might be lucky to see worm activity and then notify its customers.
All vendors indicated a focus on preventing, rather than detecting, intrusions... In the rare event that an MSSP does suspect a compromise, all vendors said that escalations are immediately executed while potential damage is contained through every countermeasure available until both the MSSP and client agree on whether a compromise actually occurred.
Of course "prevention" is king. Unfortunately, focus on "prevention" often drowns out considerations for detection. As a result, compromise is seldom "rare."
After evaluating five RFI proposals, we gave our Editor's Choice to Internet Security Systems because of it wealth of service offerings, outstanding SLA agreements, highly skilled personnel and bundled pricing options.
"Highly skilled" may indeed be true, but how does NWC know this?
My overall assessment of this "review" is that was a large amount of work for NWC to write the RFI (.doc) and for the vendors to respond. In the end, the process identified many plausibly necessary conditions for running a good MSSP, but none are sufficient to assess if the MSSP is any good at its core task: helping customers resist, detect, and respond to intrusions. (I no longer say "prevent" intrusions. "Resist" seems more accurate when prevention eventually fails.)
Take a look at the Report Card to see how the MSSPs were rated. 25% is assigned to "Service Levels." 20% is "Price." 10% is "Environmental," like physical security and facility features. That's 55% of the score based on non-technical metrics. The remaining areas, "Support" at 25% and "Security Practices" at 20%, dance around the core issue. Items like "Technical Expertise/experience," at 10%, appears to have been measured by "certifications." Throw that out the window. A portal at 7%? Again, irrelevant. Aside from 4% for equipment compatibility and 4% for research organization partnerships, the remaining metrics are all based on plans, policies, and other paperwork.
This is similar to the football analogy I offered last month. Think about the difference in focus. From the review:
[Two MSSPs] earned an edge by describing their power, air conditioning and fire-prevention setups in granular detail, including such items as contracts with diesel fuel suppliers, audits of power performance, the target temperature for SOC (security operations center) rooms, and the time (in seconds) allowed after a smoke alarm sounds for employees to leave the room before the fire-suppression system kicks in.
Great -- will these MSSPs notice if I use stolen credentials to access a monitored customer resource, install a back door, and remove proprietary information using a stateless covert channel?
It's easy for me to criticize, you might say. How would I have done such a review? Well, I have done them. Sometimes I review MSSPs, and sometimes I review in-house teams. My company's NSM Assessment and Evaluation is a two-phase process that works for in-house or outsourced security teams. I use a self-modified version of the NSA-IAM for the Assessment and the NSA-IEM for the Evaluation. The NWC review would be similar to the Assessment, since it is non-technical and hands-off. The Evaluation is technical and hands-on, and tests how well all the MSSP people, products, and processes actually perform.
I've also trained some of the analysts at VeriSign (ex-Guardent), and I've met or spoken several times with the folks at LURHQ and Verizon (ex-NetSec). I also know people at many of the other MSSPs.
If you want to know more, email me: richard [at] taosecurity [dot] com.
As far as Mike Rothman's views go, he says this:
Yes, MSS is a Commodity. So what... There seem to be very little outward differentiation and the biggest decision point is probably how long it takes you to wade through all of the pricing and packaging options. But this is good news, in that the business is mature enough to have tight operational processes - which is pretty much what you want to see in an outsourcing situation.
Ugh, that's the problem. If there's no outward differentiation, then what's being measured may be irrelevant to the core problem. As Bamm Visscher asks, "Is your MSSP just a 'Security Device Management Provider'?"
Comments