tag:blogger.com,1999:blog-4088979.post7536529650657216876..comments2023-10-16T06:06:25.012-04:00Comments on TaoSecurity Blog: No Undetectable BreachesRichard Bejtlichhttp://www.blogger.com/profile/13512184196416665417noreply@blogger.comBlogger3125tag:blogger.com,1999:blog-4088979.post-49908391389836129012009-06-16T06:39:31.853-04:002009-06-16T06:39:31.853-04:00This comment has been removed by a blog administrator.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-4088979.post-17879854032351059052007-07-19T11:36:00.000-04:002007-07-19T11:36:00.000-04:00It's unfair to knock folks that use compliance met...It's unfair to knock folks that use compliance metrics. Compliance is business. And it usually falls, at least in part, to infosec folks to implement and monitor. Compliance failures are their own incidents with their own levels of very real risk. <BR/><BR/>I would also argue that custom compliance-based metrics and canned (Big 4) compliance metrics suffer the same problems. They measure how well you adhere to policy and procedure. Thoughtful policies and procedures are the key. The metrics are secondary.<BR/><BR/>For instance, a policy might say that we monitor and respond to unauthorized use of administrative accounts. And a supporting procedure describes how we define unauthorized use, administrative accounts, describes how we will monitor these accounts, and sets an SLA/OLA for responding to detected violations. A metric might be to look at the number of detected incidents and the number of times that OLA was met or missed. That's pretty custom, and defining who, how, and when admin users are used is a good control. But is focusing on whether the response took 110 minutes or 130 minutes worth anything? Not to the security of the environment, but it could be worth real money when doing risk assessments and resource planning. So was it worth doing? I would have to say yes, even though I died a little inside just now.<BR/><BR/>Bottom line, outcome-based metrics are better at trending the actual security environment you are operating in, but they're hard to build well and sometimes even harder to collect data for. Compliance metrics are useful and have real value, even if that value is contrived from a regulatory requirement that may or may not improve the actual security environment.PaulMhttps://www.blogger.com/profile/02530533566781746778noreply@blogger.comtag:blogger.com,1999:blog-4088979.post-61655354384008164882007-07-18T22:43:00.000-04:002007-07-18T22:43:00.000-04:00Auditors like it because output is more determinis...Auditors like it because output is more deterministic. Vendors like it because they can tailor and position products as "solutions" to fill checkboxes and win earmarked budget. Consultants love it because they can understand and easily replicate it. Opertional security folks, hrmm.. perhaps a split; raises the bar on one hand, but imposes potentially needless requirements on the other. The offshoot for the end-user is that compliance is arguably a waiver of sorts in the event that something actually happens; if the checkboxes are filled, that is -- whether it actually enhances operational security or not. <BR/><BR/>Custom compliance-based metrics (i.e., versus the canned versions running about as of late), OTOH, coupled with what you're calling field-assessed and outcome-based metrics, are certainly more effective, IMO. <BR/><BR/>Finally, regarding your comment "I categorically reject the notion that it is possible to suffer sustained, completely undetectable breaches and remain unaware of the damage", I tend to agree, though considering the window for detection could be anywhere from seconds to decades, and if the latter, which it certainly has and can be, changes things.Anonymousnoreply@blogger.com