tag:blogger.com,1999:blog-4088979.post115227946697154351..comments2023-10-16T06:06:25.012-04:00Comments on TaoSecurity Blog: Control-Compliant vs Field-Assessed SecurityRichard Bejtlichhttp://www.blogger.com/profile/13512184196416665417noreply@blogger.comBlogger18125tag:blogger.com,1999:blog-4088979.post-41883962875473692342007-12-04T18:08:00.000-05:002007-12-04T18:08:00.000-05:00DoD uses red teams to attack the network ("ethical...DoD uses red teams to attack the network ("ethical hacking" some call this). Heard a recent speach by DHS Asst Secretary Garcia that they are looking at this. DoJ probably will follow suit.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-4088979.post-18403601487984841682007-02-07T12:31:00.000-05:002007-02-07T12:31:00.000-05:00I've worked in the group that Mr. heretick has res...I've worked in the group that Mr. heretick has responsibility for, and much of what is said here is true. However, understand that DOJ is most concerned with not having Congress up their butts. If they do not score "green" on every item, the AG and CIO are held accountable and pummeled until DOJ makes the phony metrics. I spent many years building and testing worthless plans that were auditor proof. So our group looked really good on paper. Everyone knew the plan would not work in real life. Heretick is pressured from above to make the metrics, and he brought the discipline to meet that mandate from DIA, where he worked to MEET THE METRICS! This is what the government does. Be afraid, be very afraid!Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-4088979.post-1155733690772165162006-08-16T09:08:00.000-04:002006-08-16T09:08:00.000-04:00I am sorry I missed this earlier. From first hand ...I am sorry I missed this earlier. From first hand experience, DOJ and Mr. Heretick concentrate way too much on metrics and not enough on 'real security'. They continue to drive the components absolutely nuts with this sort of emphasis.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-4088979.post-1153006140878752122006-07-15T19:29:00.000-04:002006-07-15T19:29:00.000-04:00If I understand the scoring system at the bottom o...If I understand the scoring system at the bottom of the article, the most severe incident that does not have loss of life barely makes it into thwe "high" category.<BR/>No loss of life, high exploitability, low countermeasures, high capability, high history high gain, low attibutability, low detectability, high sensitivity, full loss of operability and equipment - and its only high?<BR/><BR/>What can be more severe than the above situation during an incident?<BR/><BR/>Seriously flawed risk metric scoring, imhoAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-4088979.post-1152899337049345162006-07-14T13:48:00.000-04:002006-07-14T13:48:00.000-04:00I have attended some of the local Seattle ISSA cha...I have attended some of the local Seattle ISSA chapter meetings and they are usually pretty good. You can see a list of current and past Seattle chapter meetings here to get an idea of the topics they address and the speakers they invite.<BR/><BR/>http://www.issa-ps.org/index.php?option=com_content&task=blogcategory&id=14&Itemid=44Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-4088979.post-1152414189952222902006-07-08T23:03:00.000-04:002006-07-08T23:03:00.000-04:00Strike me claiming a post above...I must have not ...Strike me claiming a post above...I must have not submitted it properly. :(<BR/><BR/>- LonerVampAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-4088979.post-1152383428591983592006-07-08T14:30:00.000-04:002006-07-08T14:30:00.000-04:00LonerVamp,I am a member of my local ISSA chapter. ...LonerVamp,<BR/><BR/>I am a member of my local ISSA chapter. They usually have good speakers. I don't know about other ISSA chapters.Richard Bejtlichhttps://www.blogger.com/profile/13512184196416665417noreply@blogger.comtag:blogger.com,1999:blog-4088979.post-1152382135779468162006-07-08T14:08:00.000-04:002006-07-08T14:08:00.000-04:00Forgot to sign the post above...but that was me.On...Forgot to sign the post above...but that was me.<BR/><BR/>On a slightly different note, do you like the ISSA organization? I just recently found out about it and have been looking into membership when I move to the Seattle area. I can bet that experience is based almost fully on the local chapter, but is it fairly similar to Infragard or something? Just curious on your opinion. :)<BR/><BR/>-LonerVampAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-4088979.post-1152330344584261552006-07-07T23:45:00.000-04:002006-07-07T23:45:00.000-04:00Jake - I would never argue that having a plan or a...Jake - I would never argue that having a plan or a policy is a bad thing in itself. It's a _good_thing_ to look at the assets you are protecting and assess whether the protection is good and whether your assessment has any basis in reality. <BR/><BR/>However, I don't think playing games with numbers is a useful tool, and I don't think the report cards Federal Agencies are getting (or the Policy Audit I'm going through) are anything but a waste of time. <BR/><BR/>We already have the plan - now we have to fit it to some bureaucrat's notion of how it should be formatted. We have to describe how we are doing things that cannot be done by anybody, like secure a wireless network (they speak in absolutes, so WEP/WPA does NOT cut it per the plain meaning of the language), or insure that end-users can not modify settings of hosts they have physical possession of. <BR/><BR/>We have to assign ranking numbers and combine them in all sorts of mathematically unsound ways, to come up with answers we already know. It's idiocy.<BR/><BR/>Of course, I could recover some of the time lost by cutting back on my ranting, but what fun is that?JimmytheGeekhttps://www.blogger.com/profile/14515949902737764574noreply@blogger.comtag:blogger.com,1999:blog-4088979.post-1152326880691259782006-07-07T22:48:00.000-04:002006-07-07T22:48:00.000-04:00re: the metrics would "discourage DoJ units from r...re: the metrics would "discourage DoJ units from reporting incidents"<BR/><BR/>The CISO of the DOJ does not have the authority to compel units to report compromises of DOJ assets? Sounds like the issue may be one of a lack of authority and competence.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-4088979.post-1152306174544474442006-07-07T17:02:00.000-04:002006-07-07T17:02:00.000-04:00Jimmy,I catch your drift, but you are getting the ...Jimmy,<BR/><BR/>I catch your drift, but you are getting the main point of adopting Security Policy!<BR/><BR/>Yes, there are some security professionals that make the numbers look better than they really are, but it only hurts them in the end. They will realize this when, as Richard has put it, the company looks at the score.<BR/><BR/>You start out with some idea of what you need to protect, and how imporant they are to protect. Then after either successful attempts or caught attempts, you re-evauluate your equation. <BR/><BR/>Risk assesment allows us to reflect on our own security practices, and refine them to mitigate damange. <BR/><BR/>As quoted from the from the blog itself, "Do you want a defensible network or not?" Being able to defend against any attack is more imporant then believing you can stop them all together.<BR/><BR/>I hope you re-asses the need for a security policy, and see the importance in it. While it seems a waste of time now, you will be able to change your defenses faster and keep the attackers in check than without one.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-4088979.post-1152297925465546332006-07-07T14:45:00.000-04:002006-07-07T14:45:00.000-04:00It's the essence of a bureaucrat to point at a pol...It's the essence of a bureaucrat to point at a policy document and say, "Problem solved. Got this policy here." Now, if we just had a policy that there be no World Hunger...<BR/><BR/>There's a huge danger of skewing the metrics if rewards and punishments are tied to them. People are self-interested, occasionally rational creatures. So it takes some leadership to avoid data-corrupting gamesmanship. That's a scarce commodity. The trick is to reward steps that should generate an improvement in the numbers, rather than basing rewards on the numbers themselves. <BR/><BR/>Part of the problem with the "days since compromise" metric is that not everything is under an organization's control. The number of threats is independent of internal actions. So even if you do everything right, make the most important improvements, etc., you might have more compromises if the quantity and quality of threats (by your definition) goes up. A good leader should be able to reward a team that did everything right, even if a barrage of 0-day attacks succeed. <BR/><BR/>I think it's fundamentally a qualitative discipline, not quantitative. I like the notion of % workstations that are fully patched, but like you say, if misconfigured the patches won't help. Yikes! Computer Security is more like Ice Dancing than the running the 440!<BR/><BR/>As for the risk assessment: every time you perform arithmetic operations on ordinal numbers, God kills a kitten. Both factors of the Vulnerability figure are pulled out of the air. Seriously. How do you rank vulnerabilities you haven't heard about yet? You come up with a SWAG. Threat factors are pulled out of a more odiferous place - "Our unknown assailants are|are not experts." How the hell do you BEGIN to assign a value to that? Can you buy a Threat Meter from Fluke? Even if these were good numbers, they are rankings. You can't take the order of finishers in a race and add that number to the order of height and get anything meaningful! You can't say the second place finisher was twice as slow. All you can say is that runner finished between #1 and #3. <BR/><BR/>And what does the risk assessment try to do??? The probability of an internet connected host with a known vulnerability being compromised over a long enough period is 1. If you are simply doing triage, so the most important stuff gets protected first/best, this is a relatively unhelpful process. You already know the answers, and you'd just pick numbers that will give the results that let you do what you were going to do anyway. The only real impact this kind of thing can have is to misdirect. Say you assign low values to individual workstations, because servers are more important. The assessment drives a decision that workstations get less attention, and they wind up compromised en masse. "But the risk assessment said..." <BR/><BR/>I'm in the middle of a Security Policy audit, and it is the biggest waste of time. I could be taking steps to improve the visibility of my network and hardening its elements. Instead, I am filling up a binder with dead trees. Said binder will reflect, not drive behavior.JimmytheGeekhttps://www.blogger.com/profile/14515949902737764574noreply@blogger.comtag:blogger.com,1999:blog-4088979.post-1152294309607644002006-07-07T13:45:00.000-04:002006-07-07T13:45:00.000-04:00Anonymous,Unfortunately, private companies tolerat...Anonymous,<BR/><BR/>Unfortunately, private companies tolerate millions in losses all the time.Richard Bejtlichhttps://www.blogger.com/profile/13512184196416665417noreply@blogger.comtag:blogger.com,1999:blog-4088979.post-1152292798899777492006-07-07T13:19:00.000-04:002006-07-07T13:19:00.000-04:00The federal government has something called a "DAA...The federal government has something called a "DAA" who is supposed to be responsible for the security of these systems. All they have to do is fire one of them the next time another one is negligent and the federal government would do a 180. Until that time metric will be used to increase their budgets.<BR/><BR/>Would the private sector tollerate such things if it cost a company millions?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-4088979.post-1152289903738208152006-07-07T12:31:00.000-04:002006-07-07T12:31:00.000-04:00Second David,Those are all good items to track, bu...Second David,<BR/><BR/>Those are all good items to track, but they are still not measurements of real security. A fully patched system can still be compromised if misconfigured, misdeployed, etc.Richard Bejtlichhttps://www.blogger.com/profile/13512184196416665417noreply@blogger.comtag:blogger.com,1999:blog-4088979.post-1152289842805496642006-07-07T12:30:00.000-04:002006-07-07T12:30:00.000-04:00David Bianco (to differentiate from other David),Y...David Bianco (to differentiate from other David),<BR/><BR/>You make a good point. I do not want to promote an "ignorance is bliss" attitude. That is why an independent IG-type group would have to do the assessments and also monitor. If the end units are responsible ultimately for reporting their compromises, their incentive is to ignore/lie.Richard Bejtlichhttps://www.blogger.com/profile/13512184196416665417noreply@blogger.comtag:blogger.com,1999:blog-4088979.post-1152288411601021092006-07-07T12:06:00.000-04:002006-07-07T12:06:00.000-04:00Also not well thought out, but what about the foll...Also not well thought out, but what about the following basics:<BR/><BR/>- Percentage of Desktops with fully patched OS<BR/>- Percentage of Desktops with fully patched software<BR/>- Percentage of Servers with fully patched OS<BR/>- Percentage of Servers with fully patched software<BR/><BR/>- Percentage of Workstations and Servers with Anti-Virus and/or Anti-Spyware and basic (aka Windows XP SP2) firewall<BR/><BR/>- Number of Quarters since last internal Pen Test<BR/>- Number of Quarters since last external Pen Test<BR/><BR/>Obviously lower numbers are better...Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-4088979.post-1152286986609064842006-07-07T11:43:00.000-04:002006-07-07T11:43:00.000-04:00Great analogy. I can tell you from first-hand exp...Great analogy. I can tell you from first-hand experience that you're spot on. The current security management paradigm doesn't care so much about the effectiveness of the security system. It's mostly designed to make sure that the program looks good on paper. <BR/><BR/>I think you've left out an important factor in your suggestions for metrics, though. Simply measuring compromises is useless, because in most cases the agency will not know that they are compromised. Using "Days since last compromise of type X" or "System-days compromised" is misleading, since the world's poorest security system would look great by simply failing to detect anything amiss. In order to get good metrics, you'd also need to evaluate the effectiveness of the IDS/NSM in place, perhaps using the pen test team as a yardstick (e.g., "Number of pen test team attacks detected" or "Average response time to pen test incidents").DavidJBiancohttps://www.blogger.com/profile/09760835714791462863noreply@blogger.com