Control "Monitoring" is Not Threat Monitoring
As I write this post I'm reminded of General Hayden's advice: "Cyber" is difficult to understand, so be charitable with those who don't understand it, as well as those who claim "expertise."
It's important to remember that plenty of people are trying to act in a positive manner to defend important assets, so in that spirit I offer the following commentary.
Thanks to John Bambanek's SANS post I read NIST Drafts Cybersecurity Guidance by InformationWeek's J. Nicholas Hoover.
The article discusses the latest draft of SP 800-37 Rev. 1:
DRAFT Guide for Applying the Risk Management Framework to Federal Information Systems: A Security Life Cycle Approach.
I suspected this to be problematic given NIST's historical bias towards "controls," which I've criticized in Controls Are Not the Solution to Our Problem and Consensus Audit Guidelines Are Still Controls. The subtext for the article was:
The National Institute for Standards and Technology is urging the government to continuously monitor its own cybersecurity efforts.
As soon as I read that, I knew that NIST's definition of "monitor" and the article's definition of "monitor" did not mean the real sort of monitoring, threat monitoring, that would make a difference against modern adversaries.
The article continues:
Special Publication 800-37 fleshes out six steps federal agencies should take to tackle cybersecurity: categorization, selection of controls, implementation, assessment, authorization, and continuous monitoring...
Finally, and perhaps most significantly, the document advises federal agencies to put continuous monitoring in place. Software, firmware, hardware, operations, and threats change constantly. Within that flux, security needs to be managed in a structured way, Ross says.
"We need to recognize that we work in a very dynamic operational environment," Ross says. "That allows us to have an ongoing and continuing acceptance and understanding of risk, and that ongoing determination may change our thinking on whether current controls are sufficient."
The continuous risk management step might include use of automated configuration scanning tools, vulnerability scanning, and intrusion detection systems, as well as putting in place processes to monitor and update security guidance and assessments of system security requirements.
Note that the preceding text mentions "intrusion detection systems," but the rest of the text has nothing to do with real monitoring, i.e., detecting and responding to intrusions. I'm not just talking about network-centric approaches, by the way -- infrastructure, host, log, and other sources are all real monitoring, but this is not what NIST means by "monitoring."
To understand NIST's view of monitoring, try reading the new draft. I'll insert my comments.
APPENDIX G
CONTINUOUS MONITORING
MANAGING AND TRACKING THE SECURITY STATE OF INFORMATION SYSTEMS
A critical aspect of managing risk from information systems involves the continuous monitoring of the security controls employed within or inherited by the system.65
[65 A continuous monitoring program within an organization involves a different set of activities than Security Incident Monitoring or Security Event Monitoring programs.]
So, it sounds like activities that involve actually watching systems are not within scope for "continuous monitoring."
Conducting a thorough point-in-time assessment of the deployed security controls is a necessary but not sufficient condition to demonstrate security due diligence. An effective organizational information security program also includes a rigorous continuous monitoring program integrated into the system development life cycle. The objective of the continuous monitoring program is to determine if the set of deployed security controls continue to be effective over time in light of the inevitable changes that occur.
That sounds ok so far. I like the idea of evaluations to determine if controls are effective over time. In the next section below we get to the heart of the problem, and why I wrote this post.
An effective organization-wide continuous monitoring program includes:
• Configuration management and control processes for organizational information systems;
• Security impact analyses on actual or proposed changes to organizational information systems and environments of operation;67
• Assessment of selected security controls (including system-specific, hybrid, and common controls) based on the organization-defined continuous monitoring strategy;68
• Security status reporting to appropriate organizational officials;69 and
• Active involvement by authorizing officials in the ongoing management of information system-related security risks.
Ok, where is threat monitoring? I see configuration management, "control processes," reporting status to "officials," "active involvement by authorizing officials," and so on.
The next section tells me what NIST really considers to be "monitoring":
Priority for security control monitoring is given to the controls that have the reatest volatility and the controls that have been identified in the organization’s plan of action and milestones...
[S]ecurity policies and procedures in a particular organization may not be likely to change from one year to the next...
Security controls identified in the plan of action and milestones are also a priority in the continuous monitoring process, due to the fact that these controls have been deemed to be ineffective to some degree.
Organizations also consider specific threat information including known attack vectors (i.e., specific vulnerabilities exploited by threat sources) when selecting the set of security controls to monitor and the frequency of such monitoring...
Have you broken the code yet? Security control monitoring is a compliance activity. Granted, this is an improvement from the typical certification and accreditation debacle, where "security" is assessed via paperwork exercises every three years. Instead, .gov compliance teams will perform so-called "continuous monitoring," meaning more regular checks to see if systems are in compliance.
Is this really an improvement?
I don't think so. NIST is missing the point. Their approach advocates Control-compliant security, not field-assessed security. Their "scoreboard" is the result of a compliance audit, not the number of systems under adversary control or the amount of data exfiltrated or degraded by the adversary.
I don't care how well your defensive "controls" are informed by offense. If you don't have a Computer Incident Response Team performing continuous threat monitoring for detection and response, you don't know if your controls are working. The NIST document has a few hints about the right approach, at best, but the majority of the so-called "monitoring" guidance is another compliance activity.
Comments
There are, thankfully, still controls involved.
Controls are a good thing. They remind us what needs to be considered when evaluating risk to an organization or information system. They ensure security planners, assessors and maintainers systematically consider many different aspects of security. Every security practitioner has biases when evaluating. Some over-emphasize host based security, some under-emphasize network attacks, some forget about physical vulnerability or environmental factors entirely. The catalog of security controls is a reminder of things often forgotten.
What is the alternative to controls? Hire a horde of handy hackers hoping they holistically hit every hole? Network centric, attack based security is important but not sufficient for adequate security. Whether you call it a hardening guide or a list of things to remember it is still a catalog of controls.
Unfortunately, you are mistaken when you say "infrastructure, host, log, and other sources are all real monitoring, but this is not what NIST means by "monitoring."" All of those are parts of what is meant by continuous monitoring. Going through the NIST SP 800-53 catalog of controls the importance NIST has always placed on proper auditing and monitoring of the information system is readily apparent. Those security controls include audit record content requirements, SIEM requirements, log analysis, regular review and much more.
Some organizations have construed monitoring to mean reviewing a subset of security controls for a system. That review is part of continuous monitoring, the part that has you check to see if the security controls you think are in place are still in place. It is also a part NIST is attempting to automate through the SCAP program to allow for real time monitoring of when the information system changes.
I suspect NIST made a faulty assumption when putting together the FISMA C&A process. They assumed security professionals would look at the process as an opportunity to start systematically addressing security. Somewhere along the line something was lost in translation and we ended up with the notion that the review of one third of the security controls is what constitutes continuous monitoring. In so far as that practice enhances security it should continue, but monitoring means exactly what any thoughtful security person thinks it should mean.
Two other points.
You said:
"Whether you call it a hardening guide or a list of things to remember it is still a catalog of controls."
That is EXACTLY the problem. Finding intruders is not "a hardening guide or a list of things to remember."
You also said:
"It is also a part NIST is attempting to automate through the SCAP program to allow for real time monitoring of when the information system changes."
Detecting system change through SCAP is NOT the same task as finding intruders. Intruders may introduce change, or they may not.
Controls are certainly helpful, but I did not read anything in the NIST guide that will really make a difference as far as discovering and removing intruders in the enterprise. NIST is several years too late if they think they're going to improve security when all of their constituents are already thoroughly compromised.
When you say "monitoring" you have made clear throughout your post you mean of the "intrusion analysis" variety. That would be a specific control. Whether its good or bad (and I'm not defending the quality of the NIST draft), the NIST document is not intended to recommend or design specific controls. From the draft:
"The purpose of this publication is to provide guidelines for applying the Risk Management Framework..."
One of the steps in that framework is "security control selection and implementation." In a given situation, it may be that an organization applying the framework would select intrusion monitoring/analysis as a control. Irrespective of what controls a particular organization choose to implement, this draft is not intended to suggest one or another particular controls but rather to develop/propose a framework within which to make that decision.
It is true that the NIST when discussing "control monitoring" does not mean "intrusion monitoring". If they had meant that, they may have said that.
Your whole article is predicated on a sophomoric error of logic that all "monitoring" must be of the variety that you most closely associate with. In this case you have, on that basis, essentially criticized the NIST for doing something poorly which they were not trying to do at all (i.e. recommend specific security controls).
"the NIST document is not intended to recommend or design specific controls."
Then what's the point of SP800-53: "RECOMMENDED Security Controls for Federal Information Systems and Organizations".
SP800-53 is one of the core documents that make up the RMF.
The AU family of controls is all about intrusion monitoring from Bejtlich's point of view but an organization can easily be compliant with those controls and not have anything close to an effective intrusion monitoring program.
COMPLIANCE DOES NOT EQUAL SECURITY. How many times do we have to rehash this argument?
What you should be referring to, is the actual 'intrusion monitoring' control (SI-4 INFORMATION SYSTEM MONITORING) contained in 800-53 annex 3. However, the more important question really, is why this control isn't more deeply defined and actually enforced using quantifiable metrics.
Understandably, the bar needs to be raised. However, until the community surpasses the 'patch and check' mentality propagated by vendors and their poorly coded software (which has never scaled to large systems), it's all a moot point anyway.
Well thats a different document, now in'it?
THIS particular draft, which was criticized for not recommending "threat monitoring" also does not recommend a whole host of other security controls (e.g. firewalls, access control) because it is not intended to do so.
"COMPLIANCE DOES NOT EQUAL SECURITY. How many times do we have to rehash this argument?"
I never said any such thing, nor do I believe it. I'm not defending the quality of NIST work at all, but this is a bit like criticizing a brownie recipe for making a poor pasta substitute. Even if the cook sucks (and I really have no opinion on that in this case) this particular criticism is illogical.
But if more organizations adopted a systematic process to assess risk, address vulnerabilities, reduce footprint, monitor threats, and manage change the world would be a better place. No compliance not equal security, but you can't have security without some sort of foundation.
I would knock NIST for not producing a manageable documentation set and guidance that might actually be picked up and used by smaller organizations.
I would argue that that should be a goal of the federal government and they fail at it.
In the pursuit of academic rigor and comprehensiveness they lost usability by non-specialists. Leaving the people that need help the most without any.
I couldn't agree more with the limitations of a control-based approach, but given where the feds are now, I think you have to take the revised 800-37 as an improvement. "Continuous monitoring" is more about achieving situational awareness and linking incident response to overall security management than it is about identifying and responding to all the threats coming in. Just about every agency has an incident response capability, so without presuming to think that these are all doing a great job (clearly the rise in annual reported incidents would suggest otherwise), just the recognition that occasional attention to what's supposed to be in place and operating as intended isn't producing results. I don't know what your time at KSG was like, but one of the key themes I took away from the MPP program is in government, "all successful change is incremental." Let's not characterize the failure on NIST's part to recommend a wholesale replacement of current security program operations to mean that there couldn't or shouldn't be improvements sought within the sub-optimal control-driven model.
Where intrusion detection and prevention is concerned, I think it's disingenuous to fault individual agencies for not moving to implement continuous threat monitoring when they have no current capability to make sense of the information. IDS or IPS is of no use (and may be counter-productive) without the corresponding experts to analyze the data produced by the tools, tailor detection rules, and tune operations to minimize false positives and separate noise from actual threats.
Lastly, I assume from the alternative approach you propose that you'd fully endorse the Einstein program taken to the realization of its vision with active intrusion detection and IPS on all government networks (operated for all agencies by the NSA). I'm not sure how it is that DHS or NSA expect to field the army of analysts that would be required to support the monitoring of that volume of traffic, but the government has made it abundantly clear they neither want nor trust individual agencies (civilian ones anyway) to do their own monitoring.
You mention the specificity that NIST SP 800-37 uses when discussing monitoring. SP 800-37 is a framework document. To borrow the military metaphor: FISMA is the policy, SP 800-37 is the strategy, SP 800-53 controls are the tactics. Do we criticize a campaign strategy because it lacks specificity of how each battle should take place?
The people SP 800-37 mentions are those responsible for overseeing the security controls where the tactics take place. The specific mention of security controls selection, assessment and reporting are because those are policy requirement of FISMA (Title 44, Chapter 35, Sec. 3544). The continuously monitored security controls are where the monitoring we expect take place.
Here is a short list of the security controls that "make a difference as far as discovering and removing intruders in the enterprise":
AU-2 Auditable Events
AU-3 Content of Audit Records
AU-6 Audit Review, Analysis, and Reporting
SI-3 Malicious Code Protection
SI-4 Information System Monitoring
SI-5 Security Alerts, Advisories, and Directives
SI-7 Software and Information Integrity
There are more (most of the AU, IR, SC and SI families) but these meet your criteria best. Controls do not give every detail of how to implement the general directive. Technology changes too rapidly and needs are too varied to do that. The controls specify a goal and a general methodology but leaves the detail to the security professionals.
Take the Incident Response family as an example. The family's controls require incident response policy be established, handling mechanism established, training provided, monitoring conducted, reporting, assistance provided and response plan created. The details however are left to the organization. NIST doesn't know ahead of time what precisely is needed or what current technology will be best for the situation so it provides the requirement in general terms. They provide additional guidance to help implementation (in this case SPs 800-50, 800-61, 800-84 and 800-115). The guy doing FISMA compliance evaluates the effectiveness of those Incident Response security controls. If they aren't implemented commensurate to the risk associate with the operation of the system, they fail.
So idiots doing security still means you get some level of idiocy in implementation. NIST provides more than enough guidance to do it correctly but human fallibility is still a factor. I've yet to see any methodology in any field that eradicates human fallibility. The most provably correct OS will still fail if the computer is hacked with an ax. But with the NIST SP 800-37 strategy of continuous monitoring someone periodically and independently assesses the effectiveness of those security controls.
Still, I haven't heard the proposal that systematically addresses all the security concerns of organizations and systems. Responding to attackers is an important aspect of security but casting it as the entirety of security practice is a poor policy that results in poor practice.
Have a happy Thanksgiving Richard, Marcus and everyone else!
Then what's the point of SP800-53: "RECOMMENDED Security Controls for Federal Information Systems and Organizations".
SP800-53 is one of the core documents that make up the RMF.
Having worked in the security space for a very long time, I continue to see a complete, utter absence of STAFFING MODELS that provide some recommended staffing levels for security staff. I can deploy the latest, greatest security technology, pay out the butt to get it tuned and providing actionable data, but if I don't have the people and the skill sets to actually USE the technology then it’s a complete waste of time. But we are compliant…
People are the key to continuous monitoring and we, like so many others under the Federal microscope, can not get the bodies to meet either the compliance or continuous security monitoring requirements properly. NIST, DHS, DOE, Industry, SOMEONE, needs to conduct (and continually revise) a study of staffing resources and skill sets needed to successfully implement and manage security controls and conduct continuous monitoring.
Bueler…… Bueler…… Bueler…