Thursday, March 28, 2019

Thoughts on OSSEC Con 2019

Last week I attended my first OSSEC conference. I first blogged about OSSEC in 2007, and wrote other posts about it in the following years.

OSSEC is a host-based intrusion detection and log analysis system with correlation and active response features. It is cross-platform, such that I can run it on my Windows and Linux systems. The moving force behind the conference was a company local to me called Atomicorp.

In brief, I really enjoyed this one-day event. (I had planned to attend the workshop on the second day but my schedule did not cooperate.) The talks were almost uniformly excellent and informative. I even had a chance to talk jiu-jitsu with OSSEC creator Daniel Cid, who despite hurting his leg managed to travel across the country to deliver the keynote.

I'd like to share a few highlights from my notes.

First, I had been worried that OSSEC was in some ways dead. I saw that the Security Onion project had replaced OSSEC with a fork called Wazuh, which I learned is apparently pronounced "wazoo." To my delight, I learned OSSEC is decidedly not dead, and that Wazuh has been suffering stability problems. OSSEC has a lot of interesting development ahead of it, which you can track on their Github repo.

For example, the development roadmap includes eliminating Logstash from the pipeline used by many OSSEC users. OSSEC would feed directly into Elasticsearch. One speaker noted that Logstash has a 1.7 GB memory footprint, which astounded me.

On a related note, the OSSEC team is planning to create a new Web console, with a design goal to have it run in an "AWS t2.micro" instance. The team noted that instance offers 2 GB memory, which doesn't match what AWS says. Perhaps they meant t2.micro and 1 GB memory, or t2.small with 2 GB memory. I think they mean t2.micro with 1 GB RAM, as that is the free tier. Either way, I'm excited to see this later in 2019.

Second, I thought the presentation by security personnel from USA Today offered an interesting insight. One design goal they had for monitoring their Google Cloud Platform (GCP) was to not install OSSEC on every container or on Kubernetes worker nodes. Several times during the conference, speakers noted that the transient nature of cloud infrastructure is directly antithetical to standard OSSEC usage, whereby OSSEC is installed on servers with long uptime and years of service. Instead, USA Today used OSSEC to monitor HTTP logs from the GCP load balancer, logs from Google Kubernetes Engine, and monitored processes by watching output from successive kubectl invocations.

Third, a speaker from Red Hat brought my attention to an aspect of containers that I had not considered. Docker and containers had made software testing and deployment a lot easier for everyone. However, those who provide containers have effectively become Linux distribution maintainers. In other words, who is responsible when a security or configuration vulnerability in a Linux component is discovered? Will the container maintainers be responsive?

Another speaker emphasized the difference between "security of the cloud," offered by cloud providers, and "security in the cloud," which is supposed to be the customer's responsibility. This makes sense from a technical point of view, but I expect that in the long term this differentiation will no longer be tenable from a business or legal point of view.

Customers are not going to have the skills or interest to secure their software in the cloud, as they outsource ever more technical talent to the cloud providers and their infrastructure. I expect cloud providers to continue to develop, acquire, and offer more security services, and accelerate their competition on a "complete security environment."

I look forward to more OSSEC development and future conferences.

Wednesday, March 13, 2019

Thoughts on Cloud Security

Recently I've been reading about cloud security and security with respect to DevOps. I'll say more about the excellent book I'm reading, but I had a moment of déjà vu during one section.

The book described how cloud security is a big change from enterprise security because it relies less on IP-address-centric controls and more on users and groups. The book talked about creating security groups, and adding users to those groups in order to control their access and capabilities.

As I read that passage, it reminded me of a time long ago, in the late 1990s, when I was studying for the MCSE, then called the Microsoft Certified Systems Engineer. I read the book at left, Windows NT Security Handbook, published in 1996 by Tom Sheldon. It described the exact same security process of creating security groups and adding users. This was core to the new NT 4 role based access control (RBAC) implementation.

Now, fast forward a few years, or all the way to today, and consider the security challenges facing the majority of legacy enterprises: securing Windows assets and the data they store and access. How could this wonderful security model, based on decades of experience (from the 1960s and 1970s no less), have failed to work in operational environments?

There are many reasons one could cite, but I think the following are at least worthy of mention.

The systems enforcing the security model are exposed to intruders.


Intruders are generally able to gain code execution on systems participating in the security model.


Intruders have access to the network traffic which partially contains elements of the security model.

From these weaknesses, a large portion of the security countermeasures of the last two decades have been derived as compensating controls and visibility requirements.

The question then becomes:

Does this change with the cloud?

In brief, I believe the answer is largely "yes," thankfully. Generally, the systems upon which the security model is being enforced are not able to access the enforcement mechanism, thanks to the wonders of virtualization.

Should an intruder find a way to escape from their restricted cloud platform and gain hypervisor or management network access, then they find themselves in a situation similar to the average Windows domain network.

This realization puts a heavy burden on the cloud infrastructure operators. They major players are likely able to acquire and apply the expertise and resources to make their infrastructure far more resilient and survivable than their enterprise counterparts.

The weakness will likely be their personnel.

Once the compute and network components are sufficiently robust from externally sourced compromise, then internal threats become the next most cost-effective and return-producing vectors for dedicated intruders.

Is there anything users can do as they hand their compute and data assets to cloud operators?

I suggest four moves.

First, small- to mid-sized cloud infrastructure users will likely have to piggyback or free-ride on the initiatives and influence of the largest cloud customers, who have the clout and hopefully the expertise to hold the cloud operators responsible for the security of everyone's data.

Second, lawmakers may also need improved whistleblower protection for cloud employees who feel threatened by revealing material weaknesses they encounter while doing their jobs.

Third, government regulators will have to ensure no cloud provider assumes a monopoly, or no two providers assume a duopoloy. We may end up with the three major players and a smattering of smaller ones, as is the case with many mature industries.

Fourth, users should use every means at their disposal to select cloud operators not only on their compute features, but on their security and visibility features. The more logging and visibility exposed by the cloud provider, the better. I am excited by new features like the Azure network tap and hope to see equivalent features in other cloud infrastructure.

Remember that security has two main functions: planning/resistance, to try to stop bad things from happening, and detection/respond, to handle the failures that inevitably happen. "Prevention eventually fails" is one of my long-time mantras. We don't want prevention to fail silently in the cloud. We need ways to know that failure is happening so that we can plan and implement new resistance mechanisms, and then validate their effectiveness via detection and response.

Update: I forgot to mention that the material above assumed that the cloud users and operators made no unintentional configuration mistakes. If users or operators introduce exposures or vulnerabilities, then those will be the weaknesses that intruders exploit. We've already seen a lot of this happening and it appears to be the most common problem. Procedures and tools which constantly assess cloud configurations for exposures and vulnerabilities due to misconfiguration or poor practices are a fifth move which all involved should make.

A corollary is that complexity can drive problems. When the cloud infrastructure offers too many knobs to turn, then it's likely the users and operators will believe they are taking one action when in reality they are implementing another.