We're familiar with the plan -> prevent -> detect -> respond model. We form and practice plans and policies to guide our organizations. We prevent exploitation of vulnerabilities with security strategies like defense-in-depth, access control, least privilege, segmentation, and vulnerability management via proactive assessment and patching. We should perform detection via network security monitoring -- collecting, validating, and escalating event, session, full content, and statistical data. We respond to policy violations and intrusions (unlawful, unacceptable, or unauthorized use of our resources) via containment and mitigation. Is there anything else we can do when a threat to core infrastructure like OpenSSH arises?
Where possible, I think it would be helpful to have redundant systems to perform those critical services. Note that redundant does not mean "identical." Why have an extra Microsoft IIS server ready to replace the one just hacked by an intruder? It's much better to have an Apache server ready to go. (I'd argue it's better to have Apache be the primary and another solution as the back-up.) For DNS, pair BIND and djbdns. For email, run sendmail and postfix.
Where does this leave OpenSSH? Well, today I tried lsh. I had no problem installing it since it lives in the FreeBSD ports tree. While lsh may not be as well-scrutinized or tested as OpenSSH, it's not the vulnerable OpenSSH service waiting to be exploited. How could this be used in a production environment?
- You suspect OpenSSH may be vulnerable. You're running OpenSSH on port 22 TCP (a bad idea in my book -- why not run it somewhere else?) and the lsh daemon on port 222 TCP. Port 222 isn't allowed remotely, while 22 is.
- You make two quick changes to your firewall rules: Disable port 22 TCP inbound and allow port 222 TCP inbound. This quickly removes outsider access to vulnerable services.
- You administer critical systems using lsh (which can be accessed using standard OpenSSH clients) and patch the vulnerable OpenSSH services.
- Once done, disable port 222 TCP access and reinstate port 22 TCP.
The same approach could apply to other services, with modifications. This can be implemented on the client side too. Replace Internet Explorer (30+ unpatched holes and counting) with Mozilla Firebird.
When I was a Boy Scout, my Scoutmaster always asked one question whenever I planned a camping trip or hike: "What's plan B?" Not having alternatives was unacceptable. The security community has got to start devising plan Bs, because people rely on our services and expect them to work.