Saturday, February 20, 2010

Offshoring Incident Response

A blog reader emailed the following question.

We recently had a CISO change, and in the process of doing an initial ops review and looking at organizational structure, one of the questions the new CISO has is about the viability of offshoring incident response... I would be very interested in your views on this matter, and would appreciate any feedback you can offer.

As background, I've been involved in incident response in many different capacities: top-level military CERT, managed security services provider, fly-away consultant, government contractor, independent consultant, and top-level corporate CIRT. In other words, I've worked in insourced and outsourced environments.

I strongly advocate insourced or internal, professional incident response teams. Many technical people fixate on the technical aspects of security, as you might expect. While technical expertise is critical, it is also critical to understand the client. Depending on the size and complexity of the client, it can take an external team weeks or months to acquire the necessary understanding of the client to make a real difference. Sure, an external team can probably perform great analysis if given the right details and context. However, doing something about usually relies heavily on identifying and overcoming the various bureaucratic, cultural, financial, legal, and political challenges found in any suitable large organization. Therefore, I believe internal CIRTs are necessary for all organizations larger than a few hundred employees.

I believe it is appropriate and sometimes necessary to rely on outsourced incident response services when your organization meets one or more of these criteria during an incident.

  • Your CIRT is nonexistent.

  • Your CIRT is not staffed with enough people to meet the challenge at hand.

  • Your CIRT is not technically equipped to meet the challenge at hand.

  • Your CIRT needs help with a specific aspect of the challenge at hand.

  • Your CIRT needs external assistance due to regulatory, compliance, or other legal issues.

Furthermore, when I read the term "offshoring" I get the sense that the question may involve hiring contractors who work for the organization permanently but report to their home contracting organization. In my experience any "cost savings" in such an arrangement are a figment of the accounting imagination. I recommend full-time employees be CIRT members.

Any thoughts from blog readers?


Anonymous said...

I'm not sure if this comment is appropriate, but my guess is the person who asked the question was more concerned with your thoughts on whether such sensitive work can be outsource to companies and individuals based in other countries. I think you hit the nail on the head with regards to whether it should be outsourced in general- especially the comment on cost saving. However, I'd be interested in your thoughts on offshoring. Personally, I think it largely depends on the organization mission, the type of business they do, and where the work is being outsourced. All this assumes you've already considered the comments Richard made and decided that outsourcing is necessary or advisable first.

Anonymous said...

I had the same reaction...I agree with the general outsourcing thought. If the outsourcing decision is being made solely based on "cost savings", then there is already an inherit flaw in it. I do wonder about the sensitivity issue though. For example, if you are a US based corporation, do you want a CIRT member who is located in China (regardless of whether they are an outsourced resource or a dedicated employee)?

Anonymous said...

We solved it contractually and practically. The provider was forced to set up their own accredited CIRT. Provider's CIRT will handle "day-2-day"-cases as malware, scans etc and our team handles all severe cases as well as all forensics issues. In order to be able to follow up on provider's actions they were required to let our own team set up their own security monitoring and as well have full insight in all provider cases that is related to our company.

Anonymous said...

What is this "accredited" CIRT you speak of? Accredited by whom? and what are the requirements?

You would think doing a full certification and accreditation of a CIRT process/team would zap any of your cost savings.

Anonymous said...

The specific functions I would consider 'outsourcing' is the analytic reverse engineering of malware and a complete analysis of memory samples and specific disk artifacts that may allude the internal staff or take too much time to process. This would simply augment the CIRT capabilities.

Anonymous said...

Our company has *jack* in the CIRT-department but has a NOC in India... I'm in charge of training them to become lvl1 SOC analysts...
yay me.

But *something* should be better than nothing. Actually taking a "Globalizing IT" course right now and the given advice for starting off an international work unit (and not have them call you every 20 minutes) is apparently to create 'atomic work' items.
But in a SOC environment?
I loathe the idea of defining every "signature" they should watch for in a "day-2-day" capacity... seems counter-intuitive.

Anonymous said...

As the 'onshore' component of a team that also provides on/offshore CIRT capabilities, the problem I see is one of quality vs profit. Year one is all about setting up the services, and years 2-5 are all about doing them as cheaply as possible.

I'm fighting a losing battle in requiring an actual analysis phase (especially determining the extent of the incident impact both in terms of breadth of incident... This CIRT team tends to treat widespread outbreacks of malware as independent incidents, instead of looking for commonalities, and in the impact of data losses [I work in the personal finance industry]). This is due to contractual SLAs to 'close the call.' This then leads to no lessons learned, and more importantly, no data on the true impact of the incident [data exposed].

Anonymous said...

By accredited CERT/CSIRT I referr to this: