Thursday, December 21, 2006

Thoughts on SAS 70 and Other Standards

I'm not an auditor or CPA, thank goodness. The first time I heard of SAS 70 (Statement on Auditing Standards No. 70, Service Organizations) happened when I visited Symantec in October. Last week, however, one of my clients asked what I knew about SAS 70. I knew Symantec used its SAS 70 results as a way to avoid having every Symantec managed security service client perform its own audit of Symantec. My client wanted to know if his company might also benefit from getting a SAS 70 audit.

I found an exceptionally helpful CSO Online article by Michael Fitzgerald about SAS 70. I'd like to share some insights from it.

A spokeswoman for the body that created SAS 70 doesn't actually recommend it for security purposes. "It isn't a measure of security, it's a measure of financial controls," says Judith Sherinsky, a technical manager on the audit and test standards team at the American Institute of Certified Public Accountants (AICPA), which created SAS 70...

For security audits, Sherinsky recommends a different AICPA standard: SysTrust, an attestation engagement that includes criteria for system security. SysTrust was developed to help CPAs gauge whether systems meet the following criteria: availability, security, processing integrity, online privacy and confidentiality.


That's extraordinary. It means all the customers who get a SAS 70 audit from a service provider aren't getting any real security assurances. It sounds like Common Criteria.

The Sarbanes-Oxley Act is essentially a mandate to establish internal controls so that corporate executives can't fudge their numbers. Sarbox requires that companies verify the accuracy of their financial statements, and establishes SAS 70 Type 2 audits as a way to verify that third-party providers meet those needs...

A SAS 70 audit does not rate a company's security controls against a particular set of defined best practices. In a SAS 70 audit, the service organization being audited must first prepare a written description of its goals and objectives. The auditor then examines the service organization's description and says whether the auditor believes those goals are fairly stated, whether the controls are suitably designed to achieve the control objectives that the organization has stated for itself, whether the controls have been placed in operations (as opposed to existing only on paper), and in a Type 2 engagement, whether these controls are operating effectively.

The fact that a company has conducted a SAS 70 audit does not necessarily mean its systems are secure. In fact, a SAS 70 may confirm that a particular system is not secure, by design.

"You can have control objectives to make any statement management may want to make," says Robert Aanerud, chief risk officer and principal consultant at security consultancy HotSkills. In effect, he says, management could decide that the company is OK with bad access control, and the auditor (who must be a CPA) then needs to ensure that access control is at least bad. The SAS 70 opinion would essentially say that, yes, the company has achieved its stated control objectives.
(emphasis added)

It sounds like reading the SAS 70 report is important!

Unfortunately, consultants say many companies are skipping the hard work and treating SAS 70 as a security rubber stamp. Sharon O'Bryan, head of O'Bryan Advisory Services, says she's aware of companies taking SAS 70 reports for potential service providers, sticking them someplace and never reading them...

Service providers say they're being asked more and more often for SAS 70 audits, often instead of governance standards like Cobit or ISO 17799. That's even true for companies that handle security functions, traditionally more oriented toward granular best-practice tests than the broad audit test of SAS 70.

Michael Scher, general counsel and compliance architect at Nexum, a security product and service provider, says his company is preparing to undergo its first SAS 70 audit. "It's an efficiency-type move," Scher says. It will save his company the trouble of having to be audited by every potential client, or generate reams of documentation in answer to questions.


If SAS 70 is so bad for security, is there an alternative? The AICPA quote earlier in the article mentions SysTrust. AICPA provides a comparison brochure for SAS 70 vs SysTrust. It includes the following:

SAS 70 intended purpose: To provide user auditors with information about controls at the service organization that may affect assertions in the user organizations' financial statements. This generally enables a user auditor to perform an audit.

Trust Services intended purpose: To provide assurance that an organization's system's controls meet one or more of the Trust Services principles and related criteria. Areas addressed by the Principles include: security, online privacy, availability, confidentiality and processing integrity.


It seems SAS 70 is not at all what the customers think it is. Alternatively, they know they are not getting any security assurances, but just want a rubber stamp. Apparently this is more make-work for CPAs, since SAS 70 work must be done by a CPA.

Speaking of work for CPAs, their What Skills Do I Need to Provide SysTrust Services? site is hilarious in a sick way:

Application of Current Skills and Knowledge. CPAs have the ethical standards and principles needed to evaluate and provide assurance on the reliability of systems. CPAs have skills in evaluating evidence, determining the effectiveness of internal controls, and reporting to third parties on the results of the work performed.

New Skills and Knowledge May Be Required. CPA in public accounting and CPAs industry may require additional competencies in addition to those from the traditional accounting, audit, and tax arena in order to provide services related to SysTrust. In order to deliver SysTrust-related advice or assurance, CPAs may need to use automated techniques.


So, in brief: CPAs are ethical but are going to rely on "automated techniques" to validate security effectiveness. Right...

What about the various ISO standards? Here Wikipedia is helpful:

ISO/IEC 27001 is an information security standard published in October 2005 by the International Organization for Standardization and the International Electrotechnical Commission. Its complete name is Information technology -- Security techniques -- Information security management systems -- Requirements. The current standard replaced BS 7799-2:2002, which has now been withdrawn.

ISO/IEC 27001:2005 specifies the requirements for establishing, implementing, operating, monitoring, reviewing, maintaining and improving a documented Information Security Management System (ISMS). It specifies requirements for the management of the implementation of security controls. It is intended to be used in conjunction with ISO 17799:2005, a security Code of Practice, which offers a list of specific security controls to select from.

This is also the first standard in a proposed series of standards which will be assigned numbers within the ISO 27000 series. Others are anticipated to include a re-publication of ISO 17799, a standard for information security measurement and metrics, and potentially a version of the current BS7799-3 standard.

Prior to the release of the ISO 27001 standard, organizations could only be certified against the British Standard Institute's BS7799-2 standard. Now organizations can obtain the ISO 27001 certification, as the BS7799-2 certification is being phased out, and the standard itself has been withdrawn...

It should also be noted that this certification scheme now aligns with other ISO schemes, such as those for ISO 9001 and ISO 14001.


This blog entry has some opinion on ISO 27001 too.

Do any of you recommend a certain standard to show your company is implementing effective security practices?

13 comments:

Landon Lewis said...

In my past experiences SAS70 has been implemented by private companies that do a significant amount of business with publicly traded companies (who are SOX compliant).
These private companies may be connected to these public companies for control processes/data and also dependent upon the financial systems of the private company. This (I guess) provides the public companies with assurance that the private companies are performing some type of due diligence.
I've always found it interesting from the infosec perspective that the policy is written and the customer can only be responsible for being compliant with the policy. I did run into an instance where an auditing company was trying to force security requirements not written in the SAS70 policy. We as the customer just noted that we would put them as a recommendation.
The first approach however was first following the ISO17799 standard, but quickly the amount of work tripled for being compliant so the policy was slimmed down. I don't think this is the type of policy where private companies will spend a lot of time unless the public companies demand more from their private partners or customers.

Misha said...

SAS 70 is one of those phenomenons that has reached critical mass and is widely accepted for all sorts of purposes now. We are being forced by our customers to go through it and there isn't a whole lot we can do about it.

I would not say SAS 70 is all bad though. SAS 70 forces you to get your operations under control, and that's important. There are a lot of service providers that are secure, but could do a better job doing security reliably and repeatably well.

A SAS 70 audit is often the stimulus needed to get the proper motivation to do the right things for your operation and get management buy-in for implementing the right controls, even if they are not specifically prescribed by the standard.

I'd say it's more useful than say, HIPAA, from that perspective.

nigelmellish said...

"The first time I heard of SAS 70 (Statement on Auditing Standards No. 70, Service Organizations) happened when I visited Symantec in October...

...Do any of you recommend a certain standard to show your company is implementing effective security practices?"


Richard, this is a joke, right? You're being sarcastic?

Richard Bejtlich said...

nigelmellish,

With a dozen "standards" to choose from, what's the problem with me asking what other people recommend? And so what if I'm new to SAS 70 -- it's not even meant for security purposes. Relax.

eugk007 said...

While working for a Big 5 firm, I was sent on a SAS 70 engagement. It was a bit confusing at first, since I was a security guy and this seemed an awful lot like an accounting audit. Anyway, I assumed I was there to provide some technical security expertise. When I recommended that we at least do some basic vulnerability scanning, the engagement lead laughed and said that would require an entirely new contract - ie, more money.

robert m. richardson said...

My organization is a large web host and considered a "service provider" by most people and standards. We have gone through and passed both the SAS70 and PCI audits.

I agree that the public perception of the SAS70 is vastly different from what it was designed to accomplish; even more so with the different types of SAS70 certifications (Type 1 & Type 2).

Type 1 simply states that at one single point in time, that particular organization followed these guidlines. It doesn't mean that they follow these practices before and/or after that particular date in time. The Type 2 is more in-depth, which I feel carries more weight.

As far as security goes, I feel that the PCI Compliancy is better. Obviously, PCI was designed for the credit card infrastucture and companies that process cards online. But the audit actually involves a detailed questionaire, followed by one or more security scans and site surveys. Also, the PCI compliancy is a quarterly audit; unlike the SAS70 Type 1 which is a one time policy overview.

Pieter said...

As a security auditor at one of the big 4 firms, I have been involved with SAS70 audits for some time now, and this is my opinion:

SAS70 Type 1 is just a description of a list of controls and objectives of this control. For example. a company chooses "network segmentation" as a control with the objective of segregating critical information systems based upon trust level. During a SAS70 Type 1 audit, the auditor is going to provide assurance that at a certain point in time, this control was in place.

A SAS70 Type 2 provides assurance over a certain period of time. The same controls can be defined as in a type 1, but they will also be tested over time (over the last financial year for example).

Now, what is important to look at for a client:
- type of SAS70
- scope of the SAS70
- controls taken into account
- results of the controls

Often there are companies who only list their controls which they KNOW are working correctly, and the other (immature) controls are silently left out of the report.

I personnaly don't think SAS70 is useless. It can provide reasonable assurance (yes, I still am an auditor :) ) if the selection of the controls has been done ethically and professionally.

Anonymous said...

One primary issue with a SAS70 is that it is a test of the clients controls - that is if the client says that they do ABC then the auditor will test ABC, they will not (usually) pass an opinion on whether ABC is the correct way to do it, or in many cases whether ABC actually works.
The clearest example of this would be in the area of patch management, if the client claims that they apply patches on a regular basis and have a sign off sheet then this process is what is tested. The issue of whether or not the pacthes are actually applied (for instance by running a vulnerability scan) is never addressed. In practice we have seen this several times where an organization supposedly has good patch management policies and can produce a list showing every patch applied and to what, however when performing some technical testing we have found this to be incorrect, yet the SAS70 makes no mention of this.
The testing tends to be far too audit focused and does not address technology as deeply as it should (if it was going to be used for scurity reliance).

Systrust is only marginally better, security controls are tested but rather narrowly.

Anonymous said...

This article makes the same mistakes that every article written by security professionals makes....it tries to compare a security assessment (aka "apples") to a SAS 70 audit (aka "oranges"). And, BELIEVE IT OR NOT, security professionals always seem to conclude that apples are not oranges and apples are better than oranges...EUREKA!

There is no substitute for a SAS 70 audit in terms of providing third party verification to clients and their auditors in conjunction with a financial statement audit and/or SOX. It is, essentially, the only acceptable substitute for a CPA firm reviewing the service organizations themselves.

There is no one security standard that the industry would agree upon. There is no one standard that contemplates all of the possibilities that could be in place at a service organization. There is not authoritative body that regulates standards or enforces licensing. There is no real risk of civil or criminal liability for gross negligence related to a security assessment.

It is very possible that a company could need both types of audits. There is also nothing that prevents a company from incorporating the components of standards they have implemented into the scope of their SAS 70 audit. However, note that it generally not preferable to be very specific about security information in a report issued to third parties b/c of obvious issues as well as the fact that it guarantees feedback from all your clients on their OPINIONS about your configurations. Such conversations are better held offline if that level of detail is desired.

One final note is that if you need a SAS 70 audit, you should find an audit firm (and team) that has significant SAS 70 experience. This is hard to find, even at the "Big 4" firms. If you need a security assessment of any type, don't call a CPA firm...and don't consider SysTrust. Call a security assessment and find a team that has experience performing information security audits.

Anonymous said...

Richard,
Thumbs up, you hit the target.

Anonymous said...

Anyone looking for a security certification should look no further than ISO 27001:2005. Unlike SAS70, 27001 has a list of predefined controls (around 150, give or take a few). An organization must state whether that control applies to them, how they meet the control, and provide evidence that they have met the control as they state. If you state that a control does not apply, you must provide enough evidence to that fact to convince the auditor. My first ISO 27001 audit is in a week, and I have talked to others within my organization that have already been audited. For all of us, it has helped us tremendously to find weaknesses and to create processes to correct them.

Jeremy said...

Is there a reason the private sector does not use NIST guidance at all? Specifically NIST 800-53 from an Information Security perspective?

It is a pretty comprehensive set of security controls that are well documented.

CPA said...

SAS 70 is something completely different as Systrust or ISO27. It is a method with (very strict) rules for the CPAs to execute a service delivery audit. If the control objectives put in the SAS 70 by the organisation are strong it is the best assurance on the market (SAS 70 type 2). The devil is in the details but that is the same for all reports (e.g. ISO27, Systrust).
ISO27 certification can only for ISO27001. It is thus only a statement of "yes, they have processes in place". If they actually work you do not known.
If you would put the ISO27002 control objectives in a SAS 70, you would get; "I, a CPA, states that these control objectives are met and that the controls are working every day. If not you may sue me". That is a strong statement.