Thursday, July 30, 2009

Thoughts from Black Hat USA 2009

Black Hat USA 2009 is history. My two classes of TCP/IP Weapons School 2.0 went very well. I should be back to teach in DC, Barcelona, and Las Vegas next year. Thank you to my students for your positive feedback and cooperation in class! Despite your numbers we had little to no problems and I believe everyone learned something useful. For future classes I will add a table of contents, focus the questions, add more on my personal methodologies, and add more consistent page numbers to the class books. I added two of your comments to my Training page, and I'll add one other here:

The instructor was great. Very informative and very "in the weeds" for a Director!

That made me laugh.

I recorded my take-aways from the Briefings using my new Twitter.com/taosecurity account. Moxie Marlinspike delivered my favorite briefing. He completely demolished SSL, and he presented the material in a very understandable story. As one attendee commented to me: "he told a story we could all follow, unlike some of the other speakers." In addition to Moxie, Dan Kaminsky, and Alex Sotirov & Mike Zusman also showed SSL problems.

I paid a decent amount of attention to the "mobile" track this year. The outside world seems to not realize that the iPhone or Blackberry in your pocket is a computer. Some of the vendors don't think that way either. Apple is becoming the new Microsoft as mentioned by several people this week. Start with the page listing Apple security updates: http://support.apple.com/kb/HT1222. What kind of a URL is that?

Now look for iPhone updates:

Can you spot the problem here? How about more timely updates?

Now select the latest update and search for "arbitrary code execution". I counted 27 instances. The bottom line is that Apple needs to step up to the plate. How about creating a PSIRT like the grown-up vendors have?

A close second favorite talk was "Fighting Russian Cybercrime Mobsters" by Dmitri Alperovitch and Keith Mularski. That's the kind of threat-centric talk that everyone can understand. Jeremiah Grossman and Trey Ford again brought it strong with their latest on making money through cybercrime. The last talk I attended, by Bill Blunden, featured an updated version of the slide where he posts my picture, except he used the more recent, grayer-beard photo. Thanks Bill -- nice to meet you!

Wednesday, July 29, 2009

What is Cloud?

The slide at left was one of my favorites from Craig Balding's Cloud Security Ghost Story talk from Black Hat EU earlier this year.

I like that he shows that a "cloud" does not mean a "VM farm" run by admins who require users to endure lengthy provisioning processes, followed by requests from the IT department for the supposed "customer" to provide information and resources they would expect to get from the Cloud.

Real Clouds are provisioned via credit card, are trivial and transparent to extend, and make life easier for the customer, not a hassle!

Notes from OISF Meeting in DC

This month I was pleased to attend a public meeting of the Open Information Security Foundation in Washington, DC. I got a chance to meet several people I have known for many years through their work with Snort, such as Matt Jonkman, Will Metcalf, Victor Julien, Frank Knobbe, and two guys from a federal agency that have extended Sguil way beyond what I knew anyone was doing! The group posted DC Brainstorming Meeting Notes, but I wanted to record a few thoughts here.

OISF is a US nonprofit, a 501c(3). Their goal is to produce a new network inspection and filtering engine (IDS/IPS) that will be released under GPLv2. They can not and will not commercialize, sell, patent, copyright, or profit from the engine. Rather, others who participate in the OISF Consortium (listed on their Web site) are donating coders, equipment, and financial support in exchange for the ability to commercialize the engine.

OISF works with the Open Source Software Institute, famous for getting FIPS validation for OpenSSL -- something everybody wanted but no one wanted to fund alone. OISF is part of the DHS Homeland Open Security Technology (HOST) program. OISF has received legal guidance from the Software Freedom Law Center.

OISF has many goals for their engine, outlined in the notes I linked earlier. Most interesting is their goal for a production release by the end of this year. If they are to make this goal, I think the project needs to severely limit the requirements for the first release. I would focus on the following.

  • Developing the rules language.

  • Implementing IPv6.

  • Implementing multi-threading.


Those three tasks are monumental, but they would immediately differentiate OISF from other options. There is talk within the project of semi-Snort compatible output, so you might send OISF data to a file in Snort Unified or Unified2 format to be read by Barnyard or Barnyard2.

If you want to know more about the project, the Mailing Lists are the best option. As it develops I will discuss it here.

Tuesday, July 28, 2009

Guest Post at Fudsec.com

I was asked to write a guest post for the new Fudsec.com blog, so I published Threat-Centric Thinking on the Rise. From that post:

A lot of people have been discussing denial of service attacks against various Important Sites earlier this month. It struck me that the focus of the discussion, really to the exclusion of anything else, has been one question: "who did it?"

Think about that for a second. If this attack had happened in 1996, we would have asked "how did that happen?" In other words, network DoS was new enough to warrant a technical examination of the event. Attribution would be a concern, but most people would want to know how it happened...

Review of Voice over IP Security Posted

Amazon.com just posted my four star review of Voice over IP Security by Patrick Park. From the review:

The reviews of Voice over IP Security are fairly consistent at 4 stars, and I agree with that consensus. I've read a few books on this topic, and early titles were fairly awful. My favorite remains Hacking Exposed: VoIP, but a comparison with Voice over IP Security shows different audiences for the two books. The HE book is better suited for those assessing VoIP systems, while this book is better for engineers and those implementing VoIP systems.

Direct Financial Cost of Intrusions

Thanks to the blog reader who directed me to the Washington Times story Contractor returns money to Pentagon:

Apptis Inc., a military information technology provider, repaid $1.3 million of a $5.4 million Pentagon contract after investigators said the company provided inadequate computer security and a subcontractors system was hacked from an Internet address in China...

Apptis agreed to the repayment after the Defense Criminal Investigative Service concluded the company and a subcontractor failed to provide "proper network security and information assurance services," according to the report, released in June.

The subcontractors system under Apptis management was intruded upon "with total access to the root network" from an Internet address in China, the report said.


Wow. Can anyone think of another case where a company was "fined" by a customer for an intrusion? Usually we only hear of PCI issues.

Monday, July 20, 2009

SANS Forensics and Incident Response 2009 Summit Round-Up

I'd like to share a few thoughts from the second SANS WhatWorks Summit in Forensics and Incident Response, where I delivered the keynote. I could only attend the first day, but I thought it was definitely worthwhile. I was given a few questions which I promised to answer on this blog, so here they are.

With your background with Information Operations and cyber security, what would you advise the new U.S. Cyber Command? What should their priorities be?

I've written a lot on cyber command over the years. I believe their first priority is to create a real career path for cyber operators. Tools, tactics, and procedures are secondary to attracting and retaining talent. You can accomplish amazing feats if you have the right butts in the seats. Without that, you are guaranteed to fail. Part of that will involve identifying all of the people with cyber duties in the military. Once they have that part working, I would advise Cyber Command to think in terms of a Cyber NORAD.

Five years from now the Verizon Data Breach Report 2014 is published. What trend will be the "big red dot" in 2014? What will be your biggest surprise?

To clarify, the "big red dot" of 2009 was the huge number of records stolen by external parties, far exceeding internal intruders.

This is a really good question. I never see a future where insiders are more dangerous than outsiders. By insiders I mean people formally associated with an organization, e.g., employees, contractors, etc. Outsiders are people who are not formally associated with an organization. Insiders will remain capable of individual large incidents, but outsiders will continue to conduct repeated large and small incidents.

I will be really surprised if IPv6 is changing the way businesses operate in 2014. I think we may see internal business operations (like carrier networks) using IPv6, but I don't think we'll see a substantial user base for IPv6 by 2014. If that is not true I will be surprised.

What do you know about public/private partnerships to leverage known command and control servers? Is there any way for a CIRT to avoid third party notification by performing proactive detection?

There's a few options here. One is to join the Forum of Incident Response and Security Teams (FIRST). FIRST maintains a private mailing list that shares information among members. Another option is to look for private associations among peer businesses. A third idea is to make contact with the many volunteer and commercial security intelligence services organizations, including The Shadowserver Foundation, Support Intelligence, Secure Science, iDefense, and many others.

With the questions answered, I'd like to say I thought Summit organizer Rob Lee did a great job (again) keeping the event moving smartly. Kris Harms, Harlan Carvey, Jamie Butler/Peter Silberman, and Brendan Dolan-Gavitt all delivered great talks. The two user panels I saw (I missed the third) were also excellent.

I wanted to record a few tricks that Kris offered so I don't forget them.

  • Use the PsTools handle.exe app and grep for "pid\:" in the output to see a different sort of process list.

  • Grep handle.exe output for "Mutant" to see mutexes.

  • Pay attention to digital signature output in autorunsc.exe, particularly for results that are not signed and/or not verified; and signed but verification failed. Check hashes against fileadvisor.bit9.com.

  • Remember to teach junior analysts a methodology, like:


    1. Determine if compromised.

    2. Develop investigative leads.

    3. Build a timeline.

    4. Determine how compromised.

    5. Suggest remediation measures.

    6. Assess impact of compromise.



While listening to the speakers, it was clear to me the differences between three communities:

  1. Intrusion detectors and responders

  2. Computer forensics investigators

  3. Litigation support and ediscovery investigators


I thought this slide by Jess Garcia from One eSecurity showing one practitioner's opinion on the variety of forensics tools was interesting.



I still need to try MANDIANT Audit Viewer. Jamie Butler and Pete Silberman noted that since MANDIANT Memoryze uses live analysis to access the Windows page file, they don't run into issues found when trying to combine a dead page file with a memory capture.

I'm looking forward to next year! If you do IR, you should try to be there.



Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.

Friday, July 17, 2009

Free Issue of Linux+ Magazine Posted


A free issue of Linux+ magazine is available -- look for the link to "Free Issue: Linux in Mission Critical". It's 68 pages of Linux information, for free!


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.

Wednesday, July 15, 2009

Review of vi(1) Tips Posted

Amazon.com just posted my five star review of vi(1) Tips by Jacek Artymiak. From the review:

I agree with just about everything that appeared in KN's review. Jacek Artymiak has written a sort of "vi(1) for the Desperate" covering all of the aspects of vi I would like to see addressed. I could see this book used in an introductory Unix class where the students are
expected to try all of the examples. Jacek posted the sample files used in the book examples at devguide.net, so you can easily follow along.



Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.

Tuesday, July 14, 2009

Cisco Routers for the Desperate, 2nd Ed

A little over four years ago I reviewed the first edition of Michael W. Lucas' Cisco Routers for the Desperate. A second edition has been published, but since my first review is still posted at Amazon.com I can't post another. Also, Michael asked me to tech-edit the second edition, and I don't formally rate books in which I play a part. (Authors and others who are involved in books -- it's bad form to give your own product five stars!)

I gave the first edition four stars because it missed a few areas I thought were important. I'd give the new edition five stars if I were not involved. Michael literally fixed the areas I found in the first edition (like p 20 in the new edition reminding users to issue the "no shutdown" command) and added a new chapter on basic switch administration.

If you are one of the "poor bastards who are awake at oh-dark-thirty trying to get their router working," you should keep a copy of this book nearby. Michael is absolutely one of the best technical writers in the networking, computer, and security worlds. Plenty of others could learn how to write by studying his approach to teaching readers. Great work again Michael!


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.

White Hat Budgeting

After publishing Black Hat Budgeting last month, several readers asked me how to spend the same $1 million on defense. This is a more difficult question. As I wrote in the previous post, for $1 million per year an adversary could fund a Western-salaried black hat team that could penetrate and persist in roughly any target it chose to attack. That does not hold true for defense, i.e., for $1 million per year a defender could not fund a Western-salaried white hat team that could plan, resist, detect, and respond to any $1 million black hat team.

So, if you had $1 million to spend on defense, how could you spend it? I turned to my 2008 post Defensible Network Architecture 2.0 as a guide. One interesting aspect of the eight DNA 2.0 tenets is that half of them are IT responsibilities (or at least I would strongly argue they are): inventoried, claimed, minimized, current. All of that is just "good IT." Security can provide inputs, but IT should own those aspects. That leaves monitored, controlled, assessed, and measured.

With that's, let's allocate the funding. With such a small team we would expect people to move among the roles so they don't burn out, and so they can grow their capabilities.

  • Staff. Without people, this operation goes nowhere. We allocate $850,000 of our budget to salaries and benefits to hire the following people.


    • The team leader should have experience as an enterprise defender as a minimum. The leader can be very skilled in at least one speciality but should be familiar with all of the team's roles. The team leader needs a vision for the team while preserving business value. Because this team is so small the leader has to do strategic thinking and overall management, including the "measured" aspect of DNA 2.0. $120,000.

    • The incident response team is responsible for detecting and responding to intrusions. They perform the "monitor" aspect of DNA 2.0. We hire three people, one with Windows expertise, one with Unix expertise, and one with infrastructure expertise. $330,000.

    • The security operator is responsible for the "controlled" aspect of DNA 2.0. He or she seeks to minimize intrusions by deploying and operating countermeasures. This person is also a utility player who can learn other roles and consult as necessary. $80,000.

    • The threat operator performs an advanced security intelligence and analysis role. He or she should be able to reverse engineer malware while also paying attention to underground activities and applying that knowledge to all aspects of the team's work. $120,000.

    • The Red-Blue Team performs adversary simulation/penetration testing (red) and collaborative vulnerability assessment (blue) activities. With a team this size there is only room for two technicians. Red-Blue handles the "assessed" aspect of DNA 2.0. $80,000 for the blue, $120,000 for the red.


  • Technology. At this point we only have $150,000 left. We can spend $100,000 on technology. It should be clear that $100,000 isn't going to buy much of any commercial tools. In fact, the $1 million security operation is going to have to rely on several realities.


    • Built-in capabilities. This team is going to have to rely on capabilities built into the products deployed by other IT teams, like the computer and networking groups. This actually makes a good amount of sense. Is it really necessary to deploy another host firewall on Windows if you can use IPsec policies and/or Windows firewall? With a budget that small, these are the uncomfortable choices to be made.

    • Open source software. The $1 million security team should deploy a lot of open source software. Sguil could be the NSM suite of choice, for example. By spending money on staff who know their way around open source tools, you can go very far using what can be downloaded for free. Let the staff contribute back to the community and it's a win-win situation.

    • Commodity hardware. You can't buy hardware for free, and those NSM sensors and other open source packages need to run on something. A decent amount of the budget will be spent on hardware.

    • Cloud hosting. The Cloud becomes an attractive place to store logs, do processing, and other activities that don't scale well or work well on commodity hardware. Security concerns are lessened when the alternative is no security services.


  • Miscellaneous. The last $50,000 could be spent on incidentals, training, team awards, travel, or whatever else the group might require to attract and retain talent.


Note I did not advocate outsourcing here. You spend too much money and probably won't receive value for it.

With such a small team, there is no concept of 24x7 support. 8x5 is the best you can get. The ability of the team to detect and respond to intrusions in a timely manner is going to decrease as the enterprise grows. A team of 8 security defenders will be strained once the company size exceeds 10,000 people, at the largest.

I am much less comfortable building out this team, compared to the Black Hat Budgeting exercise. There are way too many variables involved in defending any enterprise. Most companies really are unique. However, this is a good point to stop to see if anyone has comments on this approach.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.

Monday, July 13, 2009

FreeBSD Pf and Tftp-proxy

Several IP-enabled devices in the lab use TFTP to retrieve configuration files from various locations on the Internet. This pains me. You can probably imagine what these devices are. Unfortunately I don't control how these devices work.

I run Sguil at my lab gateway to the Internet. I watch traffic right before the gateway, before it is NAT'd. I really don't care what's on the other side. I mostly care what is leaving the network, so I concentrate my NSM activities there.

I noticed one of these TFTP-enabled devices trying to retrieve a file repeatedly. I looked closer at the traffic (thanks to Sguil I keep a record of traffic leaving for the Internet) and noticed I never saw any replies. Simultaneously I received an email from tech support for this device. They told me to unplug all Internet devices from my cable modem and plug the troublesome device into the cable modem overnight (!) My answer to that: "heck no."

I decided to run an experiment with a TFTP client inside the lab and a TFTP server on the Internet. By watching traffic on the internal and external sides of the gateway, I could see TFTP requests making it to the TFTP server on the Internet, and TFTP replies coming from the server back to the gateway. However, the TFTP replies never appeared on the internal side of the gateway.

I did some research and determined that FreeBSD's Pf firewall can't handle TFTP traffic by default. Here is why:

18:13:31.205435 IP my.public.ip.addr.64212 > tftp.server.public.ip.69: 17 RRQ "test.txt" octet
18:13:31.282363 IP tftp.server.public.ip.51186 > my.public.ip.addr.64212: UDP, length 29
18:13:31.284161 IP my.public.ip.addr.57880 > tftp.server.public.ip.51186: UDP, length 4

You see the TFTP request to port 69 UDP. The reply, however, comes from port 51186 UDP to port 64212 UDP. Pf doesn't automatically know that packet 2 is associated with the TFTP request in packet 1.

Fortunately, FreeBSD and other operating systems ship with tftp-proxy(8). I tried following the example in the man page, but I ended up adding the following to the configuration file /etc/pf.conf. $local192 is the LAN from which I expect to see TFTP requests.

no nat on $ext_if to port tftp

rdr-anchor "tftp-proxy/*"

rdr on $int_if proto udp from $local192 to port tftp -> \
$int_if port 6969

anchor "tftp-proxy/*"

I added the following to /etc/inetd.conf.

acmsoda dgram udp wait root /usr/libexec/tftp-proxy tftp-proxy -v

acmsoda is the name in /etc/services for port 6969.

I had to enable /etc/inetd in /etc/rc.conf.

inetd_enable="YES"
inetd_flags="-wW -C 60 -a 172.16.2.1"

Without the -a flag, tftp-proxy would be listening on all interfaces, and I don't want that.

Now I was ready to reload Pf and restart /etc/inetd.conf.

r200a:/root# pfctl -Fa -f /etc/pf.conf

r200a:/root# /etc/rc.d/inetd restart

I checked to ensure port 6969 UDP was listening.

r200a:/root# sockstat -4 | grep 6969
root inetd 161 5 udp4 172.16.2.1:6969 *:*

Now I was able to retrieve my test file via TFTP.

tftp> get test.txt
getting from tftp.server.public.ip:test.txt to test.txt [octet]
sent RRQ
received DATA
Received 25 bytes in 0.1 seconds [2000 bits/sec]

I wanted to note that the man page recommended this addition to inetd.conf:

inetd(8) must be configured to spawn the proxy on the port that packets
are being forwarded to by pf(4). An example inetd.conf(5) entry follows:

127.0.0.1:6969 dgram udp wait root \
/usr/libexec/tftp-proxy tftp-proxy

That didn't work for me; I saw this error in /var/log/messages.

Jul 13 17:11:56 r200a inetd[99738]: 127.0.0.1:6969/udp: unknown service

By specifying the port only and using -a to bind inetd where I needed it, I avoided this error. There's probably another way around this though.

The final step will be seeing this TFTP-enabled device updating itself during the next 24 hours.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.

Sunday, July 12, 2009

Review of Practical Intrusion Analysis Posted

Amazon.com just published my three star review of Practical Intrusion Analysis by Ryan Trost. From the review:

I must start this review by stating the lead author lists me in the Acknowledgments and elsewhere in the book, which I appreciate. I also did consulting work years ago for the lead author's company, and I know the lead author to be a good guy with a unique eye for applying geography to network security data. Addison-Wesley provided me a review copy.

I did not participate in the writing process for Practical Intrusion Analysis (PIA), but after reading it I think I know how it unfolded. The lead author had enough material to write his two main sections: ch 10, Geospatial Intrusion Detection, and ch 11, Visual Data Communications. He realized he couldn't publish a 115-page book, so he enlisted five contributing authors who wrote chapters on loosely related security topics. Finally the lead author wrote two introductory sections: ch 1, Network Overview, and ch 2, Infrastructure Monitoring. This publication-by-amalgamation method seldom yields coherent or helpful material, despite the superior production efforts of a company like Addison-Wesley. To put a point on PIA's trouble, there's only a single intrusion analyzed in the book, and it's in the lead author's core section. The end result is a book you can skip, although it would be good for chapters 4 and 10 to be published separately as digital "Short Cuts" on InformIT.



Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.

Saturday, July 11, 2009

Must-Read Verizon Post Demolishes More Myths

I'm a big fan of the 2009 Verizon Data Breach Report. Today I read Compromised Assets & Data: But our company doesn’t handle credit cards... by Verizon's Bryan Sartin. It's an excellent post. I'd like to post several excerpts, emphasizing and expanding on certain points.

I find it fascinating that no matter where in the world you go, what type of company you talk to, public or private sector, you find two very common beliefs:

1. All data stolen in security breach is a result of lost assets, not systems-related intrusions.

2. I don’t handle payment cards (credit or debit) - so this stuff does not apply to me.

If you could only understand how outrageous these sound from the standpoint of the computer forensic investigator. Both thought processes couldn’t be more wrong.


I hear these refrains as well, or at least I see the effects of devoting resources to other projects. Bryan continues:

Pretty much everyone I speak to firmly believes that in the real world, companies do not get hacked into and data is never compromised as the result of a systems-based intrusion. The prevailing wisdom, if you can call it that, suggests that almost all lost records leading to fraud are the product of backup tapes that don’t make it from point A to point B, blackberries left in taxi cabs, and company-issued laptops left at train stations. This is the prevailing wisdom UNTIL a company is hacked.

In reality, hackers and fraudsters target data of value. Companies are targeted, either directly or indirectly, because they are perceived to be data rich, and data that is stolen tends to lead to some measurable form of fraud, whether it is counterfeit, identity fraud, etc...

Online data, including digital repositories of information like databases, transaction logs, and other aggregation and storage points, account for an overwhelming 94 percent of casework and 99.9 percent of all verifiable records compromised.


This is confirmation of my focus on external threats. Bryan turns to his second point:

"But my company doesn’t handle credit cards, so this doesn’t apply to me..." [I]t doesn’t matter whether you store payment card data or not. The threats affecting companies in a particular industry or sector care more about the ability to sidestep security controls reliably, than about what type of data they’ll find once inside. Every company has something of value to a hacker.

If you don't have something of value to an intruder, you probably don't do anything worth keeping you in operation.

The following excerpt is really crucial:

There is no question that our case load is biased toward payment cards. Payment card data is a premium cybercrime target because when applied in a certain manner, stolen records of sufficient content can lead to fraudulent purchases...

[B]based on our figures, I would estimate that payment cards represent as little as 1.2 – 1.5 percent of all data thefts. The remaining 98.x percent being occupied primarily by personally identifiable data (PII), then account credentials, company-proprietary data, and a few other categories in a distant fourth and fifth by incidence. Payment cards are in fact a distinct minority in data theft cases, albeit an extremely noisy minority.

The ensuing fraud is detectable and fraud analysis and detection tools have made it almost elementary to identify the likely source of a suspected payment card breach for almost 10 years.


Did you catch that? A stolen payment card intrusion is detectable. The hacked parties (online vendors, offline vendors, anyone using and storing payment card data) don't detect the actual theft of the data. Fraudulent use of the payment card data is detected by consumers and payment card providers. What about other data?

In simple terms, when payment card data is stolen – someone always finds out about it. The same cannot be said for PII and the other categories of compromised data we see...

Fraud is a direct, easily observable and easily trackable consequence of an intrusion. When an intruder steals payment card data, and the payment card data is used to commit fraud, the hacked party can be identified and notified using external means (bank or law enforcement calling).

However, the consequences of other data theft intrusions are not so easily observed nor tracked. If a competitor steals your company's intellectual property, sales plans, and other sensitive information, it may not be obvious how that competitor beat you to a deal that quarter. This is why I spoke of long-term competitiveness, because you can't tie non-payment card intrusions back to an obvious consequence or impact.

Thanks to Bryan Sartin for such a great post.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.

Review of Security Monitoring Posted

Amazon.com just posted my four star review of Security Monitoring by Chris Fry and Martin Nystrom. From the review:

I must start this review by noting that the authors of Security Monitoring (SM) cite my blog and books several times, which is appreciated. I must also mention that their boss Gavin Reid, who posted a review below, has offered to sponsor my company's application to the Forum of Incident Response and Security Teams (FIRST). O'Reilly kindly provided a review copy of SM.

I think SM should be positioned as an Introduction to Basic Security Monitoring. At just over 200 pages, it's not written to be much more than that. I'm not sure I will change the mind of the reviewer who considers my first book to be "introductory," but it might help to remember that my first book is just shy of 800 pages and covers every aspect of Network Security Monitoring.

SM is technically correct, but its approach to incident detection will fall far short of what is needed in the real world. SM concentrates on a paradigm it calls "policy-based monitoring," (abbreviated PBM here) with this goal: "to compare events discovered on the network to ensure that they are approved and acceptable... PBM is practical where acceptable conditions can be documented as policies... [Y]ou must codify acceptable behavior as policies, providing a reference point against which to survey" (pp 16-17) This sounds great, but it has several real flaws...


Please read the rest of the review for the whole story.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.

Friday, July 10, 2009

You Down with APT?

Today I had shared a phone call with a very knowledgable and respected security industry analyst. During the course of the conversation he made a few statements which puzzled me, so I asked him "do you know what APT means?" He might have thought I was referring to the Debian Advanced Package Tool or apt, but that's not what I meant. When I said Advanced Persistent Threat, it still didn't ring any bells with him. I decided to do some searching on the Web to see what was available regarding APT.

Helpfully, BusinessWeek just published Under Cyberthreat: Defense Contractors this week. The article begins like this:

Northrop Grumman's info security chief addresses the "well-resourced, highly sophisticated" attacks against makers of high-tech weaponry...

The defense industry faces "a near-existential threat from state-sponsored foreign intelligence services" that target sensitive IP, according to a report by the Internet Security Alliance, a nonprofit organization on whose board McKnight sits...

[BusinessWeek asked:] Are defense contractors being singled out in highly targeted attacks?

[McKnight responded:] It's gotten to a point where it has a name for itself: the APT or "advanced persistent threat," meaning that they are well resourced, highly sophisticated, clearly targeting companies or information, and they're not giving up in that mission.


Incidentally, McKnight practices NSM:

[BusinessWeek asked:] What kind of tools do you use to keep your network secure?

[McKnight responded:] We've focused a lot on... capabilities where you're capturing all traffic, not just bits and pieces of it.


Security company Mandiant devotes an entire site to APT, saying:

The Advanced Persistent Threat (APT) is a sophisticated and organized cyber attack to access and steal information from compromised computers.

The intruders responsible for the APT attacks target the Defense Industrial Base (DIB), financial industry, manufacturing industry, and research industry.

The attacks used by the APT intruders are not very different from any other intruder. The main differentiator is the APT intruder’s perseverance and resources. They have malicious code (malware) that circumvents common safeguards such as anti-virus and they tend to generate more activity than wanton “drive by hacks” on the Internet.

The intruders also escalate their tools and techniques as a victim firm’s capability to respond improves. Therefore, the APT attacks present different challenges than addressing common computer security breaches.

Combating the APT is a protracted event, requiring a sustained effort to rid your networks of the threat.


I briefly mentioned APT in my post last year Thoughts on 2008 SANS Forensics and IR Summit.

Aside from Northrup Grumman, Mandiant, and a few vendors (like NetWitness, one of the full capture vendors out there) mentioning APT, there's not much else available. A Google search for "advanced persistent threat" -netwitness -mandiant -Northrop yields 34 results (prior to this blog post).

APT is one of those subjects that is very important but not well understood outside the defense industry. Your best bet for a public introduction to APT is to watch for the next Webinar offered by Mandiant. Ask them to do another soon; I listened to their Webinar in May and realized many participants had never heard of APT before. If you're not down with APT, you need to be.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.

Thursday, July 02, 2009

Traffic Talk 6 Posted

My 6th edition of Traffic Talk, titled Wireshark 1.2 tutorial: Open source network analyzer's new features has been posted. From the article:

Wireshark is a staple of any network administrator's toolkit, and it can be equally useful for any network solution providers or consultants who troubleshoot business networks. Most of the readers of this tutorial have probably used Gerald Combs' open source protocol analyzer for years. In this edition of Traffic Talk, I'd like to discuss a few new features of Wireshark as present in the 1.2 version released on June 15, 2009. I use Windows XP SP3 as my test platform.

If you have any questions on the article, please post them here. Thank you.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.

Still Blogging

When I announced I would join General Electric as Director of Incident Response in June 2007, I had to post a follow-up titled I'm Not Dead. That issue even made it onto Bill Brenner's radar. Two years later I'm still at GE, glad that as of 1 January this year we have a functional and growing Computer Incident Response Team (CIRT) manned by the best incident handlers and support staff you'll find anywhere.

Sometimes work occupies time I would have previously spent blogging, reading, or writing. That's why you'll often see a flurry of blog posts when I have time on a weekend (or now, before a Company holiday). I've fallen far behind in my reading, and my writing is limited to articles. However, I will be collaborating with Keith Jones and team for Real Digital Forensics Volume 2, which should be cool. I don't have a schedule for other books beyond RDF2 at the moment.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.

Bejtlich on Black Hat Briefings Panel

The registration process for my TCP/IP Weapons School 2.0 class at Black Hat USA 2009 continues to be active, with seats almost gone in the weekday version. The weekend version has open seats. If you'd like more details, please see my post Black Hat Class Outline Posted.

I was invited to be a panelist for The Laws of Vulnerabilities Research Version 2.0: Comparing Critical Infrastructure Industries, a description of which is posted at the Black Hat Briefings speaker list. Because I'm busy during the 10 am panel time on day 1, I won't have to make the decision about which great talk I'll miss at that time! I mean, Billy Hoffman, FX, Rod Beckstrom, Dino Dai Zovi, and Chris Gates all at the same time?


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.

Review of Hacking Exposed: Windows, 3rd Ed Posted

Amazon.com just posted my four star review of Hacking Exposed: Windows, 3rd Ed. Better late than never! From the review:

I've been reading and reviewing Hacking Exposed (HE) books since 1999, and I reviewed the two previous Windows books. Hacking Exposed: Windows, 3rd Ed (HEW3E) is an excellent addition to the HE series. I agree with Chris Gates' review, but I'd like to add a few of my own points. The bottom line is that if you need a solid book on Windows technologies and how to attack and defend them, HEW3E is the right resource.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.

NSA to "Screen" .gov Now, I Predict .com Later

In my Predictions for 2008 I wrote Expect greater military involvement in defending private sector networks. Today I read a great Washington Post story titled Obama Administration to Involve NSA in Defending Civilian Agency Networks. It says in part:

The Obama administration will proceed with a Bush-era plan to use National Security Agency assistance in screening government computer traffic on private-sector networks, with AT&T as the likely test site...

President Obama said in May that government efforts to protect computer systems from attack would not involve "monitoring private sector networks or Internet traffic" and Department of Homeland Security officials say that the new program will only scrutinize data going to or from government systems...

Under a classified pilot program approved during the Bush administration, NSA data and hardware would be used to protect the networks of some civilian government agencies. Part of an initiative known as Einstein 3, the pilot called for telecommunications companies to route the Internet traffic of civilian government agencies through a monitoring box that would search for and block malicious computer codes...

The internal controversy reflects the central tension in the debate over how best to defend the nation's mostly private system of computer networks. The most effective techniques, experts say, require the automated scrutiny of e-mail and other electronic communications content -- something that commercial providers already do.

Proponents of involving the government said such efforts should harness the NSA's resources, especially its database of computer codes, or signatures, that have been linked to cyberattacks or known adversaries. The NSA has compiled the cache by, for example, electronically observing hackers trying to gain access to U.S. military systems, the officials said.

"That's the secret sauce," one official said. "It's the stuff they have that the private sector doesn't."

But it is also the prospect of NSA involvement in cybersecurity that fuels concerns of unwarranted government snooping into private communications...

The classified NSA system, known as Tutelage, has the ability to decide how to handle malicious intrusions -- to block them or watch them closely to better assess the threat, sources said. It is currently used to defend military networks.


You're thinking, "this article says NSA will not monitor purely private networks. What's the fuss?" Imagine you're the CEO, CIO/CTO, or CISO of a big company. You say "why is my company and our employees paying taxes so that the government can protect itself while my company is left outside the circled wagons?" The higher you go in corporate management, the more likely the only "security" that will be recognized will be "firewalls." So, you're going to have big-league corporate leaders telling the government that they want their companies "protected" too. This isn't really what is happening, but at that level it really doesn't matter.

The bottom line is that first the military protected itself, and now the military is going to help protect civilian government agencies. Critical private infrastructure will be next, followed by economically important companies -- think "too big to be 0wned." This will be interesting.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.