Monday, December 31, 2018

Notes on Self-Publishing a Book


In this post I would like to share a few thoughts on self-publishing a book, in case anyone is considering that option.

As I mentioned in my post on burnout, one of my goals was to publish a book on a subject other than cyber security. A friend from my Krav Maga school, Anna Wonsley, learned that I had published several books, and asked if we might collaborate on a book about stretching. The timing was right, so I agreed.

I published my first book with Pearson and Addison-Wesley in 2004, and my last with No Starch in 2013. 14 years is an eternity in the publishing world, and even in the last 5 years the economics and structure of book publishing have changed quite a bit.

To better understand the changes, I had dinner with one of the finest technical authors around, Michael W. Lucas. We met prior to my interest in this book, because I had wondered about publishing books on my own. MWL started in traditional publishing like me, but has since become a full-time author and independent publisher. He explained the pros and cons of going it alone, which I carefully considered.

By the end of 2017, Anna and I were ready to begin work on the book. I believe our first "commits" occurred in December 2017.

For this stretching book project, I knew my strengths included organization, project management, writing to express another person's message, editing, and access to a skilled lead photographer. I learned that my co-author's strengths included subject matter expertise, a willingness to be photographed for the book's many pictures, and friends who would also be willing to be photographed.

None of us was very familiar with the process of transforming a raw manuscript and photos into a finished product. When I had published with Pearson and No Starch, they took care of that process, as well as copy-editing.

Beyond turning manuscript and photos into a book, I also had to identify a publication platform. Early on we decided to self-publish using one of the many newer companies offering that service. We wanted a company that could get our book into Amazon, and possibly physical book stores as well. We did not want to try working with a traditional publisher, as we felt that we could manage most aspects of the publishing process ourselves, and augment with specialized help where needed.

After a lot of research we chose Blurb. One of the most attractive aspects of Blurb was their expert ecosystem. We decided that we would hire one of these experts to handle the interior layout process. We contacted Jennifer Linney, who happened to be local and had experience publishing books to Amazon. We met in person, discussed the project, and agreed to move forward together.

I designed the structure of the book. As a former Air Force officer, I was comfortable with the "rule of threes," and brought some recent writing experience from my abandoned PhD thesis.

I designed the book to have an introduction, the main content, and a conclusion. Within the main content, the book featured an introduction and physical assessment, three main sections, and a conclusion. The three main sections consisted of a fundamental stretching routine, an advanced stretching routine, and a performance enhancement section -- something with Indian clubs, or kettle bells, or another supplement to stretching.

Anna designed all of the stretching routines and provided the vast majority of the content. She decided to focus on three physical problem areas -- tight hips, shoulders/back, and hamstrings. We encouraged the reader to "reach three goals" -- open your hips, expand your shoulders, and touch your toes. Anna designed exercises that worked in a progression through the body, incorporating her expertise as a certified trainer and professional martial arts instructor.

Initially we tried a process whereby she would write section drafts, and I would edit them, all using Google Docs. This did not work as well as we had hoped, and we spent a lot of time stalled in virtual collaboration.

By the spring of 2018 we decided to try meeting in person on a regular basis. Anna would explain her desired content for a section, and we would take draft photographs using iPhones to serve as placeholders and to test the feasibility of real content. We made a lot more progress using these methods, although we stalled again mid-year due to schedule conflicts.

By October our text was ready enough to try taking book-ready photographs. We bought photography lights from Amazon and used my renovated basement game room as a studio. We took pictures over three sessions, with Anna and her friend Josh as subjects. I spent several days editing the photos to prepare for publication, then handed the bundled manuscript and photographs to Jennifer for a light copy-edit and layout during November.

Our goal was to have the book published before the end of the year, and we met that goal. We decided to offer two versions. The first is a "collector's edition" featuring all color photographs, available exclusively via Blurb as Reach Your Goal: Collector's Edition. The second will be available at Amazon in January, and will feature black and white photographs.

While we were able to set the price of the book directly via Blurb, we could basically only suggest a price to Ingram and hence to Amazon. Ingram is the distributor that feeds Amazon and physical book stores. I am curious to see how the book will appear in those retail locations, and how much it will cost readers. We tried to price it competitively with older stretching books of similar size. (Ours is 176 pages with over 200 photographs.)

Without revealing too much of the economic structure, I can say that it's much cheaper to sell directly from Blurb. Their cost structure allows us to price the full color edition competitively. However, one of our goals was to provide our book through Amazon, and to keep the price reasonable we had to sell the black and white edition outside of Blurb.

Overall I am very pleased with the writing process, and exceptionally happy with the book itself. The color edition is gorgeous and the black and white version is awesome too.

The only change I would have made to the writing process would have been to start the in-person collaboration from the beginning. Working together in person accelerated the transfer of ideas to paper and played to our individual strengths of Anna as subject matter expert and me as a writer.

In general, I would not recommend self-publishing if you are not a strong writer. If writing is not your forte, then I highly suggest you work with a traditional publisher, or contract with an editor. I have seen too many self-published books that read terribly. This usually happens when the author is a subject matter expert, but has trouble expressing ideas in written form.

The bottom line is that it's never been easier to make your dream of writing a book come true. There are options for everyone, and you can leverage them to create wonderful products that scale with demand and can really help your audience reach their goals!

If you want to start the new year with better flexibility and fitness, consider taking a look at our book on Blurb! When the Amazon edition is available I will update this post with a link.

Update: Here is the Amazon listing.

Cross-posted from Rejoining the Tao Blog.

Friday, December 21, 2018

Managing Burnout

This is not strictly an information security post, but the topic likely affects a decent proportion of my readership.

Within the last few years I experienced a profound professional "burnout." I've privately mentioned this to colleagues in the industry, and heard similar stories or requests for advice on how to handle burnout.

I want to share my story in the hopes that it helps others in the security scene, either by coping with existing burnout or preparing for a possible burnout.

How did burnout manifest for me? It began with FireEye's acquisition of Mandiant, almost exactly five years ago. 2013 was a big year for Mandiant, starting with the APT1 report in early 2013 and concluding with the acquisition in December.

The prospect of becoming part of a Silicon Valley software company initially seemed exciting, because we would presumably have greater resources to battle intruders. Soon, however, I found myself at odds with FireEye's culture and managerial habits, and I wondered what I was doing inside such a different company.

(It's important to note that the appointment of Kevin Mandia as CEO in June 2016 began a cultural and managerial shift. I give Kevin and his lieutenants credit for helping transform the company since then. Kevin's appointment was too late for me, but I applaud the work he has done over the last few years.)

Starting in late 2014 and progressing in 2015, I became less interested in security. I was aggravated every time I saw the same old topics arise in social or public media. I did not see the point of continuing to debate issues which were never solved. I was demoralized and frustrated.

At this time I was also working on my PhD with King's College London. I had added this stress myself, but I felt like I could manage it. I had earned two major and two minor degrees in four years as an Air Force Academy cadet. Surely I could write a thesis!

Late in 2015 I realized that I needed to balance the very cerebral art of information security with a more physical activity. I took a Krav Maga class the first week of January 2016. It was invigorating and I began a new blog, Rejoining the Tao, that month. I began to consider options outside of informations security.

In early 2016 my wife began considering ways to rejoin the W-2 workforce, after having stayed home with our kids for 12 years. We discussed the possibility of me leaving my W-2 job and taking a primary role with the kids. By mid-2016 she had a new job and I was open to departing FireEye.

By late 2016 I also realized that I was not cut out to be a PhD candidate. Although I had written several books, I did not have the right mindset or attitude to continue writing my thesis. After two years I quit my PhD program. This was the first time I had quit anything significant in my life, and it was the right decision for me. (The Churchill "never, never, never give up" speech is fine advice when defending your nation's existence, but it's stupid advice if you're not happy with the path you're following.)

In March 2017 I posted Bejtlich Moves On, where I said I was leaving FireEye. I would offer security consulting in the short term, and would open a Krav Maga school in the long-term. This was my break with the security community and I was happy to make it. I blogged on security only five more times in 2017.

(Incidentally, one very public metric for my burnout experience can be seen in my blog output. In 2015 I posted 55 articles, but in 2016 I posted only 8, and slightly more, 12, in 2017. This is my 21st post of 2018.)

I basically took a year off from information security. I did some limited consulting, but Mrs B paid the bills, with some support from my book royalties and consulting. This break had a very positive effect on my mental health. I stayed aware of security developments through Twitter, but I refused to speak to reporters and did not entertain job offers.

During this period I decided that I did not want to open a Krav Maga school and quit my school's instructor development program. For the second time, I had quit something I had once considered very important.

I started a new project, though -- writing a book that had nothing to do with information security. I will post about it shortly, as I am finalizing the cover with the layout team this weekend!

By the spring of 2018 I was able to consider returning to security. In May I blogged that I was joining Splunk, but that lasted only two months. I realized I had walked into another cultural and managerial mismatch. Near the end of that period, Seth Hall from Corelight contacted me, and by July 20th I was working there. We kept it quiet until September. I have been very happy at Corelight, finally finding an environment that matches my temperament, values, and interests.

My advice to those of you who have made it this far:

If you're feeling burnout now, you're not alone. It happens. We work in a stressful industry that will take everything that you can give, and then try to take more. It's healthy and beneficial to push back. If you can, take a break, even if it means only a partial break.

Even if you can't take a break, consider integrating non-security activities into your lifestyle -- the more physical, the better. Security is a very cerebral activity, often performed in a sedentary manner. You have a body and taking care of it will make your mind happier too.

If you're not feeling burnout now, I recommend preparing for a possible burnout in the future. In addition to the advice in the previous paragraphs, take steps now to be able to completely step away from security for a defined period. Save a proportion of your income to pay your bills when you're not working in security. I recommend at least a month, but up to six months if you can manage it.

This is good financial advice anyway, in the event you were to lose your job. This is not an emergency fund, though -- this is a planned reprieve from burnout. We are blessed in security to make above-average salaries, so I suggest saving for retirement, saving for layoffs, and saving for burnout.

Finally, it's ok to talk to other people about this. This will likely be a private conversation. I don't see too many people saying "I'm burned out!" on Twitter or in a blog post. I only felt comfortable writing this post months after I returned to regular security work.

I'm very interested in hearing what others have to say on this topic. Replying to my Twitter announcement for the blog post is probably the easiest step. I moderate the comments here and might not get to them in a timely manner.

Tuesday, December 18, 2018

The Origin of the Quote "There Are Two Types of Companies"

While listening to a webcast this morning, I heard the speaker mention

There are two types of companies: those who have been hacked, and those who don’t yet know they have been hacked.

He credited Cisco CEO John Chambers but didn't provide any source.

That didn't sound right to me. I could think of two possible antecedents. so I did some research. I confirmed my memory and would like to present what I found here.

John Chambers did indeed offer the previous quote, in a January 2015 post for the World Economic Forum titled What does the Internet of Everything mean for security? Unfortunately, neither Mr Chambers nor the person who likely wrote the article for him decided to credit the author of this quote.

Before providing proper credit for this quote, we need to decide what the quote actually says. As noted in this October 2015 article by Frank Johnson titled Are there really only “two kinds of enterprises”?, there are really (at least) two versions of this quote:

A popular meme in the information security industry is, “There are only two types of companies: those that know they’ve been compromised, and those that don’t know.”

And the second is like unto it: “There are only two kinds of companies: those that have been hacked, and those that will be.”

We see that the first is a version of what Mr Chambers said. Let's call that 2-KNOW. The second is different. Let's call that 2-BE.

The first version, 2-KNOW, can be easily traced and credited to Dmitri Alperovitch. He stated this proposition as part of the publicity around his Shady RAT report, written while he worked at McAfee. For example, this 3 August 2011 story by Ars Technica, Operation Shady RAT: five-year hack attack hit 14 countries, quotes Dmitri in the following:

So widespread are the attacks that Dmitri Alperovitch, McAfee Vice President of Threat Research, said that the only companies not at risk are those who have nothing worth taking, and that of the world's biggest firms, there are just two kinds: those that know they've been compromised, and those that still haven't realized they've been compromised.

Dmitri used slightly different language in this popular Vanity Fair article from September 2011, titled Enter the Cyber-Dragon:

Dmitri Alperovitch, who discovered Operation Shady rat, draws a stark lesson: “There are only two types of companies—those that know they’ve been compromised, and those that don’t know. If you have anything that may be valuable to a competitor, you will be targeted, and almost certainly compromised.”

No doubt former FBI Director Mueller read this report (and probably spoke with Dmitri). He delivered a speech at RSA on 1 March 2012 that introduced question 2-BE into the lexicon, plus a little more:

For it is no longer a question of “if,” but “when” and “how often.”

I am convinced that there are only two types of companies: those that have been hacked and those that will be. 

And even they are converging into one category: companies that have been hacked and will be hacked again.  

Here we see Mr Mueller morphing Dmitri's quote, 2-KNOW, into the second, 2-BE. He also introduced a third variant -- "companies that have been hacked and will be hacked again." Let's call this version 2-AGAIN.

The very beginning of Mr Mueller's quote is surely a play on Kevin Mandia's long-term commitment to the inevitability of compromise. However, as far as I could find, Kevin did not use the "two companies" language.

One article that mentions version 2-KNOW and Kevin is this December 2014 Ars Technica article titled “Unprecedented” cyberattack no excuse for Sony breach, pros say. However, the article is merely citing other statements by Kevin along with the aphorism of version 2-KNOW.

Finally, there's a fourth version introduced by Mr Mueller's successor, James Comey, as well! In a 6 October 2014 story, FBI Director: China Has Hacked Every Big US Company Mr Comey said:

Speaking to CBS' 60 Minutes, James Comey had the following to say on Chinese hackers: 

There are two kinds of big companies in the United States. There are those who've been hacked by the Chinese and those who don't know they've been hacked by the Chinese.

Let's call this last variant 2-CHINA.

To summarize, there are four versions of the "two companies" quote:

  • 2-KNOW, credited to Dmitri Alperovitch in 2011, says "There are only two types of companies—those that know they’ve been compromised, and those that don’t know."
  • 2-BE, credited to Robert Mueller in 2012, says "[T]here are only two types of companies: those that have been hacked and those that will be."
  • 2-AGAIN, credited to Robert Mueller in 2012, says "[There are only two types of companies:] companies that have been hacked and will be hacked again."
  • 2-CHINA, credited to James Comey in 2014, says "There are two kinds of big companies in the United States. There are those who've been hacked by the Chinese and those who don't know they've been hacked by the Chinese."
Now you know!


Sunday, November 25, 2018

The Origin of the Term Indicators of Compromise (IOCs)

I am an historian. I practice digital security, but I earned a bachelor's of science degree in history from the United States Air Force Academy. (1)

Historians create products by analyzing artifacts, among which the most significant is the written word.

In my last post, I talked about IOCs, or indicators of compromise. Do you know the origin of the term? I thought I did, but I wanted to rely on my historian's methodology to invalidate or confirm my understanding.

I became aware of the term "indicator" as an element of indications and warning (I&W), when I attended Air Force Intelligence Officer's school in 1996-1997. I will return to this shortly, but I did not encounter the term "indicator" in a digital security context until I encountered the work of Kevin Mandia.

In August 2001, shortly after its publication, I read Incident Response: Investigating Computer Crime, by Kevin Mandia, Chris Prosise, and Matt Pepe (Osborne/McGraw-Hill). I was so impressed by this work that I managed to secure a job with their company, Foundstone, by April 2002. I joined the Foundstone incident response team, which was led by Kevin and consisted of Matt Pepe, Keith Jones, Julie Darmstadt, and me.

I Tweeted earlier today that Kevin invented the term "indicator" (in the IR context) in that 2001 edition, but a quick review of the hard copy in my library does not show its usage, at least not prominently. I believe we were using the term in the office but that it had not appeared in the 2001 book. Documentation would seem to confirm that, as Kevin was working on the second edition of the IR book (to which I contributed), and that version, published in 2003, features the term "indicator" in multiple locations.

In fact, the earliest use of the term "indicators of compromise," appearing in print in a digital security context, appears on page 280 in Incident Response & Computer Forensics, 2nd Edition.


From other uses of the term "indicators" in that IR book, you can observe that IOC wasn't a formal, independent concept at this point, in 2003. In the same excerpt above you see "indicators of attack" mentioned.

The first citation of the term "indicators" in the 2003 book shows it is meant as an investigative lead or tip:


Did I just give up my search at this point? Of course not.

If you do time-limited Google searches for "indicators of compromise," after weeding out patent filings that reference later work (from FireEye, in 2013), you might find this document, which concludes with this statement:

Indicators of compromise are from Lynn Fischer, Lynn, "Looking for the Unexpected," Security Awareness Bulletin, 3-96, 1996. Richmond, VA: DoD Security Institute.

Here the context is the compromise of a person with a security clearance.

In the same spirit, the earliest reference to "indicator" in a security-specific, detection-oriented context appears in the patent Method and system for reducing the rate of infection of a communications network by a software worm (6 Dec 2002). Stuart Staniford is the lead author; he was later chief scientist at FireEye, although he left before FireEye acquired Mandiant (and me).

While Kevin, et al were publishing the second edition of their IR book in 2003, I was writing my first book, The Tao of Network Security Monitoring. I began chapter two with a discussion of indicators, inspired by my Air Force intelligence officer training in I&W and Kevin's use of the term at Foundstone.

You can find chapter two in its entirety online. In the chapter I also used the term "indicators of compromise," in the spirit Kevin used it; but again, it was not yet a formal, independent term.

My book was published in 2004, followed by two more in rapid succession.

The term "indicators" didn't really make a splash until 2009, when Mike Cloppert published a series on threat intelligence and the cyber kill chain. The most impactful in my opinion was Security Intelligence: Attacking the Cyber Kill Chain. Mike wrote:


I remember very much enjoying these posts, but the Cyber Kill Chain was the aspect that had the biggest impact on the security community. Mike does not say "IOC" in the post. Where he does say "compromise," he's using it to describe a victimized computer.

The stage is now set for seeing indicators of compromise in a modern context. Drum roll, please!

The first documented appearance of the term indicators of compromise, or IOCs, in the modern context, appears in basically two places simultaneously, with ultimate credit going to the same organziation: Mandiant.

The first Mandiant M-Trends report, published on 25 Jan 2010, provides the following description of IOCs on page 9:


The next day, 26 Jan 2010, Matt Frazier published Combat the APT by Sharing Indicators of Compromise to the Mandiant blog. Matt wrote to introduce an XML-based instantiation of IOCs, which could be read and created using free Mandiant tools.


Note how complicated Matt's IOC example is. It's not a file hash (alone), or a file name (alone), or an IP address, etc. It's a Boolean expression of many elements. You can read in the text that this original IOC definition rejects what some commonly consider "IOCs" to be. Matt wrote:

Historically, compromise data has been exchanged in CSV or PDFs laden with tables of "known bad" malware information - name, size, MD5 hash values and paragraphs of imprecise descriptions... (emphasis added)

On a related note, I looked for early citations of work on defining IOCs, and found a paper by Simson Garfinkel, well-respected forensic analyst. He gave credit to Matt Frazier and Mandiant, writing in 2011:

Frazier (2010) of MANDIANT developed Indicators of Compromise (IOCs), an XML-based language designed to express signatures of malware such as files with a particular MD5 hash value, file length, or the existence of particular registry entries. There is a free editor for manipulating the XML. MANDIANT has a tool that can use these IOCs to scan for malware and the so-called “Advanced Persistent Threat.”

Starting in 2010, the debate was initially about the format for IOCs, and how to produce and consume them. We can see in this written evidence from 2010, however, a definition of indicators of compromise and IOCs that contains all the elements that would be recognized in current usage.

tl;dr Mandiant invented the term indicators of compromise, or IOCs, in 2010, building off the term "indicator," introduced widely in a detection context by Kevin Mandia, no later than his 2003 incident response book.

(1) Yes, a BS, not a BA -- thank you USAFA for 14 mandatory STEM classes.

Saturday, November 24, 2018

Even More on Threat Hunting

In response to my post More on Threat Hunting, Rob Lee asked:

[D]o you consider detection through ID’ing/“matching” TTPs not hunting?

To answer this question, we must begin by clarifying "TTPs." Most readers know TTPs to mean tactics, techniques and procedures, defined by David Bianco in his Pyramid of Pain post as:

How the adversary goes about accomplishing their mission, from reconnaissance all the way through data exfiltration and at every step in between.

In case you've forgotten David's pyramid, it looks like this.


It's important to recognize that the pyramid consists of indicators of compromise (IOCs). David uses the term "indicator" in his original post, but his follow-up post from his time at Sqrrl makes this clear:

There are a wide variety of IoCs ranging from basic file hashes to hacking Tactics, Techniques and Procedures (TTPs). Sqrrl Security Architect, David Bianco, uses a concept called the Pyramid of Pain to categorize IoCs. 

At this point it should be clear that I consider TTPs to be one form of IOC.

In The Practice of Network Security Monitoring, I included the following workflow:

You can see in the second column that I define hunting as "IOC-free analysis." On page 193 of the book I wrote:

Analysis is the process of identifying and validating normal, suspicious, and malicious activity. IOCs expedite this process. Formally, IOCs are manifestations of observable or discernible adversary actions. Informally, IOCs are ways to codify adversary activity so that technical systems can find intruders in digital evidence...

I refer to relying on IOCs to find intruders as IOC-centric analysis, or matching. Analysts match IOCs to evidence to identify suspicious or malicious activity, and then validate their findings.

Matching is not the only way to find intruders. More advanced NSM operations also pursue IOC-free analysis, or hunting. In the mid-2000s, the US Air Force popularized the term hunter-killer in the digital world. Security experts performed friendly force projection on their networks, examining data and sometimes occupying the systems themselves in order to find advanced threats. 

Today, NSM professionals like David Bianco and Aaron Wade promote network “hunting trips,” during which a senior investigator with a novel way to detect intruders guides junior analysts through data and systems looking for signs of the adversary. 

Upon validating the technique (and responding to any enemy actions), the hunters incorporate the new detection method into a CIRT’s IOC-centric operations. (emphasis added)

Let's consider Chris Sanders' blog post titled Threat Hunting for HTTP User Agents as an example of my definition of hunting. 

I will build a "hunting profile" via excerpts (in italics) from his post:

Assumption: "Attackers frequently use HTTP to facilitate malicious network communication."

Hypothesis: If I find an unusual user agent string in HTTP traffic, I may have discovered an attacker.

Question: “Did any system on my network communicate over HTTP using a suspicious or unknown user agent?”

Method: "This question can be answered with a simple aggregation wherein the user agent field in all HTTP traffic for a set time is analyzed. I’ve done this using Sqrrl Query Language here:

SELECT COUNT(*),user_agent FROM HTTPProxy GROUP BY user_agent ORDER BY COUNT(*) ASC LIMIT 20

This query selects the user_agent field from the HTTPProxy data source and groups and counts all unique entries for that field. The results are sorted by the count, with the least frequent occurrences at the top."

Results: Chris offers advice on how to interpret the various user agent strings produced by the query.

This is the critical part: Chris did not say "look for *this user agent*. He offered the reader an assumption, a hypothesis, a question, and a method. It is up to the defender to investigate the results. This, for me, is true hunting.

If Chris had instead referred users to this list of malware user agents (for example) and said look for "Mazilla/4.0", then I consider that manual (human) matching. If I created a Snort or Suricata rule to look for that user agent, then I consider that automated (machine) matching.

This is where my threat hunting definition likely diverges from modern practice. Analyst Z sees the results of Chris' hunt and thinks "Chris found user agent XXXX to be malicious, so I should go look for it." Analyst Z queries his or her data and does or does not find evidence of user agent XXXX.

I do not consider analyst Z's actions to be hunting. I consider it matching. There is nothing wrong with this. In fact, one of the purposes of hunting is to provide new inputs to the matching process, so that future hunting trips can explore new assumptions, hypotheses, questions, and methods, and let the machines do the matching on IOCs already found to be suggestive of adversary activity. This is why I wrote in my 2013 book "Upon validating the technique (and responding to any enemy actions), the hunters incorporate the new detection method into a CIRT’s IOC-centric operations."

The term "hunting" is a victim of its own success, with emotional baggage. We defenders have finally found a way to make "blue team" work appealing to the wider security community. Vendors love this new way to market their products. "If you're not hunting, are you doing anything useful?" one might ask.

Compared to "I'm threat hunting!" (insert chest beating), the alternative, "I'm matching!" (womp womp), seems sad. 

Nevertheless, we must remember that threat hunting methodologies were invented to find adversary activity for which there were no IOCs. Hunting was IOC-free analysis because we didn't know what to look for. Once you know what to look for, you are matching. Both forms of detection require analysis to validate adversary activity, of course. Let's not forget that.

I'm also very thankful, however it's defined or packaged, that people are excited to search for adversary activity in their environment, whether via matching or hunting. It's a big step from the mindset of 10 years ago, which had a "prevention works" milieu.

tl;dr Because TTPs are a form of IOC, then detection via matching IOCs is a form of matching, and not hunting.

Friday, November 23, 2018

More on Threat Hunting

Earlier this week hellor00t asked via Twitter:

Where would you place your security researchers/hunt team?

I replied:

For me, "hunt" is just a form of detection. I don't see the need to build a "hunt" team. IR teams detect intruders using two major modes: matching and hunting. Junior people spend more time matching. Senior people spend more time hunting. Both can and should do both functions.

This inspired Rob Lee to blog a response, from which I extract his core argument:

[Hunting] really isn’t, to me, about detecting threats...

Hunting is a hypothesis-led approach to testing your environment for threats. The purpose, to me, is not in finding threats but in determining what gaps you have in your ability to detect and respond to them...

In short, hunting, to me, is a way to assess your security (people, process, and technology) against threats while extending your automation footprint to better be prepared in the future. Or simply stated, it’s incident response without the incident that’s done with a purpose and contributes something. 

As background for my answer, I recommend my March 2017 post The Origin of Threat Hunting, which cites my article "Become a Hunter," published in the July-August 2011 issue of Information Security Magazine. I wrote it in the spring of 2011, when I was director of incident response for GE-CIRT.

For the term "hunting," I give credit to briefers from the Air Force and NSA who, in the mid-2000s briefed "hunter-killer" missions to the Red Team/Blue Team Symposium at the Johns Hopkins University Applied Physics Lab in Laurel, MD.

As a comment to that post, Tony Sager, who ran NSA VAO at the time I was briefed at ReBl, described hunting thus:

[Hunting] was an active and sustained search for Attackers...

For us, "Hunt" meant a very planned and sustained search, taking advantage of the existing infrastructure of Red/Blue Teams and COMSEC Monitoring, as well as intelligence information to guide the search. 

For the practice of hunting, as I experienced it, I give credit to our GE-CIRT incident handlers -- David Bianco,  Ken Bradley, Tim Crothers, Tyler Hudak, Bamm Visscher, and Aaron Wade -- who took junior analysts on "hunting trips," starting in 2008-2009.

It is very clear, to me, that hunting has always been associated with detecting an adversary, not "determining what gaps you have in your ability to detect and respond to them," as characterized by Rob.

For me, Rob is describing the job of an enterprise visibility architect, which I described in a 2007 post:

[W]e are stuck with numerous platforms, operating systems, applications, and data (POAD) for which we have zero visibility. 

I suggest that enterprises consider hiring or assigning a new role -- Enterprise Visibility Architect. The role of the EVA is to identify visibility deficiencies in existing and future POAD and design solutions to instrument these resources.

A primary reason to hire an enterprise visibility architect is to build visibility in, which I described in several posts, including this one from 2009 titled Build Visibility In. As a proponent of the "monitor first" school, I will always agree that it is important to identify and address visibility gaps.

So where do we go from here?

Tony Sager, as one of my wise men, offers sage advice at the conclusion of his comment:

"Hunt" emerged as part of a unifying mission model for my Group in the Information Assurance Directorate at NSA (the defensive mission) in the mid-late 2000's. But it was also a way to unify the relationship between IA and the SIGINT mission - intelligence as the driver for Hunting. The marketplace, of course, has now brought its own meaning to the term, but I just wanted to share some history. 

In my younger days I might have expressed much more energy and emotion when encountering a different viewpoint. At this point in my career, I'm more comfortable with other points of view, so long as they do not result in harm, or a waste of my taxpayer dollars, or other clearly negative consequences. I also appreciate the kind words Rob offered toward my point of view.

tl;dr I believe the definition and practice of hunting has always been tied to adversaries, and that Rob describes the work of an enterprise visibility architect when he focuses on visibility gaps rather than adversary activity.

Update 1: If in the course of conducting a hunt you identify a visibility or resistance deficiency, that is indeed beneficial. The benefit, however, is derivative. You hunt to find adversaries. Identifying gaps is secondary although welcome.

The same would be true of hunting and discovering misconfigured systems, or previously unidentified assets, or unpatched software, or any of the other myriad facts on the ground that manifest when one applies Clausewitz's directed telescope towards their computing environment.

Friday, November 02, 2018

Cybersecurity and Class M Planets

I was considering another debate about appropriate cybersecurity measures and I had the following thought: not all networks are the same. Profound, right? This is so obvious, yet so obviously forgotten.

Too often when confronting a proposed defensive measure, an audience approaches the concept from their own preconceived notion of what assets need to be protected.

Some think about an information technology enterprise organization with endpoints, servers, and infrastructure. Others think about an industrial organization with manufacturing equipment. Others imagine an environment with no network at all, where constituents access cloud-hosted resources. Still others think in terms of being that cloud hosting environment itself.

Beyond those elements, we need to consider the number of assets, their geographic diversity, their relative value, and many other aspects that you can no doubt imagine.

This made me wonder if we need some sort of easy reference term to capture the essential nature of these sorts of diverse environments. I thought immediately of the term "class M planet," from Star Trek. From the Wikipedia entry:

[An] Earth-like planet, the Class M designation is similar to the real-world astronomical theory of life-supporting planets within the habitable zone... Class M planets are said to possess an atmosphere composed of nitrogen and oxygen as well as an abundance of liquid water necessary for carbon-based life to exist. Extensive plant and animal life often flourishes; often, a sentient race is also present. 

In contrast, consider a class Y planet:

Class Y planets are referred to as "demon" worlds, where surface conditions do not fall into any other recognized category. Such worlds are usually hostile and lethal to humanoid life. If life forms develop on these worlds they usually take on many bizarre forms, like living crystal or rock, liquid or gaseous physical states, or incorporeal, dimensional, or energy-based states. 

Given their work providing names for various offensive security activities in ATT&CK, I wonder if MITRE might consider creating a naming scheme to capture this idea? For example, a "class M" network might be an enterprise organization with endpoints, servers, and infrastructure, of a certain size. Or perhaps M1 might be "small," M2 "medium," and M3 "large," where each is associated with a user count.

Perhaps an environment with no network at all, where constituents access cloud-hosted resources, would be a class C network. (I'm not sure "network" is even the right term, if there is no "network" for which the organization is responsible.)

With such a scheme in place, we could begin a cybersecurity discussion by asking, "given a class M network, what defensive processes, people, or technology are appropriate," versus "given a class C network, what defensive processes, people, or technology are appropriate."

This is only an idea, and I'd be happy if something was already created to address this problem. Comments below are welcome (pending moderation to repel trolls and spammers.) Alternatively, reply to my announcement of this post via @taosecurity on Twitter.

Thursday, October 25, 2018

Have Network, Need Network Security Monitoring

I have been associated with network security monitoring my entire cybersecurity career, so I am obviously biased towards network-centric security strategies and technologies. I also work for a network security monitoring company (Corelight), but I am not writing this post in any corporate capacity.

There is a tendency in many aspects of the security operations community to shy away from network-centric approaches. The rise of encryption and cloud platforms, the argument goes, makes methodologies like NSM less relevant. The natural response seems to be migration towards the endpoint, because it is still possible to deploy agents on general purpose computing devices in order to instrument and interdict on the endpoint itself.

It occurred to me this morning that this tendency ignores the fact that the trend in computing is toward closed computing devices. Mobile platforms, especially those running Apple's iOS, are not friendly to introducing third party code for the purpose of "security." In fact, one could argue that iOS is one of, if not the, most security platform, thanks to this architectural decision. (Timely and regular updates, a policed applications store, and other choices are undoubtedly part of the security success of iOS, to be sure.)

How is the endpoint-centric security strategy going to work when security teams are no longer able to install third party endpoint agents? The answer is -- it will not. What will security teams be left with?

The answer is probably application logging, i.e., usage and activity reports from the software with which users interact. Most of this will likely be hosted in the cloud. Therefore, security teams responsible for protecting work-anywhere-but-remote-intensive users, accessing cloud-hosted assets, will have really only cloud-provided data to analyze and escalate.

It's possible that the endpoint providers themselves might assume a greater security role. In other words, Apple and other manufacturers provide security information directly to users. This could be like Chase asking if I really made a purchase. This model tends to break down when one is using a potentially compromised asset to ask the user if that asset is compromised.

In any case, this vision of the future ignores the fact that someone will still be providing network services. My contention is that if you are responsible for a network, you are responsible for monitoring it.

It is negligent to provide network services but ignore abuse of that service.

If you disagree and cite the "common carrier" exception, I would agree to a certain extent. However, one cannot easily fall back on that defense in an age where Facebook, Twitter, and other platforms are being told to police their infrastructure or face ever more government regulation.

At the end of the day, using modern Internet services means, by definition, using someone's network. Whoever is providing that network will need to instrument it, if only to avoid the liability associated with misuse. Therefore, anyone operating a network would do well to continue to deploy and operate network security monitoring capabilities.

We may be in a golden age of endpoint visibility, but closure of those platforms will end the endpoint's viability as a source of security logging. So long as there are networks, we will need network security monitoring.

Friday, October 05, 2018

Network Security Monitoring vs Supply Chain Backdoors

On October 4, 2018, Bloomberg published a story titled “The Big Hack: How China Used a Tiny Chip to Infiltrate U.S. Companies,” with a subtitle “The attack by Chinese spies reached almost 30 U.S. companies, including Amazon and Apple, by compromising America’s technology supply chain, according to extensive interviews with government and corporate sources.” From the article:

Since the implants were small, the amount of code they contained was small as well. But they were capable of doing two very important things: telling the device to communicate with one of several anonymous computers elsewhere on the internet that were loaded with more complex code; and preparing the device’s operating system to accept this new code. The illicit chips could do all this because they were connected to the baseboard management controller, a kind of superchip that administrators use to remotely log in to problematic servers, giving them access to the most sensitive code even on machines that have crashed or are turned off.

Companies mentioned in the story deny the details, so this post does not debate the merit of the Bloomberg reporters’ claims. Rather, I prefer to discuss how a computer incident response team (CIRT) and a chief information security officer (CISO) should handle such a possibility. What should be done when hardware-level attacks enabling remote access via the network are possible?

This is not a new question. I have addressed the architecture and practices needed to mitigate this attack model in previous writings. This scenario is a driving force behind my recommendation for network security monitoring (NSM) for any organization running a network, of any kind. This does not mean endpoint-centric security, or other security models, should be abandoned. Rather, my argument shows why NSM offers unique benefits when facing hardware supply chain attacks.

The problem is one of trust and detectability. The problem here is that one loses trust in the integrity of a computing platform when one suspects a compromised hardware environment. One way to validate whether a computing platform is trustworthy is to monitor outside of it, at places where the hardware cannot know it is being monitored, and cannot interfere with that monitoring. Software installed on the hardware is by definition untrustworthy because the hardware backdoor may have the capability to obscure or degrade the visibility and control provided by an endpoint agent.

Network security monitoring applied outside the hardware platform does not suffer this limitation, if certain safeguards are implemented. NSM suffers limitations unique to its deployment, of course, and they will be outlined shortly. By watching traffic to and from a suspected computing platform, CIRTs have a chance to identify suspicious and malicious activity, such as contact with remote command and control (C2) infrastructure. NSM data on this C2 activity can be collected and stored in many forms, such as any of the seven NSM data types: 1) full content; 2) extracted content; 3) session data; 4) transaction data; 5) statistical data; 6) metadata; and 7) alert data.

Most likely session and transaction data would have been most useful for the case at hand. Once intelligence agencies identified that command and control infrastructure used by the alleged Chinese agents in this example, they could provide that information to the CIRT, who could then query historical NSM data for connectivity between enterprise assets and C2 servers. The results of those queries would help determine if and when an enterprise was victimized by compromised hardware.

The limitations of this approach are worth noting. First, if the intruders never activated their backdoors, then there would be no evidence of communications with C2 servers. Hardware inspection would be the main way to deal with this problem. Second, the intruders may leverage popular Internet services for their C2. Historical examples include command and control via Twitter, domain fronting via Google or other Web sites, and other covert channels. Depending on the nature of the communication, it would be difficult, though not impossible, to deal with this situation, mainly through careful analysis. Third, traditional network-centric monitoring would be challenging if the intruders employed an out-of-band C2 channel, such as a cellular or radio network. This has been seen in the wild but does not appear to be the case in this incident. Technical countermeasures, whereby rooms are swept for unauthorized signals, would have to be employed. Fourth, it’s possible, albeit unlikely, that NSM sensors tasked with watching for suspicious and malicious activity are themselves hosted on compromised hardware, making their reporting also untrustworthy.

The remedy for the last instance is easier than that for the previous three. Proper architecture and deployment can radically improve the trust one can place in NSM sensors. First, the sensors should not be able to connect to arbitrary systems on the Internet. The most security conscious administrators apply patches and modifications using direct access to trusted local sources, and do not allow access for any reason other than data retrieval and system maintenance. In other words, no one browses Web sites or checks their email from NSM sensors! Second, this moratorium on arbitrary connections should be enforced by firewalls outside the NSM sensors, and any connection attempts that violate the firewall policy should generate a high-priority alert. It is again theoretically possible for an extremely advanced intruder to circumvent these controls, but this approach increases the likelihood of an adversary tripping a wire at some point, revealing his or her presence.

The bottom line is that NSM must be a part of the detection and response strategy for any organization that runs a network. Collecting and analyzing the core NSM data types, in concert with host-based security, integration with third party intelligence, and infrastructure logging, provides the best chance for CIRTs to detect and respond to the sorts of adversaries who escalate their activities to the level of hardware hacking via the supply chain. Whether or not the Bloomberg story is true, the investment in NSM merits the peace of mind a CISO will enjoy when his or her CIRT is equipped with robust network visibility.

This post first appeared on the Corelight blog.

Tuesday, September 18, 2018

Firewalls and the Need for Speed

I was looking for resources on campus network design and found these slides (pdf) from a 2011 Network Startup Resource Center presentation. These two caught my attention:



This bothered me, so I Tweeted about it.

This started some discussion, and prompted me to see what NSRC suggests for architecture these days. You can find the latest, from April 2018, here. Here is the bottom line for their suggested architecture:






What do you think of this architecture?

My Tweet has attracted some attention from the high speed network researcher community, some of whom assume I must be a junior security apprentice who equates "firewall" with "security." Long-time blog readers will laugh at that, like I did. So what was my problem with the original recommendation, and what problems do I have (if any) with the 2018 version?

First, let's be clear that I have always differentiated between visibility and control. A firewall is a poor visibility tool, but it is a control tool. It controls inbound or outbound activity according to its ability to perform in-line traffic inspection. This inline inspection comes at a cost, which is the major concern of those responding to my Tweet.

Notice how the presentation author thinks about firewalls. In the slides above, from the 2018 version, he says "firewalls don't protect users from getting viruses" because "clicked links while browsing" and "email attachments" are "both encrypted and firewalls won't help." Therefore, "since firewalls don't really protect users from viruses, let's focus on protecting critical server assets," because "some campuses can't develop the political backing to remove firewalls for the majority of the campus."

The author is arguing that firewalls are an inbound control mechanism, and they are ill-suited for the most prevalent threat vectors for users, in his opinion: "viruses," delivered via email attachment, or "clicked links."

Mail administrators can protect users from many malicious attachments. Desktop anti-virus can protect users from many malicious downloads delivered via "clicked links." If that is your worldview, of course firewalls are not important.

His argument for firewalls protecting servers is, implicitly, that servers may offer services that should not be exposed to the Internet. Rather than disabling those services, or limiting access via identity or local address restrictions, he says a firewall can provide that inbound control.

These arguments completely miss the point that firewalls are, in my opinion, more effective as an outbound control mechanism. For example, a firewall helps restrict adversary access to his victims when they reach outbound to establish post-exploitation command and control. This relies on the firewall identifying the attempted C2 as being malicious. To the extent intruders encrypt their C2 (and sites fail to inspect it) or use covert mechanisms (e.g., C2 over Twitter), firewalls will be less effective.

The previous argument assumes admins rely on the firewall to identify and block malicious outbound activity. Admins might alternatively identify the activity themselves, and direct the firewall to block outbound activity from designated compromised assets or to designated adversary infrastructure.

As some Twitter responders said, it's possible to do some or all of this without using a stateful firewall. I'm aware of the cool tricks one can play with routing to control traffic. Ken Meyers and I wrote about some of these approaches in 2005 in my book Extrusion Detection. See chapter 5, "Layer 3 Network Access Control."

Implementing these non-firewall-based security choices requries a high degree of diligence, which requires visibility. I did not see this emphasized in the NSRC presentation. For example:


These are fine goals, but I don't equate "manageability" with visibility or security. I don't think "problems and viruses" captures the magnitude of the threat to research networks.

The core of the reaction to my original Tweet is that I don't appreciate the need for speed in research networks. I understand that. However, I can't understand the requirement for "full bandwidth, un-filtered access to the Internet." That is a recipe for disaster.

On the other hand, if you define partner specific networks, and allow essentially site-to-site connectivity with exquisite network security monitoring methods and operations, then I do not have a problem with eliminating firewalls from the architecture. I do have a problem with unrestricted access to adversary infrastructure.

I understand that security doesn't exist to serve itself. Security exists to enable an organizational mission. Security must be a partner in network architecture design. It would be better to emphasize enhance monitoring for the networks discussed above, and think carefully about enabling speed without restrictions. The NSRC resources on the science DMZ merit consideration in this case.

Tuesday, September 11, 2018

Twenty Years of Network Security Monitoring: From the AFCERT to Corelight

I am really fired up to join Corelight. I’ve had to keep my involvement with the team a secret since officially starting on July 20th. Why was I so excited about this company? Let me step backwards to help explain my present situation, and forecast the future.

Twenty years ago this month I joined the Air Force Computer Emergency Response Team (AFCERT) at then-Kelly Air Force Base, located in hot but lovely San Antonio, Texas. I was a brand new captain who thought he knew about computers and hacking based on experiences from my teenage years and more recent information operations and traditional intelligence work within the Air Intelligence Agency. I was desperate to join any part of the then-five-year-old Information Warfare Center (AFIWC) because I sensed it was the most exciting unit on “Security Hill.”

I had misjudged my presumed level of “hacking” knowledge, but I was not mistaken about the exciting life of an AFCERT intrusion detector! I quickly learned the tenets of network security monitoring, enabled by the custom software watching and logging network traffic at every Air Force base. I soon heard there were three organizations that intruders knew to be wary of in the late 1990s: the Fort, i.e. the National Security Agency; the Air Force, thanks to our Automated Security Incident Measurement (ASIM) operation; and the University of California, Berkeley, because of a professor named Vern Paxson and his Bro network security monitoring software.

When I wrote my first book in 2003-2004, The Tao of Network Security Monitoring, I enlisted the help of Christopher Jay Manders to write about Bro 0.8. Bro had the reputation of being very powerful but difficult to stand up. In 2007 I decided to try installing Bro myself, thanks to the introduction of the “brolite” scripts shipped with Bro 1.2.1. That made Bro easier to use, but I didn’t do much analysis with it until I attended the 2009 Bro hands-on workshop. There I met Vern, Robin Sommer, Seth Hall, Christian Kreibich, and other Bro users and developers. I was lost most of the class, saved only by my knowledge of standard Unix command line tools like sed, awk, and grep! I was able to integrate Bro traffic analysis and logs into my TCP/IP Weapons School 2.0 class, and subsequent versions, which I taught mainly to Black Hat students. By the time I wrote my last book, The Practice of Network Security Monitoring, in 2013, I was heavily relying on Bro logs to demonstrate many sorts of network activity, thanks to the high-fidelity nature of Bro data.

In July of this year, Seth Hall emailed to ask if I might be interested in keynoting the upcoming Bro users conference in Washington, D.C., on October 10-12. I was in a bad mood due to being unhappy with the job I had at that time, and I told him I was useless as a keynote speaker. I followed up with another message shortly after, explained my depressed mindset, and asked how he liked working at Corelight. That led to interviews with the Corelight team and a job offer. The opportunity to work with people who really understood the need for network security monitoring, and were writing the world’s most powerful software to generate NSM data, was so appealing! Now that I’m on the team, I can share how I view Corelight’s contribution to the security challenges we face.

For me, Corelight solves the problems I encountered all those years ago when I first looked at Bro. The Corelight embodiment of Bro is ready to go when you deploy it. It’s developed and maintained by the people who write the code. Furthermore, Bro is front and center, not buried behind someone else’s logo. Why buy this amazing capability from another company when you can work with those who actually conceptualize, develop, and publish the code?

It’s also not just Bro, but it’s Bro at ridiculous speeds, ingesting and making sense of complex network traffic. We regularly encounter open source Bro users who spend weeks or months struggling to get their open source deployments to run at the speeds they need, typically in the tens or hundreds of Gbps. Corelight’s offering is optimized at the hardware level to deliver the highest performance, and our team works with customers who want to push Bro to the even greater levels. 

Finally, working at Corelight gives me the chance to take NSM in many exciting new directions. For years we NSM practitioners have worried about challenges to network-centric approaches, such as encryption, cloud environments, and alert fatigue. At Corelight we are working on answers for all of these, beyond the usual approaches — SSL termination, cloud gateways, and SIEM/SOAR solutions. We will have more to say about this in the future, I’m happy to say!

What challenges do you hope Corelight can solve? Leave a comment or let me know via Twitter to @corelight_inc or @taosecurity.

Sunday, July 22, 2018

Defining Counterintelligence

I've written about counterintelligence (CI) before, but I realized today that some of my writing, and the writing of others, may be confused as to exactly what CI means.

The authoritative place to find an American definition for CI is the United States National Counterintelligence and Security Center. I am more familiar with the old name of this organization, the  Office of the National Counterintelligence Executive (ONCIX).

The 2016 National Counterintelligence Strategy cites Executive Order 12333 (as amended) for its definition of CI:

Counterintelligence – Information gathered and activities conducted to identify, deceive,
exploit, disrupt, or protect against espionage, other intelligence activities, sabotage, or assassinations conducted for or on behalf of foreign powers, organizations, or persons, or their agents, or international terrorist organizations or activities. (emphasis added)

The strict interpretation of this definition is countering foreign nation state intelligence activities, such as those conducted by China's Ministry of State Security (MSS), the Foreign Intelligence Service of the Russian Federation (SVR RF), Iran's Ministry of Intelligence, or the military intelligence services of those countries and others.

In other words, counterintelligence is countering foreign intelligence. The focus is on the party doing the bad things, and less on what the bad thing is.

The definition, however, is loose enough to encompass others; "organizations," "persons," and "international terrorist organizations" are in scope, according to the definition. This is just about everyone, although criminals are explicitly not mentioned.

The definition is also slightly unbounded by moving beyond "espionage, or other intelligence activities," to include "sabotage, or assassinations." In those cases, the assumptions is that foreign intelligence agencies and their proxies are the parties likely to be conducting sabotage or assassinations. In the course of their CI work, paying attention to foreign intelligence agents, the CI team may encounter plans for activities beyond collection.

The bottom line for this post is a cautionary message. It's not appropriate to call all intelligence activities "counterintelligence." It's more appropriate to call countering adversary intelligence activities counterintelligence.

You may use similar or the same approaches as counterintelligence agents when performing your cyber threat intelligence function. For example, you may recruit a source inside a carding forum, or you may plant your own source in a carding forum. This is similar to turning a foreign intelligence agent, or inserting your own agent in a foreign intelligence service. However, activities directing against a carding forum are not counterintelligence. Activities directing against a foreign intelligence service are counterintelligence.

The nature and target of your intelligence activities are what determine if it is counterintelligence, not necessarily the methods you use. Again, this is in keeping with the stricter definition, and not becoming a victim of scope creep.


Thursday, June 28, 2018

Why Do SOCs Look Like This?

When you hear the word "SOC," or the phrase "security operations center," what image comes to mind? Do you think of analyst sitting at desks, all facing forward, towards giant screens? Why is this?

The following image is from the outstanding movie Apollo 13, a docudrama about the challenged 1970 mission to the moon.


It's a screen capture from the go for launch sequence. It shows mission control in Houston, Texas. If you'd like to see video of the actual center from 1970, check out This Is Mission Control.

Mission control looks remarkably like a SOC, doesn't it? When builders of computer security operations centers imagined what their "mission control" rooms would look like, perhaps they had Houston in mind?

Or perhaps they thought of the 1983 movie War Games?


Reality was way more boring however:


I visited NORAD under Cheyenne Mountain in 1989, I believe, when visiting the Air Force Academy as a high school senior. I can confirm it did not look like the movie depiction!

Let's return to mission control. Look at the resources available to personnel manning the mission control room. The big screens depict two main forms of data: telemetry and video of the rocket. What about the individual screens, where people sit? They are largely customized. Each station presents data or buttons specific to the role of the person sitting there. Listen to Ed Harris' character calling out the stations: booster, retro, vital, etc. For example:


This is one of the key differences between mission control and any modern computerized operations center. In the 1960s and 1970s, workstations (literally, places where people worked) had to be customized. They lacked the technology to have generic workstations where customization was done via screen, keyboard, and mouse. They also lacked the ability to display video on demand, and relied on large television screens. Personnel with specific functions sat at specific locations, because that was literally the only way they could perform their jobs.

With the advent of modern computing, every workstation is instantly customizable. There is no need to specialize. Anyone can sit anywhere, assuming computers allow one's workspace to follow their logon. In fact, modern computing allows a user to sit in spaces outside of their office. A modern mission control could be distributed.

With that in mind, what does the current version of mission control look like? Here is a picture of the modern Johnson Space Center's mission control room.



It looks similar to the 1960s-1970s version, except it's dominated by screens, keyboards, and mice.

What strikes me about every image of a "SOC" that I've ever seen is that no one is looking at the big screens. They are almost always deployed for an audience. No one in an operational role looks at them.

There are exceptions. Check out the Arizona Department of Transportation operations center.


Their "big screen" is a composite of 24 smaller screens showing traffic and roadways. No one is looking at the screen, but that sort of display is perfect for the human eye.

It's a variant of Edward Tufte's "small multiple" idea. There is no text. The eye can discern if there is a lot of traffic, or little traffic, or an accident pretty easily. It's likely more for the benefit of an audience, but it works decently well.

Compare those screens to what one is likely to encounter in a cyber SOC. In addition to a "pew pew" map and a "spinning globe of doom," it will likely look like this, from R3 Cybersecurity:


The big screens are a waste of time. No one is standing near them. No one sitting at their workstations can read what the screens show. They are purely for an audience, who can't discern what they show either.

The bottom line for this post is that if you're going to build a "SOC," don't build it based on what you've seen in the movies, or in other industries, or what a consultancy recommends. Spend some time determining your SOC's purpose, and let the workflow drive the physical setting. You may determine you don't even need a "SOC," either physically or logically, based on maturing understandings of a SOC's mission. That's a topic for a future post!

Monday, June 25, 2018

Bejtlich on the APT1 Report: No Hack Back

Before reading the rest of this post, I suggest reading Mandiant/FireEye's statement Doing Our Part -- Without Hacking Back.

I would like to add my own color to this situation.

First, at no time when I worked for Mandiant or FireEye, or afterwards, was there ever a notion that we would hack into adversary systems. During my six year tenure, we were publicly and privately a "no hack back" company. I never heard anyone talk about hack back operations. No one ever intimated we had imagery of APT1 actors taken with their own laptop cameras. No one even said that would be a good idea.

Second, I would never have testified or written, repeatedly, about our company's stance on not hacking back if I knew we secretly did otherwise. I have quit jobs because I had fundamental disagreements with company policy or practice. I worked for Mandiant from 2011 through the end of 2013, when FireEye acquired Mandiant, and stayed until last year (2017). I never considered quitting Mandiant or FireEye due to a disconnect between public statements and private conduct.

Third, I was personally involved with briefings to the press, in public and in private, concerning the APT1 report. I provided the voiceover for a 5 minute YouTube video called APT1: Exposing One of China's Cyber Espionage Units. That video was one of the most sensitive, if not the most sensitive, aspects of releasing the report. We showed the world how we could intercept adversary communications and reconstruct it. There was internal debate about whether we should do that. We decided to cover the practice in the report, as Christopher Glyer Tweeted:


In none of these briefings to the press did we show pictures or video from adversary laptops. We did show the video that we published to YouTube.

Fourth, I privately contacted former Mandiant personnel with whom I worked during the time of the APT1 report creation and distribution. Their reaction to Mr Sanger's allegations ranged from "I've never heard of that" to "completely false." I asked former Mandiant colleagues, like myself, in the event that current Mandiant or FireEye employees were told not to talk to outsiders about the case.

What do I think happened here? I agree with the theory that Mr Sanger misinterpreted the reconstructed RDP sessions for some sort of "camera access." I have no idea about the "bros" or "leather jackets" comments!

In the spirit of full disclosure, prior to publication, Mr Sanger tried to reach me to discuss his book via email. I was sick and told him I had to pass. Ellen Nakashima also contacted me; I believe she was doing research for the book. She asked a few questions about the origin of the term APT, which I answered. I do not have the book so I do not know if I am cited, or if my message was included.

The bottom line is that Mandiant and FireEye did not conduct any hack back for the APT1 report.

Update: Some of you wondered about Ellen's role. I confirmed last night that she was working on her own project.