Tuesday, March 20, 2007

Programming and Digital Security

I received the following question recently, so I thought I would anonymize the person asking the question but post my response publicly.

I have a question regarding programming languages and their relation to computer security research. I would appreciate your input on the following. In order for one to be able to "contribute" to security research, do you feel it is necessary for one to become familiar with programming languages?

I am fascinated by computer security and have read several books about stages of attack, malware, and defenses but have not read any books containing any code as I do not understand it. I therefore feel as if I am of no use if I cannot write tools or examine exploits on my own.

I would again really appreciate your input on this, and if you recommend learning programming languages, do you believe one can get away with knowing just one or do you feel an understanding of several is necessary (and if so, which one[s] would you suggest)?

These are great questions. I struggle with them as well because I am not a programmer. When I was a kid in 1980 I programmed my Timex Sinclair using BASIC. I remember looking at books on coding in Assembly for my Commodore 64 several years later so I could try writing cool "demo" programs to impress my BBS buddies. In high school I basically abandoned those sorts of pursuits and only used my C-64 for writing papers, much the same as I did in college from 1990-1994 with my first PC. At USAFA I took the mandatory programming course for freshmen, which basically involved me writing programs in PASCAL and then helping classmates get their versions to compile. (Talk about zero concepts of security!) I did manage to install an Ethernet NIC in my 486SX and play the first game of Doom on FalconNet at USAFA. I didn't own a computer for most of 1996 and early 1997, but by 1998 I was back online and doing my first hands-on security work at the Air Force CERT.

The Air Force formally trained me as an intelligence officer, so I've always been an analyst and operator. Since my first hands-on security work required inspecting network traffic, I've always been deeply involved with TCP/IP. My first jobs also held me responsible for detecting and responding to intrusions. I've enjoyed staying within defensive roles, although I've poked my head out to do "offensive" work occasionally.

I've managed to build a career without being a programmer, but I'm not satisfied with my level of skill or knowledge. If you review my terribly stalled reading list and my Amazon.com Wish List you'll see plenty of books on programming.

I think programming is important for several reasons.

  1. The development community is becoming security-aware. The big pushes I see for improvement lie with groups like US-CERT's Build Security In. Most of the interesting books being published cover software security of one sort or the other. If you want to really participate in this work you need to understand programming.

  2. Programmers can build tools to solve their problems. I have many ideas for cool tools but no ability to execute on them. Because I didn't study programming earlier I am stuck with learning a language to code my tool, then writing the tool itself. My lack of progress over the last several years indicates how tough it is to overcome this hurdle when one's primary work doesn't require programming. I end up using basic scripting capabilities to get other people's tools to solve some of my problems, which is sub-optimal.

  3. Programmers can deeply understand security-related code. Even if I can't code my own tools, it would be extremely helpful to be able to reverse engineer other people's tools or malware, see how they solve certain problems, discover flaws in open source code, and perform other code-centric functions. This is probably the most realistic place for me to apply some beginning programming knowledge. Understanding what someone else wrote can be easier than starting to write a program from scratch.

To directly answer your question, I don't think you need to be a programmer to "contribute," but if you want to be a researcher I think it would be extremely beneficial. I am not a researcher; I am an operator. Programming would still be helpful but it's not critical.

You are definitely not "of no use" because you can't program. I hope my background demonstrates you can be "useful" while not being a programmer.

Regarding languages, here are my two cents.

  • Assembly: If you want to understand shellcode, I recommend learning some Assembly.

  • C: If you want to know write or understand operating systems and many security tools, learn C.

  • Perl: Perl seems to be a prerequisite for many security jobs and a lot of custom code is written in Perl. I would prefer to avoid it but I think some familiarity with Perl is helpful.

  • Python and/or Ruby: These two newer languages are very popular. Jose Nazario, for example, writes many cool Python tools. Metasploit 2.x was written in Perl but 3.x is written in Ruby.

  • Lua: I only learned about Lua recently, but it's apparently got a role in Snort 3.x and a researcher friend showed me how he uses it in his work.

I hope this answered your questions. I hope those of you with backgrounds or skills similar to mine will take heart, if you find yourselves doubting your worth compared to your programmer peers!


Thomas said...

You can't be a web pentester and not be Java-literate. About 3 times a year, on J2EE projects, I lose about 20 minutes re-teaching myself how to write basic Java so I can communicate with application that pass serialized Java objects back and forth; if I did more web pest work, I'd say ability to write Java is also critical.

You need Javascript to do web pentest work.

You can't be a competitive shrink-wrap tester without strong C skills.

You can't be a shrink-wrap tester of any sort without a modern scripting language, of which Python, Ruby, and Perl (in that order) are your choices. Without one of those, you can't use fuzzing tools.

You can't do anything with vendor patches, except for applying them and hoping, without C and X86 assembly (literacy). You can get away with not being able to write assembly, even if you're trying to use shellcode. Assembly falls pretty naturally out of being a good C coder.

You can write security tools in any language, including Bourne Shell, and there's nothing wrong with that. You're not getting near a kernel without C. You're not going inline or passive on a real network without C.

If I was entering field now and I was serious about it, I'd get C and Python --- and by "get", I mean, "build something complex and useful in".

There are obviously plenty of security jobs that don't require any of these abilities, including operations, forensics, and network and systems security architecture. I think you start crossing the line into needing dev skills as soon as you start doing assessment work (I wouldn't hire a net pentester who couldn't break a web app).

Pete said...

I got into security starting with development and have been bouncing around. Right now I'm doing operations. The dev skills help when you need systems to automate interaction with users or debug issues especially when doing interoperability work.

I blogged about a similar issue over here

John Ward said...


I posted my thoughts on this at my site.


Anonymous said...

When it comes to the scripting languages such as Perl, Python and Ruby - I'd suggest picking it one and learning it well. You'll eventually find uses for it almost every day. I think the only reason to learn all 3 is if you have a need to audit code in all 3 languages, or are interested in languages themselves.

My personal preference is Python, and its pretty easy to mix with C if you need to go to that level.

I've also found that knowing C, at least when it comes to reading data structures written in C is very useful, and I've learned a lot about networking through C (but then again my background is embedded kernel development on network devices).

Kenneth F. Belva said...

No one mentioned that "In order for one to be able to "contribute" to security research" via programming, one should also be a great debugger.... :)

eugk007 said...

That's a good point... It's usually more helpful for a security investigator to be good at debugging and reading disassembled code, rather than writing programs from scratch.

Anonymous said...

Me myself is also not a programmer. I find programming too hard for me and i always forgot syntax. I would rather be like Richard being as a investigator or operator. But if i had a chance, i would love to take up perl as i think it is compact and portable. Not to mention java.