Using md5deep

Thank you to Harlan Carvey for reminding me of Jesse Kornblum's md5deep. md5deep is a suite of tools to recursively compute a variety of hashes. It is not limited to the MD5 algorithm. In the example below I run the sha256deep tool to provide sha256 hashes of various files. The -r flag initializes recursive behavior, and -z says display file size before the hash.

bourque:/home/analyst$ sha256deep -r -e -z *

     93506  1a6da6a2a849eb27fb7522939afab63ec59bcdb9412c2460fe611543b573d95f  
/home/analyst/2005-041-santini_air/sample

       111  43450978e07f87dfbc4918fec928209c54f4d5804367960fbde617e71ee50985
/home/analyst/2005-041-santini_air/sample.sha256

209.180.018.089.02001-156.023...: 391MB of 1405MB done, 00:01:22 left

The last entry shows sha256deep is busy computing the hash for a 1405 MB file. By passing the -e flag, I told the program to estimate time until hash completion. This is useful for processing large files. The resulting hash is eventually shown below.

1473577526  3f4eb24ae943dba4bdb1126540d309854824ac64ff6f288020c9c2bdc4793de9  
/home/analyst/2005-041-santini_air/209.180.018.089.02001-156.023.170.238.02001

md5deep and related tools simplify maintaining forensic evidence as the program can rapidly produce hashes in an investigator-friendly format. There's also a FreeBSD port. For forensic applications, you would save the hashes to a file instead of standard output.

Comments

Anonymous said…
If interested, wininterrogate http://winfingerprint.sourceforge.net can do something similar to md5deep. It only supports MD5 and SHA-1 currently but can also provide some extra information that is useful. Win32 platform only.
1:30 PM
Anonymous said…
I maintain a list of MD5 hashes for malicious or suspect files that I come across in my work. To use such a list as a comparison source be sure to have exactly two spaces between the hash and the filename. Otherwise md5deep won't work properly.
3:39 PM
Post a Comment

Popular posts from this blog

Zeek in Action Videos

Image
This is a quick note to point blog readers to my Zeek in Action YouTube video series for the Zeek network security monitoring project .  Each video addresses a topic that I think might be of interest to people trying to understand their network using Zeek and adjacent tools and approaches, like Suricata, Wireshark, and so on.  I am especially pleased with Video 6 on monitoring wireless networks . It took me several weeks to research material for this video. I had to buy new hardware and experiment with a Linux distro that I had not used before -- Parrot .  Please like and subscribe, and let me know if there is a topic you think might make a good video.
Read more

MITRE ATT&CK Tactics Are Not Tactics

Image
Just what are "tactics"? Introduction MITRE ATT&CK  is a great resource, but something about it has bothered me since I first heard about it several years ago. It's a minor point, but I wanted to document it in case it confuses anyone else. The MITRE ATT&CK Design and Philosophy document from March 2020 says the following: At a high-level, ATT&CK is a behavioral model that consists of the following core components: • Tactics, denoting short-term, tactical adversary goals during an attack; • Techniques, describing the means by which adversaries achieve tactical goals; • Sub-techniques, describing more specific means by which adversaries achieve tactical goals at a lower level than techniques; and • Documented adversary usage of techniques, their procedures, and other metadata. My concern is with MITRE's definition of "tactics" as "short-term, tactical adversary goals during an attack," which is oddly recursive. The key word in the tacti
Read more

New Book! The Best of TaoSecurity Blog, Volume 4

Image
  I've completed the TaoSecurity Blog book series . The new book is  The Best of TaoSecurity Blog, Volume 4: Beyond the Blog with Articles, Testimony, and Scholarship .  It's available now for Kindle , and I'm working on the print edition.  I'm running a 50% off promo on Volumes 1-3 on Kindle through midnight 20 April. Take advantage before the prices go back up. I described the new title thus: Go beyond TaoSecurity Blog with this new volume from author Richard Bejtlich. In the first three volumes of the series, Mr. Bejtlich selected and republished the very best entries from 18 years of writing and over 18 million blog views, along with commentaries and additional material.  In this title, Mr. Bejtlich collects material that has not been published elsewhere, including articles that are no longer available or are stored in assorted digital or physical archives. Volume 4 offers early white papers that Mr. Bejtlich wrote as a network defender, either for technical or pol
Read more