Logs from the Cloud

I received an email with the following notice today:

Amazon CloudFront Adds Access Logging Capability:

AWS today released access logs for Amazon CloudFront. Access logs are activity records that show you details about every request delivered through Amazon CloudFront. They contain a comprehensive set of information about requests for your content, including the object requested, the date and time of the request, the edge location serving the request, the client IP address, the referrer and the user agent. It’s easy to get started using access logs: you just specify the name of the Amazon S3 bucket you want to use to store the logs when you configure your Amazon CloudFront distribution. There are no fees for using the access logs, beyond normal Amazon S3 charges to write, store and retrieve the logs.



The Amazon Elastic MapReduce team has also built a sample application, CloudFront LogAnalyzer, that will analyze your Amazon CloudFront access logs. This tool lets you use the power of Amazon Elastic MapReduce to quickly turn Access Logs into the answers to the most commonly asked questions about your business. Additionally, several partners have also built solutions that help you analyze these access logs; you can find more information about these in the AWS Solutions Catalog.

Looking at the Developer Guide entry for Access Logs, we see the following sorts of data will be recorded:


The log files use the W3C extended log file format 
 (for more information, go to http://www.w3.org/TR/WD-logfile.html).

The files contain information for each record in the following order:

Date of the request (in UTC)
Time (when the server finished processing the request; in UTC)
Edge location that served the request 
 (a variable-length string with a minimum of 3 characters)
Bytes served
Client IP address (no hostname lookups occur)
HTTP access method
DNS name (either the CloudFront distribution name or your CNAME, 
 whichever the end user specified in the request)
URI stem (e.g., /images/daily-ad.jpg)
HTTP status code (e.g., 200)
Referrer
User agent

An entry might look like this:

#Version: 1.0
#Fields: date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status 
 cs(Referer) cs(User-Agent)

02/01/2009 01:13:11 FRA2 182 10.10.10.10 GET d2819bc28.cloudfront.net /view/my/file.html 200 
 www.displaymyfiles.com Mozilla/4.0%20(compatible;%20MSIE%205.0b1;%20Mac_PowerPC)

02/01/2009 01:13:12 LAX1 2390282 12.12.12.12 GET www.singalong.com /soundtrack/happy.mp3 304 
 www.unknownsingers.com Mozilla/4.0%20(compatible;%20MSIE%207.0;%20Windows%20NT%205.1)

I think this is a good start, but I'll leave it to Cloudsecurity.org for expert commentary!

Richard Bejtlich is teaching new classes in Las Vegas in 2009. Regular Las Vegas registration ends 1 July.

Comments

Craig Balding said…
Hi Richard

Thanks for the mention.

Honestly, I think you have the meat of it covered :-).

The key point for peeps to bear in mind is that this is just for Amazons Content Distribution Network service (aka CloudFront) - not AWS API usage. So this is just like Akamai giving you logs for content you've pushed to them for edge access. I guess its worth noting that the existing Amazon S3 storage service already has optional logging, disabled by default).

CloudFront logging improves on the previous situation where only usage statistics were available, hence this feature mostly helps Amazon clients that rebill CDN usage to their own customers.

From a security perspective I'm struggling to see much benefit (the corner case I can think of is if sensitive data was accidently leaked by a customer to CloudFront and they needed to know which IP's downloaded it).

If someone can see more security benefits, do share.

Cheers

Craig
1:47 AM
andy said…
I always enjoy learning how other people employ Amazon S3 and CloudFront. I am wondering if you can check out my very own tool CloudBerry Explorer that helps to manage S3 on Windows . It is a freeware. We have recently added basic support for CloudFront log to make it easier to turn on logging and analyze it.
2:44 AM
Post a Comment

Popular posts from this blog

Zeek in Action Videos

Image
This is a quick note to point blog readers to my Zeek in Action YouTube video series for the Zeek network security monitoring project .  Each video addresses a topic that I think might be of interest to people trying to understand their network using Zeek and adjacent tools and approaches, like Suricata, Wireshark, and so on.  I am especially pleased with Video 6 on monitoring wireless networks . It took me several weeks to research material for this video. I had to buy new hardware and experiment with a Linux distro that I had not used before -- Parrot .  Please like and subscribe, and let me know if there is a topic you think might make a good video.
Read more

MITRE ATT&CK Tactics Are Not Tactics

Image
Just what are "tactics"? Introduction MITRE ATT&CK  is a great resource, but something about it has bothered me since I first heard about it several years ago. It's a minor point, but I wanted to document it in case it confuses anyone else. The MITRE ATT&CK Design and Philosophy document from March 2020 says the following: At a high-level, ATT&CK is a behavioral model that consists of the following core components: • Tactics, denoting short-term, tactical adversary goals during an attack; • Techniques, describing the means by which adversaries achieve tactical goals; • Sub-techniques, describing more specific means by which adversaries achieve tactical goals at a lower level than techniques; and • Documented adversary usage of techniques, their procedures, and other metadata. My concern is with MITRE's definition of "tactics" as "short-term, tactical adversary goals during an attack," which is oddly recursive. The key word in the tacti
Read more

New Book! The Best of TaoSecurity Blog, Volume 4

Image
  I've completed the TaoSecurity Blog book series . The new book is  The Best of TaoSecurity Blog, Volume 4: Beyond the Blog with Articles, Testimony, and Scholarship .  It's available now for Kindle , and I'm working on the print edition.  I'm running a 50% off promo on Volumes 1-3 on Kindle through midnight 20 April. Take advantage before the prices go back up. I described the new title thus: Go beyond TaoSecurity Blog with this new volume from author Richard Bejtlich. In the first three volumes of the series, Mr. Bejtlich selected and republished the very best entries from 18 years of writing and over 18 million blog views, along with commentaries and additional material.  In this title, Mr. Bejtlich collects material that has not been published elsewhere, including articles that are no longer available or are stored in assorted digital or physical archives. Volume 4 offers early white papers that Mr. Bejtlich wrote as a network defender, either for technical or pol
Read more