Note: Our current website is at www.zeek.org. This website is unmaintained and contains outdated information.

Bro File Analysis Exercises

The file analysis framework (FAF) is a new feature being introduced with Bro 2.2 that provides a generalized presentation of file-related information. A goal of Bro’s file analysis is to borrow patterns/idioms from network protocol analysis, but do so in a way that’s independent from the actual network connections that transport the files. That is, programming and configuring Bro to analyze files should feel familiar to analyzing network connections in some aspects, but you don’t have to care if a file is sent over HTTP, FTP, SMTP, etc. unless you want to.

Part 1: The Default

For all the exercises, we’ll be using the faf-exercise.pcap trace, so download it first.

Exercise

Run the following command to have Bro perform the default analysis for any files it finds within the trace file:

bro -r faf-exercise.pcap

And examine the contents of the files.log that is generated.

cat files.log

Exercise

Compare and contrast this log file with the conn.log, which contains a summary/overview of analysis of each network connection.

Part 2: Add File Hashing

By default, file hashes aren’t calculated, but turning that on is simple.

Exercise

Run this command:

bro -r faf-exercise.pcap frameworks/files/hash-all-files.bro

Now re-examine to files.log to verify that MD5 and SHA1 hashes are calculated for each file.

Exercise

The "frameworks/files/hash-all-files.bro" referenced in the previous command is telling bro to now load a specific script that’s distributed with Bro, but not loaded by default. And this new script that’s loaded has the code to turn on file hashing for MD5 and SHA1. Now write your own script that tells Bro to also do SHA256 hashing. The reference documentation for the Files::add_analyzer function may be helpful.

Part 3: Extract All The Files

To have Bro extract files from the network stream and save them to the local disk for later use, there’s an "extraction" analyzer specifically designed to do that and just needs to be told which file to extract.

Exercise

Copy this Bro script and save it in a local file, say "extract-all.bro":

global ext_map: table[string] of string = {
    ["application/x-dosexec"] = "exe",
    ["text/plain"] = "txt",
    ["image/jpeg"] = "jpg",
    ["image/png"] = "png",
    ["text/html"] = "html",
} &default ="";

event file_new(f: fa_file)
    {
    local ext = "";

    if ( f?$mime_type )
        ext = ext_map[f$mime_type];

    local fname = fmt("%s-%s.%s", f$source, f$id, ext);
    Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]);
    }

Now run the command:

bro -r faf-exercise.pcap extract-all.bro

Examine the extracted files in the new extract_files/ subdirectory and determine how "Pat" likes his coffee.

Exercise

What email client did Pat appear to use to send his coffee preference?

Part 4: Tone It Down a Bit

Just because Bro can analyze and extract all files it sees on the network doesn’t mean you have to. This is where Bro being a programming language starts to get helpful — it’s easy to change the analysis depending on context available at runtime.

Exercise

Let’s say we don’t care to have any email such as Pat’s coffee preferences archived to disk. Alter the extract-all.bro script used in the previous part to only extract executable files by adding a condition inside the body of the file_new event.

Part 5: Other File Events/Info To Program Around

So far we’ve only seen the use of file_new as the entry point for file analysis programming, but there’s also several others that may be useful: file_over_new_connection, file_timeout, file_gap, and file_state_remove,. For example, in order to determine if Bro saw all the bits in a file, that check can be done in file_state_remove for protocols/connections that advertised the total file size (some may not do that).

Exercise

Write a script to determine the average file size served by host 198.189.255.75 in faf-exercise.pcap.

Part 6: Next Steps

The amount of different file analysis currently offered by the new framework by itself is quite modest, but the scripting language of Bro lends itself well to creating tools on top of it that integrate with external services that may help determine if a file seen on the network is of concern. One such example of this is the integration with Team Cymru’s Malware Hash Registry that’s enabled simply by loading the frameworks/files/detect-MHR.bro script. You shouldn’t find anything that hits in the trace file used for the previous exercises, but maybe you’ve got some other network traffic of your own you’d like check for malware?