What is Zeek?

Zeek provides a comprehensive platform for network traffic analysis, with a particular focus on semantic security monitoring at scale. While often compared to classic intrusion detection/prevention systems, Zeek takes a quite different approach by providing users with a flexible framework that facilitates customized, in-depth monitoring far beyond the capabilities of traditional systems. With initial versions already in operational deployment during the mid ‘90s, Zeek finds itself grounded in more than 20 years of research. For more information, see the Zeek Overview and our promotional document, Why Choose Zeek?.

Who’s using Zeek?

Zeek supports network operations at a broad variety of sites, including major corporations, universities, research labs, and supercomputing centers. It’s also used widely by researchers to prototype novel network analyses and, more generally, for measuring network properties.

Who’s behind Zeek?

Zeek is developed and maintained by a group of researchers and engineers sharing a joint interest in advancing today’s network monitoring capabilities to keep pace with the rapid development of the online world. The Zeek Project is head-quartered at the International Computer Science Institute (ICSI), Berkeley, CA; with a second group of core team members located at the National Center for Supercomputing Applications (NCSA), Urbana-Champaign, IL.

What is the relationship between Zeek and Bro?

In 2018, the long-established Bro system was renamed Zeek. For the rationale behind changing the name and the selection of Zeek as the new name, see the associated blog posting or the unveiling video.

Note that the name "Bro" still appears extensively within the system and its internals. The project is undertaking an ongoing process of shifting over to the use of "Zeek".

What is the International Computer Science Institute?

The International Computer Science Institute (ICSI) is a leading center for research in computer science and one of the few independent, nonprofit such research institutes in the United States. Since its inauguration in 1988, ICSI has maintained an affiliation with the University of California at Berkeley. Several of ICSI’s scientists hold joint faculty appointments at UC Berkeley, teaching graduate and undergraduate courses and supervising students who pursue their doctoral thesis research at ICSI.

For many years now, ICSI has been the home of the Zeek Project, providing it with a public face, handling logistics and administrative, and operating the Zeek infrastructure such as web server, master version control systems, and more.

What is the National Center for Supercomputing Applications?

The National Center for Supercomputing Applications (NCSA), located at the University of Illinois at Urbana-Champaign, provides powerful computers and expert support that help thousands of scientists and engineers across the country improve our world.

For over 10 years, NCSA has used Zeek as a key piece of their security infrastructure, and since 2010 they have been actively involved in the Zeek project. Now a significant part of Zeek’s development is done at the NCSA with support primarily from the National Science Foundation. For more information, see NCSA CyberSecurity .

Who’s funding Zeek?

Historically the Zeek Project has been funded mostly by R&D grants from outside entities. Current and past funding organizations include in particular the National Science Foundation and the Department of Energy. Individual members of the Zeek Project typically spend some of their time on such Zeek-related grants, while also working on other projects funded independently.

Can I support the Zeek Project with a donation?

Yes, we gladly accept unrestricted donations to support Zeek development, maintenance, and operations at ICSI. Send any questions your have to info@zeek.org.

Can I contribute functionality to Zeek?

Absolutely. As any open-source project, Zeek depends crucially on its community for moving forward. We gladly accept patches, small and large, from individuals and organizations. We consider any contributions we receive for inclusion into the mainline distribution. If you’re thinking to work on something specific, we recommend you get in touch with us first to discuss further; we’ll aim to help where we can and smooth the path for later merging into Zeek proper. That way we can also tell you early on if we believe an extension wouldn’t be a good fit for the main Zeek distribution.

On the legal side, please keep in mind that all code contributed to Zeek must be subject to the same BSD license as the system itself; we will implicitly assume so if not stated otherwise. Please note that Zeek cannot even link to libraries with incompatible licenses (such as GPL).

Please continue here for further information.

How do I report a security vulnerability within Zeek?

We are eager to work with the community to resolve security vulnerabilities within Zeek in a timely manner and to properly acknowledge the contributor(s). Review our reporting process here.

Is there a roadmap for Zeek’s development?

We usually have a rough idea of things we’d like to see going into Zeek but we do not have much of a long-time roadmap cast in stone. Specifics depend to a large degree on the interests of the people working on the code base, including members of the core project as well as external contributors. Our releases do not follow a strict timeline either, we rather follow a pragmatic "release-when-ready" strategy. This model has served us well over the years, however it means that if there is something specific you’d like to see done, your best option might be contributing it yourself … That all said, we usually list some features we target for upcoming Zeek releases in our short-term roadmap in the development section.

Does the Zeek Project offer commercial support?

The Zeek Project does not offer individual assistance.

Can I contract the Zeek Project for specific work?

We are not performing contract work but we’re happy to provide pointers to commercial options where we are aware of them. If you are offering commercial Zeek work, let us know and we’ll keep you in mind when somebody asks.

How do I contact the Zeek Project?

Either use the form on the web site or email us.

What’s Zeek’s license?

Zeek is open-source. The system, including all its subcomponents, comes with the very permissive BSD license, which allows for pretty much unrestricted use as long as you leave the attributions in the source code in place. Read the license for the details.

What’s the license for Zeek’s documentation?

All the documentation, and all our content on www.zeek.org that’s not otherwise marked, is licensed under the Creative Commons Attribution 4.0 International License. Occasionally we reuse content from external sources (e.g., trace files) that may be subject to different licenses even if not explicitly pointed out.

Can I use Zeek in my commercial products?

Yes, no problem. Indeed, we encourage you to do so. With its BSD license, there’s no restriction on integrating or bundling Zeek with your product in whatever way you see fit; in particular you do not need to publish your modifications if you’d rather not. That said, we would certainly appreciate if you credited the Zeek Project, or could at least tell us where it’s being used. If you develop custom functionality that might be useful to others, please consider contributing it back to the Zeek Project.

Please note that there are restrictions on how you can refer to your modified Zeek version; see next question.

What are the rules for using the Zeek or Bro name or logo?

In order to protect users’ trust in the system, the Zeek Project reserves the rights to the Zeek name and logo, and similarly for the older Bro name and logo. While generally we are happy to give people permission to use them under many circumstances, we maintain the right to decide on a case-by-case basis. For more information, please see our guidelines for using the Zeek and Bro marks.

Why does v2.4 fail to build on Mac OS X 10.11?

If you have problems building Zeek on OS X 10.11, then you need to install OpenSSL from your preferred OS X package manager (Homebrew, MacPorts, etc.). If Zeek still doesn’t build, then you may need to specify the path to your installation of openssl (e.g. if you’re using Homebrew), try:

./configure --with-openssl=/usr/local/opt/openssl

Network cards and taps, packet capture questions

Here are some pointers to more information:

• A lot of this topic is covered within load balancing.
• The NSMWiki has a page on Collecting Data for different OSes.

Two older papers on packet capture with commodity hardware of the time. The hardware may be outdated. The methods may still be helpful.

• An IMC 2010 paper by Lothar Braun et al. evaluates packet capture performance on commodity hardware
• Fabian Schneider’s research on Packet Capture in 10-Gigabit Ethernet Environments Using Contemporary Commodity Hardware

How can I reduce the amount of CaptureLoss or Dropped_Packets notices?

There are two kinds of loss reported by Zeek:

• Dropped packets reported by the capture library, example:

PacketFilter::Dropped_Packets 2 packets dropped after filtering,
17392297 received, 17392295 on link

• Capture loss reported by the TCP analyzer, example:

CaptureLoss::Too_Much_Loss  The capture loss script detected an
estimated loss rate above 10.476%

Dropped packets will cause both locations to report loss, but capture loss can be caused by an issue upstream from Zeek and can be reported without any dropped packets.

The first step in troubleshooting loss is to determine if you are having an upstream problem or a local problem. Look at the values reported for Dropped_Packets as well as CaptureLoss. Your system will have one of two conditions:

• Capture loss with dropped packets
• Capture loss without dropped packets

How can I configure load balancing?

Although load balancing is a complex subject, we provide a few configuration suggestions here.

What does an error message like internal error: NB-DNS error mean?

That often means that DNS is not set up correctly on the system running Zeek. Try verifying from the command line that DNS lookups work, e.g., host www.google.com.

What is the recommended way to install Zeek on OpenBSD?

The recommended way to install Zeek on OpenBSD 6.1 (or newer) is to simply install the "bro" package:

pkg_add bro

In addition to installing Zeek, this will also automatically install all packages required by Zeek. After installing Zeek, the configuration files will be in the /etc/bro directory.

How can I build Zeek from source on OpenBSD?

In order to build Zeek from source on OpenBSD 6.2 (or newer), first install the required packages:

pkg_add cmake bison swig libbind bash python findutils

If the pkg_add command prompts you to choose a python version, then the recommended version is 2.7.

Geolocation of IP addresses can optionally be added to Zeek by installing these packages:

pkg_add GeoIP geolite-city

Before attempting to build Zeek, make sure to set the following environment variable:

export LDFLAGS="-L/usr/local/lib/libbind -lbind -Wl,-rpath /usr/local/lib/libbind"

Generally, please note that we do not regularly test OpenBSD builds. We appreciate any patches that improve Zeek’s support for this platform.

How do I update my Zeek installation from git master?

Assuming that you’ve already retrieved the Zeek source code with a recursive git clone, then in the repository directory type:

git pull
git submodule update --recursive --init

Then, you’ll need to rebuild Zeek. Before doing "make install" you need to first stop all running Zeek instances (if using BroControl, this is accomplished with a "broctl stop").

How do BroControl options affect Zeek script variables?

Some (but not all) BroControl options override a corresponding Zeek script variable. For example, setting the BroControl option "LogRotationInterval" will override the value of the Zeek script variable "Log::default_rotation_interval". See the BroControl documentation to find out which BroControl options override Zeek script variables, and for more discussion on site-specific customization.

How can I set a custom capture filter?

There are two valid solutions to this. You can either statically override the filter via the broargs option in broctl, or add to the capture_filters and restrict_filters variables in a local script.

If you’re using BroControl, then you can add something like this to your broctl.cfg:

broargs = -f 'net 1.0.0.0/24 or port 443'

Alternatively, you can add something like this to one of your local Zeek scripts:

redef capture_filters += {
    ["watched network"] = "net 1.0.0.0/24",
    ["https"] = "port 443"
};

If you build up what you want to capture this way it gives Zeek the chance to automatically build your BPF filters for you, including checking each component of your filter for mistakes which it will then detect at startup and tell you which component of your filter failed. If you use the above lines to indicate the traffic you’d like to allow into Zeek, you can also set restriction filters to limit something a bit. For instance, in that 1.0.0.0/24 subnet you might want to ignore a single host. You could implement that by adding the following lines:

redef restrict_filters += {
    ["unmonitored host"] = "host 1.0.0.54"
};

The filter that would ultimately be constructed by those lines is:

((port 443) or (net 1.0.0.0/24)) and (not host 1.0.0.54)

One thing to be careful with this though is that generally when you take the stance that you are doing filtering you have to be really careful to understand your traffic. If you have any traffic with MPLS or VLAN tags, the filters given here won’t allow that traffic through. If you’re interested in doing ARP analysis you won’t see those packets either. Same goes for IPv6.

Filtering is an area where we’ve tried to make things simple by running a fully open filter, but there are a lot of dragons when you stray from that path.

How do I customize the output format of ASCII logs?

Formatting the output of ASCII logs is covered in Zeek’s documentation on the ASCII log writer.

Do you have any packet traces I can use with Zeek?

We collect a set of publicly available packet traces on our Traces page.

How can I identify backscatter?

Identifying backscatter via connections labeled as OTH is not a reliable means to detect backscatter. Backscatter is however visible by interpreting the contents of the history field in the conn.log file. The basic idea is to watch for connections that never had an initial SYN but started instead with a SYN-ACK or RST (though this latter generally is just discarded). Here are some history fields which provide backscatter examples: hAFf, r. Refer to the conn protocol analysis scripts to interpret the individual character meanings in the history field.

Is there help for understanding Zeek’s resource consumption?

There are two scripts that collect statistics on resource usage: misc/stats.bro and misc/profiling.bro. The former is quite lightweight, while the latter should only be used for debugging.

How can I capture packets as an unprivileged user?

Normally, unprivileged users cannot capture packets from a network interface, which means they would not be able to use Zeek to read/analyze live traffic. However, there are operating system specific ways to enable packet capture permission for non-root users, which is worth doing in the context of using Zeek to monitor live traffic.

sudo setcap cap_net_raw,cap_net_admin=eip /path/to/bro

sudo chgrp admin /dev/bpf*
sudo chmod g+r /dev/bpf*

sudo sh -c 'echo "own    bpf    root:admin" >> /etc/devfs.conf'
sudo sh -c 'echo "perm   bpf    0640" >> /etc/devfs.conf'
sudo service devfs restart

Why isn’t Zeek producing the logs I expect? (a note about checksums)

Normally, Zeek’s event engine will discard packets which don’t have valid checksums. This can be a problem if one wants to analyze locally generated/captured traffic on a system that offloads checksumming to the network adapter. In that case, all transmitted/captured packets will have bad checksums because they haven’t yet been calculated by the NIC, thus such packets will not undergo analysis defined in Zeek policy scripts as they normally would. Bad checksums in traces may also be a result of some packet alteration tools.

There are three options to workaround such situations and ignore bad checksums:

1. The -C command line option to bro.

2. An option called ignore_checksums that can be redefined at the policy script layer (e.g. in your $PREFIX/share/bro/site/local.bro):

   redef ignore_checksums = T;
   

3. An alternative is to disable checksum offloading for your network adapter, but this is not always possible or desirable. Disable checksum offloading on the NIC using ethtool --offload <int> rx off tx off so the correct checksums are generated to begin with. Replacing <int> with the name of your interface.

What do Zeek’s Notices mean?

The Bro Notice Index lists Zeek Notices and links to further documentation on what the notices mean.

What other software packages use Zeek?

There are a number of software packages that include or work with Zeek. See our list of related software for more information. For support, contact the corresponding developer.

Can I write an analyzer for that?

The short answer is "it depends".

In general, if Zeek sees a packet you can write an analyzer for it. Yes, Zeek is that flexible.

The longer answer: It really depends. On what you want to analyze, on the skill level, your knowldge of the network stack, the protocol, Zeek, C++, Zeek scripting, and more.

• Every protocol above the transport layer is relatively easy to analyze. Zeek offers a tool that supports writing analyzers, called BinPac. There is a BinPac tutorial to help you create an analyzer.
• BinPac cannot be used for protocol analyzers on the layers below application layer. Especially for layer two this task is difficult. A starting point could be our ARP analyzer.