For Zeek 3.1 and later, the guide for using AddressSanitizer/LeakSanitizer is here:

https://docs.zeek.org/en/latest/devel/contributors/memory-checks.html

For Zeek 3.0 and earlier, read on for how to use gperftools to find leaks.

Internally, Zeek’s memory management is pretty complex and when writing new code it’s unfortunately pretty easy to introduce memory leaks. If Zeek’s memory footprint keeps growing over time (hours or days), and there’s no script-level state table to blame for what you see, it might be a leak in Zeek’s C++ event-engine.

Zeek has been in use for many years and we are pretty confident that there aren’t any major leaks left in the more common code paths these days. However, there’s also code which isn’t used very regularly and it still happens that people run into leaks in these parts. But most importantly, for new code it is crucial to make sure that it does not introduce new leaks. The lesson we learned in the past is that even the tiniest leak can have devastating effects when it occurs for every connection Zeek analyses.

Whenever you are writing code for Zeek’s event engine, such as a new protocol analyzer, it’s a good idea to double-check that all the memory it allocates will for sure be released later. As a rule of thumb: whenever you allocate memory but cannot guarantee that it will be freed at a well-defined point of time other than termination, there’s likely something wrong.

There are various tools out there which can help with leak-checking. In the past we had most success with valgrind (good but slow with large input traces) and gperftools.

Zeek has now support built in for perftools, which we’ll summarize in the following. Note that perftools’ leak-checking works on Linux systems only at the moment. In the past, we have found quite a few leaks in Zeek with perftools and we recommend to run new code through it to see whether there’s something turning up. The nice thing about perftools is that its performance is sufficiently good that one can actually run Zeek with some non-trivial amount of traffic, which is crucial because often leaks are in code paths not triggered when analyzing just a few connections.

To use perftools for checking Zeek for leaks, get the current source code from gperftools. Compile and install it with:

> ./autogen.sh
> ./configure --enable-heap-checker
> make install

Next, you need to configure Zeek with debugging code and enable perftools support:

> ./configure --enable-debug --enable-perftools --enable-perftools-debug
> make

Make sure that the output of configure indicates that it has indeed found the perftools installation. If it didn’t find perftools, try giving it the paths to your perftools installation directly, as in the following example (replace <prefix> with whatever prefix you were using when configuring the perftools distribution):

> export LDFLAGS=-L<prefix>/lib
> export CFLAGS=-I<prefix>/include
> export CPPFLAGS=-I<prefix>/include
> export LD_LIBRARY_PATH=<prefix>/lib
> ./configure --enable-debug --enable-perftools

Once the configure output looks ok, compile Zeek as usual with make. The last preparation step is setting perftools’ environment variable HEAPCHECK to local to activate perftools’ memory checking:

> export HEAPCHECK=local

Finally, start Zeek as usual but add the option -m to the command-line, e.g,:

> bro -m -r trace tcp

You likely want to run from a trace instead of live because the memory checking decreases Zeek’s performance significantly. Once Zeek terminates (which is fine to trigger via CTRL-C if necessary), perftools will output leak information into /tmp/bro.*.heap files. (There might be output saying that something "has failed" but it seems that you can safely ignore this.)

The recorded data will only contain leaks which appeared within Zeek’s main packet processing loop; any leaks during initialization and termination are skipped as they won’t cause any trouble during live operation. At termination, Zeek will print out a pre-built command-line to start the pprof utility. pprof is perftools’ user interface to inspect the leak information, and you can just literally copy&paste the command line into your shell (except that you might want to skip the -gv option to suppress the graphical output in favor of the interactive interface).

Inside pprof, typing help shows the available commands. Most importantly, there’s top which shows the top leaks as determined by perftools. See here for more information about how to interpret its output, and here for more details about perftools’ heap checking in general.