For more than a decade now, Zeek has successfully bridged the traditional gap between academic research and large-scale production deployment. Much of the functionality now part of the core system originates in experimental research projects, often published at top-tier academic conferences. In the same spirit, we summarize a number of related ongoing research efforts below that we currently pursue at the International Computer Science Institute. Results from these projects may eventually be integrated into the production system. We also collect Zeek-related publications on this page. Over the years, Zeek has not only been used as a platform for novel detection approaches, but also facilitated numerous more general traffic analysis studies.
Research Projects
- Understanding and Exploiting Parallelism in Deep Packet Inspection:
Deep packet inspection (DPI) is a crucial tool for protecting networks from emerging and sophisticated attacks. However, it is becoming increasingly difficult to implement DPI effectively due to the rising need for more complex analysis, combined with the relentless growth in the volume of network traffic that these systems must inspect. To address this challenge, future DPI technologies must exploit the power of emerging highly concurrent multi- and many-core platforms. Unfortunately, however, current DPI systems severely limit their use of parallelism by either resorting to coarse-grained load-balancing or restricting their analysis to very simple, hard-coded detectors. In order to fully exploit parallel hardware platforms, in this project we develop a comprehensive approach that introduces parallelism across all stages of the complex DPI pipeline. We investigate application-independent scheduling strategies that take existing DPI analyses and automatically parallelize their processing. We do so by mapping them into a domain-specific intermediary representation that abstracts from specifics of the underlying hardware architecture while providing low-level consistency guarantees. Conceptually, the project’s goal is to virtualize and abstract parallelism as a fundamental primitive, just like how virtual memory abstracts away physical memory size limitations from programmers.
This project is currently funded by the National Science Foundation as a part of Award CNS-1228792. The project is a collobaration with the University of Wisconsin - Madison.
- HILTI: A High-level Intermediary Language for Traffic Analysis
Network intrusion detection systems need to balance between a set of challenges difficult to simultaneously address to their full extent: the complexity of network communication; the need to operate extremely efficiently to achieve line-rate performance; and dealing securely with untrusted input. Our project aims to build an efficient and secure bridge between dealing effectively with these challenges, and offering the high-level abstractions required for describing a security policy. Observing that NIDS implementations share a large degree of functionality, we introduce a new middle-layer into NIDS processing, consisting of two main pieces: first, an abstract machine model that is specifically tailored to the network traffic analysis domain and directly supports the field’s common abstractions and idioms in its instruction set; and second, a compilation strategy for turning programs written for the abstract machine into highly optimized, natively executable code for a given target platform, with performance comparable to manually written C code. As a broader goal, our undertaking provides the security community with a novel architecture that facilitates development and reuse of building blocks commonly required for network traffic analysis.
See the project’s home page for more information. This project was originally funded by the National Science Foundation under Award CNS-0915667, and by a grant from the Cisco Research Center .
- BinPAC++: A Next-Generation Parser Generator for Network Protocols
In earlier work, we developed BinPAC, a "A yacc for generating application protocol parsers". BinPAC is now part of the Zeek distribution. In a current effort, we are developing a significantly extended version, nick-named BinPAC++, that integrates semantic constructs into its protocol grammar language, rather than just syntax. Doing so will allow us to move much of the high-level state-tracking that application-layer analysis requires into BinPAC++ protocol specifications, making them it suitable for reuse across different host applications. Furthermore, the new BinPAC++ compiler now compiles into the instruction of the HILTI abstract machine (see above), and no longer into C++.
See the project’s home page for more information. This project is currently funded by the National Science Foundation as a part of Award CNS-0831535.
- Semantic Security Monitoring for Industrial Control Systems
Industrial control systems differ significantly from standard, general-purpose computing environments, and they face quite different security challenges. With physical "air gaps" now the exception, our critical infrastructure has become vulnerable to a broad range of potential attackers. In this project we develop novel network monitoring approaches that can detect sophisticated semantic attacks: malicious actions that drive a process into an unsafe state without however exhibiting any obvious protocol-level red flags. In one thrust, we conduct a measurement-centric study of ICS network activity, aimed at developing a deep understanding of operational semantics in terms of actors, workloads, dependencies, and state changes over time. In a second thrust, we develop domain-specific behavior models that abstract from low-level protocol activity to their semantic meaning according to the current state of the processes under control. Our goal is to integrate these models into operationally viable, real-time network monitoring that reports unexpected deviations as indicators of attacks or malfunction. A separate "Transition to Practice" phase advances our research results into deployment-ready technology by integrating it into the open-source Zeek network monitor. Overall, our work will improve security and safety of today’s critical infrastructure by providing effective, unobtrusive security monitoring tailored to their specific semantics. In addition, we tie a number of educational activities to the research and involve students at all levels.
This project is currently funded by the National Science Foundation as a part of Awards CNS-1314973 and CNS-1314891.
Past Projects
- Exploiting Multi-Core Processors for High-Performance Traffic Monitoring
The performance pressures on implementing effective network security monitoring are growing fiercely in multiple dimensions, outpacing improvements in CPU performance. The situation has now become dire with the end of Moore’s Law for single CPUs. In general, hardware vendors now turn to parallel execution—-many cores and many threads—-to sustain performance growth. But adapting network security monitoring to such parallelism raises a host of challenging issues. This project seeks to develop methodologies for effectively parallelizing in-depth security analysis of network activity. Doing so requires structuring the processing into separate, low-level threads suitable for concurrent execution, for which several key issues must be addressed: forwarding packets only when all relevant threads have finished their vetting; minimizing inter-thread communication in the presence of global analysis algorithms; optimizing memory access patterns for locality; and providing effective performance debugging tools. As a proof-of-concept, we have implemented a multi-threaded Zeek version that demonstrates the key concepts developed.
This project was funded by the National Science Foundation as part of Award CNS-0716636. We are now integrating its results into the HILTI abstract machine (see above), which will provide a concurrency model that directly supports the developed approach.
- Establishing a Cross-Institutional Platform for Cooperative Security Monitoring and Forensics
Although there has been much research in developing systems for globally sharing security information, often these approaches are fundamentally limited because their broad scope limits the trust that participants can place in the system. This project instead seeks to reap significantly greater utility by considering a more restricted scope: a system for coordinated security analysis based on exchanging information between a set of sites who have explicitly decided to work with each other. This more limited scope optimizes for the common case that in such an environment the participating sites will usually (but not always) act in a responsible manner. A key focus of the project concerns automating the steps commonly involved in security monitoring and forensic analysis while still keeping an analyst "in the loop" for significant decisions.
This project was funded by the National Science Foundation under Award CNS-0716640.
Feel free to contact us for more information about any of these projects.
Publications
We are maintaining a bibliography of Zeek-related academic publications. (Many of these refer to it using the older "Bro" name.) If you would like to see your work listed here, please send us the bibliographic information (BibTeX prefered), including a link to a PDF version if available online. We are interested in any work that extends or evaluates Zeek, as well as research that uses Zeek as a tool.
If you are writing a paper that refers to Zeek, please cite the original Bro publication as follows:
Bibliography
Page Contents
Quick Links
- 13 February 2020: Ask the Zeeksperts - Aashish Sharma
- 18 February 2020 - Portland OR: Zeek Days Workshops
- 27 February 2020: Ask the Zeeksperts
- 7-9 October 2020 - Austin, Texas: ZeekWeek 2020
