Detailed Version History
Contents
Bro
2.5.5 | 2018-08-28 10:55:18 -0500
* Release 2.5.5 (Jon Siwek, Corelight)
* Fix signed/unsigned comparison warning (Jon Siwek, Corelight)
* Add 'smtp_excessive_pending_cmds' weird (Jon Siwek, Corelight)
* Fix SMTP command string comparisons (Jon Siwek, Corelight)
* Improve handling of empty lines in several text protocol analyzers
(Jon Siwek, Corelight)
* Add rate-limiting sampling mechanism for weird events
(Jon Siwek, Corelight)
The generation of weird events, by default, are now rate-limited
according to these tunable options:
- Weird::sampling_whitelist
- Weird::sampling_threshold
- Weird::sampling_rate
- Weird::sampling_duration
The new get_reporter_stats() BIF also allows one to query the
total number of weirds generated (pre-sampling) which the new
policy/misc/weird-stats.bro script uses periodically to populate
a weird_stats.log.
There's also new reporter BIFs to allow generating weirds from the
script-layer such that they go through the same, internal
rate-limiting/sampling mechanisms:
- Reporter::conn_weird
- Reporter::flow_weird
- Reporter::net_weird
Some of the code was adapted from previous work by Johanna Amann.
2.5.4 | 2018-05-29 19:54:21 -0500
* Release 2.5.4.
* Add NCP::max_frame_size tuning option (Corelight)
* Migrate NCP analyzer to use latest analyzer API (Corelight)
* Fixes for MySQL and SMB protocol parsers (Corelight)
* MySQL: the parser for this was generally broken (not following
the specification well) and needed many changes. One addition is a
new "mysql_result_row" event that provides access to the results of
queries.
* SMB: the spec seems to explitly call out the omission of the
PrimaryDomain field on SMB_COM_SESSION_SETUP_ANDX responses (and I
don't see that field in pcaps either), so this may have just been a
typo that used to work fine in the past only due to faulty array
parsing behavior in binpac.
* Fix a memory leak in SMBv1 share mapping (Corelight)
2.5.3 | 2018-02-14 07:26:57 -0800
* Release 2.5.3. (Johanna Amann)
* Patch in Binpac submodule that fixes an integer overflow
(Philippe Antoine/Catena cyber).
2.5.2 | 2017-10-16 13:37:47 -0700
* Release 2.5.2
* Patch OOB write in content-line analyzer.
A combination of packets can trigger an out of bound write of '0' byte
in the content-line analyzer. Addresses BIT-1856/CVE-2017-1000458.
(Frank Meier/Johanna Amann)
2.5.1 | 2017-06-26 15:55:45 -0700
* Release 2.5.1.
2.5.1-beta | 2017-06-07 12:40:10 -0700
* Release 2.5.1-beta.
* Remove non-existing links; this broke documentation build. (Johanna Amann)
* Fix at_least in version.bro. (Johanna Amann)
2.5-140 | 2017-05-12 15:31:32 -0400
* Lessen cluster load due to notice suppression.
(Johanna Amann, Justin Azoff)
2.5-137 | 2017-05-04 11:37:48 -0500
* Add plugin hooks for log init and writing: HookLogInit and HookLogWrite.
(Corelight)
* TLS: Fix compile warning (comparison between signed/unsigned).
This was introduced with the addition of new TLS1.3 extensions. (Johanna Amann)
2.5-134 | 2017-05-01 10:34:34 -0500
* Add rename, unlink, and rmdir bifs. (Corelight)
2.5-131 | 2017-04-21 14:27:16 -0700
* Guard more format strings with __attribute__((format)). (Johanna Amann)
* Add support for two TLS 1.3 extensions.
New events:
- event ssl_extension_supported_versions(c: connection, is_orig: bool, versions: index_vec)
- event ssl_extension_psk_key_exchange_modes(c: connection, is_orig: bool, modes: index_vec) (Johanna Amann)
2.5-125 | 2017-04-17 22:02:39 +0200
* Documentation updates for loading Bro scripts. (Seth Hall)
2.5-123 | 2017-04-10 13:30:14 -0700
* Fix some failing tests by increasing delay times. (Daniel Thayer)
* Threading Types: add a bit of documentation to subnet type. (Johanna Amann)
* Fixing couple issues reported by Coverity. (Robin Sommer)
2.5-119 | 2017-04-07 10:30:09 -0700
* Fix the test group name in some broker test files. (Daniel Thayer)
* NetControl: small rule_error changes (test, call fix). (Johanna Amann)
* SSL: update dpd signature for TLS1.3. (Johanna Amann)
2.5-115 | 2017-03-23 07:25:41 -0700
* Fix a test that was failing on some platforms. (Daniel Thayer)
* Remove test for cluster catch and release. This test keeps failing
intermittently because of timing issues that are surprisingly hard
to fix. (Johanna Amann)
* Fix some Coverity warnings. (Daniel Thayer)
2.5-106 | 2017-03-13 11:19:03 -0700
* print version string to stdout on --version, instead
of printing it to stderr. Output is not an error output. (Pete)
* Fix compiler warning raised by llvm8. (Johanna Amann)
* Fix coverity warning in Ascii reader. (Johanna Amann)
2.5-101 | 2017-03-09 12:20:11 -0500
* Input's framework's ascii reader is now more resilient.
By default, the ASCII reader does not fail on errors anymore.
If there is a problem parsing a line, a reporter warning is
written and parsing continues. If the file is missing or can't
be read, the input thread just tries again on the next heartbeat.
(Seth Hall, Johanna Amann)
2.5-92 | 2017-03-03 10:44:14 -0800
* Move most threading to C++11 primitives (mostly). (Johanna Amann)
* Fix a test that sometimes fails on FreeBSD. (Daniel Thayer)
* Remove build time warnings. (Seth Hall)
2.5-84 | 2017-02-27 15:08:55 -0500
* Change semantics of Broker's remote logging to match old communication
framework. (Robin Sommer)
* Add and fix documentation for HookSetupAnalyzerTree (Johanna Amann)
2.5-76 | 2017-02-23 10:19:57 -0800
* Kerberos ciphertext had some additional ASN.1 content being lumped
in. (Vlad Grigorescu)
* Updated Windows version detection to include Windows 10. (Fatema
Bannatwala, Keith Lehigh, Mike, Seth Hall).
2.5-70 | 2017-02-20 00:20:02 -0500
* Rework the RADIUS base script.
Fixes BIT-1769 which improves logging behavior when replies aren't
seen. Also added a `framed_addr` field to indicate if the radius
server is hinting at an address for the client and a `ttl` field to
show how quickly the server is responding. (Seth Hall)
2.5-68 | 2017-02-18 13:59:05 -0500
* Refactored base krb scripts. (Seth Hall)
* New script to log ticket hashes in krb log
(policy/protocols/krb/ticket-logging.bro). Also, add
ciphertext to ticket data structure. (John E. Rollinson)
2.5-62 | 2017-02-15 15:56:38 -0800
* Fix case in which scripts were able to access unitialized variables
in certain cases. Addresses BIT-1785. (Jon Siwek)
2.5-60 | 2017-02-15 15:19:20 -0800
* Implement ERSPAN support.
There is a small caveat to this implementation. The ethernet
header that is carried over the tunnel is ignored. If a user
tries to do MAC address logging, it will only show the MAC
addresses for the outer tunnel and the inner MAC addresses
will be stripped and not available anywhere. (Seth Hall)
* Tiny mime-type fix from Dan Caselden. (Seth Hall)
* Update failing intel framework test. (Johanna Amann)
2.5-55 | 2017-02-10 09:50:43 -0500
* Fixed intel expiration reset. Reinserting the same indicator did not reset
the expiration timer for the indicator in the underlying data store.
Addresses BIT-1790. (Jan Grashoefer)
2.5-51 | 2017-02-06 10:15:56 -0500
* Fix memory leak in file analyzer. (Johanna Amann)
* Fix a series of problems with the to_json function.
Addresses BIT-1788. (Daniel Thayer)
2.5-44 | 2017-02-03 16:38:10 -0800
* Change snap lengths of some tests. (Johanna Amann)
* Fix layer 2 connection flipping. If connection flipping occured in
Sessions.cc code (invoked e.g. when the original SYN is missing),
layer 2 flipping was not performed. (Johanna Amann)
2.5-39 | 2017-02-01 14:03:08 -0800
* Fix file analyzer memory management, and a delay in disabling file analyzers.
File analyzers are no longer deleted immediately; this is delayed until
a file opject is destroyed. Furthermore, no data is sent to file analyzers
anymore after they have been disabled.
2.5-33 | 2017-02-01 10:07:47 -0500
* New file types sigs. (Keith Lehigh)
* Change snaplen of test trace from 1,000,000 to 10,000
Recent versions of libpcap are unhappy with values bigger than 262,144
and will refuse reading the file. (Johanna Amann)
2.5-30 | 2017-01-26 13:24:36 -0800
* Extend file extraction log, adding extracted_cutoff and extracted_size
fields. (Seth Hall)
* Add new TLS extension type (cached_info) (Johanna Amann)
* Remove brocon event; it caused test failures. (Johanna Amann)
* Add missing paths to SMB Log::create_streams calls. (Johanna Amann)
* Tiny xlsx file signature fix. (Dan Caselden)
* Allow access to global variables using GLOBAL:: namespace.
Addresses BIT-1758. (Francois Pennaneac)
2.5-17 | 2016-12-07 14:51:37 -0800
* Broxygen no longer attempts to do tilde expansion of PATH, giving
an error message instead if bro is located in a PATH component
that starts with a tilde. Broxygen also no longer attempts to get
the mtime of the bro executable when bro is not invoced with the
"-X" option. (Daniel Thayer)
* Fix failing tests, compiler warnings and build issues on OpenBSD.
(Daniel Thayer)
2.5-9 | 2016-12-05 11:39:54 -0800
* Fix validation of OCSP replies inside of Bro. (Johanna Amann)
At one place in the code, we did not check the correct return
code. This makes it possible for a reply to get a response of
"good", when the OCSP reply is not actually signed by the
responder in question.
This also instructs OCSP verication to skip certificate chain
validation, which we do ourselves earlier because the OCSP verify
function cannot do it correctly (no way to pass timestamp).
2.5-6 | 2016-11-29 12:51:04 -0800
* Fix a build failure on OpenBSD relating to pcap_pkthdr. Also fixes
an include issue on OpenBSD. (Daniel Thayer)
* Fix compile error in krb-types.pac. (Johanna Amann)
* Update krb-types.pac: KerberosString formatting for the principal
principal name is now compliant with RFC 4120 section 5.2.2. (jamesecorrenti)
2.5 | 2016-11-16 14:51:59 -0800
* Release 2.5.
2.5-beta2-17 | 2016-11-14 17:59:19 -0800
* Add missing '@load ./pubkey-hashes' to
policy/frameworks/intel/seen. (Robin Sommer)
2.5-beta2-15 | 2016-11-14 17:52:55 -0800
* Remove unused "bindist" make target. (Daniel Thayer)
* Improve the "How to Upgrade" page in the Bro docs. (Daniel Thayer)
* Update the quickstart guide for the deploy command. (Daniel Thayer)
* Improved installation instructions for Mac OS X. (Daniel Thayer)
* Lots of more small updates to documentation. (Daniel Thayer)
2.5-beta2 | 2016-11-02 12:13:11 -0700
* Release 2.5-beta2.
2.5-beta-135 | 2016-11-02 09:47:20 -0700
* SMB fixes and cleanup. Includes better SMB error handling, improved DCE_RPC
handling in edge cases where drive_mapping is not seen. The concept of unknown
shares has been removed with this change. Also fixes SMB tree connect handling and
removes files that are not parsed. SMB2 error parsing is disabled because it never
was implemented correctly. (Seth Hall)
* Including a test for raw NTLM in SMB (Seth Hall)
* Updates for SMB auth handling from Martin van Hensbergen.
- Raw NTLM (not in GSSAPI) over SMB is now handled correctly.
- The encrypted NTLM session key is now passed into scriptland
through the ntlm_authenticate event. (Seth Hall)
* Add a files framework signature for VIM tmp files. (Seth Hall)
* Version parsing scripts now supports several beta versions. (Johanna Amann)
2.5-beta-123 | 2016-11-01 09:40:49 -0700
* Add a new site policy script local-logger.bro. (Daniel Thayer)
2.5-beta-121 | 2016-10-31 14:24:33 -0700
* Python 3 compatibility fixes for documentation building. (Daniel Thayer)
2.5-beta-114 | 2016-10-27 09:00:24 -0700
* Fix for Sphinx >= 1.4 compability. (Robin Sommer)
2.5-beta-113 | 2016-10-27 07:44:25 -0700
* XMPP: Fix detection of StartTLS when using namespaces. (Johanna
Amann)
2.5-beta-110 | 2016-10-26 09:42:11 -0400
* Improvements DCE_RPC analyzer to make it perform fragment handling correctly
and generally be more resistent to unexpected traffic. (Seth Hall)
2.5-beta-102 | 2016-10-25 09:43:45 -0700
* Update number of bytes in request/response of smb1-com-open-andx.pac. (balintm)
* Fix a IPv4 CIDR specifications and payload-size condition of signature matching.
(Robin Sommer)
* Python 3 compatibility fix for coverage-calc script. (Daniel Thayer)
2.5-beta-93 | 2016-10-24 11:11:07 -0700
* Fix alignment issue of ones_complement_checksum. This error
occured reproducibly newer compilers when called from
icmp6_checksum. (Johanna Amann)
2.5-beta-91 | 2016-10-20 11:40:37 -0400
* Fix istate.pybroccoli test on systems using Python 3. (Daniel Thayer)
2.5-beta-89 | 2016-10-18 21:50:51 -0400
* SSH analyzer changes: the events are now restructured a bit. There is a new
event ssh_auth_attempted, that is raised each time authentication is tried.
ssh_auth_failed is still only being raised once per connection. There also
is an additional event ssh_auth_result giving more information about the
number of times that authentication was tried and if it succeded/failed in
the end. The number of authentication attemps are now part of ssh.log.
Addresses BIT-1641. (Vlad Grigorescu)
2.5-beta-79 | 2016-10-13 15:58:48 -0700
* Fix MD5 problem with FreeBSD 11.0 and clang 3.8. The apparent
cause is some confusion in clang when using a static char inside a
static inline function that is refered to several compilation
units. (Johanna Amann)
* Initial TLS 1.3 support, as of draft-16. (Johanna Amann)
2.5-beta-73 | 2016-10-13 14:03:04 -0700
* Added missing README files for documentation. (Daniel Thayer)
* List new log files in the log-files.rst document. (Daniel Thayer)
2.5-beta-67 | 2016-10-10 08:28:38 -0700
* Fixes for DCE_RPC analyzer. (Seth Hall)
- DCE_RPC fragmentation handling returns.
- Fixed some general parsing issues.
- Fixed an issue with the DCE_RPC signature not working for IPv6
connections.
2.5-beta-64 | 2016-10-10 08:20:42 -0700
* Fix httpd.py test script for Py3 compatibility. (Daniel Thayer)
* Tiny fix for a DCE_RPC script issue. Fixes BIT-1688. (Seth Hall)
* Fix for plugins/hooks test. (Johanna Amann)
* Update a TLS constants in preparation for TLS 1.3, and rename a
few names that had never been formally assigned yet. (Johanna
Amann)
* Fixing Broxygen indexing confusion for plugins. Broxygen now
indexes plugin scripts as, e.g., "Bro_Netmap/scripts/init.bro".
Addresses BIT-1693. (Robin Sommer)
2.5-beta-54 | 2016-10-06 14:24:01 -0700
* Fixing documentation piece on the interesting-hostname script.
(Robin Sommer)
* Improve the SMB documentation. (Vlad Grigorescu)
2.5-beta-46 | 2016-10-06 14:11:03 -0700
* Fixing Broxygen indexing confusion for plugins. Scripts in plugins now
get an artificial index prefix: "plugin_name/", followed by the script's
relative path inside the plugin's top-level directory. For example,
"/opt/bro/lib/bro/plugins/Bro_Netmap/scripts/init.bro" now turns into
"Bro_Netmap/scripts/init.bro" for Broxygen purposes (whereas it used to
be just "init.bro"). (Robin Sommer)
* Fix a couple of tests, addressing issues of the newly introduced version.bro
as well as small FreeBSD 11.0 issues. (Johanna Amann)
* Update documentation license. (Johanna Amann)
* Add a convenient way to access version information to Bro.
@if ( Version::number >= 20500 )
or
@if ( Version::at_least("2.5") )
Version::info contains detailed information about the running version of
Bro, including beta flags, etc. (Johanna Amann)
2.5-beta-35 | 2016-10-02 17:38:31 -0400
* Normalize http host in seen script. (Johanna Amann)
2.5-beta-33 | 2016-10-02 14:42:22 -0400
* Handle removing non-existent intel items. (Jan Grashoefer)
2.5-beta-29 | 2016-09-28 18:18:35 -0700
* Prettifying reporter output in case no epxression is associated
with runtime error. (Robin Sommer)
2.5-beta-28 | 2016-09-27 11:44:33 -0700
* Check if the number of fields in a log write are equal to the
number of fields required. Addresses BIT-1683. (Johanna Amann)
* Fix a small memory leak for disabled log writers. (Johanna Amann)
* Fix loaded-scripts.bro to work with arbitrary indentation levels.
Addresses BIT-1691 (Johanna Amann)
* In interesting-hostnames.bro, move lookup_addr() outside of when
statement to avoid expensive cloning of full connection record.
Addresses BIT-1670. (Justin Azoff)
2.5-beta-21 | 2016-09-26 10:15:23 -0700
* Fix a debugger bug where it would not support statements like print(3).
Addresses BIT-1703. (Vlad Grigorescu)
2.5-beta-19 | 2016-09-19 17:16:40 -0700
* Kerberos updates (Vlad Grigorescu):
* Make PA_EncType_Info salt field optional.
* Add support for parsing ENCTYPE_INFO pre-auth data (same as
ENCTYPE_INFO2).
2.5-beta-17 | 2016-09-19 15:26:14 -0700
* Clarified string and fa_file documentation (Moshe Kaplan)
2.5-beta-12 | 2016-09-06 07:35:38 -0700
* Added a missing fclose in scan.l. Addresses BIT-1690.
(Daniel Thayer).
* Fix issue with file_extraction_limit event. (Seth Hall)
* Fix a crash when a user disables DCE_RPC while enabling SMB.
(Seth Hall)
2.5-beta-6 | 2016-08-19 07:50:10 -0700
* Clarify explanation of mime_entity_data event. (Moshe Kaplan)
* Update NEWS, correcting typos formatting and adding more
information. (Daniel Thayer)
* Remove old ack_above_hole event from scripts.
Fixes BIT-1673. (Johanna Amann)
2.5-beta | 2016-08-17 10:37:49 -0700
* Release 2.5-beta.
* Fix sphinx build errors (Johanna Amann)
* Change failure in utf16_bytestring_to_utf8_val to be a conn weird. (Seth Hall)
* Fix test failure caused by uninitialized memory. (Johanna Amann)
* SMB: fix rounding error due to value truncation when converting timestamps. (Johanna Amann)
2.4-947 | 2016-08-16 12:10:02 -0700
* Fix issues with handling of indermediate sumstats updates. (Justin Azoff)
* Address coverity errors. (Johanna Amann)
2.4-943 | 2016-08-15 17:03:14 -0700
* Add 'bro-config' script. (Jon Siwek)
* add certificate to external list for debian 8 to fix test. (Johanna Amann)
* KRB: fix field value missing error for msg$client_name. (Johanna Amann)
* Tiny SMB cleanup in pipe handling as well as NTLM fixes. (Seth Hall).
* Fix a number of format specifier errors. (Johanna Amann)
* Make several tests more stable. (Johanna Amann)
2.4-927 | 2016-08-11 21:49:06 -0700
* Make component tags generated during component initialization stable.
Before, it was dependent on the order a compiler called constructors.
This makes a few tests work with gcc. (Johanna Amann)
* Make x509 intel seen script more robust (Johanna Amann)
* Input: DisableFrontend was not called upon Init failure. (Johanna Amann)
* DCE_RPC code simplification. (Seth Hall)
2.4-921 | 2016-08-10 20:29:48 -0700
* Add logging framework ext-data mechanism. It is now possible to
extend logs by adding new data columns by them - either to specific
ones, or globally to all logs. This can, e.g., be used to add node
names to all logs. (Seth Hall)
* Add unrolling separator & field name map to logging framework.
One can now use logging separators other than ".", as well as
change specific column names in logs. (Seth Hall)
* Fix memory leak in EnumType. (Johanna Amann)
* Fix configure warning when compiling with --enable-broker. (Johanna Amann)
* Add netcontrol-connectors to aux directory. (Johanna Amann)
* Update Mozilla CA list. (Johanna Amann)
* update scripts loaded by default in local.bro. Traceroute is now disabled
by default, stats and capture-loss enabled by default. (Johanna Amann)
2.4-907 | 2016-08-09 15:42:17 -0400
* Updating NEWS.
2.4-905 | 2016-08-09 08:19:37 -0700
* GSSAPI analyzer now forwards authentication blobs more correctly.
(Seth Hall)
* The KRB analyzer now includes support for the PA_ENCTYPE_INFO2
pre-auth data type. (Seth Hall)
* Add an argument to "disable_analyzer" function to not do a
reporter message by default. (Seth Hall)
2.4-902 | 2016-08-08 16:50:35 -0400
* Adding SMB analyzer. (Seth Hall, Vlad Grigorescu and many others)
* NetControl: allow reasons in remove_rule calls. Addresses BIT-1655
(Johanna Amann)
2.4-893 | 2016-08-05 15:43:04 -0700
* Remove -z/--analysis option. (Johanna Amann)
* Remove already defunct code for XML serialization. (Johanna Amann)
2.4-885 | 2016-08-05 15:03:59 -0700
* Reverting SMB analyzer merge. (Robin Sommer)
2.4-883 | 2016-08-05 12:57:26 -0400
* Add a new node type for logging with the cluster framework scripts by
adding a new Bro node type for doing logging (this is intended to
reduce the load on the manager). If a user chooses not to specify a
logger node in the cluster configuration, then the manager will
write logs locally as usual. (Daniel Thayer)
2.4-874 | 2016-08-05 12:43:06 -0400
* SMB analyzer (Seth Hall, Vlad Grigorescu and many others)
2.4-759 | 2016-08-05 09:32:42 -0400
* Intel framework improvements (Jan Grashoefer)
* Added expiration for intelligence items.
* Improved intel notices.
* Added hook to allow extending the intel log.
* Added support for subnets to intel-framework.
2.4-742 | 2016-08-02 15:28:31 -0700
* Fix duplicate SSH authentication failure events. Addresses BIT-1641.
(Robin Sommer)
* Remove OpenSSL dependency for plugins. (Robin Sommer)
2.4-737 | 2016-08-02 11:38:07 -0700
* Fix some Coverity warnings. (Robin Sommer)
2.4-735 | 2016-08-02 11:05:36 -0700
* Added string slicing examples to documentation. (Moshe Kaplan)
2.4-733 | 2016-08-01 09:09:29 -0700
* Fixing a CMake dependency issue for the pcap bifs. (Robin Sommer)
2.4-732 | 2016-08-01 08:33:00 -0700
* Removing pkg/make-*-packages scripts. BIT-1509 #closed (Robin
Sommer)
2.4-731 | 2016-08-01 08:14:06 -0700
* Correct endianness of IP addresses in SNMP. Addresses BIT-1644.
(Anony Mous)
2.4-729 | 2016-08-01 08:00:54 -0700
* Fix behavior of connection_pending event. It is now really only
raised when Bro is terminating. Also adds a test-case that raises
the event. (Johanna Amann)
* Retired remove -J/-K options (set md5/hash key) from the manpage.
They had already been removed from the code. (Johanna Amann)
* NetControl: Add catch-and-release event when IPs are forgotten.
This adds an event catch_release_forgotten() that is raised once
Catch & Release ceases block management for an IP address because
the IP has not been seen in traffic during the watch interval.
(Johanna Amann)
2.4-723 | 2016-07-26 15:04:26 -0700
* Add error events to input framework. (Johanna Amann)
This change introduces error events for Table and Event readers.
Users can now specify an event that is called when an info,
warning, or error is emitted by their input reader. This can,
e.g., be used to raise notices in case errors occur when reading
an important input stream.
Example:
event error_event(desc: Input::TableDescription, msg: string, level: Reporter::Level)
{
...
}
event bro_init()
{
Input::add_table([$source="a", $error_ev=error_event, ...]);
}
Addresses BIT-1181.
* Calling Error() in an input reader now automatically will disable
the reader and return a failure in the Update/Heartbeat calls.
(Johanna Amann)
* Convert all errors in the ASCII formatter into warnings (to show
that they are non-fatal. (Johanna Amann)
* Enable SQLite shared cache mode. This allows all threads accessing
the same database to share sqlite objects. See
https://www.sqlite.org/sharedcache.html. Addresses BIT-1325.
(Johanna Amann)
* NetControl: Adjust default priority of ACTION_DROP hook to standad
level. (Johanna Amann)
* Fix types when constructing SYN_packet record. Fixes BIT-1650.
(Grant Moyer).
2.4-715 | 2016-07-23 07:27:05 -0700
* SQLite writer: Remove unused string formatting function. (Johanna Amann)
* Deprecated the ElasticSearch log writer. (Johanna Amann)
2.4-709 | 2016-07-15 09:05:20 -0700
* Change Bro's hashing for short inputs and Bloomfilters from H3 to
Siphash, which produces much better results for HLL in particular.
(Johanna Amann)
* Fix a long-standing bug which truncated hash values to 32-bit on
most machines. (Johanna Amann)
* Fixes to HLL. Addresses BIT-1612. (Johanna Amann)
* Add test checking the quality of HLL. (Johanna Amann)
* Remove the -K/-J options for setting keys. (Johanna Amann)
* SSL: Fix memory management problem. (Johanna Amann)
2.4-693 | 2016-07-12 11:29:17 -0700
* Change TCP analysis to process connections without the initial SYN as
non-partial connections. Addresses BIT-1492. (Robin Sommer).
2.4-691 | 2016-07-12 09:58:38 -0700
* SSL: add support for signature_algorithms extension. (Johanna
Amann)
2.4-688 | 2016-07-11 11:10:33 -0700
* Disable broker by default. To enable it, use --enable-broker.
Addresses BIT-1645. (Daniel Thayer)
2.4-686 | 2016-07-08 19:14:43 -0700
* Added flagging of retransmission to the connection history.
Addresses BIT-977. (Robin Sommer)
2.4-683 | 2016-07-08 14:55:04 -0700
* Extendign connection history field to flag with '^' when Bro flips
a connection's endpoints. Addresses BIT-1629. (Robin Sommer)
2.4-680 | 2016-07-06 09:18:21 -0700
* Remove ack_above_hole() event, which was a subset of content_gap
and led to plenty noise. Addresses BIT-688. (Robin Sommer)
2.4-679 | 2016-07-05 16:35:53 -0700
* Fix segfault when an existing enum identifier is added again with
a different value. Addresses BIT-931. (Robin Sommer)
* Escape the empty indicator in logs if it occurs literally as a
field's actual content. Addresses BIT-931. (Robin Sommer)
2.4-676 | 2016-06-30 17:27:54 -0700
* A larger series of NetControl updates. (Johanna Amann)
* Add NetControl framework documentation to the Bro manual.
* Use NetControl for ACTION_DROP of notice framework. So far,
this action did nothing by default.
* Rewrite of catch-and-release.
* Fix several small logging issues.
* find_rules_subnet() now works in cluster mode. This
introduces two new events, NetControl::rule_new and
NetControl::rule_destroyed, which are raised when rules are
first added and then deleted from the internal state
tracking.
* Fix acld whitelist command.
* Add rule existance as a state besides added and failure.
* Suppress duplicate "plugin activated" messages.
* Make new Broker plugin options accessible.
* Add predicates to Broker plugin.
* Tweak SMTP scripts to not to pull in the notice framework.
2.4-658 | 2016-06-30 16:55:32 -0700
* Fix a number of documentation building errors. (Johanna Amann)
* Input/Logging: Make bool conversion operator explicit. (Johanna Amann)
* Add new TLS ciphers from RFC 7905. (Johanna Amann)
2.4-648 | 2016-06-21 18:33:22 -0700
* Fix memory leaks. Reported by Dk Jack. (Johanna Amann)
2.4-644 | 2016-06-21 13:59:05 -0400
* Fix an off-by-one error when grabbing x-originating-ip header in
email. (Seth Hall, Aashish Sharma)
2.4-642 | 2016-06-18 13:18:23 -0700
* Fix potential mismatches when ignoring duplicate weirds. (Johanna Amann)
* Weird: Rewrite internals of weird logging. (Johanna Amann)
- "flow weirds" now actually log information about the flow
that they occur in.
- weirds can now be generated by calling Weird::weird() with
the info record directly, allowing more fine-granular passing
of information. This is e.g. used for DNS weirds.
Addresses BIT-1578 (Johanna Amann)
* Exec: fix reader cleanup when using read_files, preventing file
descriptors from leaking every time it was used. (Johanna Amann)
* Raw Writer: Make code more c++11-y, remove raw pointers. (Johanna
Amann)
* Add separate section with logging changes to NEWS. (Seth Hall)
2.4-635 | 2016-06-18 01:40:17 -0400
* Add some documentation for modbus data types. Addresses
BIT-1216. (Seth Hall)
* Removed app-stats scripts. Addresses BIT-1171. (Seth Hall)
2.4-631 | 2016-06-16 16:45:10 -0400
* Fixed matching mail address intel and added test (Jan Grashoefer)
* A new utilities script named email.bro with some utilities
for parsing out email addresses from strings. (Seth Hall)
* SMTP "rcptto" and "mailfrom" fields now do some minimal
parsing to clean up email addresses. (Seth Hall)
* Added "cc" to the SMTP log and feed it into the Intel framework
with the policy/frameworks/intel/seen/smtp.bro script. (Seth Hall)
2.4-623 | 2016-06-15 17:31:12 -0700
* &default values are no longer overwritten with uninitialized
by the input framework. (Jan Grashoefer)
2.4-621 | 2016-06-15 09:18:02 -0700
* Fixing memory leak in changed table expiration code. (Robin
Sommer)
* Fixing test portability. (Robin Sommer)
* Move the HTTP "filename" field (which was never filled out
anyways) to "orig_filenames" and "resp_filenames". (Seth Hall)
* Add a round trip time (rtt) field to dns.log. (Seth Hall)
* Add ACE archive files to the identified file types. Addresses
BIT-1609. (Stephen Hosom)
2.4-613 | 2016-06-14 18:10:37 -0700
* Preventing the event processing from looping endlessly when an
event reraised itself during execution of its handlers. (Robin
Sommer)
2.4-612 | 2016-06-14 17:42:52 -0700
* Improved handling of 802.11 headers. (Jan Grashoefer)
2.4-609 | 2016-06-14 17:15:28 -0700
* Fixed table expiration evaluation. The expiration attribute
expression is now evaluated for every use. Thus later adjustments
of the value (e.g. by redefining a const) will now take effect.
Values less than 0 will disable expiration. (Jan Grashoefer)
2.4-606 | 2016-06-14 16:11:07 -0700
* Fix parsing precedence of "hook" expression. Addresses BIT-1619
(Johanna Amann)
* Update the "configure" usage message for --with-caf (Daniel
Thayer)
2.4-602 | 2016-06-13 08:16:34 -0700
* Fixing Covertity warning (CID 1356391). (Robin Sommer)
* Guarding against reading beyond packet data when accessing L2
address in Radiotap header. (Robin Sommer)
2.4-600 | 2016-06-07 15:53:19 -0700
* Fixing typo in BIF macros. Reported by Jeff Barber. (Robin Sommer)
2.4-599 | 2016-06-07 12:37:32 -0700
* Add new functions haversine_distance() and haversine_distance_ip()
for calculating geographic distances. They requires that Bro be
built with libgeoip. (Aashish Sharma/Daniel Thayer).
2.4-597 | 2016-06-07 11:46:45 -0700
* Fixing memory leak triggered by new MAC address logging. (Robin
Sommer)
2.4-596 | 2016-06-07 11:07:29 -0700
* Don't create debug.log immediately upon startup (BIT-1616).
(Daniel Thayer)
2.4-594 | 2016-06-06 18:11:16 -0700
* ASCII Input: Accept DOS/Windows newlines. Addresses BIT-1198
(Johanna Amann)
* Fix BinPAC exception in RFB analyzer. (Martin van Hensbergen)
* Add URL decoding for the unofficial %u00AE style of encoding. (Seth Hall)
* Remove the unescaped_special_char HTTP weird. (Seth Hall)
2.4-588 | 2016-06-06 17:59:34 -0700
* Moved link-layer addresses into endpoints. The link-layer
addresses are now part of the connection endpoints following the
originator/responder pattern. (Jan Grashoefer)
* Link-layer addresses are extracted for 802.11 plus RadioTap. (Jan
Grashoefer)
* Fix coverity error (uninitialized variable) (Johanna Amann)
* Use ether_ntoa instead of ether_ntoa_r
The latter is thread-safe, but a GNU addition which does not exist on
OS-X. Since the function only is called in the main thread, it should
not matter if it is or is not threadsafe. (Johanna Amann)
* Fix FreeBSD/OSX compile problem due to headers (Johanna Amann)
2.4-581 | 2016-05-30 10:58:19 -0700
* Adding missing new script file mac-logging.bro. (Robin Sommer)
2.4-580 | 2016-05-29 13:41:10 -0700
* Add Ethernet MAC addresses to connection record. c$eth_src and
c$eth_dst now contain the Ethernet address if available. A new
script protocols/conn/mac-logging.bro adds these to conn.log when
loaded. (Robin Sommer)
2.4-579 | 2016-05-29 08:54:57 -0700
* Fixing Coverity warning. Addresses CID 1356116. (Robin Sommer)
* Fixing FTP cwd getting overlue long. (Robin Sommer)
* Clarifying notice documentation. Addresses BIT-1405. (Robin
Sommer)
* Changing protocol_{confirmation,violation} events to queue like
any other event. Addresses BIT-1530. (Robin Sommer)
* Normalizing test baseline. (Robin Sommer)
* Do not use scientific notations when printing doubles in logs.
Addresses BIT-1558. (Robin Sommer)
2.4-573 | 2016-05-23 13:21:03 -0700
* Ignoring packets with negative timestamps. Addresses BIT-1562 and
BIT-1443. (Robin Sommer)
2.4-572 | 2016-05-23 12:45:23 -0700
* Fix for a table refering to a expire function that's not defined.
Addresses BIT-1597. (Robin Sommer)
2.4-571 | 2016-05-23 08:26:43 -0700
* Fixing a few Coverity warnings. (Robin Sommer)
2.4-569 | 2016-05-18 07:39:35 -0700
* DTLS: Use magix constant from RFC 5389 for STUN detection.
(Johanna Amann)
* DTLS: Fix binpac bug with DTLSv1.2 client hellos. (Johanna Amann)
* DTLS: Fix interaction with STUN. Now the DTLS analyzer cleanly
skips all STUN messages. (Johanna Amann)
* Fix the way that child analyzers are added. (Johanna Amann)
2.4-563 | 2016-05-17 16:25:21 -0700
* Fix duplication of new_connection_contents event. Addresses
BIT-1602 (Johanna Amann)
* SMTP: Support SSL upgrade via X-ANONYMOUSTLS This seems to be a
non-standardized microsoft extension that, besides having a
different name, works pretty much the same as StartTLS. We just
treat it as such. (Johanna Amann)
* Fixing control framework's net_stats and peer_status commands. For
the latter, this removes most of the values returned, as we don't
have access to them anymore. (Robin Sommer)
2.4-555 | 2016-05-16 20:10:15 -0700
* Fix failing plugin tests on OS X 10.11. (Daniel Thayer)
* Fix failing test on Debian/FreeBSD. (Johanna Amann)
2.4-552 | 2016-05-12 08:04:33 -0700
* Fix a bug in receiving remote logs via broker. (Daniel Thayer)
* Fix Bro and unit tests when broker is not enabled. (Daniel Thayer)
* Added interpreter error for local event variables. (Jan Grashoefer)
2.4-544 | 2016-05-07 12:19:07 -0700
* Switching all use of gmtime and localtime to use reentrant
variants. (Seth Hall)
2.4-541 | 2016-05-06 17:58:45 -0700
* A set of new built-in function for gathering execution statistics:
get_net_stats(), get_conn_stats(), get_proc_stats(),
get_event_stats(), get_reassembler_stats(), get_dns_stats(),
get_timer_stats(), get_file_analysis_stats(), get_thread_stats(),
get_gap_stats(), get_matcher_stats().
net_stats() resource_usage() have been superseded by these. (Seth
Hall)
* New policy script misc/stats.bro that records Bro execution
statistics in a standard Bro log file. (Seth Hall)
* A series of documentation improvements. (Daniel Thayer)
* Rudimentary XMPP StartTLS analyzer. It parses certificates out of
XMPP connections using StartTLS. It aborts processing if StartTLS
is not found. (Johanna Amann)
2.4-507 | 2016-05-03 11:18:16 -0700
* Fix incorrect type tags in Bro broker source code. These are just
used for error reporting. (Daniel Thayer)
* Update docs and tests of the fmt() function. (Daniel Thayer)
2.4-500 | 2016-05-03 11:16:50 -0700
* Updating submodule(s).
2.4-498 | 2016-04-28 11:34:52 -0700
* Rename Broker::print to Broker::send_print and Broker::event to
Broker::send_event to avoid using reserved keywords as function
names. (Daniel Thayer)
* Add script wrapper functions for Broker BIFs. This faciliates
documenting them through Broxygen. (Daniel Thayer)
* Extend, update, and clean up Broker tests. (Daniel Thayer)
* Intel: Allow to provide uid/fuid instead of conn/file. (Johanna
Amann)
* Provide file IDs for hostname matches in certificates. (Johanna
Amann)
* Rudimentary IMAP StartTLS analyzer. It parses certificates out of
IMAP connections using StartTLS. It aborts processing if StartTLS
is not found. (Johanna Amann)
2.4-478 | 2016-04-28 09:56:24
* Fix parsing of x509 pre-y2k dates. (Johanna Amann)
* Fix small error in bif documentation. (Johanna Amann)
* Fix unknown data link type error message. (Vitaly Repin)
* Correcting spelling errors. (Jeannette Dopheide)
* Minor cleanup in ARP analyzer. (Johanna Amann)
* Fix parsing of pre-y2k dates in X509 certificates. (Johanna Amann)
* Fix small error in get_current_packet documentation. (Johanna Amann)
2.4-471 | 2016-04-25 15:37:15 -0700
* Add DNS tests for huge TLLs and CAA. (Johanna Amann)
* Add DNS "CAA" RR type and event. (Mark Taylor)
* Fix DNS response parsing: TTLs are unsigned. (Mark Taylor)
2.4-466 | 2016-04-22 16:25:33 -0700
* Rename BrokerStore and BrokerComm to Broker. Also split broker main.bro
into two scripts. (Daniel Thayer)
* Add get_current_packet_header bif. (Jan Grashoefer)
2.4-457 | 2016-04-22 08:36:27 -0700
* Fix Intel framework not checking the CERT_HASH indicator type. (Johanna Amann)
2.4-454 | 2016-04-14 10:06:58 -0400
* Additional mime types for file identification and a few fixes. (Seth Hall)
New file mime types:
- .ini files
- MS Registry policy files
- MS Registry files
- MS Registry format files (e.g. DESKTOP.DAT)
- MS Outlook PST files
- Apple AFPInfo files
Mime type fixes:
- MP3 files with ID3 tags.
- JSON and XML matchers were extended
* Avoid a macro name conflict on FreeBSD. (Seth Hall, Daniel Thayer)
2.4-452 | 2016-04-13 01:15:20 -0400
* Add a simple file entropy analyzer. (Seth Hall)
* Analyzer and bro script for RFB/VNC protocol (Martin van Hensbergen)
This analyzer parses the Remote Frame Buffer
protocol, usually referred to as the 'VNC protocol'.
It supports several dialects (3.3, 3.7, 3.8) and
also handles the Apple Remote Desktop variant.
It will log such facts as client/server versions,
authentication method used, authentication result,
height, width and name of the shared screen.
2.4-430 | 2016-04-07 13:36:36 -0700
* Fix regex literal in scripting documentation. (William Tom)
2.4-428 | 2016-04-07 13:33:08 -0700
* Confirm protocol in SNMP/SIP only if we saw a response SNMP/SIP
packet. (Vlad Grigorescu)
2.4-424 | 2016-03-24 13:38:47 -0700
* Only load openflow/netcontrol if compiled with broker. (Johanna Amann)
* Adding canonifier to test. (Robin Sommer)
2.4-422 | 2016-03-21 19:48:30 -0700
* Adapt to recent change in CAF CMake script. (Matthias Vallentin)
* Deprecate --with-libcaf in favor of --with-caf, as already done in
Broker. (Matthias Vallentin)
2.4-418 | 2016-03-21 12:22:15 -0700
* Add protocol confirmation to MySQL analyzer. (Vlad Grigorescu)
* Check that there is only one of &read_expire, &write_expire,
&create_expire. (Johanna Amann)
* Fixed &read_expire for subnet-indexed tables, plus test case. (Jan
Grashoefer)
* Add filter_subnet_table() that works similar to matching_subnet()
but returns a filtered view of the original set/table only
containing the changed subnets. (Jan Grashoefer)
* Fix bug in tablue values' tracking read operations. (Johanna
Amann)
* Update TLS constants and extensions from IANA. (Johanna Amann)
2.4-406 | 2016-03-11 14:27:47 -0800
* Add NetControl and OpenFlow frameworks. (Johanna Amann)
2.4-313 | 2016-03-08 07:47:57 -0800
* Remove old string functions in C++ code. This removes the
functions: strcasecmp_n, strchr_n, and strrchr_n. (Johanna Amann)
2.4-307 | 2016-03-07 13:33:45 -0800
* Add "disable_analyzer_after_detection" and remove
"skip_processing_after_detection". Addresses BIT-1545.
(Aaron Eppert & Johanna Amann)
* Add bad_HTTP_request_with_version weird (William Glodek)
2.4-299 | 2016-03-04 12:51:55 -0800
* More detailed installation instructions for FreeBSD 9.X. (Johanna Amann)
* Update CMake OpenSSL checks. (Johanna Amann)
* "SUBSCRIBE" is a valid SIP. message per RFC 3265. Addresses
BIT-1529. (Johanna Amann)
* Update documentation for connection log's RSTR. Addresses BIT-1535
(Johanna Amann)
2.4-284 | 2016-02-17 14:12:15 -0800
* Fix sometimes failing dump-events test. (Johanna Amann)
2.4-282 | 2016-02-13 10:48:21 -0800
* Add missing break in in StartTLS case of IRC analyzer. Found by
Aaron Eppert. (Johanna Amann)
2.4-280 | 2016-02-13 10:40:16 -0800
* Fix memory leaks in stats.cc and smb.cc. (Johanna Amann)
2.4-278 | 2016-02-12 18:53:35 -0800
* Better multi-space separator handline. (Mark Taylor & Johanna Amann)
2.4-276 | 2016-02-10 21:29:33 -0800
* Allow IRC commands to not have parameters. (Mark Taylor)
2.4-272 | 2016-02-08 14:27:58 -0800
* fix memory leaks in find_all() and IRC analyzer. (Dirk Leinenbach)
2.4-270 | 2016-02-08 13:00:57 -0800
* Removed duplicate parameter for IRC "QUIT" event handler. (Mark Taylor)
2.4-267 | 2016-02-01 12:38:32 -0800
* Add testcase for CVE-2015-3194. (Johanna Amann)
* Fix portability issue with use of mktemp. (Daniel Thayer)
2.4-260 | 2016-01-28 08:05:27 -0800
* Correct irc_privmsg_message event handling bug. (Mark Taylor)
* Update copyright year for Sphinx. (Johanna Amann)
2.4-253 | 2016-01-20 17:41:20 -0800
* Support of RadioTap encapsulation for 802.11 (Seth Hall)
Radiotap support should be fully functional with Radiotap
packets that include IPv4 and IPv6. Other radiotap packets are
silently ignored.
2.4-247 | 2016-01-19 10:19:48 -0800
* Fixing C++11 compiler warnings. (Seth Hall)
* Updating plugin documentation building. (Johanna Amann)
2.4-238 | 2016-01-15 12:56:33 -0800
* Add HTTP version information to HTTP log file. (Aaron Eppert)
* Add NOTIFY as a valid SIP message, per RFC 3265. (Aaron Eppert)
* Improve HTTP parser's handling of requests that don't have a URI.
(William Glodek/Robin Sommer)
* Fix crash when deleting non existing record member. Addresses
BIT-1519. (Johanna Amann)
2.4-228 | 2015-12-19 13:40:09 -0800
* Updating BroControl submodule.
2.4-227 | 2015-12-18 17:47:24 -0800
* Update host name in windows-version-detection.bro. (Aaron Eppert)
* Update installation instructions to mention OpenSSL dependency for
newer OS X version. (Johanna Amann)
* Change a stale bro-ids.org to bro.org. (Johanna Amann)
* StartTLS support for IRC. (Johanna Amann)
* Adding usage guard to canonifier script. (Robin Sommer)
2.4-217 | 2015-12-04 16:50:46 -0800
* SIP scripts code cleanup. (Seth Hall)
- Daniel Guerra pointed out a type issue for SIP request and
response code length fields which is now corrected.
- Some redundant code was removed.
- if/else tree modified to use switch instead.
2.4-214 | 2015-12-04 16:40:15 -0800
* Delaying BinPAC initializaton until afte plugins have been
activated. (Robin Sommer)
2.4-213 | 2015-12-04 15:25:48 -0800
* Use better data structure for storing BPF filters. (Robin Sommer)
2.4-211 | 2015-11-17 13:28:29 -0800
* Making cluster reconnect timeout configurable. (Robin Sommer)
* Bugfix for child process' communication loop. (Robin Sommer)
2.4-209 | 2015-11-16 07:31:22 -0800
* Updating submodule(s).
2.4-207 | 2015-11-10 13:34:42 -0800
* Fix to compile with OpenSSL that has SSLv3 disalbed. (Christoph
Pietsch)
* Fix potential race condition when logging VLAN info to conn.log.
(Daniel Thayer)
2.4-201 | 2015-10-27 16:11:15 -0700
* Updating NEWS. (Robin Sommer)
2.4-200 | 2015-10-26 16:57:39 -0700
* Adding missing file. (Robin Sommer)
2.4-199 | 2015-10-26 16:51:47 -0700
* Fix problem with the JSON Serialization code. (Aaron Eppert)
2.4-188 | 2015-10-26 14:11:21 -0700
* Extending rexmit_inconsistency() event to receive an additional
parameter with the packet's TCP flags, if available. (Robin
Sommer)
2.4-187 | 2015-10-26 13:43:32 -0700
* Updating NEWS for new plugins. (Robin Sommer)
2.4-186 | 2015-10-23 15:07:06 -0700
* Removing pcap options for AF_PACKET support. Addresses BIT-1363.
(Robin Sommer)
* Correct a typo in controller.bro documentation. (Daniel Thayer)
* Extend SSL DPD signature to allow alert before server_hello.
(Johanna Amann)
* Make join_string_vec work with vectors containing empty elements.
(Johanna Amann)
* Fix support for HTTP CONNECT when server adds headers to response.
(Eric Karasuda).
* Load static CA list for validation tests too. (Johanna Amann)
* Remove cluster certificate validation script. (Johanna Amann)
* Fix a bug in diff-remove-x509-names canonifier. (Daniel Thayer)
* Fix test canonifiers in scripts/policy/protocols/ssl. (Daniel
Thayer)
2.4-169 | 2015-10-01 17:21:21 -0700
* Fixed parsing of V_ASN1_GENERALIZEDTIME timestamps in x509
certificates. (Yun Zheng Hu)
* Improve X509 end-of-string-check code. (Johanna Amann)
* Refactor X509 generalizedtime support and test. (Johanna Amann)
* Fix case of offset=-1 (EOF) for RAW reader. Addresses BIT-1479.
(Johanna Amann)
* Improve a number of test canonifiers. (Daniel Thayer)
* Remove unnecessary use of TEST_DIFF_CANONIFIER. (Daniel Thayer)
* Fixed some test canonifiers to read only from stdin
* Remove unused test canonifier scripts. (Daniel Thayer)
* A potpourri of updates and improvements across the documentation.
(Daniel Thayer)
* Add configure option to disable Broker Python bindings. Also
improve the configure summary output to more clearly show whether
or not Broker Python bindings will be built. (Daniel Thayer)
2.4-131 | 2015-09-11 12:16:39 -0700
* Add README.rst symlink. Addresses BIT-1413 (Vlad Grigorescu)
2.4-129 | 2015-09-11 11:56:04 -0700
* hash-all-files.bro depends on base/files/hash (Richard van den Berg)
* Make dns_max_queries redef-able, and bump default to 25. Addresses
BIT-1460 (Vlad Grigorescu)
2.4-125 | 2015-09-03 20:10:36 -0700
* Move SIP analyzer to flowunit instead of datagram Addresses
BIT-1458 (Vlad Grigorescu)
2.4-122 | 2015-08-31 14:39:41 -0700
* Add a number of out-of-bound checks to layer 2 code. Addresses
BIT-1463 (Johanna Amann)
* Fix error in 2.4 release notes regarding SSH events. (Robin
Sommer)
2.4-118 | 2015-08-31 10:55:29 -0700
* Fix FreeBSD build errors (Johanna Amann)
2.4-117 | 2015-08-30 22:16:24 -0700
* Fix initialization of a pointer in RDP analyzer. (Daniel
Thayer/Robin Sommer)
2.4-115 | 2015-08-30 21:57:35 -0700
* Enable Bro to leverage packet fanout mode on Linux. (Kris
Nielander).
## Toggle whether to do packet fanout (Linux-only).
const Pcap::packet_fanout_enable = F &redef;
## If packet fanout is enabled, the id to sue for it. This should be shared amongst
## worker processes processing the same socket.
const Pcap::packet_fanout_id = 0 &redef;
## If packet fanout is enabled, whether packets are to be defragmented before
## fanout is applied.
const Pcap::packet_fanout_defrag = T &redef;
* Allow libpcap buffer size to be set via configuration. (Kris Nielander)
## Number of Mbytes to provide as buffer space when capturing from live
## interfaces.
const Pcap::bufsize = 128 &redef;
* Move the pcap-related script-level identifiers into the new Pcap
namespace. (Robin Sommer)
snaplen -> Pcap::snaplen
precompile_pcap_filter() -> Pcap::precompile_pcap_filter()
install_pcap_filter() -> Pcap::install_pcap_filter()
pcap_error() -> Pcap::pcap_error()
2.4-108 | 2015-08-30 20:14:31 -0700
* Update Base64 decoding. (Jan Grashoefer)
- A new built-in function, decode_base64_conn() for Base64
decoding. It works like decode_base64() but receives an
additional connection argument that will be used for
reporting decoding errors into weird.log (instead of
reporter.log).
- FTP, POP3, and HTTP analyzers now likewise log Base64
decoding errors to weird.log.
- The built-in functions decode_base64_custom() and
encode_base64_custom() are now deprecated. Their
functionality is provided directly by decode_base64() and
encode_base64(), which take an optional parameter to change
the Base64 alphabet.
* Fix potential crash if TCP header was captured incompletely.
(Robin Sommer)
2.4-103 | 2015-08-29 10:51:55 -0700
* Make ASN.1 date/time parsing more robust. (Johanna Amann)
* Be more permissive on what characters we accept as an unquoted
multipart boundary. Addresses BIT-1459. (Johanna Amann)
2.4-99 | 2015-08-25 07:56:57 -0700
* Add ``Q`` and update ``I`` documentation for connection history
field. Addresses BIT-1466. (Vlad Grigorescu)
2.4-96 | 2015-08-21 17:37:56 -0700
* Update SIP analyzer. (balintm)
- Allows space on both sides of ':'.
- Require CR/LF after request/reply line.
2.4-94 | 2015-08-21 17:31:32 -0700
* Add file type detection support for video/MP2T. (Mike Freemon)
2.4-93 | 2015-08-21 17:23:39 -0700
* Make plugin install honor DESTDIR= convention. (Jeff Barber)
2.4-89 | 2015-08-18 07:53:36 -0700
* Fix diff-canonifier-external to use basename of input file.
(Daniel Thayer)
2.4-87 | 2015-08-14 08:34:41 -0700
* Removing the yielding_teredo_decapsulation option. (Robin Sommer)
2.4-86 | 2015-08-12 17:02:24 -0700
* Make Teredo DPD signature more precise. (Martina Balint)
2.4-84 | 2015-08-10 14:44:39 -0700
* Add hook 'HookSetupAnalyzerTree' to allow plugins access to a
connection's initial analyzer tree for customization. (James
Swaro)
* Plugins now look for a file "__preload__.bro" in the top-level
script directory. If found, they load it first, before any scripts
defining BiF elements. This can be used to define types that the
BiFs already depend on (like a custom type for an event argument).
(Robin Sommer)
2.4-81 | 2015-08-08 07:38:42 -0700
* Fix a test that is failing very frequently. (Daniel Thayer)
2.4-78 | 2015-08-06 22:25:19 -0400
* Remove build dependency on Perl (now requiring Python instad).
(Daniel Thayer)
* CID 1314754: Fixing unreachable code in RSH analyzer. (Robin
Sommer)
* CID 1312752: Add comment to mark 'case' fallthrough as ok. (Robin
Sommer)
* CID 1312751: Removing redundant assignment. (Robin Sommer)
2.4-73 | 2015-07-31 08:53:49 -0700
* BIT-1429: SMTP logs now include CC: addresses. (Albert Zaharovits)
2.4-70 | 2015-07-30 07:23:44 -0700
* Updated detection of Flash and AdobeAIR. (Jan Grashoefer)
* Adding tests for Flash version parsing and browser plugin
detection. (Robin Sommer)
2.4-63 | 2015-07-28 12:26:37 -0700
* Updating submodule(s).
2.4-61 | 2015-07-28 12:13:39 -0700
* Renaming config.h to bro-config.h. (Robin Sommer)
2.4-58 | 2015-07-24 15:06:07 -0700
* Add script protocols/conn/vlan-logging.bro to record VLAN data in
conn.log. (Aaron Brown)
* Add field "vlan" and "inner_vlan" to connection record. (Aaron
Brown)
* Save the inner vlan in the Packet object for Q-in-Q setups. (Aaron
Brown)
* Increasing plugin API version for recent packet source changes.
(Robin Sommer)
* Slightly earlier protocol confirmation for POP3. (Johanna Amann)
2.4-46 | 2015-07-22 10:56:40 -0500
* Fix broker python bindings install location to track --prefix.
(Jon Siwek)
2.4-45 | 2015-07-21 15:19:43 -0700
* Enabling Broker by default. This means CAF is now a required
dependency, altjough for now at least, there's still a switch
--disable-broker to turn it off.
* Requiring a C++11 compiler, and turning on C++11 support. (Robin
Sommer)
* Tweaking the listing of hooks in "bro -NN" for consistency. (Robin
Sommer)
2.4-41 | 2015-07-21 08:35:17 -0700
* Fixing compiler warning. (Robin Sommer)
* Updates to IANA TLS registry. (Johanna Amann)
2.4-38 | 2015-07-20 15:30:35 -0700
* Refactor code to use a common Packet type throught. (Jeff
Barber/Robin Sommer)
* Extend parsing layer 2 and keeping track of layer 3 protoco. (Jeff Barber)
* Add a raw_packet() event that generated for all packets and
include layer 2 information. (Jeff Barber)
2.4-27 | 2015-07-15 13:31:49 -0700
* Fix race condition in intel test. (Johanna Amann)
2.4-24 | 2015-07-14 08:04:11 -0700
* Correct Perl package name on FreeBSD in documentation.(Justin Azoff)
* Adding an environment variable to BTest configuration for external
scripts. (Robin Sommer)
2.4-20 | 2015-07-03 10:40:21 -0700
* Adding a weird for when truncated packets lead TCP reassembly to
ignore content. (Robin Sommer)
2.4-19 | 2015-07-03 09:04:54 -0700
* A set of tests exercising IP defragmentation and TCP reassembly.
(Robin Sommer)
2.4-17 | 2015-06-28 13:02:41 -0700
* BIT-1314: Add detection for Quantum Insert attacks. The TCP
reassembler can now keep a history of old TCP segments using the
tcp_max_old_segments option. An overlapping segment with different
data will then generate an rexmit_inconsistency event. The default
for tcp_max_old_segments is zero, which disabled any additional
buffering. (Yun Zheng Hu/Robin Sommer)
2.4-14 | 2015-06-28 12:30:12 -0700
* BIT-1400: Allow '<' and '>' in MIME multipart boundaries. The spec
doesn't actually seem to permit these, but they seem to occur in
the wild. (Jon Siwek)
2.4-12 | 2015-06-28 12:21:11 -0700
* BIT-1399: Trying to decompress deflated HTTP content even when
zlib headers are missing. (Seth Hall)
2.4-10 | 2015-06-25 07:11:17 -0700
* Correct a name used in a header identifier (Justin Azoff)
2.4-8 | 2015-06-24 07:50:50 -0700
* Restore the --load-seeds cmd-line option and enable the short
options -G/-H for --load-seeds/--save-seeds. (Daniel Thayer)
2.4-6 | 2015-06-19 16:26:40 -0700
* Generate protocol confirmations for Modbus, making it appear as a
confirmed service in conn.log. (Seth Hall)
* Put command line options in alphabetical order. (Daniel Thayer)
* Removing dead code for no longer supported -G switch. (Robin
Sommer) (Robin Sommer)
2.4 | 2015-06-09 07:30:53 -0700
* Release 2.4.
* Fixing tiny thing in NEWS. (Robin Sommer)
2.4-beta-42 | 2015-06-08 09:41:39 -0700
* Fix reporter errors with GridFTP traffic. (Robin Sommer)
2.4-beta-40 | 2015-06-06 08:20:52 -0700
* PE Analyzer: Change how we calculate the rva_table size. (Vlad Grigorescu)
2.4-beta-39 | 2015-06-05 09:09:44 -0500
* Fix a unit test to check for Broker requirement. (Jon Siwek)
2.4-beta-38 | 2015-06-04 14:48:37 -0700
* Test for Broker termination. (Robin Sommer)
2.4-beta-37 | 2015-06-04 07:53:52 -0700
* BIT-1408: Improve I/O loop and Broker IOSource. (Jon Siwek)
2.4-beta-34 | 2015-06-02 10:37:22 -0700
* Add signature support for F4M files. (Seth Hall)
2.4-beta-32 | 2015-06-02 09:43:31 -0700
* A larger set of documentation updates, fixes, and extentions.
(Daniel Thayer)
2.4-beta-14 | 2015-06-02 09:16:44 -0700
* Add memleak btest for attachments over SMTP. (Vlad Grigorescu)
* BIT-1410: Fix flipped tx_hosts and rx_hosts in files.log. Reported
by Ali Hadi. (Vlad Grigorescu)
* Updating the Mozilla root certs. (Seth Hall)
* Updates for the urls.bro script. Fixes BIT-1404. (Seth Hall)
2.4-beta-6 | 2015-05-28 13:20:44 -0700
* Updating submodule(s).
2.4-beta-2 | 2015-05-26 08:58:37 -0700
* Fix segfault when DNS is not available. Addresses BIT-1387. (Frank
Meier and Robin Sommer)
2.4-beta | 2015-05-07 21:55:31 -0700
* Release 2.4-beta.
* Update local-compat.test (Johanna Amann)
2.3-913 | 2015-05-06 09:58:00 -0700
* Add /sbin to PATH in btest.cfg and remove duplicate default_path.
(Daniel Thayer)
2.3-911 | 2015-05-04 09:58:09 -0700
* Update usage output and list of command line options. (Daniel
Thayer)
* Fix to ssh/geo-data.bro for unset directions. (Vlad Grigorescu)
* Improve SIP logging and remove reporter messages. (Seth Hall)
2.3-905 | 2015-04-29 17:01:30 -0700
* Improve SIP logging and remove reporter messages. (Seth Hall)
2.3-903 | 2015-04-27 17:27:59 -0700
* BIT-1350: Improve record coercion type checking. (Jon Siwek)
2.3-901 | 2015-04-27 17:25:27 -0700
* BIT-1384: Remove -O (optimize scripts) command-line option, which
hadn't been working for a while already. (Jon Siwek)
2.3-899 | 2015-04-27 17:22:42 -0700
* Fix the -J/--set-seed cmd-line option. (Daniel Thayer)
* Remove unused -l, -L, and -Z cmd-line options. (Daniel Thayer)
2.3-892 | 2015-04-27 08:22:22 -0700
* Fix typos in the Broker BIF documentation. (Daniel Thayer)
* Update installation instructions and remove outdated references.
(Johanna Amann)
* Easier support for systems with tcmalloc_minimal installed. (Seth
Hall)
2.3-884 | 2015-04-23 12:30:15 -0500
* Fix some outdated documentation unit tests. (Jon Siwek)
2.3-883 | 2015-04-23 07:10:36 -0700
* Fix -N option to work with builtin plugins as well. (Robin Sommer)
2.3-882 | 2015-04-23 06:59:40 -0700
* Add missing .pac dependencies for some binpac analyzer targets.
(Jon Siwek)
2.3-879 | 2015-04-22 10:38:07 -0500
* Fix compile errors. (Jon Siwek)
2.3-878 | 2015-04-22 08:21:23 -0700
* Fix another compiler warning in DTLS. (Johanna Amann)
2.3-877 | 2015-04-21 20:14:16 -0700
* Adding missing include. (Robin Sommer)
2.3-876 | 2015-04-21 16:40:10 -0700
* Attempt at fixing a potential std::length_error exception in RDP
analyzer. Addresses BIT-1337. (Robin Sommer)
* Fixing compile problem caused by overeager factorization. (Robin
Sommer)
2.3-874 | 2015-04-21 16:09:20 -0700
* Change details of escaping when logging/printing. (Seth Hall/Robin
Sommer)
- Log files now escape non-printable characters consistently
as "\xXX'. Furthermore, backslashes are escaped as "\\",
making the representation fully reversible.
- When escaping via script-level functions (escape_string,
clean), we likewise now escape consistently with "\xXX" and
"\\".
- There's no "alternative" output style anymore, i.e., fmt()
'%A' qualifier is gone.
Addresses BIT-1333.
* Remove several BroString escaping methods that are no longer
useful. (Seth Hall)
2.3-864 | 2015-04-21 15:24:02 -0700
* A SIP protocol analyzer. (Vlad Grigorescu)
Activity gets logged into sip.log. It generates the following
events:
event sip_request(c: connection, method: string, original_URI: string, version: string);
event sip_reply(c: connection, version: string, code: count, reason: string);
event sip_header(c: connection, is_orig: bool, name: string, value: string);
event sip_all_headers(c: connection, is_orig: bool, hlist: mime_header_list);
event sip_begin_entity(c: connection, is_orig: bool);
event sip_end_entity(c: connection, is_orig: bool);
The analyzer support SIP over UDP currently.
* BIT-1343: Factor common ASN.1 code from RDP, SNMP, and Kerberos
analyzers. (Jon Siwek/Robin Sommer)
2.3-838 | 2015-04-21 13:40:12 -0700
* BIT-1373: Fix vector index assignment reference count bug. (Jon Siwek)
2.3-836 | 2015-04-21 13:37:31 -0700
* Fix SSH direction field being unset. Addresses BIT-1365. (Vlad
Grigorescu)
2.3-835 | 2015-04-21 16:36:00 -0500
* Clarify Broker examples. (Jon Siwek)
2.3-833 | 2015-04-21 12:38:32 -0700
* A Kerberos protocol analyzer. (Vlad Grigorescu)
Activity gets logged into kerberos.log. It generates the following
events:
event krb_as_request(c: connection, msg: KRB::KDC_Request);
event krb_as_response(c: connection, msg: KRB::KDC_Response);
event krb_tgs_request(c: connection, msg: KRB::KDC_Request);
event krb_tgs_response(c: connection, msg: KRB::KDC_Response);
event krb_ap_request(c: connection, ticket: KRB::Ticket, opts: KRB::AP_Options);
event krb_priv(c: connection, is_orig: bool);
event krb_safe(c: connection, is_orig: bool, msg: KRB::SAFE_Msg);
event krb_cred(c: connection, is_orig: bool, tickets: KRB::Ticket_Vector);
event krb_error(c: connection, msg: KRB::Error_Msg);
2.3-793 | 2015-04-20 20:51:00 -0700
* Add decoding of PROXY-AUTHORIZATION header to HTTP analyze,
treating it the same as AUTHORIZATION. (Josh Liburdi)
* Remove deprecated fields "hot" and "addl" from the connection
record. Remove the functions append_addl() and
append_addl_marker(). (Robin Sommer)
* Removing the NetFlow analyzer, which hasn't been used anymore
since then corresponding command-line option went away. (Robin
Sommer)
2.3-787 | 2015-04-20 19:15:23 -0700
* A file analyzer for Portable Executables. (Vlad Grigorescu/Seth
Hall).
Activity gets logged into pe.log. It generates the following
events:
event pe_dos_header(f: fa_file, h: PE::DOSHeader);
event pe_dos_code(f: fa_file, code: string);
event pe_file_header(f: fa_file, h: PE::FileHeader);
event pe_optional_header(f: fa_file, h: PE::OptionalHeader);
event pe_section_header(f: fa_file, h: PE::SectionHeader);
2.3-741 | 2015-04-20 13:12:39 -0700
* API changes to file analysis mime type detection. Removed
"file_mime_type" and "file_mime_types" event, replacing them with
a new event called "file_metadata_inferred". Addresses BIT-1368.
(Jon Siwek)
* A large series of improvements for file type identification. This
inludes a many signature updates (new types, cleanup, performance
improvments) and splitting out signatures into subfiles. (Seth
Hall)
* Fix an issue with files having gaps before the bof_buffer is
filled, which could lead to file type identification not working
correctly. (Seth Hall)
* Fix an issue with packet loss in HTTP file reporting for file type
identification wasn't working correctly zero-length bodies. (Seth
Hall)
* X.509 certificates are now populating files.log with the mime type
application/pkix-cert. (Seth Hall)
* Normalized some FILE_ANALYSIS debug messages. (Seth Hall)
2.3-725 | 2015-04-20 12:54:54 -0700
* Updating submodule(s).
2.3-724 | 2015-04-20 14:11:02 -0500
* Fix uninitialized field in raw input reader. (Jon Siwek)
2.3-722 | 2015-04-20 12:59:03 -0500
* Remove unneeded documentation cross-referencing. (Jon Siwek)
2.3-721 | 2015-04-20 12:47:05 -0500
* BIT-1380: Improve Broxygen output of &default expressions.
(Jon Siwek)
2.3-720 | 2015-04-17 14:18:26 -0700
* Updating NEWS.
2.3-716 | 2015-04-17 13:06:37 -0700
* Add seeking functionality to raw reader. One can now add an option
"offset" to the config map. Positive offsets are interpreted to be
from the beginning of the file, negative from the end of the file
(-1 is end of file). Only works for raw reader in streaming or
manual mode. Does not work with executables. Addresses BIT-985.
(Johanna Amann)
* Allow setting packet and byte thresholds for connections. (Johanna Amann)
This extends the ConnSize analyzer to be able to raise events when
each direction of a connection crosses a certain amount of bytes
or packets.
Thresholds are set using:
- set_conn_bytes_threshold(c$id, [num-bytes], [direction]);
- set_conn_packets_threshold(c$id, [num-packets], [direction]);
They raise the events, respectively:
- event conn_bytes_threshold_crossed(c: connection, threshold: count, is_orig: bool)
- event conn_packets_threshold_crossed(c: connection, threshold: count, is_orig: bool)
Current thresholds can be examined using get_conn_bytes_threshold()
and get_conn_packets_threshold().
Only one threshold can be set per connection.
* Add high-level API for packet/bytes thresholding in
base/protocols/conn/thresholds.bro that holds lists of thresholds
and raises an event for each threshold exactly once. (Johanna
Amann)
* Fix a bug where child packet analyzers of the TCP analyzer
where not found using FindChild.
* Update GridFTP analyzer to use connection thresholding instead of
polling. (Johanna Amann)
2.3-709 | 2015-04-17 12:37:32 -0700
* Fix addressing the dreaded "internal error: unknown msg type 115
in Poll()". (Jon Siwek)
This patch removes the error handling code for overload conditions
in the main process that could cause trouble down the road. The
"chunked_io_buffer_soft_cap" script variable can now tune when the
client process begins shutting down peer connections, and the
default setting is now double what it used to be. Addresses
BIT-1376.
2.3-707 | 2015-04-17 10:57:59 -0500
* Add more info about Broker to NEWS. (Jon Siwek)
2.3-705 | 2015-04-16 08:16:45 -0700
* Update Mozilla CA list. (Johanna Amann)
* Update tests to have them keep using older certificates where
appropiate. (Johanna Amann)
2.3-699 | 2015-04-16 09:51:58 -0500
* Fix the to_count function to use strtoull versus strtoll.
(Jon Siwek)
2.3-697 | 2015-04-15 09:51:15 -0700
* Removing error check verifying that an ASCII writer has been
properly finished. Instead of aborting, we now just clean up in
that case and proceed. Addresses BIT-1331. (Robin Sommer)
2.3-696 | 2015-04-14 15:56:36 -0700
* Update sqlite to 3.8.9
2.3-695 | 2015-04-13 10:34:42 -0500
* Fix iterator invalidation in broker::Manager dtor. (Jon Siwek)
* Add paragraph to plugin documentation. (Robin Sommer)
2.3-693 | 2015-04-11 10:56:31 -0700
* BIT-1367: improve coercion of anonymous records in set constructor.
(Jon Siwek)
* Allow to specify ports for sftp log rotator. (Johanna Amann)
2.3-690 | 2015-04-10 21:51:10 -0700
* Make sure to always delete the remote serializer. Addresses
BIT-1306 and probably also BIT-1356. (Robin Sommer)
* Cleaning up --help. -D and -Y/y were still listed, even though
they had no effect anymore. Removing some dead code along with -D.
Addresses BIT-1372. (Robin Sommer)
2.3-688 | 2015-04-10 08:10:44 -0700
* Update SQLite to 3.8.8.3.
2.3-687 | 2015-04-10 07:32:52 -0700
* Remove stale signature benchmarking code (-L command-line option).
(Jon Siwek)
* BIT-844: fix UDP payload signatures to match packet-wise. (Jon
Siwek)
2.3-682 | 2015-04-09 12:07:00 -0700
* Fixing input readers' component type. (Robin Sommer)
* Tiny spelling correction. (Seth Hall)
2.3-680 | 2015-04-06 16:02:43 -0500
* BIT-1371: remove CMake version check from binary package scripts.
(Jon Siwek)
2.3-679 | 2015-04-06 10:16:36 -0500
* Increase some unit test timeouts. (Jon Siwek)
* Fix Coverity warning in RDP analyzer. (Jon Siwek)
2.3-676 | 2015-04-02 10:10:39 -0500
* BIT-1366: improve checksum offloading warning.
(Frank Meier, Jon Siwek)
2.3-675 | 2015-03-30 17:05:05 -0500
* Add an RDP analyzer. (Josh Liburdi, Seth Hall, Johanna Amann)
2.3-640 | 2015-03-30 13:51:51 -0500
* BIT-1359: Limit maximum number of DTLS fragments to 30. (Johanna Amann)
2.3-637 | 2015-03-30 12:02:07 -0500
* Increase timeout duration in some broker tests. (Jon Siwek)
2.3-636 | 2015-03-30 11:26:32 -0500
* Updates related to SSH analysis. (Jon Siwek)
- Some scripts used wrong SSH module/namespace scoping on events.
- Fix outdated notice documentation related to SSH password guessing.
- Add a unit test for SSH pasword guessing notice.
2.3-635 | 2015-03-30 11:02:45 -0500
* Fix outdated documentation unit tests. (Jon Siwek)
2.3-634 | 2015-03-30 10:22:45 -0500
* Add a canonifier to a unit test's output. (Jon Siwek)
2.3-633 | 2015-03-25 18:32:59 -0700
* Log::write in signature framework was missing timestamp.
(Andrew Benson/Michel Laterman)
2.3-631 | 2015-03-25 11:03:12 -0700
* New SSH analyzer. (Vlad Grigorescu)
2.3-600 | 2015-03-25 10:23:46 -0700
* Add defensive checks in code to calculate log rotation intervals.
(Pete Nelson).
2.3-597 | 2015-03-23 12:50:04 -0700
* DTLS analyzer. (Johanna Amann)
* Implement correct parsing of TLS record fragmentation. (Johanna
Amann)
2.3-582 | 2015-03-23 11:34:25 -0700
* BIT-1313: In debug builds, "bro -B <x>" now supports "all" and
"help" for "<x>". "all" enables all debug streams. "help" prints a
list of available debug streams. (John Donnelly/Robin Sommer).
* BIT-1324: Allow logging filters to inherit default path from
stream. This allows the path for the default filter to be
specified explicitly through $path="..." when creating a stream.
Adapted the existing Log::create_stream calls to explicitly
specify a path value. (Jon Siwek)
* BIT-1199: Change the way the input framework deals with values it
cannot convert into BroVals, raising error messages instead of
aborting execution. (Johanna Amann)
* BIT-788: Use DNS QR field to better identify flow direction. (Jon
Siwek)
2.3-572 | 2015-03-23 13:04:53 -0500
* BIT-1226: Fix an example in quickstart docs. (Jon siwek)
2.3-570 | 2015-03-23 09:51:20 -0500
* Correct a spelling error (Daniel Thayer)
* Improvement to SSL analyzer failure mode. (Johanna Amann)
2.3-565 | 2015-03-20 16:27:41 -0500
* BIT-978: Improve documentation of 'for' loop iterator invalidation.
(Jon Siwek)
2.3-564 | 2015-03-20 11:12:02 -0500
* BIT-725: Remove "unmatched_HTTP_reply" weird. (Jon Siwek)
2.3-562 | 2015-03-20 10:31:02 -0500
* BIT-1207: Add unit test to catch breaking changes to local.bro
(Jon Siwek)
* Fix failing sqlite leak test (Johanna Amann)
2.3-560 | 2015-03-19 13:17:39 -0500
* BIT-1255: Increase default values of
"tcp_max_above_hole_without_any_acks" and "tcp_max_initial_window"
from 4096 to 16384 bytes. (Jon Siwek)
2.3-559 | 2015-03-19 12:14:33 -0500
* BIT-849: turn SMTP reporter warnings into weirds,
"smtp_nested_mail_transaction" and "smtp_unmatched_end_of_data".
(Jon Siwek)
2.3-558 | 2015-03-18 22:50:55 -0400
* DNS: Log the type number for the DNS_RR_unknown_type weird. (Vlad Grigorescu)
2.3-555 | 2015-03-17 15:57:13 -0700
* Splitting test-all Makefile target into Bro tests and test-aux.
(Robin Sommer)
2.3-554 | 2015-03-17 15:40:39 -0700
* Deprecate &rotate_interval, &rotate_size, &encrypt. Addresses
BIT-1305. (Jon Siwek)
2.3-549 | 2015-03-17 09:12:18 -0700
* BIT-1077: Fix HTTP::log_server_header_names. Before, it just
re-logged fields from the client side. (Jon Siwek)
2.3-547 | 2015-03-17 09:07:51 -0700
* Update certificate validation script to cache valid intermediate
chains that it encounters on the wire and use those to try to
validate chains that might be missing intermediate certificates.
(Johanna Amann)
2.3-541 | 2015-03-13 15:44:08 -0500
* Make INSTALL a symlink to doc/install/install.rst (Jon siwek)
* Fix Broxygen coverage. (Jon Siwek)
2.3-539 | 2015-03-13 14:19:27 -0500
* BIT-1335: Include timestamp in default extracted file names.
And add a policy script to extract all files. (Jon Siwek)
* BIT-1311: Identify GRE tunnels as Tunnel::GRE, not Tunnel::IP.
(Jon Siwek)
* BIT-1309: Add Connection class getter methods for flow labels.
(Jon Siwek)
2.3-536 | 2015-03-12 16:16:24 -0500
* Fix Broker leak tests. (Jon Siwek)
2.3-534 | 2015-03-12 10:59:49 -0500
* Update NEWS file. (Jon Siwek)
2.3-533 | 2015-03-12 10:18:53 -0500
* Give broker python bindings default install path within --prefix.
(Jon Siwek)
2.3-530 | 2015-03-10 13:22:39 -0500
* Fix broker data stores in absence of --enable-debug. (Jon Siwek)
2.3-529 | 2015-03-09 13:14:27 -0500
* Fix format specifier in SSL protocol violation. (Jon Siwek)
2.3-526 | 2015-03-06 12:48:49 -0600
* Fix build warnings, clarify broker requirements, update submodule.
(Jon Siwek)
* Rename comm/ directories to broker/ (Jon Siwek)
* Rename broker-related namespaces. (Jon Siwek)
* Improve remote logging via broker by only sending fields w/ &log.
(Jon Siwek)
* Disable a stream's remote logging via broker if it fails. (Jon Siwek)
* Improve some broker communication unit tests. (Jon Siwek)
2.3-518 | 2015-03-04 13:13:50 -0800
* Add bytes_recvd to stats.log recording the number of bytes
received, according to packet headers. (Mike Smiley)
2.3-516 | 2015-03-04 12:30:06 -0800
* Extract most specific Common Name from SSL certificates (Johanna
Amann)
* Send CN and SAN fields of SSL certificates to the Intel framework.
(Johanna Amann)
2.3-511 | 2015-03-02 18:07:17 -0800
* Changes to plugin meta hooks for function calls. (Gilbert Clark)
- Add frame argument.
- Change return value to tuple unambigiously whether hook
returned a result.
2.3-493 | 2015-03-02 17:17:32 -0800
* Extend the SSL weak-keys policy file to also alert when
encountering SSL connections with old versions as well as unsafe
cipher suites. (Johanna Amann)
* Make the notice suppression handling of other SSL policy files a
tad more robust. (Johanna Amann)
2.3-491 | 2015-03-02 17:12:56 -0800
* Updating docs for recent addition of local_resp. (Robin Sommer)
2.3-489 | 2015-03-02 15:29:30 -0800
* Integrate Broker, Bro's new communication library. (Jon Siwek)
See aux/broker/README for more information on Broker, and
doc/frameworks/comm.rst for the corresponding Bro script API.
Broker support is by default off for now; it can be enabled at
configure time with --enable-broker. It requires CAF
(https://github.com/actor-framework/actor-framework); for now iot
needs CAF's "develop" branch. Broker also requires a C++11
compiler.
Broker will become a mandatory dependency in future Bro versions.
* Add --enable-c++11 configure flag to compile Bro's source code in
C++11 mode with a corresponding compiler. (Jon Siwek)
2.3-451 | 2015-02-24 16:37:08 -0800
* Updating submodule(s).
2.3-448 | 2015-02-23 16:58:10 -0800
* Updating NEWS. (Robin Sommer)
2.3-447 | 2015-02-23 16:28:30 -0800
* Fix potential crash in logging framework when deserializing
WriterInfo from remote. where config is present. Testcase crashes
on unpatched versions of Bro. (Aaron Eppert)
* Fix wrong value test in WriterBackend. (Aaron Eppert)
2.3-442 | 2015-02-23 13:29:30 -0800
* Add a "local_resp" field to conn.log, along the lines of the
existing "local_orig". (Mike Smiley)
2.3-440 | 2015-02-23 11:39:17 -0600
* Updating plugin docs to recent changes. (Robin Sommer)
* Updating plugin tests to recent changes. (Robin Sommer)
* Making plugin names case-insensitive for some internal comparisions.
Makes plugin system more tolerant against spelling inconsistencies
are hard to catch otherwise. (Robin Sommer)
* Explicitly removing some old scripts on install that have moved
into plugins to prevent them causing confusion. (Robin Sommer)
* BIT-1312: Removing setting installation plugin path from
bro-path-dev.sh. Also, adding to existing BRO_PLUGIN_PATH rather
than replacing. (Robin Sommer)
* Creating the installation directory for plugins at install time.
(Robin Sommer)
2.3-427 | 2015-02-20 13:49:33 -0800
* Removing dependency on PCAP_NETMASK_UNKNOWN to compile with
libpcap < 1.1.1. (Robin Sommer)
2.3-426 | 2015-02-20 12:45:51 -0800
* Add 'while' statement to Bro language. Really. (Jon Siwek)
2.3-424 | 2015-02-20 12:39:10 -0800
* Add the ability to remove surrounding braces from the JSON
formatter. (Seth Hall)
2.3-419 | 2015-02-13 09:10:44 -0600
* BIT-1011: Update the SOCKS analyzer to support user/pass login.
(Nicolas Retrain, Seth Hall, Jon Siwek)
- Add a new field to socks.log: "password".
- Two new events: "socks_login_userpass_request" and
"socks_login_userpass_reply".
- Two new weirds for unsupported SOCKS authentication method or
version.
- A new test for authenticated socks traffic.
2.3-416 | 2015-02-12 12:18:42 -0600
* Submodule update - newest sqlite version (Johanna Amann)
* Fix use of deprecated gperftools headers. (Jon Siwek)
2.3-413 | 2015-02-08 18:23:05 -0800
* Fixing analyzer tag types for some Files::* functions. (Robin Sommer)
* Changing load order for plugin scripts. (Robin Sommer)
2.3-411 | 2015-02-05 10:05:48 -0600
* Fix file analysis of files with total size below the bof_buffer size
never delivering content to stream analyzers. (Seth Hall)
* Add/fix log fields in x509 diff canonifier. (Jon Siwek)
* "id" not defined for debug code when using -DPROFILE_BRO_FUNCTIONS
(Mike Smiley)
2.3-406 | 2015-02-03 17:02:45 -0600
* Add x509 canonifier to a unit test. (Jon Siwek)
2.3-405 | 2015-02-02 11:14:24 -0600
* Fix memory leak in new split_string* functions. (Jon Siwek)
2.3-404 | 2015-01-30 14:23:27 -0800
* Update documentation (broken links, outdated tests). (Jon Siwek)
* Deprecate split* family of BIFs. (Jon Siwek)
These functions are now deprecated in favor of alternative versions that
return a vector of strings rather than a table of strings.
Deprecated functions:
- split: use split_string instead.
- split1: use split_string1 instead.
- split_all: use split_string_all instead.
- split_n: use split_string_n instead.
- cat_string_array: see join_string_vec instead.
- cat_string_array_n: see join_string_vec instead.
- join_string_array: see join_string_vec instead.
- sort_string_array: use sort instead instead.
- find_ip_addresses: use extract_ip_addresses instead.
Changed functions:
- has_valid_octets: uses a string_vec parameter instead of string_array.
Addresses BIT-924.
* Add a new attribute: &deprecated. While scripts are parsed, a
warning is raised for each usage of an identifier marked as
&deprecated. This also works for BIFs. Addresses BIT-924,
BIT-757. (Jon Siwek)
2.3-397 | 2015-01-27 10:13:10 -0600
* Handle guess_lexer exceptions in pygments reST directive (Jon Siwek)
2.3-396 | 2015-01-23 10:49:15 -0600
* DNP3: fix reachable assertion and buffer over-read/overflow.
CVE number pending. (Travis Emmert, Jon Siwek)
* Update binpac: Fix potential out-of-bounds memory reads in generated
code. CVE-2014-9586. (John Villamil and Chris Rohlf - Yahoo
Paranoids, Jon Siwek)
* Fixing (harmless) Coverity warning. (Robin Sommer)
2.3-392 | 2015-01-15 09:44:15 -0800
* Small changes to EC curve names in a newer draft. (Johanna Amann)
2.3-390 | 2015-01-14 13:27:34 -0800
* Updating MySQL analyses. (Vlad Grigorescu)
- Use a boolean success instead of a result string.
- Change the affected_rows response detail string to a "rows" count.
- Fix the state tracking to log incomplete command.
* Extend DNP3 to support communication over UDP. (Hui Lin)
* Fix a bug in DNP3 determining the length of an object in some
cases. (Hui Lin)
2.3-376 | 2015-01-12 09:38:10 -0600
* Improve documentation for connection_established event. (Jon Siwek)
2.3-375 | 2015-01-08 13:10:09 -0600
* Increase minimum required CMake version to 2.8. (Jon Siwek)
2.3-374 | 2015-01-07 10:03:17 -0600
* Improve documentation of the Intelligence Framework. (Daniel Thayer)
2.3-371 | 2015-01-06 09:58:09 -0600
* Update/improve file mime type identification. (Seth Hall)
- Change to the default BOF buffer size to 3000 (was 1024).
- Reorganized MS signatures into a separate file.
- Remove all of the x-c detections. Nearly all false positives.
- Improve TAR detections, removing old, back up TAR detections.
- Remove one of the x-elc detections that was too loose
and caused many false positives.
- Improved lots of the signatures and added new ones. (Seth Hall)
* Add support for file reassembly in the file analysis framework
(Seth Hall, Jon Siwek).
- The reassembly behavior can be modified per-file by enabling or
disabling the reassembler and/or modifying the size of the
reassembly buffer.
- Changed the file extraction analyzer to use stream-wise input to
avoid issues with the chunk-wise approach not immediately
triggering the file_new event due to mime-type detection delay.
Before, early chunks frequently ended up lost. Extraction also
will now explicitly NUL-fill gaps in the file instead of
implicitly relying on pwrite to do it.
2.3-349 | 2015-01-05 15:21:13 -0600
* Fix race condition in unified2 file analyzer startup. (Jon siwek)
2.3-348 | 2014-12-31 09:19:34 -0800
* Changing Makefile's test-all to run test-all for broctl, which now
executes trace-summary tests as well. (Robin Sommer)
2.3-345 | 2014-12-31 09:06:15 -0800
* Correct a typo in the Notice framework doc. (Daniel Thayer)
2.3-343 | 2014-12-12 12:43:46 -0800
* Fix PIA packet replay to deliver copy of IP header. This prevented
one from writing a packet-wise analyzer that needs access to IP
headers and can be attached to a connection via signature match.
Addresses BIT-1298 (Jon Siwek)
2.3-338 | 2014-12-08 13:56:19 -0800
* Add man page for Bro. (Raúl Benencia)
* Updating doc baselines. (Robin Sommer)
2.3-334 | 2014-12-03 14:22:07 -0800
* Fix compound assignment to require proper L-value. Addresses
BIT-1295. (Jon Siwek)
2.3-332 | 2014-12-03 14:14:11 -0800
* Make using local IDs in @if directives an error. Addresses
BIT-1296. (Jon Siwek)
2.3-330 | 2014-12-03 14:10:39 -0800
* Fix some "make doc" warnings and update some doc tests. (Daniel
Thayer)
2.3-328 | 2014-12-02 08:13:10 -0500
* Update windows-version-detection.bro to add support for
Windows 10. (Michal Purzynski)
2.3-326 | 2014-12-01 12:10:27 -0600
* BIFScanner: fix invalid characters in generated preprocessor macros.
(Hilko Bengen)
* BIT-1294: fix exec.bro from mutating Input::end_of_data event
parameters. (Johanna Amann)
* Add/invoke "distclean" for testing directories. (Raúl Benencia)
* Delete prebuilt python bytecode files from git. (Jon Siwek)
* Add Windows detection based on CryptoAPI HTTP traffic as a software
framework policy script. (Vlad Grigorescu)
2.3-316 | 2014-11-25 17:35:06 -0800
* Make the SSL analyzer skip further processing once encountering
situations which are very probably non-recoverable. (Johanna
Amann)
2.3-313 | 2014-11-25 14:27:07 -0800
* Make SSL v2 protocol tests more strict. In its former state they
triggered on http traffic over port 443 sometimes. Found by Michał
Purzyński. (Johanna Amann)
* Fix X509 analyzer to correctly return ECDSA as the key_type for
ECDSA certs. Bug found by Michał Purzyński. (Johanna Amann)
2.3-310 | 2014-11-19 10:56:59 -0600
* Disable verbose bison output. (Jon Siwek)
2.3-309 | 2014-11-18 12:17:53 -0800
* New decompose_uri() function in base/utils/urls that splits a URI
into its pieces. (Anthony Kasza).
2.3-305 | 2014-11-18 11:09:04 -0800
* Improve coercion of &default expressions. Addresses BIT-1288. (Jon
Siwek)
2.3-303 | 2014-11-18 10:53:04 -0800
* For DH key exchanges, use p as the parameter for weak key
exchanges. (Johanna Amann)
2.3-301 | 2014-11-11 13:47:27 -0800
* Add builtin function enum_to_int() that converts an enum into a
integer. (Christian Struck)
2.3-297 | 2014-11-11 11:50:47 -0800
* Removing method from SSL analyzer that's no longer used. (Robin
Sommer)
2.3-296 | 2014-11-11 11:42:38 -0800
* A new analyzer parsing the MySQL wire protocol. Activity gets
logged into mysql.log. Supports protocol versions 9 and 10. (Vlad
Grigorescu)
2.3-280 | 2014-11-05 09:46:33 -0500
* Add Windows detection based on CryptoAPI HTTP traffic as a
software framework policy script. (Vlad Grigorescu)
2.3-278 | 2014-11-03 18:55:18 -0800
* Add new curves from draft-ietf-tls-negotiated-ff-dhe to SSL
analysis. (Johanna Amann)
2.3-274 | 2014-10-31 17:45:25 -0700
* Adding call to new binpac::init() function. (Robin Sommer)
2.3-272 | 2014-10-31 16:29:42 -0700
* Fix segfault if when statement's RHS is unitialized. Addresses
BIT-1176. (Jon Siwek)
* Fix checking vector indices via "in". Addresses BIT-1280. (Jon
Siwek)
2.3-268 | 2014-10-31 12:12:22 -0500
* BIT-1283: Fix crash when using &encrypt. (Jon Siwek)
2.3-267 | 2014-10-31 10:35:02 -0500
* BIT-1284: Allow arbitrary when statement timeout expressions
(Jon Siwek)
2.3-266 | 2014-10-31 09:21:28 -0500
* BIT-1166: Add configure options to fine tune local state dirs used
by BroControl. (Jon Siwek)
2.3-264 | 2014-10-30 13:25:57 -0500
* Fix some minor Coverity Scan complaints. (Jon Siwek)
2.3-263 | 2014-10-28 15:09:10 -0500
* Fix checking of fwrite return values (Johanna Amann)
2.3-260 | 2014-10-27 12:54:17 -0500
* Fix errors/warnings when compiling with -std=c++11 (Jon Siwek)
2.3-259 | 2014-10-27 10:04:04 -0500
* Documentation fixes. (Vicente Jimenez Aguilar and Stefano Azzalini)
2.3-256 | 2014-10-24 15:33:45 -0700
* Adding missing test baseline. (Robin Sommer)
2.3-255 | 2014-10-24 13:39:44 -0700
* Fixing unstable active-http test. (Robin Sommer)
2.3-254 | 2014-10-24 11:40:51 -0700
* Fix active-http.bro to deal reliably with empty server responses,
which will now be passed back as empty files. (Christian Struck)
2.3-248 | 2014-10-23 14:20:59 -0700
* Change order in which a plugin's scripts are loaded at startup.
(Robin Sommer)
2.3-247 | 2014-10-21 13:42:38 -0700
* Updates to the SSL analyzer. (Johanna Amann)
* Mark everything below 2048 bit as a weak key.
* Fix notice suppression.
* Add information about server-chosen protocol to ssl.log, if
provided by application_layer_next_protocol.
* Add boolean flag to ssl.log signaling if a session was
resumed. Remove the (usually not really that useful) session
ID that the client sent.
2.3-240 | 2014-10-21 13:36:33 -0700
* Fix Coverity-reported issues in DNP3 analyzer. (Seth Hall)
2.3-238 | 2014-10-16 06:51:49 -0700
* Fix multipart HTTP/MIME entity file analysis so that (1) singular
CR or LF characters in multipart body content are no longer
converted to a full CRLF (thus corrupting the file) and (2) it
also no longer considers the CRLF before the multipart boundary as
part of the content. Addresses BIT-1235. (Jon Siwek)
2.3-235 | 2014-10-15 10:20:47 -0500
* BIT-1273: Add error message for bad enum declaration syntax.
(Jon Siwek)
2.3-234 | 2014-10-14 14:42:09 -0500
* Documentation fixes. (Steve Smoot)
2.3-233 | 2014-10-09 16:00:27 -0500
* Change find-bro-logs unit test to follow symlinks. (Jon Siwek)
* Add error checks and messages to a test script (Daniel Thayer)
2.3-230 | 2014-10-08 08:15:17 -0700
* Further baseline normalization for plugin test portability. (Robin
Sommer)
2.3-229 | 2014-10-07 20:18:11 -0700
* Fix for test portability. (Robin Sommer)
2.3-228 | 2014-10-07 15:32:37 -0700
* Include plugin unit tests into the top-level btest configuration. (Robin Sommer)
* Switching the prefix separator for packet source/dumper plugins
once more, now to "::". Addresses BIT-1267. (Robin Sommer)
* Fix for allowing a packet source/dumper plugin to support multiple
prefixes with a colon. (Robin Sommer)
2.3-225 | 2014-10-07 15:13:35 -0700
* Updating plugin documentation. (Robin Sommer)
2.3-224 | 2014-10-07 14:32:17 -0700
* Improved the log file reference documentation. (Jeannette Dopheide
and Daniel Thayer)
* Improves shockwave flash file signatures. (Seth Hall)
- This moves the signatures out of the libmagic imported signatures
and into our own general.sig.
- Expand the detection to LZMA compressed flash files.
* Add new script language reference documentation on operators,
statements, and directives. Also improved the documentation on
types and attributes by splitting them into two docs, and
providing more examples and adding a chart on the top of each page
with links to each type and attribute for easier access to the
information. (Daniel Thayer)
* Split the types and attributes reference doc into two docs.
(Daniel Thayer)
2.3-208 | 2014-10-03 09:38:52 -0500
* BIT-1268: Fix uninitialized router_list argument in
dhcp_offer/dhcp_ack. (Jon Siwek)
2.3-207 | 2014-10-02 16:39:17 -0700
* Updating plugin docs. (Robin Sommer)
* Fix packet sources being treated as idle when a packet is
available. Addresses BIT-1266. (Jon Siwek)
* Fix regression causing the main loop to spin more frequently.
Addresses BIT-1266. (Jon Siwek)
2.3-203 | 2014-09-29 20:06:54 -0700
* Fix to use length parameter in DNP3 time conversion correctly now.
(Robin Sommer)
2.3-202 | 2014-09-29 17:05:18 -0700
* New SSL extension type from IANA and a few other SSL const
changes. (Johanna Amann)
* Make unexpected pipe errors fatal as precaution. Addresses
BIT-1260. (Jon Siwek)
* Adding a function for DNP3 to translate the timestamp format. (Hui
Lin)
2.3-197 | 2014-09-29 10:42:01 -0500
* Fix possible seg fault in TCP reassembler. (Jon Siwek)
2.3-196 | 2014-09-25 17:53:27 -0700
* Changing prefix for packet sources/dumper from ':' to '%'.
Addresses BIT-1249. (Robin Sommer)
* Remove timeouts from remote communication loop. The select() now
blocks until there's work to do instead of relying on a small
timeout value which can cause unproductive use of cpu cycles. (Jon
Siwek)
* Improve error message when failing to activate a plugin. Also fix
a unit test helper script that checks plugin availability. (Jon
Siwek)
2.3-183 | 2014-09-24 10:08:04 -0500
* Add a "node" field to Intel::Seen struture and intel.log to
indicate which node discovered a hit on an intel item. (Seth Hall)
* BIT-1261: Fixes to plugin quick start doc. (Jon Siwek)
2.3-180 | 2014-09-22 12:52:41 -0500
* BIT-1259: Fix issue w/ duplicate TCP reassembly deliveries.
(Jon Siwek)
2.3-178 | 2014-09-18 14:29:46 -0500
* BIT-1256: Fix file analysis events from coming after bro_done().
(Jon Siwek)
2.3-177 | 2014-09-17 09:41:27 -0500
* Documentation fixes. (Chris Mavrakis)
2.3-174 | 2014-09-17 09:37:09 -0500
* Fixed some "make doc" warnings caused by reST formatting
(Daniel Thayer).
2.3-172 | 2014-09-15 13:38:52 -0500
* Remove unneeded allocations for HTTP messages. (Jon Siwek)
2.3-171 | 2014-09-15 11:14:57 -0500
* Fix a compile error on systems without pcap-int.h. (Jon Siwek)
2.3-170 | 2014-09-12 19:28:01 -0700
* Fix incorrect data delivery skips after gap in HTTP Content-Range.
Addresses BIT-1247. (Jon Siwek)
* Fix file analysis placement of data after gap in HTTP
Content-Range. Addresses BIT-1248. (Jon Siwek)
* Fix issue w/ TCP reassembler not delivering some segments.
Addresses BIT-1246. (Jon Siwek)
* Fix MIME entity file data/gap ordering and raise http_entity_data
in line with data arrival. Addresses BIT-1240. (Jon Siwek)
* Implement file ID caching for MIME_Mail. (Jon Siwek)
* Fix a compile error. (Jon Siwek)
2.3-161 | 2014-09-09 12:35:38 -0500
* Bugfixes and test updates/additions. (Robin Sommer)
* Interface tweaks and docs for PktSrc/PktDumper. (Robin Sommer)
* Moving PCAP-related bifs to iosource/pcap.bif. (Robin Sommer)
* Moving some of the BPF filtering code into base class.
This will allow packet sources that don't support BPF natively to
emulate the filtering via libpcap. (Robin Sommer)
* Removing FlowSrc. (Robin Sommer)
* Removing remaining pieces of the 2ndary path, and left-over
files of packet sorter. (Robin Sommer)
* A bunch of infrastructure work to move IOSource, IOSourceRegistry
(now iosource::Manager) and PktSrc/PktDumper code into iosource/,
and over to a plugin structure. (Robin Sommer)
2.3-137 | 2014-09-08 19:01:13 -0500
* Fix Broxygen's rendering of opaque types. (Jon Siwek)
2.3-136 | 2014-09-07 20:50:46 -0700
* Change more http links to https. (Johanna Amann)
2.3-134 | 2014-09-04 16:16:36 -0700
* Fixed a number of issues with OCSP reply validation. Addresses
BIT-1212. (Johanna Amann)
* Fix null pointer dereference in OCSP verification code in case no
certificate is sent as part as the ocsp reply. Addresses BIT-1212.
(Johanna Amann)
2.3-131 | 2014-09-04 16:10:32 -0700
* Make links in documentation templates protocol relative. (Johanna
Amann)
2.3-129 | 2014-09-02 17:21:21 -0700
* Simplify a conditional with equivalent branches. (Jon Siwek)
* Change EDNS parsing code to use rdlength more cautiously. (Jon
Siwek)
* Fix a memory leak when bind() fails due to EADDRINUSE. (Jon Siwek)
* Fix possible buffer over-read in DNS TSIG parsing. (Jon Siwek)
2.3-124 | 2014-08-26 09:24:19 -0500
* Better documentation for sub_bytes (Jimmy Jones)
* BIT-1234: Fix build on systems that already have ntohll/htonll
(Jon Siwek)
2.3-121 | 2014-08-22 15:22:15 -0700
* Detect functions that try to bind variables from an outer scope
and raise an error saying that's not supported. Addresses
BIT-1233. (Jon Siwek)
2.3-116 | 2014-08-21 16:04:13 -0500
* Adding plugin testing to Makefile's test-all. (Robin Sommer)
* Converting log writers and input readers to plugins.
DataSeries and ElasticSearch plugins have moved to the new
bro-plugins repository, which is now a git submodule in the
aux/plugins directory. (Robin Sommer)
2.3-98 | 2014-08-19 11:03:46 -0500
* Silence some doc-related warnings when using `bro -e`.
Closes BIT-1232. (Jon Siwek)
* Fix possible null ptr derefs reported by Coverity. (Jon Siwek)
2.3-96 | 2014-08-01 14:35:01 -0700
* Small change to DHCP documentation. In server->client messages the
host name may differ from the one requested by the client.
(Johanna Amann)
* Split DHCP log writing from record creation. This allows users to
customize dhcp.log by changing the record in their own dhcp_ack
event. (Johanna Amann)
* Update PATH so that documentation btests can find bro-cut. (Daniel
Thayer)
* Remove gawk from list of optional packages in documentation.
(Daniel Thayer)
* Fix for redefining built-in constants. (Robin Sommer)
2.3-86 | 2014-07-31 14:19:58 -0700
* Fix for redefining built-in constants. (Robin Sommer)
* Adding missing check that a plugin's API version matches what Bro
defines. (Robin Sommer)
* Adding NEWS entry for plugins. (Robin Sommer)
2.3-83 | 2014-07-30 16:26:11 -0500
* Minor adjustments to plugin code/docs. (Jon Siwek)
* Dynamic plugin support. (Rpbin Sommer)
Bro now supports extending core functionality, like protocol and
file analysis, dynamically with external plugins in the form of
shared libraries. See doc/devel/plugins.rst for an overview of the
main functionality. Changes coming with this:
- Replacing the old Plugin macro magic with a new API.
- The plugin API changed to generally use std::strings instead
of const char*.
- There are a number of invocations of PLUGIN_HOOK_
{VOID,WITH_RESULT} across the code base, which allow plugins
to hook into the processing at those locations.
- A few new accessor methods to various classes to allow
plugins to get to that information.
- network_time cannot be just assigned to anymore, there's now
function net_update_time() for that.
- Redoing how builtin variables are initialized, so that it
works for plugins as well. No more init_net_var(), but
instead bifcl-generated code that registers them.
- Various changes for adjusting to the now dynamic generation
of analyzer instances.
- same_type() gets an optional extra argument allowing record type
comparision to ignore if field names don't match. (Robin Sommer)
- Further unify file analysis API with the protocol analyzer API
(assigning IDs to analyzers; adding Init()/Done() methods;
adding subtypes). (Robin Sommer)
- A new command line option -Q that prints some basic execution
time stats. (Robin Sommer)
- Add support to the file analysis for activating analyzers by
MIME type. (Robin Sommer)
- File::register_for_mime_type(tag: Analyzer::Tag, mt:
string): Associates a file analyzer with a MIME type.
- File::add_analyzers_for_mime_type(f: fa_file, mtype:
string): Activates all analyzers registered for a MIME
type for the file.
- The default file_new() handler calls
File::add_analyzers_for_mime_type() with the file's MIME
type.
2.3-20 | 2014-07-22 17:41:02 -0700
* Updating submodule(s).
2.3-19 | 2014-07-22 17:29:19 -0700
* Implement bytestring_to_coils() in Modbus analyzer so that coils
gets passed to the corresponding events. (Hui Lin)
* Add length field to ModbusHeaders. (Hui Lin)
2.3-12 | 2014-07-10 19:17:37 -0500
* Include yield of vectors in Broxygen's type descriptions.
Addresses BIT-1217. (Jon Siwek)
2.3-11 | 2014-07-10 14:49:27 -0700
* Fixing DataSeries output. It was using a now illegal value as its
default compression level. (Robin Sommer)
2.3-7 | 2014-06-26 17:35:18 -0700
* Extending "make test-all" to include aux/bro-aux. (Robin Sommer)
2.3-6 | 2014-06-26 17:24:10 -0700
* DataSeries compilation issue fixed. (mlaterman)
* Fix a reference counting bug in ListVal ctor. (Jon Siwek)
2.3-3 | 2014-06-26 15:41:04 -0500
* Support tilde expansion when Bro tries to find its own path. (Jon
Siwek)
2.3-2 | 2014-06-23 16:54:15 -0500
* Remove references to line numbers in tutorial text. (Daniel Thayer)
2.3 | 2014-06-16 09:48:25 -0500
* Release 2.3.
2.3-beta-33 | 2014-06-12 11:59:28 -0500
* Documentation improvements/fixes. (Daniel Thayer)
2.3-beta-24 | 2014-06-11 15:35:31 -0500
* Fix SMTP state tracking when server response is missing.
(Robin Sommer)
2.3-beta-22 | 2014-06-11 12:31:38 -0500
* Fix doc/test that broke due to a Bro script change. (Jon Siwek)
* Remove unused --with-libmagic configure option. (Jon Siwek)
2.3-beta-20 | 2014-06-10 18:16:51 -0700
* Fix use-after-free in some cases of reassigning a table index.
Addresses BIT-1202. (Jon Siwek)
2.3-beta-18 | 2014-06-06 13:11:50 -0700
* Add two more SSL events, one triggered for each handshake message
and one triggered for the tls change cipherspec message. (Johanna
Amann)
* Small SSL bug fix. In case SSL::disable_analyzer_after_detection
was set to false, the ssl_established event would fire after each
data packet once the session is established. (Johanna Amann)
2.3-beta-16 | 2014-06-06 13:05:44 -0700
* Re-activate notice suppression for expiring certificates.
(Johanna Amann)
2.3-beta-14 | 2014-06-05 14:43:33 -0700
* Add new TLS extension type numbers from IANA (Johanna Amann)
* Switch to double hashing for Bloomfilters for better performance.
(Matthias Vallentin)
* Bugfix to use full digest length instead of just one byte for
Bloomfilter's universal hash function. Addresses BIT-1140.
(Matthias Vallentin)
* Make buffer for X509 certificate subjects larger. Addresses
BIT-1195 (Johanna Amann)
2.3-beta-5 | 2014-05-29 15:34:42 -0500
* Fix misc/load-balancing.bro's reference to
PacketFilter::sampling_filter (Jon Siwek)
2.3-beta-4 | 2014-05-28 14:55:24 -0500
* Fix potential mem leak in remote function/event unserialization.
(Jon Siwek)
* Fix reference counting bug in table coercion expressions (Jon Siwek)
* Fix an "unused value" warning. (Jon Siwek)
* Remove a duplicate unit test baseline dir. (Jon Siwek)
2.3-beta | 2014-05-19 16:36:50 -0500
* Release 2.3-beta
* Clean up OpenSSL data structures on exit. (Johanna Amann)
* Fixes for OCSP & x509 analysis memory leak issues. (Johanna Amann)
* Remove remaining references to BROMAGIC (Daniel Thayer)
* Fix typos and formatting in event and BiF documentation (Daniel Thayer)
* Update intel framework plugin for ssl server_name extension API
changes. (Johanna Amann, Justin Azoff)
* Fix expression errors in SSL/x509 scripts when unparseable data
is in certificate chain. (Johanna Amann)
2.2-478 | 2014-05-19 15:31:33 -0500
* Change record ctors to only allow record-field-assignment
expressions. (Jon Siwek)
2.2-477 | 2014-05-19 14:13:00 -0500
* Fix X509::Result record's "result" field to be set internally as type int instead of type count. (Johanna Amann)
* Fix a couple of doc build warnings (Daniel Thayer)
2.2-470 | 2014-05-16 15:16:32 -0700
* Add a new section "Cluster Configuration" to the docs that is
intended as a how-to for configuring a Bro cluster. Most of this
content was moved here from the BroControl doc (which is now
intended as more of a reference guide for more experienced users)
and the load balancing FAQ on the website. (Daniel Thayer)
* Update some doc tests and line numbers (Daniel Thayer)
2.2-457 | 2014-05-16 14:38:31 -0700
* New script policy/protocols/ssl/validate-ocsp.bro that adds OSCP
validation to ssl.log. The work is done by a new bif
x509_ocsp_verify(). (Johanna Amann)
* STARTTLS support for POP3 and SMTP. The SSL analyzer takes over
when seen. smtp.log now logs when a connection switches to SSL.
(Johanna Amann)
* Replace errors when parsing x509 certs with weirds. (Johanna
Amann)
* Improved Heartbleed attack/scan detection. (Johanna Amann)
* Let TLS analyzer fail better when no longer in sync with the data
stream. (Johanna Amann)
2.2-444 | 2014-05-16 14:10:32 -0500
* Disable all default AppStat plugins except facebook. (Jon Siwek)
* Update for the active http test to force it to use ipv4. (Seth Hall)
2.2-441 | 2014-05-15 11:29:56 -0700
* A new RADIUS analyzer. (Vlad Grigorescu)
It produces a radius.log and generates two events:
event radius_message(c: connection, result: RADIUS::Message);
event radius_attribute(c: connection, attr_type: count, value: string);
2.2-427 | 2014-05-15 13:37:23 -0400
* Fix dynamic SumStats update on clusters (Johanna Amann)
2.2-425 | 2014-05-08 16:34:44 -0700
* Fix reassembly of data w/ sizes beyond 32-bit capacities. (Jon Siwek)
Reassembly code (e.g. for TCP) now uses int64/uint64 (signedness
is situational) data types in place of int types in order to
support delivering data to analyzers that pass 2GB thresholds.
There's also changes in logic that accompany the change in data
types, e.g. to fix TCP sequence space arithmetic inconsistencies.
Another significant change is in the Analyzer API: the *Packet and
*Undelivered methods now use a uint64 in place of an int for the
relative sequence space offset parameter.
Addresses BIT-348.
* Fixing compiler warnings. (Robin Sommer)
* Update SNMP analyzer's DeliverPacket method signature. (Jon Siwek)
2.2-417 | 2014-05-07 10:59:22 -0500
* Change handling of atypical OpenSSL error case in x509 verification. (Jon Siwek)
* Fix memory leaks in X509 certificate parsing/verification. (Jon Siwek)
* Fix new []/delete mismatch in input::reader::Raw::DoClose(). (Jon Siwek)
* Fix buffer over-reads in file_analysis::Manager::Terminate() (Jon Siwek)
* Fix buffer overlows in IP address masking logic. (Jon Siwek)
That could occur either in taking a zero-length mask on an IPv6 address
(e.g. [fe80::]/0) or a reverse mask of length 128 on any address (e.g.
via the remask_addr BuiltIn Function).
* Fix new []/delete mismatch in ~Base64Converter. (Jon Siwek)
2.2-410 | 2014-05-02 12:49:53 -0500
* Replace an unneeded OPENSSL_malloc call. (Jon Siwek)
2.2-409 | 2014-05-02 12:09:06 -0500
* Clean up and documentation for base SNMP script. (Jon Siwek)
* Update base SNMP script to now produce a snmp.log. (Seth Hall)
* Add DH support to SSL analyzer. When using DHE or DH-Anon, sever
key parameters are now available in scriptland. Also add script to
alert on weak certificate keys or weak dh-params. (Johanna Amann)
* Add a few more ciphers Bro did not know at all so far. (Johanna Amann)
* Log chosen curve when using ec cipher suite in TLS. (Johanna Amann)
2.2-397 | 2014-05-01 20:29:20 -0700
* Fix reference counting for lookup_ID() usages. (Jon Siwek)
2.2-395 | 2014-05-01 20:25:48 -0700
* Fix missing "irc-dcc-data" service field from IRC DCC connections.
(Jon Siwek)
* Correct a notice for heartbleed. The notice is thrown correctly,
just the message conteined wrong values. (Johanna Amann)
* Improve/standardize some malloc/realloc return value checks. (Jon
Siwek)
* Improve file analysis manager shutdown/cleanup. (Jon Siwek)
2.2-388 | 2014-04-24 18:38:07 -0700
* Fix decoding of MIME quoted-printable. (Mareq)
2.2-386 | 2014-04-24 18:22:29 -0700
* Do a Intel::ADDR lookup for host field if we find an IP address
there. (jshlbrd)
2.2-381 | 2014-04-24 17:08:45 -0700
* Add Java version to software framework. (Brian Little)
2.2-379 | 2014-04-24 17:06:21 -0700
* Remove unused Val::attribs member. (Jon Siwek)
2.2-377 | 2014-04-24 16:57:54 -0700
* A larger set of SSL improvements and extensions. Addresses
BIT-1178. (Johanna Amann)
- Fixes TLS protocol version detection. It also should
bail-out correctly on non-tls-connections now
- Adds support for a few TLS extensions, including
server_name, alpn, and ec-curves.
- Adds support for the heartbeat events.
- Add Heartbleed detector script.
- Adds basic support for OCSP stapling.
* Fix parsing of DNS TXT RRs w/ multiple character-strings.
Addresses BIT-1156. (Jon Siwek)
2.2-353 | 2014-04-24 16:12:30 -0700
* Adapt HTTP partial content to cache file analysis IDs. (Jon Siwek)
* Adapt SSL analyzer to generate file analysis handles itself. (Jon
Siwek)
* Adapt more of HTTP analyzer to use cached file analysis IDs. (Jon
Siwek)
* Adapt IRC/FTP analyzers to cache file analysis IDs. (Jon Siwek)
* Refactor regex/signature AcceptingSet data structure and usages.
(Jon Siwek)
* Enforce data size limit when checking files for MIME matches. (Jon
Siwek)
* Refactor file analysis file ID lookup. (Jon Siwek)
2.2-344 | 2014-04-22 20:13:30 -0700
* Refactor various hex escaping code. (Jon Siwek)
2.2-341 | 2014-04-17 18:01:41 -0500
* Fix duplicate DNS log entries. (Robin Sommer)
2.2-341 | 2014-04-17 18:01:01 -0500
* Refactor initialization of ASCII log writer options. (Jon Siwek)
* Fix a memory leak in ASCII log writer. (Jon Siwek)
2.2-338 | 2014-04-17 17:48:17 -0500
* Disable input/logging threads setting their names on every
heartbeat. (Jon Siwek)
* Fix bug when clearing Bloom filter contents. Reported by
@colonelxc. (Matthias Vallentin)
2.2-335 | 2014-04-10 15:04:57 -0700
* Small logic fix for main SSL script. (Johanna Amann)
* Update DPD signatures for detecting TLS 1.2. (Johanna Amann)
* Remove unused data member of SMTP_Analyzer to silence a Coverity
warning. (Jon Siwek)
* Fix missing @load dependencies in some scripts. Also update the
unit test which is supposed to catch such errors. (Jon Siwek)
2.2-326 | 2014-04-08 15:21:51 -0700
* Add SNMP datagram parsing support.This supports parsing of SNMPv1
(RFC 1157), SNMPv2 (RFC 1901/3416), and SNMPv2 (RFC 3412). An
event is raised for each SNMP PDU type, though there's not
currently any event handlers for them and not a default snmp.log
either. However, simple presence of SNMP is currently visible now
in conn.log service field and known_services.log. (Jon Siwek)
2.2-319 | 2014-04-03 15:53:25 -0700
* Improve __load__.bro creation for .bif.bro stubs. (Jon Siwek)
2.2-317 | 2014-04-03 10:51:31 -0400
* Add a uid field to the signatures.log. Addresses BIT-1171
(Anthony Verez)
2.2-315 | 2014-04-01 16:50:01 -0700
* Change logging's "#types" description of sets to "set". Addresses
BIT-1163 (Johanna Amann)
2.2-313 | 2014-04-01 16:40:19 -0700
* Fix a couple nits reported by Coverity.(Jon Siwek)
* Fix potential memory leak in IP frag reassembly reported by
Coverity. (Jon Siwek)
2.2-310 | 2014-03-31 18:52:22 -0700
* Fix memory leak and unchecked dynamic cast reported by Coverity.
(Jon Siwek)
* Fix potential memory leak in x509 parser reported by Coverity.
(Johanna Amann)
2.2-304 | 2014-03-30 23:05:54 +0200
* Replace libmagic w/ Bro signatures for file MIME type
identification. Addresses BIT-1143. (Jon Siwek)
Includes:
- libmagic is no longer used at all. All MIME type detection is
done through new Bro signatures, and there's no longer a means
to get verbose file type descriptions. The majority of the
default file magic signatures are derived from the default magic
database of libmagic ~5.17.
- File magic signatures consist of two new constructs in the
signature rule parsing grammar: "file-magic" gives a regular
expression to match against, and "file-mime" gives the MIME type
string of content that matches the magic and an optional strength
value for the match.
- Modified signature/rule syntax for identifiers: they can no
longer start with a '-', which made for ambiguous syntax when
doing negative strength values in "file-mime". Also brought
syntax for Bro script identifiers in line with reality (they
can't start with numbers or include '-' at all).
- A new built-in function, "file_magic", can be used to get all
file magic matches and their corresponding strength against a
given chunk of data.
- The second parameter of the "identify_data" built-in function
can no longer be used to get verbose file type descriptions,
though it can still be used to get the strongest matching file
magic signature.
- The "file_transferred" event's "descr" parameter no longer
contains verbose file type descriptions.
- The BROMAGIC environment variable no longer changes any behavior
in Bro as magic databases are no longer used/installed.
- Removed "binary" and "octet-stream" mime type detections. They
don' provide any more information than an uninitialized
mime_type field which implicitly means no magic signature
matches and so the media type is unknown to Bro.
- The "fa_file" record now contains a "mime_types" field that
contains all magic signatures that matched the file content
(where the "mime_type" field is just a shortcut for the
strongest match).
- Reverted back to minimum requirement of CMake 2.6.3 from 2.8.0.
* The logic for adding file ids to {orig,resp}_fuids fields of the
http.log incorrectly depended on the state of
{orig,resp}_mime_types fields, so sometimes not all file ids
associated w/ the session were logged. (Jon Siwek)
* Fix MHR script's use of fa_file$mime_type before checking if it's
initialized. (Jon Siwek)
2.2-294 | 2014-03-30 22:08:25 +0200
* Rework and move X509 certificate processing from the SSL protocol
analyzer to a dedicated file analyzer. This will allow us to
examine X509 certificates from sources other than SSL in the
future. Furthermore, Bro now parses more fields and extensions
from the certificates (e.g. elliptic curve information, subject
alternative names, basic constraints). Certificate validation also
was improved, should be easier to use and exposes information like
the full verified certificate chain. (Johanna Amann)
This update changes the format of ssl.log and adds a new x509.log
with certificate information. Furthermore all x509 events and
handling functions have changed.
2.2-271 | 2014-03-30 20:25:17 +0200
* Add unit tests covering vector/set/table ctors/inits. (Jon Siwek)
* Fix parsing of "local" named table constructors. (Jon Siwek)
* Improve type checking of records. Addresses BIT-1159. (Jon Siwek)
2.2-267 | 2014-03-30 20:21:43 +0200
* Improve documentation of Bro clusters. Addresses BIT-1160.
(Daniel Thayer)
2.2-263 | 2014-03-30 20:19:05 +0200
* Don't include locations into serialization when cloning values.
(Robin Sommer)
2.2-262 | 2014-03-30 20:12:47 +0200
* Refactor SerializationFormat::EndWrite and ChunkedIO::Chunk memory
management. (Jon Siwek)
* Improve SerializationFormat's write buffer growth strategy. (Jon
Siwek)
* Add --parse-only option to exit after parsing scripts. May be
useful for syntax-checking tools. (Jon Siwek)
2.2-256 | 2014-03-30 19:57:28 +0200
* For the summary statistics framewirk, change all &create_expire
attributes to &read_expire in the cluster part. (Johanna Amann)
2.2-254 | 2014-03-30 19:55:22 +0200
* Update instructions on how to build Bro docs. (Daniel Thayer)
2.2-251 | 2014-03-28 08:37:37 -0400
* Quick fix to the ElasticSearch writer. (Seth Hall)
2.2-250 | 2014-03-19 17:20:55 -0400
* Improve performance of MHR script by reducing cloned Vals in
a "when" scope. (Jon Siwek)
2.2-248 | 2014-03-19 14:47:40 -0400
* Make SumStats work incrementally and non-blocking in non-cluster
mode, but force it to operate by blocking if Bro is shutting
down. (Seth Hall)
2.2-244 | 2014-03-17 08:24:17 -0700
* Fix compile errror on FreeBSD caused by wrong include file order.
(Johanna Amann)
2.2-240 | 2014-03-14 10:23:54 -0700
* Derive results of DNS lookups from from input when in BRO_DNS_FAKE
mode. Addresses BIT-1134. (Jon Siwek)
* Fixing a few cases of undefined behaviour introduced by recent
formatter work.
* Fixing compiler error. (Robin Sommer)
* Fixing (very unlikely) double delete in HTTP analyzer when
decapsulating CONNECTs. (Robin Sommer)
2.2-235 | 2014-03-13 16:21:19 -0700
* The Ascii writer has a new option LogAscii::use_json for writing
out logs as JSON. (Seth Hall)
* Ascii input reader now supports all config options as per-input
stream "config" values. (Seth Hall)
* Refactored formatters and updated the the writers a bit. (Seth
Hall)
2.2-229 | 2014-03-13 14:58:30 -0700
* Refactoring analyzer manager code to reuse
ApplyScheduledAnalyzers(). (Robin Sommer)
2.2-228 | 2014-03-13 14:25:53 -0700
* Teach async DNS lookup builtin-functions about BRO_DNS_FAKE.
Addresses BIT-1134. (Jon Siwek)
* Enable fake DNS mode for test suites.
* Improve analysis of TCP SYN/SYN-ACK reversal situations. (Jon
Siwek)
- Since it's just the handshake packets out of order, they're no
longer treated as partial connections, which some protocol analyzers
immediately refuse to look at.
- The TCP_Reassembler "is_orig" state failed to change, which led to
protocol analyzers sometimes using the wrong value for that.
- Add a unit test which exercises the Connection::FlipRoles() code
path (i.e. the SYN/SYN-ACK reversal situation).
Addresses BIT-1148.
* Fix bug in Connection::FlipRoles. It didn't swap address values
right and also didn't consider that analyzers might be scheduled
for the new connection tuple. Reported by Kevin McMahon. Addresses
BIT-1148. (Jon Siwek)
2.2-221 | 2014-03-12 17:23:18 -0700
* Teach configure script --enable-jemalloc, --with-jemalloc.
Addresses BIT-1128. (Jon Siwek)
2.2-218 | 2014-03-12 17:19:45 -0700
* Improve DBG_LOG macro (perf. improvement for --enable-debug mode).
(Jon Siwek)
* Silences some documentation warnings from Sphinx. (Jon Siwek)
2.2-215 | 2014-03-10 11:10:15 -0700
* Fix non-deterministic logging of unmatched DNS msgs. Addresses
BIT-1153 (Jon Siwek)
2.2-213 | 2014-03-09 08:57:37 -0700
* No longer accidentally attempting to parse NBSTAT RRs as SRV RRs
in DNS analyzer. (Seth Hall)
* Fix DNS SRV responses and a small issue with NBNS queries and
label length. (Seth Hall)
- DNS SRV responses never had the code written to actually
generate the dns_SRV_reply event. Adding this required
extending the event a bit to add extra information. SRV responses
now appear in the dns.log file correctly.
- Fixed an issue where some Microsoft NetBIOS Name Service lookups
would exceed the max label length for DNS and cause an incorrect
"DNS_label_too_long" weird.
2.2-210 | 2014-03-06 22:52:36 -0500
* Improve SSL logging so that connections are logged even when the
ssl_established event is not generated as well as other small SSL
fixes. (Johanna Amann)
2.2-206 | 2014-03-03 16:52:28 -0800
* HTTP CONNECT proxy support. The HTTP analyzer now supports
handling HTTP CONNECT proxies. (Seth Hall)
* Expanding the HTTP methods used in the DPD signature to detect
HTTP traffic. (Seth Hall)
* Fixing removal of support analyzers. (Robin Sommer)
2.2-199 | 2014-03-03 16:34:20 -0800
* Allow iterating over bif functions with result type vector of any.
This changes the internal type that is used to signal that a
vector is unspecified from any to void. Addresses BIT-1144
(Johanna Amann)
2.2-197 | 2014-02-28 15:36:58 -0800
* Remove test code. (Robin Sommer)
2.2-194 | 2014-02-28 14:50:53 -0800
* Remove packet sorter. Addresses BIT-700. (Johanna Amann)
2.2-192 | 2014-02-28 09:46:43 -0800
* Update Mozilla root bundle. (Johanna Amann)
2.2-190 | 2014-02-27 07:34:44 -0800
* Adjust timings of a few leak tests. (Johanna Amann)
2.2-187 | 2014-02-25 07:24:42 -0800
* More Google TLS extensions that are being actively used. Johanna(
Amann)
* Remove unused, and potentially unsafe, function
ListVal::IncludedInString. (Johanna Amann)
2.2-184 | 2014-02-24 07:28:18 -0800
* New TLS constants from
https://tools.ietf.org/html/draft-bmoeller-tls-downgrade-scsv-01.
(Johanna Amann)
2.2-180 | 2014-02-20 17:29:14 -0800
* New SSL alert descriptions from
https://tools.ietf.org/html/draft-ietf-tls-applayerprotoneg-04.
(Johanna Amann)
* Update SQLite. (Johanna Amann)
2.2-177 | 2014-02-20 17:27:46 -0800
* Update to libmagic version 5.17. Addresses BIT-1136. (Jon Siwek)
2.2-174 | 2014-02-14 12:07:04 -0800
* Support for MPLS over VLAN. (Chris Kanich)
2.2-173 | 2014-02-14 10:50:15 -0800
* Fix misidentification of SOCKS traffic that in particiular seemed
to happen a lot with DCE/RPC traffic. (Vlad Grigorescu)
2.2-170 | 2014-02-13 16:42:07 -0800
* Refactor DNS script's state management to improve performance.
(Jon Siwek)
* Revert "Expanding the HTTP methods used in the signature to detect
HTTP traffic." (Robin Sommer)
2.2-167 | 2014-02-12 20:17:39 -0800
* Increase timeouts of some unit tests. (Jon Siwek)
* Fix memory leak in modbus analyzer. Would happen if there's a
'modbus_read_fifo_queue_response' event handler. (Jon Siwek)
* Add channel_id TLS extension number. This number is not IANA
defined, but we see it being actively used. (Johanna Amann)
* Test baseline updates for DNS change. (Robin Sommer)
2.2-158 | 2014-02-09 23:45:39 -0500
* Change dns.log to include only standard DNS queries. (Jon Siwek)
* Improve DNS analysis. (Jon Siwek)
- Fix parsing of empty question sections (when QDCOUNT == 0). In this
case, the DNS parser would extract two 2-byte fields for use in either
"dns_query_reply" or "dns_rejected" events (dependent on value of
RCODE) as qclass and qtype parameters. This is not correct, because
such fields don't actually exist in the DNS message format when
QDCOUNT is 0. As a result, these events are no longer raised when
there's an empty question section. Scripts that depends on checking
for an empty question section can do that in the "dns_message" event.
- Add a new "dns_unknown_reply" event, for when Bro does not know how
to fully parse a particular resource record type. This helps fix a
problem in the default DNS scripts where the logic to complete
request-reply pair matching doesn't work because it's waiting on more
RR events to complete the reply. i.e. it expects ANCOUNT number of
dns_*_reply events and will wait until it gets that many before
completing a request-reply pair and logging it to dns.log. This could
cause bogus replies to match a previous request if they happen to
share a DNS transaction ID. (Jon Siwek)
- The previous method of matching queries with replies was still
unreliable in cases where the reply contains no answers. The new code
also takes extra measures to avoid pending state growing too large in
cases where the condition to match a query with a corresponding reply is
never met, but yet DNS messages continue to be exchanged over the same
connection 5-tuple (preventing cleanup of the pending state). (Jon Siwek)
* Updates to httpmonitor and mimestats documentation. (Jeannette Dopheide)
* Updates to Logs and Cluster documentation (Jeannette Dopheide)
2.2-147 | 2014-02-07 08:06:53 -0800
* Fix x509-extension test sometimes failing. (Johanna Amann)
2.2-144 | 2014-02-06 20:31:18 -0800
* Fixing bug in POP3 analyzer. With certain input the analyzer could
end up trying to write to non-writable memory. (Robin Sommer)
2.2-140 | 2014-02-06 17:58:04 -0800
* Fixing memory leaks in input framework. (Robin Sommer)
* Add script to detect filtered TCP traces. Addresses BIT-1119. (Jon
Siwek)
2.2-137 | 2014-02-04 09:09:55 -0800
* Minor unified2 script documentation fix. (Jon Siwek)
2.2-135 | 2014-01-31 11:09:36 -0800
* Added some grammar and spelling corrections to Installation and
Quick Start Guide. (Jeannette Dopheide)
2.2-131 | 2014-01-30 16:11:11 -0800
* Extend file analysis API to allow file ID caching. This allows an
analyzer to either provide file IDs associated with some file
content or to cache a file ID that was already determined by
script-layer logic so that subsequent calls to the file analysis
interface can bypass costly detours through script-layer. This
can yield a decent performance improvement for analyzers that are
able to take advantage of it and deal with streaming content (like
HTTP, which has been adapted accordingly). (Jon Siwek)
2.2-128 | 2014-01-30 15:58:47 -0800
* Add leak test for Exec module. (Johanna Amann)
* Fix file_over_new_connection event to trigger when entire file is
missed. (Jon Siwek)
* Improve TCP connection size reporting for half-open connections.
(Jon Siwek)
* Improve gap reporting in TCP connections that never see data. We
no longer accomodate SYN/FIN/RST-filtered traces by not reporting
missing data. The behavior can be reverted by redef'ing
"detect_filtered_trace". (Jon Siwek)
* Improve TCP FIN retransmission handling. (Jon Siwek)
2.2-120 | 2014-01-28 10:25:23 -0800
* Fix and extend x509_extension() event, which now actually returns
the extension. (Johanna Amann)
New event signauture:
event x509_extension(c: connection, is_orig: bool, cert:X509, extension: X509_extension_info)
2.2-117 | 2014-01-23 14:18:19 -0800
* Fixing initialization context in anonymous functions. (Robin
Sommer)
2.2-115 | 2014-01-22 12:11:18 -0800
* Add unit tests for new Bro Manual docs. (Jon Siwek)
* New content for the "Using Bro" section of the manual. (Rafael
Bonilla/Jon Siwek)
2.2-105 | 2014-01-20 12:16:48 -0800
* Support GRE tunnel decapsulation, including enhanced GRE headers.
GRE tunnels are treated just like IP-in-IP tunnels by parsing past
the GRE header in between the delivery and payload IP packets.
Addresses BIT-867. (Jon Siwek)
* Simplify FragReassembler memory management. (Jon Siwek)
2.2-102 | 2014-01-20 12:00:29 -0800
* Include file information (MIME type and description) into notice
emails if available. (Justin Azoff)
2.2-100 | 2014-01-20 11:54:58 -0800
* Fix caching of recently validated SSL certifcates. (Justin Azoff)
2.2-98 | 2014-01-20 11:50:32 -0800
* For notice suppresion, instead of storing the entire notice in
Notice::suppressing, just store the time the notice should be
suppressed until. This saves significant memory but can no longer
raise end_suppression, which has been removed. (Justin Azoff)
2.2-96 | 2014-01-20 11:41:07 -0800
* Integrate libmagic 5.16. Bro now now always relies on
builtin/shipped magic library/database. (Jon Siwek)
* Bro now requires a CMake 2.8.x, but no longer a pre-installed
libmagic. (Jon Siwek)
2.2-93 | 2014-01-13 09:16:51 -0800
* Fixing compile problems with some versions of libc++. Reported by
Craig Leres. (Robin Sommer)
2.2-91 | 2014-01-13 01:33:28 -0800
* Improve GeoIP City database support. When trying to open a city
database, it now considers both the "REV0" and "REV1" versions of
the city database instead of just the former. (Jon Siwek)
* Broxygen init fixes. Addresses BIT-1110. (Jon Siwek)
- Don't check mtime of bro binary if BRO_DISABLE_BROXYGEN env var set.
- Fix failure to locate bro binary if invoking from a relative
path and '.' isn't in PATH.
* Fix for packet writing to make it use the global snap length.
(Seth Hall)
* Fix for traffic with TCP segmentation offloading with IP header
len field being set to zero. (Seth Hall)
* Canonify output of a unit test. (Jon Siwek)
* A set of documentation updates. (Daniel Thayer)
- Fix typo in Bro 2.2 NEWS on string indexing.
- Fix typo in the Quick Start Guide, and clarified the
instructions about modifying crontab.
- Add/fix documentation for missing/misnamed event parameters.
- Fix typos in BIF documentation of hexstr_to_bytestring.
- Update the documentation of types and attributes.
- Documented the new substring extraction functionality.
- Clarified the description of "&priority" and "void".
2.2-75 | 2013-12-18 08:36:50 -0800
* Fixing segfault with mismatching set &default in record fields.
(Robin Sommer)
2.2-74 | 2013-12-16 08:49:55 -0800
* Improve warnings emitted from raw/execute input reader. (Jon
Siwek)
* Further improve core.when-interpreter-exceptions unit test. (Jon
Siwek)
2.2-72 | 2013-12-12 07:12:47 -0800
* Improve the core.when-interpreter-exceptions unit test to prevent
it from occasionally timing out. (Jon Siwek)
2.2-70 | 2013-12-10 15:02:50 -0800
* Fix (harmless) uninitialized field in basename/dirname util
wrapper. (Jon Siwek)
2.2-68 | 2013-12-09 15:19:37 -0800
* Several improvements to input framework error handling for more
robustness and more helpful error messages. Includes tests for
many cases. (Johanna Amann)
2.2-66 | 2013-12-09 13:54:16 -0800
* Fix table &default reference counting for record ctor expressions.
(Jon Siwek)
* Close signature files after done parsing. (Jon Siwek)
* Fix unlikely null ptr deref in broxygen::Manager. (Jon Siwek)
* FreeBSD build fix addendum: unintended variable shadowing. (Jon
Siwek)
* Fix build on FreeBSD. basename(3)/dirname(3) const-ness may vary
w/ platform. (Jon Siwek)
* Updated software framework to support parsing IE11 user-agent
strings. (Seth Hall)
* Fix the irc_reply event for several server message types. (Seth
Hall)
* Fix memory leak in input framework. If the input framework was
used to read event streams and those streams contained records
with more than one field, not all elements of the threading Values
were cleaned up. Addresses BIT-1103. (Johanna Amann)
* Minor Broxygen improvements. Addresses BIT-1098. (Jon Siwek)
2.2-51 | 2013-12-05 07:53:37 -0800
* Improve a unit test involving 'when' conditionals. (Jon Siwek)
2.2-48 | 2013-12-04 13:45:47 -0800
* Support omission of string slice low/high indices, BIT-1097.
Omission of the low index defaults to 0:
s = "12345"; s[:3] == "123"
Omission of the high index defaults to length of the string:
s = "12345"; s[3:] == "45" (Jon Siwek)
* Tweak to SMTP script to adjust for new string slicing behaviour.
(Robin Sommer)
* Test updates. (Robin Sommer)
2.2-44 | 2013-12-04 12:41:51 -0800
* Fix string slice notation. Addresses BIT-1097. (Jon Siwek)
Slice ranges were not correctly determined for negative indices
and also off by one in general (included one more element at the
end of the substring than what actually matched the index range).
It's now equivalent to Python slice notation. Accessing a string
at a single index is also the same as Python except that an
out-of-range index returns an empty string instead of throwing an
expection.
2.2-41 | 2013-12-04 12:40:51 -0800
* Updating tests. (Robin Sommer)
2.2-40 | 2013-12-04 12:16:38 -0800
* ssl_client_hello() now receives a vector of ciphers, instead of a
set, to preserve their order. (Johanna Amann)
2.2-38 | 2013-12-04 12:10:54 -0800
* New script misc/dump-events.bro, along with core support, that
dumps events Bro is raising in an easily readable form for
debugging. (Robin Sommer)
* Prettyfing Describe() for record types. If a record type has a
name and ODesc is set to short, we now print the name instead of
the full field list. (Robin Sommer)
2.2-35 | 2013-12-04 10:10:32 -0800
* Rework the automated script-reference documentation generation
process, broxygen. Addresses BIT-701 and BIT-751. (Jon Siwek)
Highlights:
- Remove --doc-scripts and -Z options to toggle documentation
mode. The parser is now always instrumented to gather
documentation from comments of the form "##", "##!", or
"##<".
- Raw comments are available at runtime through several BIF
functions: get_*_comments;
- Add --broxygen and -X options to toggle generating
reST-format documentation output, driven by a config file
argument.
- Add a "broxygen" Sphinx extension domain, allowing certain
pieces of documentation to be generated on-the-fly via
invoking a Bro process. Re-organized/cleaned up the Sphinx
source tree in doc/ to use this in some places.
2.2-11 | 2013-12-03 10:56:28 -0800
* Unit test for broccoli vector support. (Jon Siwek)
* Changed ordering of Bro type tag enum, which was out of sync. (Jon
Siwek)
2.2-9 | 2013-11-18 14:03:21 -0800
* Update local.bro for Bro >= 2.2. The commented out Notice::policy
example didn't work anymore. (Daniel Thayer)
2.2-6 | 2013-11-15 07:05:15 -0800
* Make "install-example-configs" target use DESTDIR. (Jon Siwek)
2.2-5 | 2013-11-11 13:47:54 -0800
* Fix the irc_reply event for certain server message types. (Seth
Hall)
* Fixed Segmentation fault in SQLite Writer. (Jon Crussell)
2.2 | 2013-11-07 10:25:50 -0800
* Release 2.2.
* Removing location information from ssh.log in external tests.
(Robin Sommer)
2.2-beta-199 | 2013-11-07 00:36:46 -0800
* Fixing warnings during doc build. (Robin Sommer)
2.2-beta-198 | 2013-11-06 22:54:30 -0800
* Update docs and tests for a recent change to detect-MHR.bro
(Daniel Thayer)
* Update tests and baselines for sumstats docs. (Daniel Thayer)
2.2-beta-194 | 2013-11-06 14:39:50 -0500
* Remove resp_size from the ssh log. Refactor when we write out to
the log a bit. Geodata now works reliably. (Vlad Grigorescu)
* Update VirusTotal URL to work with changes to their website and
changed it to a redef. (Vlad Grigorescu)
* Added a document for the SumStats framework. (Seth Hall)
2.2-beta-184 | 2013-11-03 22:53:42 -0800
* Remove swig-ruby from required packages section of install doc.
(Daniel Thayer)
2.2-beta-182 | 2013-11-01 05:26:05 -0700
* Adding source and original copyright statement to Mozilla cert
list. (Robin Sommer)
* Canonfying an intel test to not depend on output order. (Robin
Sommer)
2.2-beta-177 | 2013-10-30 04:54:54 -0700
* Fix thread processing/termination conditions. (Jon Siwek)
2.2-beta-175 | 2013-10-29 09:30:09 -0700
* Return the Dir module to file name tracking instead of inode
tracking to avoid missing files that reuse a formerly seen inode.
(Seth Hall)
* Deprecate Broccoli Ruby bindings and no longer build them by
default; use --enable-ruby to do so. (Jon Siwek)
2.2-beta-167 | 2013-10-29 06:02:38 -0700
* Change percent_lost in capture-loss from a string to a double.
(Vlad Grigorescu)
* New version of the threading queue deadlock fix. (Robin Sommer)
* Updating README with download/git information. (Robin Sommer)
2.2-beta-161 | 2013-10-25 15:48:15 -0700
* Add curl to list of optional dependencies. It's used by the
active-http.bro script. (Daniel Thayer)
* Update test and baseline for a recent doc test fix. (Daniel
Thayer)
2.2-beta-158 | 2013-10-25 15:05:08 -0700
* Updating README with download/git information. (Robin Sommer)
2.2-beta-157 | 2013-10-25 11:11:17 -0700
* Extend the documentation of the SQLite reader/writer framework.
(Johanna Amann)
* Fix inclusion of wrong example file in scripting tutorial.
Reported by Michael Auger @LM4K. (Johanna Amann)
* Alternative fix for the thrading deadlock issue to avoid potential
performance impact. (Johanna Amann)
2.2-beta-152 | 2013-10-24 18:16:49 -0700
* Fix for input readers occasionally dead-locking. (Robin Sommer)
2.2-beta-151 | 2013-10-24 16:52:26 -0700
* Updating submodule(s).
2.2-beta-150 | 2013-10-24 16:32:14 -0700
* Change temporary ASCII reader workaround for getline() on
Mavericks to permanent fix. (Johanna Amann)
2.2-beta-148 | 2013-10-24 14:34:35 -0700
* Add gawk to list of optional packages. (Daniel Thayer)
* Add more script package README files. (Daniel Thayer)
* Add NEWS about new features of BroControl and upgrade info.
(Daniel Thayer)
* Intel framework notes added to NEWS. (Seth Hall)
* Temporary OSX Mavericks libc++ issue workaround for getline()
problem in ASCII reader. (Johanna Amann)
* Change test of identify_data BIF to ignore charset as it may vary
with libmagic version. (Jon Siwek)
* Ensure that the starting BPF filter is logged on clusters. (Seth
Hall)
* Add UDP support to the checksum offload detection script. (Seth
Hall)
2.2-beta-133 | 2013-10-23 09:50:16 -0700
* Fix record coercion tolerance of optional fields. (Jon Siwek)
* Add NEWS about incompatible local.bro changes, addresses BIT-1047.
(Jon Siwek)
* Fix minor formatting problem in NEWS. (Jon Siwek)
2.2-beta-129 | 2013-10-23 09:47:29 -0700
* Another batch of documentation fixes and updates. (Daniel Thayer)
2.2-beta-114 | 2013-10-18 14:17:57 -0700
* Moving the SQLite examples into separate Bro files to turn them
into sphinx-btest tests. (Robin Sommer)
2.2-beta-112 | 2013-10-18 13:47:13 -0700
* A larger chunk of documentation fixes and cleanup. (Daniel Thayer)
Apart from many smaller improves this includes in particular:
* Add README files for most Bro frameworks and base/protocols.
* Add README files for base/protocols.
* Update installation instructions.
* Improvements to file analysis docs and conversion to using
btest sphinx.
2.2-beta-80 | 2013-10-18 13:18:05 -0700
* SQLite reader/writer documentation. (Johanna Amann)
* Check that the SQLite reader is only used in MANUAL reading mode.
(Johanna Amann)
* Rename the SQLite writer "dbname" configuration option to
"tablename". (Johanna Amann)
* Remove the "dbname" configuration option from the SQLite reader as
it wasn't used there. (Johanna Amann)
2.2-beta-73 | 2013-10-14 14:28:25 -0700
* Fix misc. Coverity-reported issues (leaks, potential null pointer
deref, dead code, uninitialized values,
time-of-check-time-of-use). (Jon Siwek)
* Add check for sqlite3 command to tests that require it. (Daniel
Thayer)
2.2-beta-68 | 2013-10-14 09:26:09 -0700
* Add check for curl command to active-http.test. (Daniel Thayer)
2.2-beta-64 | 2013-10-14 09:20:04 -0700
* Review usage of Reporter::InternalError, addresses BIT-1045.
Replaced some with InternalWarning or AnalyzerError, the later
being a new method which signals the analyzer to not process
further input. (Jon Siwek)
* Add new event for TCP content file write failures:
"contents_file_write_failure". (Jon Siwek)
2.2-beta-57 | 2013-10-11 17:23:25 -0700
* Improve Broxygen end-of-sentence detection. (Jon Siwek)
2.2-beta-55 | 2013-10-10 13:36:38 -0700
* A couple of new TLS extension numbers. (Johanna Amann)
* Suport for three more new TLS ciphers. (Johanna Amann)
* Removing ICSI notary from default site config. (Robin Sommer)
2.2-beta-51 | 2013-10-07 17:33:56 -0700
* Polishing the reference and scripting sections of the manual.
(Robin Sommer)
* Fixing the historical CHANGES record. (Robin Sommer)
* Updating copyright notice. (Robin Sommer)
2.2-beta-38 | 2013-10-02 11:03:29 -0700
* Fix uninitialized (or unused) fields. (Jon Siwek)
* Remove logically dead code. (Jon Siwek)
* Remove dead/unfinished code in unary not expression. (Jon Siwek)
* Fix logic for failed DNS TXT lookups. (Jon Siwek)
* A couple null ptr checks. (Jon Siwek)
* Improve return value checking and error handling. (Jon Siwek)
* Remove unused variable assignments. (Jon Siwek)
* Prevent division/modulo by zero in scripts. (Jon Siwek)
* Fix unintentional always-false condition. (Jon Siwek)
* Fix invalidated iterator usage. (Jon Siwek)
* Fix DNS_Mgr iterator mismatch. (Jon Siwek)
* Set safe umask when creating script profiler tmp files. (Jon Siwek)
* Fix nesting/indent level whitespace mismatch. (Jon Siwek)
* Add checks to avoid improper negative values use. (Jon Siwek)
2.2-beta-18 | 2013-10-02 10:28:17 -0700
* Add support for further TLS cipher suites. (Johanna Amann)
2.2-beta-13 | 2013-10-01 11:31:55 -0700
* Updating bifcl usage message. (Robin Sommer)
* Fix bifcl getopt() usage. (Jon Siwek)
2.2-beta-8 | 2013-09-28 11:16:29 -0700
* Fix a "make doc" warning. (Daniel Thayer)
2.2-beta-4 | 2013-09-24 13:23:30 -0700
* Fix for setting REPO in Makefile. (Robin Sommer)
* Whitespace fix. (Robin Sommer)
* Removing :doc: roles so that we can render this with docutils
directly. (Robin Sommer)
2.2-beta | 2013-09-23 20:57:48 -0700
* Update 'make dist' target. (Jon Siwek)
2.1-1387 | 2013-09-23 11:54:48 -0700
* Change submodules to fixed URL. (Jon Siwek)
* Updating NEWS. (Robin Sommer)
* Fixing an always false condition. (Robin Sommer)
* Fix required for compiling with clang 3.3. (Robin Sommer)
2.1-1377 | 2013-09-20 14:38:15 -0700
* Updates to the scripting introduction. (Scott Runnels)
* Kill raw input reader's child by process group to reliably clean
it up. (Jon Siwek)
2.1-1368 | 2013-09-19 20:07:57 -0700
* Add more links in the GeoLocation document (Daniel Thayer)
2.1-1364 | 2013-09-19 15:12:08 -0700
* Add links to Intelligence Framework documentation. (Daniel Thayer)
* Update Mozilla root CA list. (Johanna Amann, Jon Siwek)
* Update documentation of required packages. (Daniel Thayer)
2.1-1359 | 2013-09-18 15:01:50 -0700
* Make client and server random available on script-level. Addresses
BIT-950. (Eric Wustrow)
2.1-1357 | 2013-09-18 14:58:52 -0700
* Update HLL API and its documentation. (Johanna Amann)
* Fix case in HLL where hll_error_margin could be undefined.
(Johanna Amann)
2.1-1352 | 2013-09-18 14:42:28 -0700
* Fix a number of compiler warnings. (Daniel Thayer)
* Fix cmake warning about ENABLE_PERFTOOLS not being used. (Daniel
Thayer)
2.1-1344 | 2013-09-16 16:20:55 -0500
* Refactor Analyzer::AddChildAnalyzer and usages. (Jon Siwek)
* Minor refactor to SSL BinPAC grammer. (Jon Siwek)
* Minor refactor to Broxygen enum comments. (Jon Siwek)
* Fix possible (unlikely) use of uninitialized value. (Jon Siwek)
* Fix/improve dereference-before-null-checks. (Jon Siwek)
* Fix out-of-bounds memory accesses, and remove a
variable-length-array usage. (Jon Siwek)
* Fix potential mem leak. (Jon Siwek)
* Fix double-free and deallocator mismatch. (Jon Siwek)
* Fix another function val reference counting bug. (Jon Siwek)
2.1-1335 | 2013-09-12 16:13:53 -0500
* Documentation fixes (Daniel Thayer, Jon Siwek)
* Fix various potential memory leaks. (Jon Siwek)
* Fix significant memory leak in function unserialization. (Jon Siwek)
* Fix use-after-free and invalid/mismatch deallocator bugs. (Jon Siwek)
* Fixed an issue with the HLL_UNIQUE SumStats plugin that caused a reporter error. (Seth Hall)
* Make the notice $actions field have a default empty set to avoid having to check for it's presence. (Seth Hall)
* Fix signatures that use identifiers of type table. (Jon Siwek)
* Fix memory leak if a DNS request fails to be made. (Jon Siwek)
* Fix memory leak in DNS TXT lookups. (Jon Siwek)
* Fix raw execution input reader's signal blocking which resulted in lingering processes. (Jon Siwek)
2.1-1306 | 2013-08-31 16:06:05 -0700
* Reorganized and signifcantly extended documentation. This includes
two new chapters contributed by Scott Runnels.
2.1-1216 | 2013-08-31 10:39:40 -0700
* Support for probabilistic set cardinality, using the HyperLogLog
algorithm. (Johanna Amann, Soumya Basu)
Bro now provides the following BiFs:
hll_cardinality_init(err: double, confidence: double): opaque of cardinality
hll_cardinality_add(handle: opaque of cardinality, elem: any): bool
hll_cardinality_merge_into(handle1: opaque of cardinality, handle2: opaque of cardinality): bool
hll_cardinality_estimate(handle: opaque of cardinality): double
hll_cardinality_copy(handle: opaque of cardinality): opaque of cardinality
2.1-1154 | 2013-08-30 08:27:45 -0700
* Fix global opaque val segfault. Addresses BIT-1071. (Jon Siwek)
* Fix malloc/delete mismatch. (Jon Siwek)
* Fix invalid pointer dereference in AsciiFormatter. (Jon Siwek)
2.1-1150 | 2013-08-29 13:43:01 -0700
* Fix input framework memory leaks. (Jon Siwek)
* Fix memory leak in SOCKS analyzer for bad addr types. (Jon Siwek)
* Fix Bloom filter memory leaks. (Jon Siwek)
2.1-1144 | 2013-08-28 18:51:06 -0700
* Add bits_per_uid unit test. Addresses BIT-1016. (Jon Siwek)
* UID optimizations. Addresses BIT-1016. (Jon Siwek)
* Added a $unique_max field to Reducers for the SumStats::UNIQUE
calculation, and using the new option in scan.bro and the FTP
bruteforce detection. (Seth Hall)
2.1-1137 | 2013-08-27 13:26:44 -0700
* Add BiF hexstr_to_bytestring() that does exactly the opposite of
bytestring_to_hexstr(). (Johanna Amann)
2.1-1135 | 2013-08-27 12:16:26 -0700
* More SumStats fixes. (Seth Hall)
* Increase UIDs to 96 bits. (Jon Siwek)
- The bit-length is adjustable via redef'ing bits_per_uid.
- Prefix 'C' is added to connection UIDS (including IP tunnels)
and 'F' to files.
Addresses BIT-1016.
2.1-1128 | 2013-08-24 10:27:29 -0700
* Remove code relict in input framework. (Jon Siwek)
* Fix documentation for mkdir BIF. (Jon Siwek)
* File extraction tweaks. (Jon Siwek)
- Default extraction limit of 100MB now provided via a tuning
script loaded in local.bro so that command-line Bro is unlimited
by default.
- Extraction directory is now created on request of file
extraction rather than unconditionally in bro_init(). (Jon
Siwek)
2.1-1124 | 2013-08-23 16:33:52 -0700
* Fixed a number of object bugs DNP3 analyzer. (Hui Lin)
2.1-1122 | 2013-08-22 16:52:27 -0700
* Use macros to create file analyzer plugin classes. (Jon Siwek)
* Add options to limit extracted file sizes w/ 100MB default. (Jon
Siwek)
2.1-1117 | 2013-08-22 08:44:12 -0700
* A number of input framework fixes and corresponding test stability
improvements. (Jon Siwek)
* Make memory leak tests able to time out. (Jon Siwek)
* Fix a compiler warning regarding strncat misuse. (Jon Siwek)
2.1-1103 | 2013-08-21 19:11:34 -0400
* A number of sumstats fixes. (Seth Hall, Vlad Grigorescu)
* Fix memory leak w/ when statements. Addresses BIT-1058. (Jon
Siwek)
* Switching to relative submodule paths (Robin Sommer)
2.1-1089 | 2013-08-19 11:25:11 -0700
* Fix bloom filters' dependence on size_t. (Jon Siwek, Matthias
Vallentin).
2.1-1081 | 2013-08-19 11:19:33 -0700
* New BiF levenshtein_distance() to compute the Levenshtein distance
between two strings. (Anthony Kasza)
2.1-1078 | 2013-08-19 09:29:30 -0700
* Moving sqlite code into new external 3rdparty submodule. Johanna(
Amann)
2.1-1074 | 2013-08-14 10:29:54 -0700
* Fix timer type enum and timer name array mismatch. (Jon Siwek)
2.1-1072 | 2013-08-14 10:28:51 -0700
* Adding the unified2 analyzer that reads unified2 files from disk,
turning them into events. (Seth Hall)
* Fixing intel framework tests. (Seth Hall)
2.1-1059 | 2013-08-13 23:52:41 -0400
* Add file name support to intel framework. (Seth Hall)
* Add file support to intel framework and slightly restructure
intel http handling. (Seth Hall)
2.1-1052 | 2013-08-12 14:38:14 -0700
* Fixing bug in DNP3 analyzer flagged by compiler warning. (Robin
Sommer)
2.1-1050 | 2013-08-12 11:37:44 -0700
* Experimental DNP3 analyzer. This includes only very basic
script-level support at the moment, but quite a number of events
are provided. (Hui Lin, Robin Sommer)
2.1-1041 | 2013-08-09 15:32:22 -0700
* Update coverage baselines for canonical load order of scripts.
(Jon Siwek)
2.1-1039 | 2013-08-09 15:30:15 -0700
* Fix mem leak in DHCP analyzer. (Jon Siwek)
* Fix a unit test outdated by recent sumstats changes. (Jon Siwek)
2.1-1036 | 2013-08-05 17:29:11 -0400
* Fix the SSL infinite loop I just created. (Seth Hall)
2.1-1035 | 2013-08-05 16:44:50 -0400
* Change to SSL log delay to cause the log to write even if delay times out. (Seth Hall)
2.1-1034 | 2013-08-03 20:27:43 -0700
* A set of DHCP extensions. (Vlad Grigorescu)
- Leases are logged to dhcp.log as they are seen.
- scripts/policy/protocols/dhcp/known-devices-and-hostnames.bro
- Added DPD sig.
2.1-1027 | 2013-08-03 01:57:37 -0400
* Fix a major memory issue in the SumStats framework.
2.1-1026 | 2013-08-02 22:35:09 -0400
* Fix the SumStats top-k plugin and test. (Seth Hall)
* Rework of SumStats API to reduce high instantaneous memory
use on clusters. (Seth Hall)
* Large update for the SumStats framework.
- On-demand access to sumstats results through "return from"
functions named SumStats::request and Sumstats::request_key.
Both functions are tested in standalone and clustered modes.
- $name field has returned to SumStats which simplifies cluster
code and makes the on-demand access stuff possible.
- Clustered results can only be collected for 1 minute from their
time of creation now instead of time of last read.
- Thresholds use doubles instead of counts everywhere now.
- Calculation dependency resolution occurs at start up time now
instead of doing it at observation time which provide a minor
cpu performance improvement. A new plugin registration mechanism
was created to support this change.
- AppStats now has a minimal doc string and is broken into hook-based
plugins.
- AppStats and traceroute detection added to local.bro (Seth Hall)
2.1-1009 | 2013-08-02 17:19:08 -0700
* A number of exec module and raw input reader fixes. (Jon Siwek)
2.1-1007 | 2013-08-01 15:41:54 -0700
* More function documentation. (Johanna Amann)
2.1-1004 | 2013-08-01 14:37:43 -0700
* Adding a probabilistic data structure for computing "top k"
elements. (Johanna Amann)
The corresponding functions are:
topk_init(size: count): opaque of topk
topk_add(handle: opaque of topk, value: any)
topk_get_top(handle: opaque of topk, k: count)
topk_count(handle: opaque of topk, value: any): count
topk_epsilon(handle: opaque of topk, value: any): count
topk_size(handle: opaque of topk): count
topk_sum(handle: opaque of topk): count
topk_merge(handle1: opaque of topk, handle2: opaque of topk)
topk_merge_prune(handle1: opaque of topk, handle2: opaque of topk)
2.1-971 | 2013-08-01 13:28:32 -0700
* Fix some build errors. (Jon Siwek)
* Internal refactoring of how plugin components are tagged/managed.
(Jon Siwek)
* Fix various documentation, mostly related to file analysis. (Jon
Siwek)
* Changing the Bloom filter hashing so that it's independent of
CompositeHash. (Robin Sommer)
2.1-951 | 2013-08-01 11:19:23 -0400
* Small fix to deal with a bug in the SSL log delay mechanism.
2.1-948 | 2013-07-31 20:08:28 -0700
* Fix segfault caused by merging an empty bloom-filter with a
bloom-filter already containing values. (Johanna Amann)
2.1-945 | 2013-07-30 10:05:10 -0700
* Make hashers serializable. (Matthias Vallentin)
* Add docs and use default value for hasher names. (Matthias
Vallentin)
2.1-939 | 2013-07-29 15:42:38 -0700
* Added Exec, Dir, and ActiveHTTP modules. (Seth Hall)
base/utils/exec.bro provides a module to start external processes
asynchronously and retrieve their output on termination.
base/utils/dir.bro uses it to monitor a directory for changes, and
base/utils/active-http.bro for providing an interface for querying
remote web servers.
2.1-930 | 2013-07-29 15:06:07 -0700
* Major file analysis overhaul in naming and appearance, along with
fixes and test updates. (Seth Hall and Jon Siwek)
Includes:
* Added protocol description functions that provide a super
compressed log representation. (Seth Hall)
* Added mime types to http.log (Seth Hall)
* Add jar files to the default MHR lookups. (Seth Hall)
* Adding CAB files for MHR checking. (Seth Hall)
* Improve malware hash registry script.
- Include a link to a virustotal search in the notice sub message field.
- Give all information returned from Team Cymru in the notice message.
- Add more file types to match on to the default set.
* Make the custom libmagic database a git submodule.
* Add an is_orig parameter to file_over_new_connection event.
* Recorrected the module name to Files.
* Added Files::analyzer_name to get a more readable name for a
file analyzer.
* Improved and just overall better handled multipart mime
transfers in HTTP and SMTP. HTTP now has orig_fuids and
resp_fuids log fields since multiple "files" can be transferred
with multipart mime in a single request/response pair. SMTP has
an fuids field which has file unique IDs for all parts
transferred. FTP and IRC have a log field named fuid added
because only a single file can be transferred per irc and ftp
log line.
2.1-895 | 2013-07-29 14:07:35 -0700
* Adding a test for a DNSKEY RR. (Robin Sommer)
2.1-894 | 2013-07-29 16:44:41 -0400
* Updates for the Intel Framework. (Seth Hall)
- policy/frameworks/intel/seen is the new location for the
scripts that push data into the intel framework for checking.
- The new policy/frameworks/intel/do_notice script adds an
example mechanism for data driven notices.
- Remove the Intel insertion after heuristically detecting SSH
bruteforcing.
- Intel importing format has changed (refer to docs).
- All string matching is now case insensitive.
- SMTP intel script has been updated to extract email
addresses correctly.
- Small fix sneaking into the smtp base script to actually
extract individual email addresses in the To: field
correctly.
2.1-888 | 2013-07-25 12:02:41 -0700
* Protection about broken traces with empty pcap headers. (Matt
Thompson)
2.1-887 | 2013-07-25 11:33:27 -0700
* Support for Bloom filter. (Matthias Vallentin)
Bro now provides the following BiFs:
bloomfilter_basic_init(fp: double, capacity: count, name: string &default=""): opaque of bloomfilter
bloomfilter_counting_init(k: count, cells: count, max: count, name: string &default=""): opaque of bloomfilter
bloomfilter_add(bf: opaque of bloomfilter, x: any)
bloomfilter_lookup(bf: opaque of bloomfilter, x: any): count
bloomfilter_merge(bf1: opaque of bloomfilter, bf2: opaque of bloomfilter): opaque of bloomfilter
bloomfilter_clear(bf: opaque of bloomfilter)
Note that currently Bloom filters from separate Bro instances
(e.g., from different cluster nodes) cannot be merged.
2.1-826 | 2013-07-25 10:12:26 -0700
* bif files declared with bif_target() are now automatically
compiled in. No more manual includes to pull them in. (Robin
Sommer)
* Covenience make target in testing/btest to update the three
coverage tests that usually need tweaking when scripts get
added/removed. (Robin Sommer)
2.1-824 | 2013-07-22 14:25:14 -0400
* Fixed a scriptland state issue that manifested especially badly on proxies. (Seth Hall)
* Another test fix. (Robin Sommer)
* Canonyfying the output of core.print-bpf-filters. (Robin Sommer)
2.1-820 | 2013-07-18 12:30:04 -0700
* Extending external canonifier to remove fractional values from
capture_loss.log. (Robin Sommer)
* Canonifying internal order for plugins and their components to
make it deterministic. (Robin Sommer)
* Small raw reader tweaks that got left our earlier. (Robin Sommer)
2.1-814 | 2013-07-15 18:18:20 -0700
* Fixing raw reader crash when accessing nonexistant file, and
memory leak when reading from file. Addresses #1038. (Johanna
Amann)
2.1-811 | 2013-07-14 08:01:54 -0700
* Bump sqlite to 3.7.17. (Johanna Amann)
* Small test fixes. (Seth Hall)
* Fix a bug where the same analyzer tag was reused for two different
analyzers. (Seth Hall)
* Moved DPD signatures into script specific directories. Left out
the BitTorrent signatures pending further updates to that
analyzer. (Seth Hall)
2.1-802 | 2013-07-10 10:55:14 -0700
* Const adjustment for methods. (Jon Siwek)
2.1-798 | 2013-07-08 13:05:37 -0700
* Rewrite of the packet filter framework. (Seth Hall)
This includes:
- Plugin interface for adding filtering mechanisms.
- Integrated the packet filter framework with the analyzer
framework to retrieve well-known ports from there.
- Support for BPF-based load balancing (IPv4 and IPv6). This will
tie in with upcoming BroControl support for configuring this.
- Support for BPF-based connection sampling.
- Support for "shunting" traffic with BPF filters.
- Replaced PacketFilter::all_packets with
PacketFilter::enable_auto_protocol_capture_filters.
2.1-784 | 2013-07-04 22:28:48 -0400
* Add a call to lookup_connection in SSH scripts to update connval. (Seth Hall)
* Updating submodule(s). (Robin Sommer)
2.1-782 | 2013-07-03 17:00:39 -0700
* Remove the SSL log queueing mechanism that was included with the
log delay mechanism. (Seth Hall)
2.1-780 | 2013-07-03 16:46:26 -0700
* Rewrite of the RAW input reader for improved robustness and new
features. (Johanna Amann) This includes:
- Send "end_of_data" event for all kind of streams.
- Send "process_finished" event with exit code of child
process at process termination.
- Expose name of input stream to readers.
- Better error handling.
- New "force_kill" option which SIGKILLs processes on reader termination.
- Supports reading from stdout and stderr simultaneously.
- Support sending data to stdin of child process.
- Streaming reads from external commands work without blocking.
2.1-762 | 2013-07-03 16:33:22 -0700
* Fix to correct support for TLS 1.2. Addresses #1020. (Seth Hall,
with help from Rafal Lesniak).
2.1-760 | 2013-07-03 16:31:36 -0700
* Teach broxygen to generate protocol analyzer plugin reference.
(Jon Siwek)
* Adding 'const' to a number of C++ methods. (Jon Siwek)
2.1-757 | 2013-07-03 16:28:10 -0700
* Fix redef of table index from clearing table.
`redef foo["x"] = 1` now acts like `redef foo += { ["x"] = 1 }`
instead of `redef foo = { ["x"] = 1 }`.
Addresses #1013. (Jon Siwek)
2.1-755 | 2013-07-03 16:22:43 -0700
* Add a general file analysis overview/how-to document. (Jon Siwek)
* Improve file analysis doxygen comments. (Jon Siwek)
* Improve tracking of HTTP file extraction. http.log now has files
taken from request and response bodies in different fields for
each, and can now track multiple files per body. That is, the
"extraction_file" field is now "extracted_request_files" and
"extracted_response_files". Addresses #988. (Jon Siwek)
* Fix HTTP multipart body file analysis. Each part now gets assigned
a different file handle/id. (Jon Siwek)
* Remove logging of analyzers field of FileAnalysis::Info. (Jon
Siwek)
* Remove extraction counter in default file extraction scripts. (Jon
Siwek)
* Remove FileAnalysis::postpone_timeout.
FileAnalysis::set_timeout_interval can now perform same function.
(Jon Siwek)
* Make default get_file_handle handlers &priority=5 so they're
easier to override. (Jon Siwek)
* Add input interface to forward data for file analysis. The new
Input::add_analysis function is used to automatically forward
input data on to the file analysis framework. (Jon Siwek)
* File analysis framework interface simplifications. (Jon Siwek)
- Remove script-layer data input interface (will be managed directly
by input framework later).
- Only track files internally by file id hash. Chance of collision
too small to justify also tracking unique file string.
2.1-741 | 2013-06-07 17:28:50 -0700
* Fixing typo that could cause an assertion to falsely trigger.
(Robin Sommer)
2.1-740 | 2013-06-07 16:37:32 -0700
* Fix for CMake 2.6.x. (Robin Sommer)
2.1-738 | 2013-06-07 08:38:13 -0700
* Remove invalid free on non-allocated pointer in hash function
object. Addresses #1018. (Matthias Vallentin)
2.1-736 | 2013-06-06 10:05:20 -0700
* New "magic constants" @DIR and @FILENAME that expand to the
directory path of the current script and just the script file name
without path, respectively. (Jon Siwek)
2.1-731 | 2013-06-04 21:19:08 -0700
* Reorginization of internal protocol analyzer code. We're moving
them to a modularized structure, based on a plugin model. Along
with this change comes generic plugin infrastructure that we'll
later extend to other Bro component as well. For now all plugins
are compiled in statically, but in the future we plan to also
enable dynamic loading at run time. (Robin Sommer)
* Ignoring file ids in external tests. (Robin Sommer)
2.1-675 | 2013-06-02 20:03:19 -0700
* Fix a compiler warning. (Robin Sommer)
* Allow named vector/set/table/record constructors. Addresses #983.
(Jon Siwek)
* Adding Makefile target test-all that also runs the BroControl test
suite. (Robin Sommer)
2.1-664 | 2013-05-28 21:37:46 -0700
* Dangling pointer fix. Addresses #1004. (Jon Siwek)
2.1-659 | 2013-05-24 17:24:18 -0700
* Fix broken/missing documentation. (Jon Siwek)
* Fixing test that would fail without ES/curl support. (Robin
Sommer)
2.1-656 | 2013-05-17 15:58:07 -0700
* Fix mutex lock problem for writers. (Johanna Amann)
2.1-654 | 2013-05-17 13:49:52 -0700
* Tweaks to sqlite3 configuration to address threading issues.
(Johanna Amann)
2.1-651 | 2013-05-17 13:37:16 -0700
* Fix uninitialized DPM member. (Jon Siwek)
* Fix issue with transaction ID reuse in a single DNS connection. (Seth Hall)
* New function added to the queue.bro script to support peeking at
the new gettable item in the queue without removing it. (Seth Hall)
2.1-647 | 2013-05-17 07:47:14 -0700
* Fixing Broxygen generation to have BROMAGIC set. (Robin Sommer)
* Fix for 'fchmod undeclared here' on FreeBSD. (Robin Sommer)
* CMake policy fix to avoid errors with older versions. (Robin
Sommer)
2.1-641 | 2013-05-15 18:15:09 -0700
* Test update. (Robin Sommer)
2.1-640 | 2013-05-15 17:24:09 -0700
* Support for cleaning up threads that have terminated. (Johanna
Amann and Robin Sommer). Includes:
- Both logging and input frameworks now clean up threads once
they aren't further needed anymnore.
- New function Log::remove_stream() that removes a logging
stream, stopping all writer threads that are associated with
it. Note, however, that removing a *filter* from a stream
still doesn't clean up any threads. The problem is that
because of the output paths potentially being created
dynamically it's unclear if the writer thread will still be
needed in the future.
2.1-626 | 2013-05-15 16:09:31 -0700
* Add "reservoir" sampler for SumStats framework. This maintains
a set of N uniquely distributed random samples. (Johanna Amann)
2.1-619 | 2013-05-15 16:01:42 -0700
* SQLite reader and writer combo. This allows to read/write
persistent data from on disk SQLite databases. The current
interface is quite low-level, we'll add higher-level abstractions
in the future. (Johanna Amann)
2.1-576 | 2013-05-15 14:29:09 -0700
* Initial version of new file analysis framework. This moves most of
the processing of file content from script-land into the core,
where it belongs. Much of this is an internal change, and at this
point the new code has essentially feature-equality with the old
one. More script-level changes to come. (Jon Siwek)
2.1-502 | 2013-05-10 19:29:37 -0700
* Allow default function/hook/event parameters. Addresses #972. (Jon
Siwek)
* Change the endianness parameter of bytestring_to_count() BIF to
default to false (big endian). (Jon Siwek)
2.1-500 | 2013-05-10 19:22:24 -0700
* Fix to prevent merge-hook of SumStat's unique plugin from damaging
source data. (Johanna Amann)
2.1-498 | 2013-05-03 17:44:08 -0700
* Table lookups return copy of non-const &default vals. This
prevents unintentional modifications to the &default value itself.
Addresses #981. (Jon Siwek)
2.1-496 | 2013-05-03 15:54:47 -0700
* Fix memory leak and unnecessary allocations in OpaqueVal.
Addresses #986. (Matthias Vallentin)
2.1-492 | 2013-05-02 12:46:26 -0700
* Work-around for sumstats framework not propagating updates after
intermediate check in cluster environments. (Johanna Amann)
* Always apply tcp_connection_attempt. Before this change it was
only applied when a connection_attempt() event handler was
defined. (Robin Sommer)
* Fixing coverage.bare-mode-errors test. (Robin Sommer)
2.1-487 | 2013-05-01 18:03:22 -0700
* Always apply tcp_connection_attempt timer, even if no
connection_attempt() event handler is defined. (Robin Sommer)
2.1-486 | 2013-05-01 15:28:45 -0700
* New framework for computing summary statistics in
base/framework/sumstats. This replaces the metrics frameworks, and
comes with a number of applications build on top, see NEWS. More
documentation to follow. (Seth Hall)
2.1-397 | 2013-04-29 21:19:00 -0700
* Fixing memory leaks in CompHash implementation. Addresses #987.
(Robin Sommer)
2.1-394 | 2013-04-27 15:02:31 -0700
* Fixed a bug in the vulnerable software script and added a test.
(Seth Hall)
* Fix schedule statements used outside event handlers. Addresses
#974. (Jon Siwek)
* Fix record coercion for default inner record fields. Addresses
#973. (Jon Siwek)
* Add bytestring_to_count function to bro.bif. Addresses #968. (Yun
Zheng Hu)
2.1-386 | 2013-03-22 12:41:50 -0700
* Added reverse() function to strings.bif. (Yun Zheng Hu)
2.1-384 | 2013-03-22 12:10:14 -0700
* Fix record constructors in table initializer indices. Addresses
#660. (Jon Siwek)
2.1-382 | 2013-03-22 12:01:34 -0700
* Add support for 802.1ah (Q-in-Q). Addresses #641. (Seth Hall)
2.1-380 | 2013-03-18 12:18:10 -0700
* Fix gcc compile warnings in base64 encoder and benchmark reader.
(Johanna Amann)
2.1-377 | 2013-03-17 17:36:09 -0700
* Fixing potential leak in DNS error case. (Vlad Grigorescu)
2.1-375 | 2013-03-17 13:14:26 -0700
* Add base64 encoding functionality, including new BiFs
encode_base64() and encode_base64_custom(). (Johanna Amann)
* Replace call to external "openssl" in extract-certs-pem.bro with
that encode_base64(). (Johanna Amann)
* Adding a test for extract-certs-pem.pem. (Robin Sommer)
* Renaming Base64Decoder to Base64Converter. (Robin Sommer)
2.1-366 | 2013-03-17 12:35:59 -0700
* Correctly handle DNS lookups for software version ranges. (Seth
Hall)
* Improvements to vulnerable software detection. (Seth Hall)
- Add a DNS based updating method. This needs to be tested
still.
- Vulnerable version ranges are used now instead of only single
versions. This can deal with software with multiple stable
major versions.
* Update software version parsing and comparison to account for a
third numeric subversion. Also, $addl is now compared numerically
if the value is actually numeric. (Seth Hall)
2.1-361 | 2013-03-13 07:18:22 -0700
* Add check for truncated link frames. Addresses #962. (Jacob
Baines)
* Fix large memory allocation in IP fragment reassembly. Addresses
#961. (Jacob Baines)
2.1-357 | 2013-03-08 09:18:35 -0800
* Fix race-condition in table-event test. (Johanna Amann)
* s/bro-ids.org/bro.org/g. (Robin Sommer)
2.1-353 | 2013-03-07 13:31:37 -0800
* Fix function type-equivalence requiring same parameter names.
Addresses #957. (Jon Siwek)
2.1-351 | 2013-03-07 13:27:29 -0800
* Fix new/delete mismatch. Addresses #958. (Jacob Baines)
* Fix compiler warnings. (Jon Siwek)
2.1-347 | 2013-03-06 16:48:44 -0800
* Remove unused parameter from vector assignment method. (Johanna Amann)
* Remove the byte_len() and length() bifs. (Johanna Amann)
2.1-342 | 2013-03-06 15:42:52 -0800
* Moved the Notice::notice event and Notice::policy table to both be
hooks. See documentation and NEWS for information. (Seth Hall).
2.1-338 | 2013-03-06 15:10:43 -0800
* Fix init of local sets/vectors via curly brace initializer lists.
(Jon Siwek)
2.1-336 | 2013-03-06 15:08:06 -0800
* Fix memory leaks resulting from 'when' and 'return when'
statements. Addresses #946. (Jon Siwek)
* Fix three bugs with 'when' and 'return when' statements. Addresses
#946. (Jon Siwek)
2.1-333 | 2013-03-06 14:59:47 -0800
* Add parsing for GTPv1 extension headers and control messages. (Jon Siwek)
This includes:
- A new generic gtpv1_message() event generated for any GTP
message type.
- Specific events for the create/update/delete PDP context
request/response messages.
Addresses #934.
2.1-331 | 2013-03-06 14:54:33 -0800
* Fix possible null pointer dereference in identify_data BIF. Also
centralized libmagic calls for consistent error handling/output.
(Jon Siwek)
* Fix build on OpenBSD 5.2. (Jon Siwek)
2.1-328 | 2013-02-05 01:34:29 -0500
* New script to query the ICSI Certificate Notary
(http://notary.icsi.berkeley.edu/) over DNS and add information
to the SSL log at runtime. (Matthias Vallentin)
* Add delayed logging to SSL base scripts. (Matthias Vallentin)
2.1-319 | 2013-02-04 09:45:34 -0800
* Update input tests to use exit_only_after_terminate. (Johanna
Amann)
* New option exit_only_after_terminate to prevent Bro from exiting.
If set, the main loop won't terminate before somebody calls
terminate(). (Robin Sommer)
2.1-311 | 2013-02-01 08:03:01 -0800
* Updating submodule(s).
2.1-310 | 2013-01-30 20:09:27 -0800
* Add an error for record coercions that would orphan a field. (Jon
Siwek)
* Fixing several scripts where a field in an inlined record was
never removed after a code refactor. (Jon Siwek)
2.1-307 | 2013-01-25 13:50:57 -0800
* Fix runaway reference counting bug in record coercion. (Jon Siwek)
* Fix memory leak in some reporter messaging cases. (Jon Siwek)
2.1-304 | 2013-01-23 19:43:27 -0800
* Making a test portable. (Robin Sommer)
2.1-302 | 2013-01-23 16:17:29 -0800
* Refactoring ASCII formatting/parsing from loggers/readers into a
separate AsciiFormatter class. (Johanna Amann)
* Fix uninitialized locals in event/hook handlers from having a
value. Addresses #932. (Jon Siwek)
* Add a null value check in CompositeHash::ComputeHash. Addresses
#930. (Jon Siwek)
* Change reporter messages to more reliably print to stderr.
Addressed #930 (and revisits #836). (Jon Siwek)
* Changing test=suite's btest call to use "-j" instead of "-j 5".
(Robin Sommer)
* Require "case" blocks to end with either "break", "return", or a
new "fallthrough" statement that passes control on to the
subsequent case. This gives us the best mix of safety,
readability, and flexibility. Addresses #754. (Jon Siwek)
2.1-279 | 2013-01-18 17:18:22 -0800
* Revert "Trick for parallelizing input framework unit tests." The
old way of doing the tests seems more reliable for now. (Jon
Siwek)
* Fixing variable size issues with http response code in
ElasticSearch writer. (Gilbert Clark)
* Removing unused class member. (Robin Sommer)
* Add opaque type-ignoring for the accept_unsupported_types input
framework option. (Johanna Amann)
2.1-271 | 2013-01-08 10:18:57 -0800
* Change substring index notation to use a colon. String slice
notation is now written as `s[1:2]`. Addresses #422. (Jon Siwek)
2.1-268 | 2013-01-07 09:43:44 -0800
* Fix memory leak in OpaqueType::DoUnserialize. (Jon Siwek)
2.1-265 | 2012-12-20 17:38:42 -0800
* Add array-style index accessor for strings. Addresses #422. (Jon
Siwek)
The index expression can take up to two indices for the start and
end index of the substring to return (e.g. "mystring[1,3]").
Negative indices are allowed, with -1 representing the last
character in the string. The indexing is not cyclic -- if the
starting index is >= the length of the string an empty string is
returned, and if the ending index is >= the length of the string
then it's interpreted as the last index of the string. Assigning
to substrings accessed like this isn't allowed.
2.1-263 | 2012-12-20 16:22:09 -0800
* Bro's language now has a new set of types "opaque of X". (Matthias
Vallentin)
Opaque values can be passed around like other values but they can
only be manipulated with BiF functions, not with other operators.
Currently, the following opaque types are supported:
- opaque of md5
- opaque of sha1
- opaque of sha256
- opaquey of entropy.
They go along with the corrsponding BiF functions md5_*, sha1_*,
sha256_*, and entropy_*, respectively. Note that these functions
have changed their signatures to work with opaques types rather
than global state as it was before.
2.1-240 | 2012-12-20 15:21:07 -0800
* Improve error for invalid use of types as values. Addresses #923.
(Jon Siwek)
2.1-238 | 2012-12-20 15:11:25 -0800
* Finish implementation of script-layer switch statement. Addresses
#754. (Jon Siwek)
They behave like C-style switches except case labels can be
comprised of multiple literal constants delimited by commas. Only
atomic types are allowed for now. Case label bodies that don't
execute a "return" or "break" statement will fall through to
subsequent cases. A default case label is allowed.
* Fix a case where c$resp$size is misrepresented. Addresses #730.
(Jon Siwek)
2.1-234 | 2012-12-20 12:12:19 -0800
* Fix return value of hook calls that have no handlers. For this
case, the return value is always true. (Jon Siwek)
* Fix to_port() BIF for port strings with a port number of zero.
(Jon Siwek)
2.1-231 | 2012-12-14 14:51:35 -0800
* Make const variables actually constant. Both local and global
variables declared with "const" could be modified, but now
expressions that would modify them generate an error message at
parse-time. Addresses #922. (Jon Siwek)
2.1-229 | 2012-12-14 14:46:12 -0800
* Fix memory leak in ASCII reader when encoutering errors in input.
(Johanna Amann)
* Improvements for the "bad checksums" detector to make it detect
bad TCP checksums. (Seth Hall)
2.1-223 | 2012-12-12 14:25:15 -0800
* Trick for parallelizing input framework unit tests. Instead of
loading listen.bro to block until files are read, just read a pcap
file in pseudo-realtime. (Jon Siwek)
* Fix reliability of a unit test that relies on when statements.
(Jon Siwek)
* Remove unused attributes. (Daniel Thayer)
- Removed attributes &postprocessor and &match from documentation and source code.
- Removed undocumented attribute &attr from source code.
- Removed internal attribute "(&tracked)" from documentation.
2.1-218 | 2012-12-10 14:45:04 -0800
* Add GPRS Tunnelling Protocol (GTPv1) decapsulation. This currently
supports automatic decapsulation of GTP-U packets on UDP port 2152.
The GTPv1 headers for such tunnels can be inspected by handling
the "gtpv1_g_pdu_packet" event, which has a parameter of type
"gtpv1_hdr". Addresses #690. (Jon Siwek; derived from patch by
Carsten Langer)
* Change BinPAC exceptions in AYIYA/GTP analyzers to do
"protocol_violation". (Jon Siwek)
2.1-212 | 2012-12-07 19:42:03 -0800
* Changing the HTTP parser to accept request methods in alignment
with the RFC. (Robin Sommer)
2.1-209 | 2012-12-05 16:44:04 -0800
* Adapting the HTTP request line parsing to only accept methods
consisting of letters [A-Za-z]. (Robin Sommer)
2.1-207 | 2012-12-05 15:47:32 -0800
* Reporting warnings if kill/waitpid fail in communication system.
(Bill Parker)
* Replace() bzero with memset(). (Bill Parker)
* Merge remote-tracking branch 'vlad/topic/vladg/http-verbs'
* vlad/topic/vladg/http-verbs:
A test for HTTP methods, including some horribly illegal requests.
Remove hardcoded HTTP verbs from the analyzer (#741)
I added a "bad_HTTP_request" weird for HTTP request lines that don't
have more than a single word.
Closes #741. (Robin Sommer)
* A test for HTTP methods, including some horribly illegal requests. (Vlad Grigorescu)
* Remove hardcoded HTTP verbs from the analyzer (#741) (Vlad Grigorescu)
2.1-203 | 2012-12-05 14:36:56 -0800
* Fix segfault: Synchronization of state between connecting peers
now skips over identifiers that aren't initialized with a value
yet. Addresses #66. (Jon Siwek)
* Fix segfault: Delete correct entry in error case in input
framework. (Johanna Amann)
* Bad record constructor initializers now give an error. Addresses
#34. (Jon Siwek)
* Invalid vector indices now generate error message. Addresses #24.
(Jon Siwek)
* Bump CPack RPM package requirement to Python >= 2.6.0. (Jon Siwek)
* Interpreter exceptions occurring in "when" blocks are now handled.
Addresses #779 (Jon Siwek)
2.1-195 | 2012-12-03 14:50:33 -0800
* Catching out-of-memory in patricia tree code. (Bill Parker)
2.1-194 | 2012-12-03 14:36:26 -0800
* Renaming ASCII writer filter option 'only_single_header_row' to
'tsv'. Also clarifying usage. Closes #912. (Robin Sommer)
2.1-193 | 2012-12-03 14:11:14 -0800
* Fix a set of bugs with table/set attributes. (Jon Siwek)
- Identifiers that are initialized with set()/table() constructor
expressions now inherit attributes from the expression. Before,
statements like
const i: set[string] = set() &redef;
associated the attribute with the set() constructor, but not the
"i" identifier, preventing redefinition. Addresses #866.
- Allow &default attribute to apply to tables initialized as empty
(via either "{ }" or "table()") or if the expression supplied to it
can evaluate to a type that's promotable to the same yield type as
the table.
2.1-191 | 2012-12-03 14:08:56 -0800
* Add test of record() constructor to table initializer unit test.
(Jon Siwek)
* Fix table(), set(), vector() constructors in table initializer
lists. Also adds type checking of yield values to table()
constructor and fixes the type checking of yield values in
vector() constructor. Addresses #5. (Jon Siwek)
2.1-188 | 2012-12-03 14:04:29 -0800
* Hook functions now callable with "hook" expression (i.e., hook is
no longer a statement). The return value of the call is an
implicit boolean value of T if all hook handlers ran, or F if one
hook handler exited as a result of a break statement and
potentially prevented other handlers from running.
Scripts don't need to declare hooks with an explicit return type of bool
(internally, that's assumed), and any values given to (optional) return
statements in handler definitions are just ignored.
Addresses #918. (Jon Siwek)
* Clarification in hook documentation. (Jon Siwek)
2.1-184 | 2012-12-03 13:59:50 -0800
* Slightly fix up file name extraction from Content-Disposition
headers. (Seth Hall)
* Adding -b flag to bro in unit tests so they run faster.
* Fixed a DNS attribute issue. Reported by Matt Thompson. (Seth
Hall)
* Adding NEWS placeholder for hooks and CSV mode. (Robin Sommer)
2.1-178 | 2012-11-23 19:35:32 -0800
* The ASCII writer now supports a new filter config option
"only_single_header_row" that turns the output into CSV format
when set to "T". (Carsten Langer)
* Add new function flavor called a "hook". This new flavor of
function behaves like a "synchronous event". See
doc/scripts/builtins.rst more details on usage. (Jon Siwek)
* Improve auto-generated enum documentation. The names of enum types
are tracked so that variables holding a value of a given enum type
can generate a reference to it instead of just listing the type as
a generic "enum". (Jon Siwek)
2.1-171 | 2012-11-23 18:24:15 -0800
* Fix ambiguity between composite table index and record ctor
expressions. If a table type is "global t = table[conn_id, bool]
of count", then checking membership like "[c$id, is_orig] in t"
now works. Addresses #80. (Jon Siwek)
2.1-169 | 2012-11-23 18:21:32 -0800
* Fix some warnings from sphinx when building docs. (Jon Siwek)
2.1-167 | 2012-11-14 13:19:17 -0800
* Add a new BIF "bytestring_to_double" for converting from a binary
representation of a double. Addresses #908. (Carsten Langer/Daniel
Thayer)
2.1-162 | 2012-11-13 17:29:00 -0800
* Fix modbus register array parsing. (Jon Siwek)
* Adjustments to modbus test cases. (Jon Siwek)
2.1-157 | 2012-11-08 16:22:00 -0800
* Fix for lookup_hostname BIF. (Jon Siwek)
* Fix for modbus test portability. (Robin Sommer)
2.1-152 | 2012-11-05 16:52:34 -0800
* Initial version of a completely reworked intelligence framework.
See doc/intel.rst for more information. (Seth Hall)
* Experimental Modbus analyzer. See policy/protocols/modbus/* for
example policies. (Dina Hadziosmanovic, Seth Hall)
2.1-112 | 2012-11-05 13:58:20 -0800
* New base script for detecting cases of checksum offloading.
Reporter messages will now tell if one has bad checksums. (Seth
Hall)
* Clarifying ownership rules for BroString constructors. (Robin
Sommer)
2.1-109 | 2012-11-05 13:39:34 -0800
* Add detection rate threshold for MHR. (Vlad Grigorescu)
* lookup_hostname_txt fixes. (Vlad Grigorescu)
2.1-104 | 2012-11-01 10:37:50 -0700
* A new built-in function lookup_hostname_txt() provides support for
DNS TXT queries. (Vlad Grigorescu)
2.1-101 | 2012-10-31 14:30:26 -0700
* Documentation reorg: The install info has been consolidated into a
single document (INSTALL), the upgrade info has been moved from
the FAQ to a section in the install doc, and the "upgrading from
1.5 to 2.0" document has been updated (and renamed) to also
include 2.0 to 2.1 upgrade info. (Daniel Thayer)
2.1-96 | 2012-10-31 14:23:50 -0700
* Renaming option defining the frequency of alarm summary mails to
'Logging::default_alarm_mail_interval'. (Daniel Thayer)
2.1-91 | 2012-10-24 16:04:47 -0700
* Adding PPPoE support to Bro. (Seth Hall)
2.1-87 | 2012-10-24 15:40:06 -0700
* Adding missing &redef for some TCP options. Addresses #905, #906,
#907. (Carsten Langer)
2.1-86 | 2012-10-24 15:37:11 -0700
* Add parsing rules for IPv4/IPv6 subnet literal constants.
Addresses #888. (Jon Siwek)
2.1-84 | 2012-10-19 15:12:56 -0700
* Added a BiF strptime() to wrap the corresponding C function. (Seth
Hall)
2.1-82 | 2012-10-19 15:05:40 -0700
* Add IPv6 support to signature header conditions. (Jon Siwek)
- "src-ip" and "dst-ip" conditions can now use IPv6 addresses/subnets.
They must be written in colon-hexadecimal representation and enclosed
in square brackets (e.g. [fe80::1]). Addresses #774.
- "icmp6" is now a valid protocol for use with "ip-proto" and "header"
conditions. This allows signatures to be written that can match
against ICMPv6 payloads. Addresses #880.
- "ip6" is now a valid protocol for use with the "header" condition.
(also the "ip-proto" condition, but it results in a no-op in that
case since signatures apply only to the inner-most IP packet when
packets are tunneled). This allows signatures to match specifically
against IPv6 packets (whereas "ip" only matches against IPv4 packets).
- "ip-proto" conditions can now match against IPv6 packets. Before,
IPv6 packets were just silently ignored which meant DPD based on
signatures did not function for IPv6 -- protocol analyzers would only
get attached to a connection over IPv6 based on the well-known ports
set in the "dpd_config" table.
2.1-80 | 2012-10-19 14:48:42 -0700
* Change how "gridftp" gets added to service field of connection
records. In addition to checking for a finished SSL handshake over
an FTP connection, it now also requires that the SSL handshake
occurs after the FTP client requested AUTH GSSAPI, more
specifically identifying the characteristics of GridFTP control
channels. Addresses #891. (Jon Siwek)
* Allow faster rebuilds in certain cases. Previously, when
rebuilding with a different "--prefix" or "--scriptdir", all Bro
source files were recompiled. With this change, only util.cc is
recompiled. (Daniel Thayer)
2.1-76 | 2012-10-12 10:32:39 -0700
* Add support for recognizing GridFTP connections as an extension to
the standard FTP analyzer. (Jon Siwek)
This is enabled by default and includes:
- An analyzer for GSI mechanism of GSSAPI FTP AUTH method. GSI
authentication involves an encoded TLS/SSL handshake over the
FTP control session. For FTP sessions that attempt GSI
authentication, the *service* field of the connection log will
include "gridftp" (as well as also "ftp" and "ssl").
- Add an example of a GridFTP data channel detection script. It
relies on the heuristics of GridFTP data channels commonly
default to SSL mutual authentication with a NULL bulk cipher
and that they usually transfer large datasets (default
threshold of script is 1 GB). The script also defaults to
skip_further_processing() after detection to try to save
cycles analyzing the large, benign connection.
For identified GridFTP data channels, the *services* fields of
the connection log will include "gridftp-data".
* Add *client_subject* and *client_issuer_subject* as &log'd fields
to SSL::Info record. Also add *client_cert* and
*client_cert_chain* fields to track client cert chain. (Jon Siwek)
* Add a script in base/protocols/conn/polling that generalizes the
process of polling a connection for interesting features. The
GridFTP data channel detection script depends on it to monitor
bytes transferred. (Jon Siwek)
2.1-68 | 2012-10-12 09:46:41 -0700
* Rename the Input Framework's update_finished event to end_of_data.
It will now not only fire after table-reads have been completed,
but also after the last event of a whole-file-read (or
whole-db-read, etc.). (Johanna Amann)
* Fix for DNS log problem when a DNS response is seen with 0 RRs.
(Seth Hall)
2.1-64 | 2012-10-12 09:36:41 -0700
* Teach --disable-dataseries/--disable-elasticsearch to ./configure.
Addresses #877. (Jon Siwek)
* Add --with-curl option to ./configure. Addresses #877. (Jon Siwek)
2.1-61 | 2012-10-12 09:32:48 -0700
* Fix bug in the input framework: the config table did not work.
(Johanna Amann)
2.1-58 | 2012-10-08 10:10:09 -0700
* Fix a problem with non-manager cluster nodes applying
Notice::policy. This could, for example, result in duplicate
emails being sent if Notice::emailed_types is redef'd in local.bro
(or any script that gets loaded on all cluster nodes). (Jon Siwek)
2.1-56 | 2012-10-03 16:04:52 -0700
* Add general FAQ entry about upgrading Bro. (Jon Siwek)
2.1-53 | 2012-10-03 16:00:40 -0700
* Add new Tunnel::delay_teredo_confirmation option that indicates
that the Teredo analyzer should wait until it sees both sides of a
connection using a valid Teredo encapsulation before issuing a
protocol_confirmation. Default is on. Addresses #890. (Jon Siwek)
2.1-50 | 2012-10-02 12:06:08 -0700
* Fix a typing issue that prevented the ElasticSearch timeout to
work. (Matthias Vallentin)
* Use second granularity for ElasticSearch timeouts. (Matthias
Vallentin)
* Fix compile issues with older versions of libcurl, which don't
offer *_MS timeout constants. (Matthias Vallentin)
2.1-47 | 2012-10-02 11:59:29 -0700
* Fix for the input framework: BroStrings were constructed without a
final \0, which makes them unusable by basically all internal
functions (like to_count). (Johanna Amann)
* Remove deprecated script functionality (see NEWS for details).
(Daniel Thayer)
2.1-39 | 2012-09-29 14:09:16 -0700
* Reliability adjustments to istate tests with network
communication. (Jon Siwek)
2.1-37 | 2012-09-25 14:21:37 -0700
* Reenable some tests that previously would cause Bro to exit with
an error. (Daniel Thayer)
* Fix parsing of large integers on 32-bit systems. (Daniel Thayer)
* Serialize language.when unit test with the "comm" group. (Jon
Siwek)
2.1-32 | 2012-09-24 16:24:34 -0700
* Fix race condition in language/when.bro test. (Daniel Thayer)
2.1-26 | 2012-09-23 08:46:03 -0700
* Add an item to FAQ page about broctl options. (Daniel Thayer)
* Add more language tests. We now have tests of all built-in Bro
data types (including different representations of constant
values, and max./min. values), keywords, and operators (including
special properties of certain operators, such as short-circuit
evaluation and associativity). (Daniel Thayer)
* Fix construction of ip6_ah (Authentication Header) record values.
Authentication Headers with a Payload Len field set to zero would
cause a crash due to invalid memory allocation because the
previous code assumed Payload Len would always be great enough to
contain all mandatory fields of the header. (Jon Siwek)
* Update compile/dependency docs for OS X. (Jon Siwek)
* Adjusting Mac binary packaging script. Setting CMAKE_PREFIX_PATH
helps link against standard system libs instead of ones that come
from other package manager (e.g. MacPorts). (Jon Siwek)
* Adjusting some unit tests that do cluster communication. (Jon Siwek)
* Small change to non-blocking DNS initialization. (Jon Siwek)
* Reorder a few statements in scan.l to make 1.5msecs etc work.
Adresses #872. (Johanna Amann)
2.1-6 | 2012-09-06 23:23:14 -0700
* Fixed a bug where "a -= b" (both operands are intervals) was not
allowed in Bro scripts (although "a = a - b" is allowed). (Daniel
Thayer)
* Fixed a bug where the "!=" operator with subnet operands was
treated the same as the "==" operator. (Daniel Thayer)
* Add sleeps to configuration_update test for better reliability.
(Jon Siwek)
* Fix a segfault when iterating over a set when using malformed
index. (Daniel Thayer)
2.1 | 2012-08-28 16:46:42 -0700
* Make bif.identify_magic robust against FreeBSD's libmagic config.
(Robin Sommer)
* Remove automatic use of gperftools on non-Linux systems.
--enable-perftools must now explicity be supplied to ./configure
on non-Linux systems to link against the tcmalloc library.
* Fix uninitialized value for 'is_partial' in TCP analyzer. (Jon
Siwek)
* Parse 64-bit consts in Bro scripts correctly. (Johanna Amann)
* Output 64-bit counts correctly on 32-bit machines (Johanna Amann)
* Input framework fixes, including: (Johanna Amann)
- One of the change events got the wrong parameters.
- Escape commas in sets and vectors that were unescaped before
tokenization.
- Handling of zero-length-strings as last element in a set was
broken (sets ending with a ,).
- Hashing of lines just containing zero-length-strings was broken.
- Make set_separators different from , work for input framework.
- Input framework was not handling counts and ints out of
32-bit-range correctly.
- Errors in single lines do not kill processing, but simply ignore
the line, log it, and continue.
* Update documentation for builtin types. (Daniel Thayer)
- Add missing description of interval "msec" unit.
- Improved description of pattern by clarifying the issue of
operand order and difference between exact and embedded
matching.
* Documentation fixes for signature 'eval' conditions. (Jon Siwek)
* Remove orphaned 1.5 unit tests. (Jon Siwek)
* Add type checking for signature 'eval' condition functions. (Jon
Siwek)
* Adding an identifier to the SMTP blocklist notices for duplicate
suppression. (Seth Hall)
2.1-beta-45 | 2012-08-22 16:11:10 -0700
* Add an option to the input framework that allows the user to chose
to not die upon encountering files/functions. (Johanna Amann)
2.1-beta-41 | 2012-08-22 16:05:21 -0700
* Add test serialization to "leak" unit tests that use
communication. (Jon Siwek)
* Change to metrics/basic-cluster unit test for reliability. (Jon
Siwek)
* Fixed ack tracking which could overflow quickly in some
situations. (Seth Hall)
* Minor tweak to coverage.bare-mode-errors unit test to work with a
symlinked 'scripts' dir. (Jon Siwek)
2.1-beta-35 | 2012-08-22 08:44:52 -0700
* Add testcase for input framework reading sets (rather than
tables). (Johanna Amann)
2.1-beta-31 | 2012-08-21 15:46:05 -0700
* Tweak to rotate-custom.bro unit test. (Jon Siwek)
* Ignore small mem leak every rotation interval for dataseries logs.
(Jon Siwek)
2.1-beta-28 | 2012-08-21 08:32:42 -0700
* Linking ES docs into logging document. (Robin Sommer)
2.1-beta-27 | 2012-08-20 20:06:20 -0700
* Add the Stream record to Log:active_streams to make more dynamic
logging possible. (Seth Hall)
* Fix portability of printing to files returned by
open("/dev/stderr"). (Jon Siwek)
* Fix mime type diff canonifier to also skip mime_desc columns. (Jon
Siwek)
* Unit test tweaks/fixes. (Jon Siwek)
- Some baselines for tests in "leaks" group were outdated.
- Changed a few of the cluster/communication tests to terminate
more explicitly instead of relying on btest-bg-wait to kill
processes. This makes the tests finish faster in the success case
and makes the reason for failing clearer in the that case.
* Fix memory leak of serialized IDs when compiled with
--enable-debug. (Jon Siwek)
2.1-beta-21 | 2012-08-16 11:48:56 -0700
* Installing a handler for running out of memory in "new". Bro will
now print an error message in that case rather than abort with an
uncaught exception. (Robin Sommer)
2.1-beta-20 | 2012-08-16 11:43:31 -0700
* Fixed potential problems with ElasticSearch output plugin. (Seth
Hall)
2.1-beta-13 | 2012-08-10 12:28:04 -0700
* Reporter warnings and error now print to stderr by default. New
options Reporter::warnings_to_stderr and
Reporter::errors_to_stderr to disable. (Seth Hall)
2.1-beta-9 | 2012-08-10 12:24:29 -0700
* Add more BIF tests. (Daniel Thayer)
2.1-beta-6 | 2012-08-10 12:22:52 -0700
* Fix bug in input framework with an edge case. (Johanna Amann)
* Fix small bug in input framework test script. (Johanna Amann)
2.1-beta-3 | 2012-08-03 10:46:49 -0700
* Merge branch 'master' of ssh://git.bro-ids.org/bro (Robin Sommer)
* Fix configure script to exit with non-zero status on error (Jon
Siwek)
* Improve ASCII output performance. (Robin Sommer)
2.1-beta | 2012-07-30 11:59:53 -0700
* Improve log filter compatibility with remote logging. Addresses
#842. (Jon Siwek)
2.0-907 | 2012-07-30 09:13:36 -0700
* Add missing breaks to switch cases in
ElasticSearch::HTTPReceive(). (Jon Siwek)
2.0-905 | 2012-07-28 16:24:34 -0700
* Fix log manager hanging on waiting for pending file rotations,
plus writer API tweak for failed rotations. Addresses #860. (Jon
Siwek and Robin Sommer)
* Tweaking logs-to-elasticsearch.bro so that it doesn't do anything
if ES server is unset. (Robin Sommer)
2.0-902 | 2012-07-27 12:42:13 -0700
* New variable in logging framework Log::active_streams to indicate
Log:ID enums which are currently active. (Seth Hall)
* Reworked how the logs-to-elasticsearch scripts works to stop
abusing the logging framework. (Seth Hall)
* Fix input test for recent default change on fastpath. (Robin
Sommer)
2.0-898 | 2012-07-27 12:22:03 -0700
* Small (potential performance) improvement for logging framework. (Seth Hall)
* Script-level rotation postprocessor fix. This fixes a problem with
writers that don't have a postprocessor. (Seth Hall)
* Update input framework documentation to reflect want_record
change. (Johanna Amann)
* Fix crash when encountering an InterpreterException in a predicate
in logging or input Framework. (Johanna Amann)
* Input framework: Make want_record=T the default for events
(Johanna Amann)
* Changing the start/end markers in logs to open/close now
reflecting wall clock. (Robin Sommer)
2.0-891 | 2012-07-26 17:15:10 -0700
* Reader/writer API: preventing plugins from receiving further
messages after a failure. (Robin Sommer)
* New test for input framework that fails to find a file. (Robin
Sommer)
* Improving error handling for threads. (Robin Sommer)
* Tweaking the custom-rotate test to produce stable output. (Robin
Sommer)
2.0-884 | 2012-07-26 14:33:21 -0700
* Add comprehensive error handling for close() calls. (Jon Siwek)
* Add more test cases for input framework. (Johanna Amann)
* Input framework: make error output for non-matching event types
much more verbose. (Johanna Amann)
2.0-877 | 2012-07-25 17:20:34 -0700
* Fix double close() in FilerSerializer class. (Jon Siwek)
* Fix build warnings. (Daniel Thayer)
* Fixes to ElasticSearch plugin to make libcurl handle http
responses correctly. (Seth Hall)
* Fixing FreeBSD compiler error. (Robin Sommer)
* Silencing compiler warnings. (Robin Sommer)
2.0-871 | 2012-07-25 13:08:00 -0700
* Fix complaint from valgrind about uninitialized memory usage. (Jon
Siwek)
* Fix differing log filters of streams from writing to same
writer/path (which now produces a warning, but is otherwise
skipped for the second). Addresses #842. (Jon Siwek)
* Fix tests and error message for to_double BIF. (Daniel Thayer)
* Compile fix. (Robin Sommer)
2.0-866 | 2012-07-24 16:02:07 -0700
* Correct a typo in usage message. (Daniel Thayer)
* Fix file permissions of log files (which were created with execute
permissions after a recent change). (Daniel Thayer)
2.0-862 | 2012-07-24 15:22:52 -0700
* Fix initialization problem in logging class. (Jon Siwek)
* Input framework now accepts escaped ASCII values as input (\x##),
and unescapes appropiately. (Johanna Amann)
* Make reading ASCII logfiles work when the input separator is
different from \t. (Johanna Amann)
* A number of smaller fixes for input framework. (Johanna Amann)
2.0-851 | 2012-07-24 15:04:14 -0700
* New built-in function to_double(s: string). (Scott Campbell)
2.0-849 | 2012-07-24 11:06:16 -0700
* Adding missing include needed on some systems. (Robin Sommer)
2.0-846 | 2012-07-23 16:36:37 -0700
* Fix WriterBackend::WriterInfo serialization, reenable ascii
start/end tags. (Jon Siwek)
2.0-844 | 2012-07-23 16:20:59 -0700
* Reworking parts of the internal threading/logging/input APIs for
thread-safety. (Robin Sommer)
* Bugfix for SSL version check. (Johanna Amann)
* Changing a HTTP DPD from port 3138 to 3128. Addresses #857. (Robin
Sommer)
* ElasticSearch logging writer. See logging-elasticsearch.rst for
more information. (Vlad Grigorescu and Seth Hall).
* Give configure a --disable-perftools option to disable Perftools
support even if found. (Robin Sommer)
* The ASCII log writer now includes "#start <timestamp>" and "#end
<timestamp> lines in the each file. (Robin Sommer)
* Renamed ASCII logger "header" options to "meta". (Robin Sommer)
* ASCII logs now escape '#' at the beginning of log lines. Addresses
#763. (Robin Sommer)
* Fix bug, where in dns.log rcode always was set to 0/NOERROR when
no reply package was seen. (Johanna Amann)
* Updating to Mozilla's current certificate bundle. (Seth Hall)
2.0-769 | 2012-07-13 16:17:33 -0700
* Fix some Info:Record field documentation. (Vlad Grigorescu)
* Fix overrides of TCP_ApplicationAnalyzer::EndpointEOF. (Jon Siwek)
* Fix segfault when incrementing whole vector values. Also removed
RefExpr::Eval(Val*) method since it was never called. (Jon Siwek)
* Remove baselines for some leak-detecting unit tests. (Jon Siwek)
* Unblock SIGFPE, SIGILL, SIGSEGV and SIGBUS for threads, so that
they now propagate to the main thread. Adresses #848. (Johanna
Amann)
2.0-761 | 2012-07-12 08:14:38 -0700
* Some small fixes to further reduce SOCKS false positive logs. (Seth Hall)
* Calls to pthread_mutex_unlock now log the reason for failures.
(Johanna Amann)
2.0-757 | 2012-07-11 08:30:19 -0700
* Fixing memory leak. (Seth Hall)
2.0-755 | 2012-07-10 16:25:16 -0700
* Add sorting canonifier to rotate-custom unit test. Addresses #846.
(Jon Siwek)
* Fix many compiler warnings. (Daniel Thayer)
* Fix segfault when there's an error/timeout resolving DNS requests.
Addresses #846. (Jon Siwek)
* Remove a non-portable test case. (Daniel Thayer)
* Fix typos in input framework doc. (Daniel Thayer)
* Fix typos in DataSeries documentation. (Daniel Thayer)
* Bugfix making custom rotate functions work again. (Robin Sommer)
* Tiny bugfix for returning writer name. (Robin Sommer)
* Moving make target update-doc-sources from top-level Makefile to
btest Makefile. (Robin Sommer)
2.0-733 | 2012-07-02 15:31:24 -0700
* Extending the input reader DoInit() API. (Johanna Amann). It now
provides a Info struct similar to what we introduced for log
writers, including a corresponding "config" key/value table.
* Fix to make writer-info work when debugging is enabled. (Johanna
Amann)
2.0-726 | 2012-07-02 15:19:15 -0700
* Extending the log writer DoInit() API. (Robin Sommer)
We now pass in a Info struct that contains:
- the path name (as before)
- the rotation interval
- the log_rotate_base_time in seconds
- a table of key/value pairs with further configuration options.
To fill the table, log filters have a new field "config: table[string]
of strings". This gives a way to pass arbitrary values from
script-land to writers. Interpretation is left up to the writer.
* Split calc_next_rotate() into two functions, one of which is
thread-safe and can be used with the log_rotate_base_time value
from DoInit().
* Updates to the None writer. (Robin Sommer)
- It gets its own script writers/none.bro.
- New bool option LogNone::debug to enable debug output. It then
prints out all the values passed to DoInit().
- Fixed a bug that prevented Bro from terminating.
2.0-723 | 2012-07-02 15:02:56 -0700
* Extract ICMPv6 NDP options and include in ICMP events. This adds
a new parameter of type "icmp6_nd_options" to the ICMPv6 neighbor
discovery events. Addresses #833. (Jon Siwek)
* Set input frontend type before starting the thread. This means
that the thread type will be output correctly in the error
message. (Johanna Amann)
2.0-719 | 2012-07-02 14:49:03 -0700
* Fix inconsistencies in random number generation. The
srand()/rand() interface was being intermixed with the
srandom()/random() one. The later is now used throughout. (Jon
Siwek)
* Changed the srand() and rand() BIFs to work deterministically if
Bro was given a seed file. Addresses #825. (Jon Siwek)
* Updating input framework unit tests to make them more reliable and
execute quicker. (Jon Siwek)
* Fixed race condition in writer and reader initializations. (Jon
Siwek)
* Small tweak to make test complete quicker. (Jon Siwek)
* Drain events before terminating log/thread managers. (Jon Siwek)
* Fix strict-aliasing warning in RemoteSerializer.cc. Addresses
#834. (Jon Siwek)
* Fix typos in event documentation. (Daniel Thayer)
* Fix typos in NEWS for Bro 2.1 beta. (Daniel Thayer)
2.0-709 | 2012-06-21 10:14:24 -0700
* Fix exceptions thrown in event handlers preventing others from running. (Jon Siwek)
* Add another SOCKS command. (Seth Hall)
* Fixed some problems with the SOCKS analyzer and tests. (Seth Hall)
* Updating NEWS in preparation for beta. (Robin Sommer)
* Accepting different AF_INET6 values for loopback link headers.
(Robin Sommer)
2.0-698 | 2012-06-20 14:30:40 -0700
* Updates for the SOCKS analyzer (Seth Hall).
- A SOCKS log!
- Now supports SOCKSv5 in the analyzer and the DPD sigs.
- Added protocol violations.
* Updates to the tunnels framework. (Seth Hall)
- Make the uid field optional since it's conceptually incorrect
for proxies being treated as tunnels to have it.
- Reordered two fields in the log.
- Reduced the default tunnel expiration interface to something
more reasonable (1 hour).
* Make Teredo bubble packet parsing more lenient. (Jon Siwek)
* Fix a crash in NetSessions::ParseIPPacket(). (Jon Siwek)
2.0-690 | 2012-06-18 16:01:33 -0700
* Support for decapsulating tunnels via the new tunnel framework in
base/frameworks/tunnels.
Bro currently supports Teredo, AYIYA, IP-in-IP (both IPv4 and
IPv6), and SOCKS. For all these, it logs the outher tunnel
connections in both conn.log and tunnel.log, and proceeds to
analyze the inner payload as if it were not tunneled, including
also logging it in conn.log (with a new tunnel_parents column
pointing back to the outer connection(s)). (Jon Siwek, Seth Hall,
Gregor Maier)
* The options "tunnel_port" and "parse_udp_tunnels" have been
removed. (Jon Siwek)
2.0-623 | 2012-06-15 16:24:52 -0700
* Changing an error in the input framework to a warning. (Robin
Sommer)
2.0-622 | 2012-06-15 15:38:43 -0700
* Input framework updates. (Johanna Amann)
- Disable streaming reads from executed commands. This lead to
hanging Bros because pclose apparently can wait for eternity if
things go wrong.
- Automatically delete disabled input streams.
- Documentation.
2.0-614 | 2012-06-15 15:19:49 -0700
* Remove an old, unused diff canonifier. (Jon Siwek)
* Improve an error message in ICMP analyzer. (Jon Siwek)
* Fix a warning message when building docs. (Daniel Thayer)
* Fix many errors in the event documentation. (Daniel Thayer)
2.0-608 | 2012-06-11 15:59:00 -0700
* Add more error handling code to logging of enum vals. Addresses
#829. (Jon Siwek)
2.0-606 | 2012-06-11 15:55:56 -0700
* Fix summary lines for BIF documentation and corrected the
description of "fmt" and "floor" BIFs. (Daniel Thayer)
* Fix val_size BIF tests and improve docs. (Daniel Thayer)
2.0-602 | 2012-06-07 15:06:19 -0700
* Include header for usleep(), caused compile failure on Archlinux. (Jon Siwek)
* Revert "Fixed a bug with the MIME analyzer not removing whitespace
on wrapped headers." Needs discussion. (Robin Sommer)
2.0-598 | 2012-06-06 11:47:00 -0700
* Add @load-sigs directive for loading signature files (addresses
#551). This can be used to load signatures relative to the current
scripts (e.g., "@load-sigs ./foo.sig"). (Jon Siwek)
2.0-596 | 2012-06-06 11:41:00 -0700
* Fixes for some BiFs and their documentation. (Daniel Thayer)
* Many new unit tests for BiFs. (Daniel Thayer)
2.0-579 | 2012-06-06 11:04:46 -0700
* Memory leak fixes for bad usages of VectorVal ctor. (Jon Siwek)
* Fixed a bug with the MIME analyzer not removing whitespace on
wrapped headers. (Seth Hall)
* Change Input::update_finished lookup to happen at init time. (Jon Siwek)
* Fix going through the internal_handler() function which will now
set the event as "used" (i.e. it's marked as being raised
somewhere). Addresses #823. (Jon Siwek)
* Fix format specifier on RemoteSerializer::Connect. This caused
32-bit systems to show a warning at compile-time, and fail when
connecting to peers. (Jon Siwek)
* Fixes for running tests in parallel. (Robin Sommer)
2.0-571 | 2012-05-30 19:12:43 -0700
* Updating submodule(s).
2.0-570 | 2012-05-30 19:08:18 -0700
* A new input framework enables scripts to read in external data
dynamically on the fly as Bro is processing network traffic.
(Johanna Amann)
Currently, the framework supports reading ASCII input that's
structured similar as Bro's log files as well as raw blobs of
data. Other formats will come in the future.
See doc/input.rst for more information (this will be extended
further soon).
2.0-395 | 2012-05-30 17:03:31 -0700
* Remove unnecessary assert in ICMP analyzer which could lead to
aborts. Addresses #822.
* Improve script debugger backtrace and print commands. (Jon Siwek)
* Switching default DS compression to gzip. (Robin Sommer)
* Improve availability of IPv6 flow label in connection records.
This adds a "flow_label" field to the "endpoint" record type,
which is used for both the "orig" and "resp" fields of
"connection" records. The new "connection_flow_label_changed"
event also allows tracking of changes in flow labels: it's raised
each time one direction of the connection starts using a different
label. (Jon Siwek)
* Add unit tests for Broccoli SSL and Broccoli IPv6 connectivity.
(Jon Siwek)
* Remove AI_ADDRCONFIG getaddrinfo hints flag for listening sockets.
(Jon Siwek)
* Undo unnecessary communication protocol version bump. (Jon Siwek)
* Add support to Bro for connecting with peers over IPv6. (Jon Siwek)
- Communication::listen_ipv6 needs to be redef'd to true in order
for IPv6 listening sockets to be opened.
- Added Communication::listen_retry option as an interval at which
to retry binding to socket addresses that were already in use.
- Added some explicit baselines to check in the istate.events and
istate.events-ssl tests -- the SSL test was incorrectly passing
because it compared two empty files. (The files being empty
because "http/base" was given as an argument to Bro which it
couldn't handle because that script doesn't exist anymore).
- Support for communication over non-global IPv6 addresses. This
usually requires specifying an additional zone identifier (see
RFC 4007). The connect() and listen() BIFs have been changed to
accept this zone identifier as an argument.
2.0-377 | 2012-05-24 16:46:06 -0700
* Documentation fixes. (Jon Siwek and Daniel Thayer)
2.0-372 | 2012-05-17 13:59:45 -0700
* Fix compile errors. (Jon Siwek)
* Linking in the DS docs. (Robin Sommer)
* Fix mobility checksums unit test. (Jon Siwek)
2.0-367 | 2012-05-17 12:42:30 -0700
* Adding support for binary output via DataSeries. See
logging-dataseries.rst for more information. (Gilbert Clark and
Robin Sommer)
* Adding target update-doc-sources to top-level Makefile that runs
genDocSourcesList.sh. (Robin Sommer)
* Moving trace for rotation test into traces directory. (Robin Sommer)
* Fixing a rotation race condition at termination. (Robin Sommer)
* Extending log post-processor call to include the name of the
writer. (Robin Sommer)
* In threads, an internal error now immediately aborts. Otherwise,
the error won't make it back to the main thread for a while and
subsequent code in the thread would still execute. (Robin Sommer)
* DataSeries cleanup. (Robin Sommer)
* Fixing threads' DoFinish() method. It wasn't called reliably. Now,
it's always called before the thread is destroyed (assuming
processing has went normally so far). (Robin Sommer)
2.0-341 | 2012-05-17 09:54:30 -0700
* Add a comment to explain the ICMPv6 error message types. (Daniel Thayer)
* Quieting external test output somehwat. (Robin Sommer)
2.0-336 | 2012-05-14 17:15:44 -0700
* Don't print the various "weird" events to stderr. Address #805.
(Daniel Thayer)
* Generate icmp_error_message event for ICMPv6 error msgs.
Previously, icmp_sent was being generated, but icmp_error_message
contains more info.
* Improved documentation comments for icmp-related events. (Daniel
Thayer)
2.0-330 | 2012-05-14 17:05:56 -0700
* Add `addr_to_uri` script-level function that adds brackets to an
address if it's IPv6 and will be included in a URI or when a
":<port>" needs to be appended to it. (Jon Siwek)
* Also add a test case for content extraction. (Jon Siwek)
* Fix typos and improve INSTALL document. (Daniel Thayer)
* Switching to new btest command TEST-SERIALIZE for communication
tests. (Robin Sommer)
2.0-323 | 2012-05-04 21:04:34 -0700
* Add SHA1 and SHA256 hashing BIFs. Addresses #542.
* Refactor all internal MD5 stuff to use OpenSSL's. (Jon Siwek)
* Changes to open-file caching limits and uncached file unserialization. (Jon Siwek)
- Unserializing files that were previously kicked out of the open-file
cache would cause them to be fopen'd with the original access
permissions which is usually 'w' and causes truncation. They
are now opened in 'a' mode. (addresses #780)
- Add 'max_files_in_cache' script option to manually set the maximum
amount of opened files to keep cached. Mainly this just helped
to create a simple test case for the above change.
- Remove unused NO_HAVE_SETRLIMIT preprocessor switch.
- On systems that don't enforce a limit on number of files opened for
the process, raise default max size of open-file cache from
32 to 512.
2.0-319 | 2012-05-03 13:24:44 -0700
* SSL bugfixes and cleanup. (Seth Hall)
- SSL related files and classes renamed to remove the "binpac" term.
- A small fix for DPD scripts to make the DPD log more helpful if
there are multiple continued failures.
- Fixed the SSL analyzer to make it stop doing repeated violation
messages for some handshake failures.
- Added a $issuer_subject to the SSL log.
- Created a basic test for SSL.
- Fixed parsing of TLS server extensions. (Seth Hall)
2.0-315 | 2012-05-03 11:44:17 -0700
* Add two more TLS extension values that we see in live traffic.
(Johanna Amann)
* Fixed IPv6 link local unicast CIDR and added IPv6 loopback to
private address space. (Seth Hall)
* Fixed a problem where cluster workers were still processing
notices in some cases. (Seth Hall)
* Added a configure option to specify the 'etc' directory. Addresses
#801. (Daniel Thayer)
2.0-306 | 2012-04-24 14:37:00 -0700
* Add further TLS extension values "extended_random" and
"heartbeat". (Seth Hall)
* Fix problem with extracting FTP passwords and add "ftpuser" as
another anonymous username. (Seth Hall, discovered by Patrik
Lundin).
2.0-303 | 2012-04-19 10:01:06 -0700
* Changes related to ICMPv6 Neighbor Discovery messages. (Jon Siwek)
- The 'icmp_conn' record now contains an 'hlim' field since hop limit
in the IP header is an interesting field for at least these ND
messages.
- Fixed and extended 'icmp_router_advertisement' event parameters.
- Changed 'icmp_neighbor_advertisement' event parameters to add
more of the known boolean flags.
2.0-301 | 2012-04-17 17:58:55 -0700
* Bro now support ICMPv6. (Matti Mantere, Jon Siwek, Robin Sommer,
Daniel Thayer).
Overall, Bro now raises the following ICMP events for v4 and v6 as
appropiate:
event icmp_sent(c: connection, icmp: icmp_conn);
event icmp_echo_request(c: connection, icmp: icmp_conn, id: count, seq: count, payload: string);
event icmp_echo_reply(c: connection, icmp: icmp_conn, id: count, seq: count, payload: string);
event icmp_error_message(c: connection, icmp: icmp_conn, code: count, context: icmp_context);
event icmp_unreachable(c: connection, icmp: icmp_conn, code: count, context: icmp_context);
event icmp_packet_too_big(c: connection, icmp: icmp_conn, code: count, context: icmp_context);
event icmp_time_exceeded(c: connection, icmp: icmp_conn, code: count, context: icmp_context);
event icmp_parameter_problem(c: connection, icmp: icmp_conn, code: count, context: icmp_context);
event icmp_router_solicitation(c: connection, icmp: icmp_conn);
event icmp_router_advertisement(c: connection, icmp: icmp_conn, hop_limit: count, managed: bool, router_lifetime: count, reachable_time: interval, retrans_timer: interval);
event icmp_neighbor_solicitation(c: connection, icmp: icmp_conn, tgt:addr);
event icmp_neighbor_advertisement(c: connection, icmp: icmp_conn, tgt:addr);
event icmp_redirect(c: connection, icmp: icmp_conn, tgt: addr, dest: addr);
The `icmp_conn` record got a new boolean field 'v6' that indicates
whether the ICMP message is v4 or v6.
This change also includes further low-level work on existing IP
and ICMP code, including a reorganization of how ICMPv4 is
handled.
2.0-281 | 2012-04-17 17:40:39 -0700
* Small updates for the bittorrent analyzer to support 64bit types
in binpac. (Seth Hall)
* Removed the attempt at bittorrent resynchronization. (Seth Hall)
2.0-276 | 2012-04-17 17:35:56 -0700
* Add more support for <netinet/ip6.h>'s that lack some structure
definitions. (Jon Siwek)
2.0-273 | 2012-04-16 18:08:56 -0700
* Removing QR flag from DNS log in response, which should not have
been there in the first place. (Seth Hall)
* Sync up patricia.c/h with pysubnettree repo. (Daniel Thayer)
* Adding missing leak groups to a couple tests. Also activating leak
checking for proxy in basic-cluster test. (Robin Sommer)
2.0-267 | 2012-04-09 17:47:28 -0700
* Add support for mobile IPv6 Mobility Header (RFC 6275). (Jon
Siwek)
- Enabled through a new --enable-mobile-ipv6 configure-time
option. If not enabled, the mobility header (routing type 2) and
Home Address Destination option are ignored.
- Accessible at script-layer through 'mobile_ipv6_message' event.
* Refactor IP_Hdr routing header handling, add MobileIPv6 Home
Address handling. Packets that use the Home Address Destination
option use that option's address as the connection's originator.
(Jon Siwek)
* Revert TCP checksumming to cache common data, like it did before.
(Jon Siwek)
* Improve handling of IPv6 routing type 0 extension headers. (Jon
Siwek)
- flow_weird event with name argument value of "routing0_hdr" is raised
for packets containing an IPv6 routing type 0 header because this
type of header is now deprecated according to RFC 5095.
- Packets with a routing type 0 header and non-zero segments left
now use the last address in that header in order to associate
with a connection/flow and for calculating TCP/UDP checksums.
- Added a set of IPv4/IPv6 TCP/UDP checksum unit tests (Jon Siwek)
* Fix table expiry for values assigned in bro_init() when reading
live. (Jon Siwek)
2.0-257 | 2012-04-05 15:32:43 -0700
* Fix CMake from warning about unused ENABLE_PERFTOOLS_DEBUG
variable. (Jon Siwek)
* Fix handling of IPv6 atomic fragments. (Jon Siwek)
* Fix that prevents Bro processes that do neither local logging nor
request remote logs from spawning threads. (Robin Sommer)
* Fixing perftools-debug support. (Robin Sommer)
* Reverting SocketComm change tuning I/O behaviour. (Robin Sommer)
* Adding notice_policy.log canonification for external tests. (Robin Sommer)
2.0-245 | 2012-04-04 17:25:20 -0700
* Internal restructuring of the logging framework: we now spawn
threads doing the I/O. From a user's perspective not much should
change, except that the OS may now show a bunch of Bro threads.
(Gilbert Clark and Robin Sommer).
* When building Bro, we now always link in tcmalloc if it's found at
configure time. If it's installed but not picked up,
--with-perftools may help. (Robin Sommer)
* Renaming the configure option --enable-perftools to
--enable-perftool-debug to indicate that the switch is only
relevant for debugging the heap. It's not needed to pick up
tcmalloc for better performance. (Robin Sommer)
2.0-184 | 2012-03-28 15:11:11 -0700
* Improve handling of IPv6 Routing Type 0 headers. (Jon Siwek)
- For RH0 headers with non-zero segments left, a
"routing0_segleft" flow_weird event is raised (with a
destination indicating the last address in the routing header),
and an "rh0_segleft" event can also be handled if the other
contents of the packet header are of interest. No further
analysis is done as the complexity required to correctly
identify destination endpoints of connections doesn't seem worth
it as RH0 has been deprecated by RFC 5095.
- For RH0 headers without any segments left, a "routing0_header"
flow_weird event is raised, but further analysis still occurs as
normal.
2.0-182 | 2012-03-28 15:01:57 -0700
* Remove dead tcp_checksum function from net_util. (Jon Siwek)
* Change routing0_data_to_addrs BIF to return vector of addresses.
The order of addresses in type 0 routing headers is
interesting/important. (Jon Siwek)
2.0-179 | 2012-03-23 17:43:31 -0700
* Remove the default "tcp or udp or icmp" filter. In default mode,
Bro would load the packet filter script framework which installs a
filter that allows all packets, but in bare mode (the -b option),
this old filter would not follow IPv6 protocol chains and thus
filter out packets with extension headers. (Jon Siwek)
* Update PacketFilter/Discarder code for IP version independence.
(Jon Siwek)
* Fix some IPv6 header related bugs. (Jon Siwek)
* Add IPv6 fragment reassembly. (Jon Siwek)
* Add handling for IPv6 extension header chains. Addresses #531.
(Jon Siwek)
- The script-layer 'pkt_hdr' type is extended with a new 'ip6' field
representing the full IPv6 header chain.
- The 'new_packet' event is now raised for IPv6 packets. Addresses
#523.
- A new event called 'ipv6_ext_header' is raised for any IPv6
packet containing extension headers.
- A new event called 'esp_packet' is raised for any packets using
ESP ('new_packet' and 'ipv6_ext_header' events provide
connection info, but that info can't be provided here since the
upper-layer payload is encrypted).
- The 'unknown_protocol' weird is now raised more reliably when
Bro sees a transport protocol or IPv6 extension header it can't
handle. Addresses #522.
* Add unit tests for IPv6 fragment reassembly, ipv6_ext_headers and
esp_packet events. (Jon Siwek)
* Adapt FreeBSD's inet_ntop implementation for internal use. Now we
get consistent text representations of IPv6 addresses across
platforms. (Jon Siwek)
* Update documentation for new syntax of IPv6 literals. (Jon Siwek)
2.0-150 | 2012-03-13 16:16:22 -0700
* Changing the regular expression to allow Site::local_nets in
signatures. (Julien Sentier)
* Removing a line of dead code. Found by . Closes #786. (Julien
Sentier)
2.0-146 | 2012-03-13 15:39:38 -0700
* Change IPv6 literal constant syntax to require encasing square
brackets. (Jon Siwek)
2.0-145 | 2012-03-09 15:10:35 -0800
* Remove the match expression. 'match' and 'using' are no longer
keywords. Addressed #753. (Jon Siwek)
2.0-143 | 2012-03-09 15:07:42 -0800
* Fix a BRO_PROFILER_FILE/mkstemp portability issue. Addresses #794.
(Jon Siwek)
2.0-139 | 2012-03-02 09:33:04 -0800
* Changes to how script coverage integrates with test suites. (Jon Siwek)
- BRO_PROFILER_FILE now passes .X* templated filenames to mkstemp
for generating unique coverage state files.
- Rearranging Makefile targets. The general rule is that if the
all/brief target fails out due to a test failure, then the dependent
coverage target won't run, but can still be invoked directly later.
(e.g. make brief || make coverage)
* Standardized on the &default function for SSL constants. (Seth
Hall)
* Adding btest group "leaks" to leak tests. (Robin Sommer)
* Adding btest group "comm" to communication tests for parallelizing
execution with new btest version. (Robin Sommer)
* Sorting all output for diffing in the external tests. (Robin
Sommer)
* Cleaned up dead code from the old SSL analyzers. Reported by
Julien Sentier. (Seth Hall)
* Update/add tests for broccoli IPv6 addr/subnet support. Addresses
#448. (Jon Siwek)
* Remove connection compressor. Addresses #559. (Jon Siwek)
* Refactor IP_Hdr class ctors. Addresses #532. (Jon Siwek)
2.0-121 | 2012-02-24 16:34:17 -0800
* A number of smaller memory fixes and code cleanups. (Julien
Sentier)
* Add to_subnet bif. Fixes #782). (Jon Siwek)
* Fix IPAddr::Mask/ReverseMask not allowing argument of 0. (Jon
Siwek)
* Refactor IPAddr v4 initialization from string. Fixes #775. (Jon Siwek)
* Parse the dotted address string directly instead of canonicalizing
and passing to inet_pton. (Jon Siwek)
2.0-108 | 2012-02-24 15:21:07 -0800
* Refactoring a number of usages of new IPAddr class. (Jon Siwek)
* Fixed a bug in remask_addr bif. (Jon Siwek)
2.0-106 | 2012-02-24 15:02:20 -0800
* Raise minimum required CMake version to 2.6.3. (Jon Siwek)
2.0-104 | 2012-02-24 14:59:12 -0800
* Add test case for FTP over IPv4. (Daniel Thayer)
* Fix IPv6 URLs in ftp.log. (Daniel Thayer)
* Add a test for FTP over IPv6 (Daniel Thayer)
* Fix parsing of FTP EPRT command and EPSV response. (Daniel Thayer)
2.0-95 | 2012-02-22 05:27:34 -0800
* GeoIP installation documentation update. (Seth Hall)
* Decrease strictness of parsing IPv4 strings into addrs. Fixes #775. (Jon Siwek)
* Fix memory leak in DNS manager. Fixes #777. (Jon Siwek)
* Fix IPAddr/IPPrefix serialization bugs. (Jon Siwek)
* Fix compile error. (Jon Siwek)
2.0-86 | 2012-02-17 15:41:06 -0800
* Changing ARP detection to always kick in even if no analyzer is
activated. (Robin Sommer)
* DNS name lookups performed by Bro now also query AAAA records.
DNS_Mgr handles combining the results of the A and AAAA queries
for a given hostname such that at the scripting layer, the name
resolution can yield a set with both IPv4 and IPv6 addresses. (Jon
Siwek)
* Add counts_to_addr and addr_to_counts conversion BIFs. (Jon Siwek)
* Change HashKey threshold for using H3 to 36 bytes. (Jon Siwek)
* Remove mention of --enable-brov6 in docs. (Daniel Thayer)
* Remove --enable-brov6 from configure usage text (Daniel Thayer)
* Add a test and baseline for addr_to_ptr_name BiF. (Daniel Thayer)
* Adding a test and baseline for ptr_name_to_addr BiF. (Seth Hall)
* Fix the ptr_name_to_addr BiF to work with IPv6 (Daniel Thayer)
* Fix a memory leak that perftools now complains about. (Jon Siwek)
* Remove --enable-brov6 flag, IPv6 now supported by default. (Jon Siwek)
Some script-layer changes of note:
- dns_AAAA_reply event signature changed: the string representation
of an IPv6 addr is easily derived from the addr value, it doesn't
need to be another parameter. This event also now generated directly
by the DNS analyzer instead of being "faked" into a dns_A_reply event.
- Removed addr_to_count BIF. It used to return the host-order
count representation of IPv4 addresses only. To make it more
generic, we might later add a BIF to return a vector of counts
in order to support IPv6.
- Changed the result of enclosing addr variables in vertical pipes
(e.g. |my_addr|) to return the bit-width of the address type which
is 128 for IPv6 and 32 for IPv4. It used to function the same
way as addr_to_count mentioned above.
- Remove bro_has_ipv6 BIF
2.0-57 | 2012-02-10 00:02:35 -0800
* Fix typos in the documentation. (Daniel Thayer)
* Fix compiler warning about Brofiler ctor init list order. (Jon Siwek)
* Fix missing optional field access in webapp signature_match handler. (Jon Siwek)
2.0-41 | 2012-02-03 04:10:53 -0500
* Updates to the Software framework to simplify the API. (Johanna
Amann)
2.0-40 | 2012-02-03 01:55:27 -0800
* Fix typos in documentation. (Daniel Thayer)
* Fix sorting of lines in Brofiler coverage.log. (Daniel Thayer)
2.0-38 | 2012-01-31 11:50:53 -0800
* Canonify sorting of lines in Brofiler coverage.log. (Daniel
Thayer)
2.0-36 | 2012-01-27 10:38:14 -0800
* New "Brofiler" mode that tracks and records script statements
executed during runtime. (Jon Siwek)
Use the BROFILER_FILE environment variable to point to a file in
which statement usage statistics from Bro script-layer can be
output.
Script statements that should be ignored can be marked with a "#
@no-test" comment. For example:
print "don't cover"; # @no-test
if ( F )
{ # @no-test
...
}
* Integrated coverage measurement into test-suite. (Jon Siwek)
2.0-20 | 2012-01-25 16:34:51 -0800
* BiF cleanup (Matthias Vallentin)
- Rename NFS3::mode2string to a more generic file_mode().
- Unify do_profiling()/make_connection_persistent()/expect_connection()
to return any (i.e., nothing) instead of bools.
- Perform type checking on count-to-port conversion. Related to #684.
- Remove redundant connection_record() BiF. The same
functionality is provided by lookup_connection().
- Remove redundant active_connection() BiF. The same
functionality is provided by connection_exists().
- exit() now takes the exit code as argument.
- to_port() now received a string instead of a count.
2.0-9 | 2012-01-25 13:47:13 -0800
* Allow local table variables to be initialized with {} list
expressions. (Jon Siwek)
2.0-7 | 2012-01-25 13:38:09 -0800
* Teach CompHash to allow indexing by records with vector/table/set
fields. Addresses #464. (Jon Siwek)
2.0-5 | 2012-01-25 13:25:19 -0800
* Fixed a bug resulting in over-logging of detected webapps. (Seth Hall)
* Make communication log baseline test more reliable. (Jon Siwek)
* Fixed some broken links in documentation. (Daniel Thayer)
2.0 | 2012-01-11 13:52:22 -0800
* Adding script reference documentation. (The Team).
2.0-beta-194 | 2012-01-10 10:44:32 -0800
* Added an option for filtering out URLs before they are turned into
HTTP::Incorrect_File_Type notices. (Seth Hall)
* Fix ref counting bug in BIFs that call internal_type. Addresses
#740. (Jon Siwek)
* Adding back the stats.bro file. (Seth Hall)
2.0-beta-188 | 2012-01-10 09:49:29 -0800
* Change SFTP/SCP log rotators to use 4-digit year in filenames
Fixes #745. (Jon Siwek)
* Adding back the stats.bro file. Addresses #656. (Seth Hall)
2.0-beta-185 | 2012-01-09 18:00:50 -0800
* Tweaks for OpenBSD support. (Jon Siwek)
2.0-beta-181 | 2012-01-08 20:49:04 -0800
* Add SFTP log postprocessor that transfers logs to remote hosts.
Addresses #737. (Jon Siwek)
* Add FAQ entry about disabling NIC offloading features. (Jon Siwek)
* Add a file NEWS with release notes. (Robin Sommer)
2.0-beta-177 | 2012-01-05 15:01:07 -0800
* Replace the --snaplen/-l command line option with a
scripting-layer option called "snaplen" (which can also be
redefined on the command line, e.g. `bro -i eth0 snaplen=65535`).
* Reduce snaplen default from 65535 to old default of 8192. Fixes
#720. (Jon Siwek)
2.0-beta-174 | 2012-01-04 12:47:10 -0800
* SSL improvements. (Seth Hall)
- Added the ssl_session_ticket_handshake event back.
- Fixed a few bugs.
- Removed the SSLv2.cc file since it's not used.
2.0-beta-169 | 2012-01-04 12:44:39 -0800
* Tuning the pretty-printed alarm mails, which now include the
covered time range into the subject. (Robin Sommer)
* Adding top-level "test" target to Makefile. (Robin Sommer)
* Adding SWIG as dependency to INSTALL. (Robin Sommer)
2.0-beta-155 | 2012-01-03 15:42:32 -0800
* Remove dead code related to record type inheritance. (Jon Siwek)
2.0-beta-152 | 2012-01-03 14:51:34 -0800
* Notices now record the transport-layer protocol. (Johanna Amann)
2.0-beta-150 | 2012-01-03 14:42:45 -0800
* CMake 2.6 top-level 'install' target compat. Fixes #729. (Jon Siwek)
* Minor fixes to test process. Addresses #298.
* Increase timeout interval of communication-related btests. (Jon Siwek)
2.0-beta-145 | 2011-12-19 11:37:15 -0800
* Empty fields are now logged as "(empty)" by default. (Robin
Sommer)
* In log headers, only escape information when necessary. (Robin
Sommer)
2.0-beta-139 | 2011-12-19 07:06:29 -0800
* The hostname notice email extension works now, plus a general
mechanism for adding delayed information to notices. (Seth Hall)
* Fix &default fields in records not being initialized in coerced
assignments. Addresses #722. (Jon Siwek)
* Make log headers include the type of data stored inside a set or
vector ("vector[string]"). (Johanna Amann)
2.0-beta-126 | 2011-12-18 15:18:05 -0800
* DNS updates. (Seth Hall)
- Fixed some bugs with capturing data in the base DNS script.
- Answers and TTLs are now vectors.
- A warning that was being generated (dns_reply_seen_after_done)
from transaction ID reuse is fixed.
* SSL updates. (Seth Hall)
- Added is_orig fields to the SSL events and adapted script.
- Added a field named last_alert to the SSL log.
- The x509_certificate function has an is_orig field now instead
of is_server and its position in the argument list has moved.
- A bit of reorganization and cleanup in the core analyzer. (Seth
Hall)
2.0-beta-121 | 2011-12-18 15:10:15 -0800
* Enable warnings for malformed Broxygen xref roles. (Jon Siwek)
* Fix Broxygen confusing scoped IDs at start of line as function
parameter. (Jon Siwek)
* Allow Broxygen markup "##<" for more general use. (Jon Siwek)
2.0-beta-116 | 2011-12-16 02:38:27 -0800
* Cleanup some misc Broxygen css/js stuff. (Jon Siwek)
* Add search box to Broxygen docs. Fixes #726. (Jon Siwek)
* Fixed major bug with cluster synchronization, which was not
working. (Seth Hall)
* Fix missing action in notice policy for looking up GeoIP data.
(Jon Siwek)
* Better persistent state configuration warning messages (fixes
#433). (Jon Siwek)
* Renaming HTTP::SQL_Injection_Attack_Against to
HTTP::SQL_Injection_Victim. (Seth Hall).
* Fixed DPD signatures for IRC. Fixes #311. (Seth Hall)
* Removing Off_Port_Protocol_Found notice. (Seth Hall)
* Teach Broxygen to more generally reference attribute values by name. (Jon Siwek)
* SSH::Interesting_Hostname_Login cleanup. Fixes #664. (Seth Hall)
* Fixed bug that was causing the malware hash registry script to
break. (Seth Hall)
* Remove remnant of libmagic optionality. (Jon Siwek)
2.0-beta-98 | 2011-12-07 08:12:08 -0800
* Adapting test-suite's diff-all so that it expands globs in both
current and baseline directory. Closes #677. (Robin Sommer)
2.0-beta-97 | 2011-12-06 11:49:29 -0800
* Omit loading local-<node>.bro scripts from base cluster framework.
Addresses #663 (Jon Siwek)
2.0-beta-94 | 2011-12-03 15:57:19 -0800
* Adapting attribute serialization when talking to Broccoli. (Robin
Sommer)
2.0-beta-92 | 2011-12-03 15:56:03 -0800
* Changes to Broxygen master script package index. (Jon Siwek)
- Now only lists packages as those directories in the script hierarchy
that contain an __load__.bro file.
- Script packages (dirs with a __load__.bro file), can now include
a README (in reST format) that will automatically be appended
under the link to a specific package in the master package
index.
2.0-beta-88 | 2011-12-02 17:00:58 -0800
* Teach LogWriterAscii to use BRO_LOG_SUFFIX environemt variable.
Addresses #704. (Jon Siwek)
* Fix double-free of DNS_Mgr_Request object. Addresses #661.
* Add a remote_log_peer event which comes with an event_peer record
parameter. Addresses #493. (Jon Siwek)
* Remove example redef of SMTP::entity_excerpt_len from local.bro.
Fixes error emitted when loading local.bro in bare mode. (Jon
Siwek)
* Add missing doc targets to top Makefile; remove old doc/Makefile.
Fixes #705. (Jon Siwek)
* Turn some globals into constants. Addresses #633. (Seth Hall)
* Rearrange packet filter and DPD documentation. (Jon Siwek)
2.0-beta-72 | 2011-11-30 20:16:09 -0800
* Fine-tuning the Sphinx layout to better match www. (Jon Siwek and
Robin Sommer)
2.0-beta-69 | 2011-11-29 16:55:31 -0800
* Fixing ASCII logger to escape the unset-field place holder if
written out literally. (Robin Sommer)
2.0-beta-68 | 2011-11-29 15:23:12 -0800
* Lots of documentation polishing. (Jon Siwek)
* Teach Broxygen the ".. bro:see::" directive. (Jon Siwek)
* Teach Broxygen :bro:see: role for referencing any identifier in
the Bro domain. (Jon Siwek)
* Teach Broxygen to generate an index of Bro notices. (Jon Siwek)
* Fix order of include directories. (Jon Siwek)
* Catch if logged vectors do not contain only atomic types.
(Johanna Amann)
2.0-beta-47 | 2011-11-16 08:24:33 -0800
* Catch if logged sets do not contain only atomic types. (Johanna
Amann)
* Promote libz and libmagic to required dependencies. (Jon Siwek)
* Fix parallel make from top-level to work on more platforms. (Jon
Siwek)
* Add decode_base64_custom(). Addresses #670 (Jon Siwek)
* A bunch of Sphinx-doc reorgs and polishing. (Jon Siwek)
2.0-beta-28 | 2011-11-14 20:09:28 -0800
* Binary packaging script tweaks. We now require CMake 2.8.6. (Jon Siwek)
* More default "weird" tuning for the "SYN_with_data" notice. (Seth
Hall)
* Tiny bugfix for http file extraction along with test. (Seth Hall)
2.0-beta-21 | 2011-11-06 19:27:22 -0800
* Quickstart doc fixes. (Jon Siwek)
2.0-beta-19 | 2011-11-03 17:41:00 -0700
* Fixing packet filter test. (Robin Sommer)
2.0-beta-12 | 2011-11-03 15:21:08 -0700
* No longer write to the PacketFilter::LOG stream if not reading
traffic. (Seth Hall)
2.0-beta-10 | 2011-11-03 15:17:08 -0700
* Notice framework documentation update. (Seth Hall)
* Fixing compiler warnings (addresses #388) (Jon Siwek)
2.0-beta | 2011-10-27 17:46:28 -0700
* Preliminary fix for SSH login detection: we need a counted measure
of payload bytes (not ack tracking and not with the IP header
which is what we have now). (Seth Hall)
* Fixing send_id() problem. We no longer update &redef functions.
Updating code on the fly isn't fully supported. (Robin Sommer)
* Tuning the format of the pretty-printed alarm summaries. (Robin
Sommer)
1.6-dev-1508 | 2011-10-26 17:24:50 -0700
* Updating submodule(s). (Robin Sommer)
1.6-dev-1507 | 2011-10-26 15:10:18 -0700
* Baseline updates. (Robin Sommer)
1.6-dev-1506 | 2011-10-26 14:48:43 -0700
* Updating submodule(s). (Robin Sommer)
1.6-dev-1505 | 2011-10-26 14:43:58 -0700
* A new base script that pretty-prints alarms in the regular
summary. (Robin Sommer)
* Adding a dummy log writer WRITER_NONE that just discards
everything. (Robin Sommer)
1.6-dev-1498 | 2011-10-26 14:30:15 -0700
* Adding instructions to local.bro how to do ACTION_ALARM by
default. (Seth Hall)
1.6-dev-1495 | 2011-10-26 10:15:58 -0500
* Updated unit test baselines. (Seth Hall)
1.6-dev-1491 | 2011-10-25 20:22:56 -0700
* Updating submodule(s). (Robin Sommer)
1.6-dev-1482 | 2011-10-25 19:08:32 -0700
* Fixing bug in log managers predicate evaluation. (Robin Sommer)
1.6-dev-1481 | 2011-10-25 18:17:03 -0700
* Fix a problem with DNS servers being logged that aren't actually
servers. (Seth Hall)
* Changed generated root cert DN format for RFC2253 compliance. (Jon
Siwek)
* Removed :bro doc directives from notice documentation. (Seth Hall)
* New notice framework docs. (Seth Hall)
* Adding sub messages to emails. (Seth Hall)
* Adding extra fields to smtp and http to track transaction depth.
(Seth Hall)
* Fix for SSH login detection heuristic. (Seth Hall)
* Removed some fields from http analysis that weren't commonly
needed or were wrong. (Seth Hall)
* Updated/fixed MSIE version parsing in the software framework.
(Seth Hall)
* Update Mozilla trust roots to index certs by subject distinguished
name. (Jon Siwek)
* weird.bro rewrite. (Seth Hall)
* More notice email tuning. (Seth Hall)
* Slightly restructured http file hashing to fix a bug. (Seth Hall)
* Changed the notice name for interesting ssh logins to correctly
reflect semantics of the notice. (Seth Hall)
* Field name change to notice framwork. $result -> $action
- $result is renamed to $action to reflect changes to the notice
framework since there is already another result-like field
($suppress_for) and there may be more in the future.
- Slipped in a change to add connection information to notice
emails too. (Seth Hall)
* Small script refinements and documentation updates. (Seth Hall)
* Pass over upgrade guide. (Robin Sommer)
1.6-dev-1430 | 2011-10-21 10:39:09 -0700
* Fixing crash with unknown debug streams. Closes #643. (Robin
Sommer)
* Code to better handle interpreter errors, which can now be turned
into non-fatal runtime errors rather than immediate aborts. (Robin
Sommer).
* Remove old make-src-packages script. (Jon Siwek)
* Fixing a bunch of format strings. Closes #567. (Robin Sommer)
* Cleaning up some distribution files. (Robin Sommer)
* Various test, doc, and installation fixes/tweaks. (Seth Hall, Jon
Siwek and Robin Sommer).
* Varios smaller policy fixes and tweaks (Seth Hall).
* Moving docs from web server into distribution. (Robin Sommer)
* Fixing more (small) memory leaks. (Robin Sommer)
* Profiling support for DNS_Mgr and triggers. With
misc/profiling.bro, both now report a line in prof.log with some
counters on usage. (Robin Sommer)
* Fixing DNS memory leaks. Closes #534. (Robin Sommer)
* Fix code for disabling analyzers. Closes #577. (Robin Sommer)
* Changed communication option from listen_encrypted to listen_ssl.
(Seth Hall)
* Modification to the Communication framework API. (Seth Hall)
- Simplified the communication API and made it easier to change
to encrypted connections by not having separate variables to
define encrypted and unencrypted ports.
- Now, to enable listening without configuring nodes just
load the frameworks/communication/listen script.
- If encrypted listening is desired set the following:
redef Communication::listen_encrypted=T;
* Connection compressor now disabled by default. Addresses #559.
(Robin Sommer)
1.6-dev-1372 | 2011-10-06 18:09:17 -0700
* Filtering some potentially high-volume DNS weirds. (Robin Sommer)
* DNS now raises DPD events. Closes #577. (Robin Sommer)
* Fixing a bunch of compiler warnings. (Robin Sommer)
* Remote logs are auto-flushed if the last write was longer than a
second ago. Addresses #498. (Robin Sommer)
* Fix missing from previous MIME commit. (Robin Sommer)
1.6-dev-1366 | 2011-10-06 17:05:21 -0700
* Make CompHash computation/recovery for functions deterministic.
Closes #636. (Jon Siwek)
* Removing unnecessary @load in local.bro. (Robin Sommer)
* Optimizing some MIME code. (Robin Sommer)
* Speed improvements in logging code. (Robin Sommer)
* Consolidating some node-specific functionality from scripts in
broctl repo. (Jon Siwek)
* Another fix the for 1xx script code. (Robin Sommer)
1.6-dev-1352 | 2011-10-05 16:20:51 -0700
* Fix for optional HTTP::Info status_code. (Jon Siwek)
* Teaking some external testing scripts. (Jon Siwek)
* HTTP bug fix reported by Martin Holste. (Seth Hall)
* More script tuning. (Seth Hall)
- Moved some of the weird events back to the base/ directory.
- SSL fixes, updates, and performance optimization.
* More adjustment to reduce Weird volumes. (Seth Hall)
* Fixed an error when calculating x509 certificate hashes (reported
by Martin Holste). (Seth Hall)
* Clean up to cluster framework to make event handling clearer.
(Seth Hall)
* Fixed a bug in the notice framework. (Seth Hall)
* Bug fix for FTP analysis script. (Seth Hall)
1.6-dev-1333 | 2011-09-29 22:29:51 -0700
* Fixing a number of memory leaks. (Robin Sommer)
* Loaded_scripts.log is indented with spaces now and makes more
sense to look at. (Seth Hall)
* Teach HTTP parser to derive content length of multipart/byteranges
bodies. Addresses #488. (Jon Siwek)
* Change logging of HTTP 1xx responses to occur in their own
columns. Addresses #411. (Jon Siwek)
* Fix handling of HTTP 1xx response codes. Addresses #411).
* Taking advantage of yet another trick to get installed browser
plugins. (Seth Hall)
- With the software-browser-plugins script you can watch for Omniture
advertising servers to grab the list of installed plugins.
- I reorganized the plugin detection a bit too to abstract it better.
- Removed the WEB_ prefix from all of the Software::Type HTTP enums.
They were essentially redundant due to the full name already being
HTTP::SERVER (for example).
1.6-dev-1316 | 2011-09-28 16:50:05 -0700
* Unit test cleanup. Updated README and collected coverage-related
tests in a common dir. (Jon Siwek)
* Fixes for known-services. (Seth Hall)
* Ported and 2.0ized the capture-loss script. (Seth Hall)
* Communication fix and extension.(Robin Sommer)
- Removing unnecessary log flushing. Closes #498.
- Adding new BiF disconnect() that shuts a connection to a peer down.
- terminate_connection() now first flushes any still buffered log
messages.
* Fix for high SSL memory usage by adding &transient attribute to
top-level SSL pac array type. Closes #574. (Robin Sommer)
* Fix a small bug in the metrics framework. (Seth Hall)
* Temporarily removing scripts that aren't ready to be included.
Will return before next release. (Seth Hall)
* New SSL policy scripts. (Seth Hall)
- protocols/ssl/expiring-certs uses time based information from
certificates to determine if they will expire soon, have already
expired, or haven't yet become valid.
- protocols/ssl/extract-certs-pem is a script for taking certs off
the line and converting them to PEM certificates with the openssl
command line tool then dumping them to a file.
* Notice::type_suppression_intervals: table[Notice::Type] of
interval can be used to modify the suppression intervals for
entire types of notices. (Seth Hall)
* EOF SSL protocol violations are only generated a single time now.
(Seth Hall)
* Script level fixes. (Seth Hall)
- Fixed a type name conflict in the Known namespace.
- Fixed a DPD framework bug that was causing Reporter messages.
- Fixed the notice_policy log.
- Predicate functions are now logged.
- Predicate functions are now optional. If not given, it's assumed that
the result should always apply. (Seth Hall)
- Fix a problem with accidental and mistaken HTTP log lines.
1.6-dev-1293 | 2011-09-22 19:44:37 -0700
* Smaller script tweaks. (Seth Hall)
* Duplicate notice suppression. (Seth Hall)
- Duplicate notices are discovered with the new Notice::Info
field $identifier. It's a string that is left up to the
notice implementor to define which would indicate a
fundamentally duplicate notice. The field is optional and
if it's not included it's not possible for notice
suppression to take place.
- Duplicate notices are suppressed by default for the interval
defined by the Notice::default_suppression_interval variable
(1 hour by default).
- A new notice action was defined ACTION_NO_SUPPRESS to prevent
suppression for a specific notice instance. A convenience set
named not_suppressed_types was also created to not suppress
entire notice types.
- A new field was added to the PolicyItem type to modify the length
of time a notice should be suppressed if the predicate matches.
The field is named $suppress_for. This name makes the code more
readable like this: $suppress_for = 1day
- New events were created to give visibility into the notice
framework's suppression activity.
- event Notice::begin_suppression(n: Notice::Info)
- event Notice::suppressed(n: Notice::Info)
- event Notice::end_suppression(n: Notice::Info)
- The suppression.bro script doesn't have a baseline because
it is causing a segfault in Bro. This one test is the
reason that this is being integrated into a branch instead
of master. (Seth Hall)
* Fix crash on exit. Addresses #607. (Jon Siwek)
* Fix PktSrc setting next_timestamp even when no packet available.
(Jon Siwek)
* Fix lack of NUL-termination in to_upper/to_lower BIF's return val.
(Jon Siwek)
* Fixing unit tests and some minor bugs. (Jon Siwek)
* Fix broctl cluster log rotation. Addresses #619. (Jon Siwek)
* Added session ID to the SSL logging. (Seth Hall)
* Adding "install-aux" target + updating bro-aux submodule. (Jon
Siwek)
* Cleaning up INSTALL and README. (Jon Siwek)
* Remove $Id$ tags. (Jon Siwek)
* Remove policy.old directory. Addresses #511. (Jon Siwek)
* Small rework with ssl base script to reduce memory usage. (Seth
Hall)
* Updated the mozilla root certs. (Seth Hall)
1.6-dev-1261 | 2011-09-15 17:13:55 -0700
* Memory leak fixes. Addresses #574 (Jon Siwek)
* Add configure options for ruby/bindings integration. (Jon Siwek)
* Fix filter path_func to allow record argument as a subset of
stream's columns. Addresses #600. (Jon Siwek)
* Log rotation is now controlled directly through Filter records. (Jon Siwek)
* Fix indexing for record types with optional fields. Addresses #378
(Jon Siwek)
1.6-dev-1248 | 2011-09-15 16:01:32 -0700
* Removed custom malloc() implementation for FreeBSD. Closes #557.
(Jon Siwek)
* Testing/external scripts no longer compute MD5 checksums for SMTP
entities. (Robin Sommer)
* External tests no longer include the full content of mismatching
files in the diagnostics output. (Robin Sommer)
1.6-dev-1241 | 2011-09-14 22:51:52 -0400
* Fixing a major memory utilization issues with SSL analysis. (Seth
Hall)
* Enhancements to HTTP analysis: (Seth Hall)
- More options for the header-names.bro script.
- New script for logging header names and values. Closes #519.
(Seth Hall)
- HTTP body size measurement added to http.log.
- The value of the content-length headers has now been removed
in the default output but it could be added back locally at an
installation by a user.
- Added fields to indicate if some parsing interruption happened
during the body transfer. Closes #581 (Seth Hall)
* Misc smaller usability and correctness updates: (Seth Hall)
- Removed an notice definition from the base SSL scripts.
- Moved a logging stream ID into the export section for known-services
and bumped priority for creating the stream.
- Adding configuration knobs for the SQL injection attack detection
script and renaming the HTTP::SQL_Injection_Attack notice to
HTTP::SQL_Injection_Attack_Against
- Bumped priority when creating Known::CERTS_LOG.
- Fixing a warning from the cluster framework. (Seth Hall)
* Bugfix for log writer, which didn't escape binary stuff in some
situations. Closes #585. (Robin Sommer)
* A larget set of changes to the testing/external infrastructure.
The traces for external test-suites are no longer kept inside the
repositories themselves but downloaded separately via curl. This
is because git is pretty bad at dealing with large files. See the
README for more information. (Robin Sommer)
1.6-dev-1221 | 2011-09-08 08:41:17 -0700
* Updates for documentation framework and script docs. (Jon Siwek)
* The script level PF_RING support isn't working so removing it.
(Seth Hall)
* Delete SSL certificates from memory after ssl_established event.
(Seth Hall)
* Small fixes for SSL analysis. (Seth Hall)
1.6-dev-1212 | 2011-09-07 16:15:28 -0700
* Internally, the UID generation can now return values from
different pool for better reproducability in testing mode.
(Gilbert Clark).
* Added new BiF unique_id_from(pool: string, prefix: string) that
allows the user to specify a randomness pool. (Gilbert Clark)
1.6-dev-1198 | 2011-09-07 11:03:36 -0700
* Extended header for ASCII log that make it easier for scripts to
parse Bro log files. (Gilbert Clark)
* Potential fix for rotation crashes. Addresses #588. (Robin Sommer)
* Added PF_RING load balancing support to the scripting layer,
enabled by loading the misc/pf-ring-load-balancing script. (Seth
Hall)
* Added a BiF setenv() for setting environment variables. (Seth
Hall)
1.6-dev-1184 | 2011-09-04 09:34:50 -0700
* FindPCAP now links against thread library when necessary (e.g.
PF_RING's libpcap). (Jon Siwek)
* Install binaries with an RPATH. (Jon Siwek)
* Fix for a case where nested records weren't coerced even though
possible. (Jon Siwek)
* Changed ASCII writer to delay creation of log after rotation until
next write.
* Changed default snaplen to 65535 and added a -l/--snaplen command
line option to set it explicitly. Addresses #447. (Jon Siwek)
* Various updates to logging framework. (Seth Hall)
* Changed presentation of enum labels to include namespace. (Jon
Siwek)
* HTTP analyzer is now enabled with any of the HTTP events. (Seth
Hall)
* Fixed missing format string that caused some segfaults. (Gregor
Maier)
* ASCII writer nows prints time interval with 6 decimal places.
(Gregor Maier)
* Added a Reporter::fatal BIF. (Jon Siwek)
* Fixes for GeoIP support. Addresses #538. (Jon Siwek)
* Fixed excessive memory usage of SSL analyzer on connections with
gaps. (Gregor Maier)
* Added a log postprocessing function that can SCP rotated logs to
remote hosts. (Jon Siwek)
* Added a BiF for getting the current Bro version string. (Jon
Siwek)
* Misc. doc/script/test cleanup. (Jon Siwek)
* Fixed bare-mode @load dependency problems. (Jon Siwek)
* Fixed check_for_unused_event_handlers option. (Jon Siwek)
* Fixing some more bare-mode @load dependency issues (Jon Siwek)
* Reorganizing btest/policy directory to match new scripts/
organization. Addresses #545 (Jon Siwek)
* bro scripts generated from bifs now install to
$prefix/share/bro/base. Addresses #545 (Jon Siwek)
* Changeed/fixed some cluster script error reporting. (Jon Siwek)
* Various script normalization. (Jon Siwek)
* Add a test that checks each individual script can be loaded in
bare-mode. Adressess #545. (Jon Siwek)
* Tune when c$conn is set. Addresses #554. (Gregor Maier)
* Add ConnSize_Analyzer's fields to conn.log. (Gregor Maier)
* Fixing bug in "interesting hostnames" detection. (Seth Hall)
* Adding metrics framework intermediate updates. (Seth Hall)
1.6-dev-1120 | 2011-08-19 19:00:15 -0700
* Fix for the CompHash fix. (Robin Sommer)
1.6-dev-1118 | 2011-08-18 14:11:55 -0700
* Fixing key size calculation in composite hash code. (Robin Sommer)
1.6-dev-1116 | 2011-08-18 10:05:07 -0700
* Remove the 'net' type from Bro (addresses #535).
* Fix H3 assumption of an 8-bit byte/char. (Jon Siwek)
* Allow reading from interface without additional script arguments.
Explicitly passing in '-' as an additional command line argument
still allows reading a script from stdin. (Jon Siwek)
* SSH bruteforcing detection now done with metrics framework. (Seth
Hall)
* Updates for SQL injection attack detection to match the metrics
framework updates. (Seth Hall)
* Metrics framework now works on cluster setups. (Seth Hall)
* Reclassifying more DNS manager errors as non-fatal errors. (Robin
Sommer)
* Fix ConnSize_Analyzer when used in conjunction with connection
compressor. (Gregor Maier)
* Fix reporter using part of the actual message as a format string.
(Jon Siwek)
1.6-dev-1095 | 2011-08-13 11:59:07 -0700
* A larger number of script documentation updates. Closes #543. (Jon
Siwek)
* Workaround for FreeBSD CMake port missing debug flags. (Jon Siwek)
* piped_exec() can now deal with null bytes. (Seth Hall)
* Fix vector initialization for lists of records with optional
types. Closes #485. (Jon Siwek)
* Fix redef'ing records with &default empty set fields. Closes #460.
(Jon Siwek)
* Fix ConnSize_Analyzer when used in conjunction with the connection
compressor. (Gregor Maier)
* Fix reporter using part of the actual message as a format string.
(Jon Siwek)
* Fixing reporter's location tracking. Closes #492. (Robin Sommer)
* Turning DNS errors into warnings. Closes #255. (Robin Sommer)
* Logging's path_func now receives the log record as argument.
Closes #555. (Robin Sommer)
* Functions can now be logged; their full body gets recorded.
Closes #506. (Robin Sommer)
* Bugfix for hostname notice email extension. (Seth Hall)
* Updates for notice framework. (Seth Hall)
- New ACTION_ADD_GEODATA to add geodata to notices in an extension
field named remote_location.
- Loading extend-email/hostnames by default now that it only does
anything when the ACTION_EMAIL action is applied (finally).
* Updates to local.bro (Seth Hall)
* Added the profiling script. (Seth Hall)
* Updates for SSH scripts. (Seth Hall)
* ConnSize analyzer is turned on by default now. (Seth Hall)
* Updates for the build system and site local scripts for cluster.
(Seth Hall)
* HTTP now uses the extract_filename_from_content_disposition function. (Seth Hall)
* Major SMTP script refactor. Closes #509. (Jon Siwek and Seth Hall)
* New variable Site::local_nets_table in utils/site for mapping
address to defined local subnet.
* Metrics framework updates, more to come. (Seth Hall)
1.6-dev-1061 | 2011-08-08 18:25:27 -0700
* A set of new/changed tests regarding the new policy script
organisation. (Robin Sommer)
1.6-dev-1058 | 2011-08-08 16:15:18 -0700
* Reorganisation of the scripts that Bro loads by default. (Seth
Hall)
- policy/ renamed to scripts/
- By default BROPATH now contains:
- scripts/
- scripts/policy
- scripts/site
- The scripts in scripts/base/protocols/ only do logging and state
building.
- All of scripts/base/ is loaded by by default. This can however
be disabled by switching Bro into "bare mode" using the new
command-line option --bare-mode (or -b). The cripts in
scripts/base/ don't use relative path loading to ease use of
bare mode (to copy and paste that script).
- The scripts in scripts/base/frameworks/ add functionality
without causing any additional overhead.
- All "detection" activity happens through scripts in
scripts/policy/.
- bro.init was renamed to base/init-bare.bro, and base/all.bro was
renamed to init-default.bro.
- local.bro now loads more functionality from policy/ and adds
more documentation. (Seth Hall)
* Adding default_path_func() to the logging framework that makes the
default naming scheme script-level controlled. (Robin Sommer)
* Reworking logging's postprocessor logic so that postprocessor
commands are no longer run by the log writers themselves, but
instead by a script level function. (Robin Sommer)
* The communication subsystem is now by default off and must be
enabled explicitly with a new BiF, enable_communication(). Closes
#540. (Robin Sommer)
* The hostname notice email extension now only add hostnames for
emailed noticed. (Seth Hall)
* Cleaning up doc generation. (Seth Hall)
1.6-dev-1044 | 2011-08-05 19:07:32 -0700
* Fixing memory (and CPU) leak in log writer.
* Fixing crash in memory profiling. (Robin Sommer)
* Fix compiler warning. (Robin Sommer)
* Fixing missing sync in cluster setup. (Robin Sommer)
1.6-dev-1038 | 2011-08-05 18:25:44 -0700
* Smaller updates to script docs and their generation. (Jon Siwek)
* When using a `print` statement to write to a file that has raw output
enabled, NUL characters in string are no longer interpreted into "\0",
no newline is appended afterwards, and each argument to `print` is
written to the file without any additional separation. (Jon Siwek)
* Test portatibility tweaks. (Jon Siwek)
* Fixing PktSrc::Statistics() which retured bogus information
offline mode. Closes #500. (Jon Siwek)
* --with-perftools configure option now assumes --enable-perftools.
Closes #527. (Jon Siwek)
1.6-dev-1018 | 2011-07-31 21:30:31 -0700
* Updating CHANGES. (Robin Sommer)
1.6-dev-1016 | 2011-07-30 18:34:28 -0700
* Install example config files dynamically. They'll only get
installed when the distribution version differs from existing
version on disk. (Jon Siwek)
* Fixed memory leak in SSL analyzer. (Seth Hall)
* Beginning rework of metrics interface. (Seth Hall)
* New/updated unit tests for scripts. (Jon Siwek)
* New/updated documentstion for scripts. (Jon Siwek)
* A number of fixes for scripts in utils/. (Jon Siwek)
1.6-dev.244 Thu Jul 28 17:08:21 PDT 2011
- mask_addr() now returns subnet (addresses #512). (Jon Siwek)
- Normalize Notice::Type identifiers per convention (closes #484).
(Jon Siwek)
- Fixing default-loaded-scripts test for BSD systems. (Jon Siwek)
- New piped_exec() BiF for pipeing data into an external command. (Jon
Siwek)
1.6-dev.242 Mon Jul 25 21:42:39 PDT 2011
- Adding a documentation coverage test. (Jon Siwek)
- The CMake targets for generating reST docs from policy scripts are
now automatically generated via the genDocSourcesList.sh script.
(Jon Siwek)
- Fixed a number of script error. (Jon Siwek)
- Fixes to relative @load'ing. (Jon Siwek)
- Fixes to tests. (Robin Sommer)
1.6-dev.240 Sun Jul 24 15:14:26 PDT 2011
- Updated tests and test baselines. (Jon Siwek)
- ASCII log writer now prints time values w/ constant 6 digit
precision. (Jon Siwek)
- Many policy script updates acrsso the board (Seth Hall).
- Moving devel-tools to bro-aux. (Robin Sommer)
- BugFix for disable_analyzer(), which could cause crashes with some
analyzers. (Robin Sommer)
- Bugfix for potential segfault in DebugLogger. (Robin Sommer)
1.6-dev.226 Thu Jul 21 15:23:39 PDT 2011
- Extensions to the @load and @unload process. (Jon Siwek)
* Make @load statements recognize relative paths. For example a
script can do "@load ./foo" to load a script named foo.bro that
lives in the same directory or "@load ../bar" to load a script
named bar.bro in the parent directory, even if those directories
are not contained in BROPATH.
* Reimplementation of the @prefixes statement. (Closes #486)
Any added prefixes are now used *after* all input files have
been parsed to look for a prefixed, flattened version of the
input file somewhere in BROPATH and, if found, load it. For
example, if "lcl" is in @prefixes, and site.bro is loaded, then
a file named "lcl.site.bro" that's in BROPATH would end up being
automatically loaded as well. Packages work similarly, e.g.
loading "protocols/http" means a file named
"lcl.protocols.http.bro" in BROPATH gets loaded automatically.
* Fix @unload'd files from generating bro_script_loaded event.
* Updates to tests.
1.6-dev.225 Wed Jul 20 17:10:41 PDT 2011
- IRC improvements (Jon Siwek). Including:
* Shorten what's displayed in the IRC's log mime_type column for
DCC transfers.
* Add IRC unit tests.
* Fix IRC analyzer supplying wrong type to irc_dcc_message event.
* Removed irc_client and irc_server events.
* Added is_orig arguments to all other irc events.
* Fix analyzer not recognizing Turbo DCC extension message format.
* Fix analyzer not generating irc_dcc_message event when irc_privmsg_message
event doesn't have a handler registered.
- Fixing tests that need a diff canonifier. (Jon Siwek)
1.6-dev.223 Tue Jul 19 19:10:36 PDT 2011
- Adding a script to update CHANGES and VERSION. (Robin Sommer)
1.6-dev.218 Tue Jul 19 18:16:44 PDT 2011
- Comprehensive policy script overhaul/rewrite. (Seth Hall)
Changes are too extensive to list individually.
- Removing undocumented -H command line flag. (Robin Sommer)
- Fixing many tests. (Everybody)
- Fixing 0-chunk bug in remote logging. (Robin Sommer)
- $PATH is now appropriately set by the bro-path-dev.(sh|csh) scripts.
(Seth Hall)
- Making valgrind a bit more happy. (Robin Sommer)
- New BiF record_field_vals() that returns the fields of a record in a
table with meta-information. (Robin Sommer)
- Adding a script in aux/devel-tools that extracts a connection from a
trace based on uid. (Robin Sommer)
- Fixing bug causing crash when running without arguments. (Robin Sommer)
- A new event bro_script_loaded() raised for each policy script
loaded. Also removing the -l command-line option as that can now be
done at the script-level. (Robin Sommer)
- Fixing memory leaks. (Gilbert Clark, Seth Hall, Robin Sommer)
- Many SSL analysis improvements and fixes. (Seth Hall)
- Fixing bug with event priorities potentially being ignored for the
handler. (Robin Sommer)
- Overhauling the internal reporting of messages to the user. The new
Reporter class is now in charge of reporting all errors, warnings,
informational messages, weirds, and syslogs; and it passes
everything through the script layer. (Robin Sommer)
* Removed the alarm statement and the alarm_hook event. (Robin Sommer)
- Adding new policy file test-all.bro that loads all other policies.
This is for testing only. (Robin Sommer)
- A new framework for doing regression testing with larger traces and
more complex Bro configurations in testing/external. (Robin Sommer)
- Many updates to script doc generation. (Jon Siwek)
1.6-dev.146 Sat Jun 25 18:12:27 PDT 2011
- DNS mapping are now becoming invalid when an entry's TTL expires.
(Thomas Other)
- Reworking how Bro tracks which scripts are already loaded. Rather
than paths, Bro now tracks inode numbers. (Jon Siwek)
- New BiF netstats() to query packet capture statistics. The netstats
script now uses the new BiF to periocally report packets drops. The
net_stats_update() event and the heartbeat_interval global went
away. (Seth Hall)
- Fixing bug with logging &optional records. Closes #476. (Robin
Sommer)
- Fixing istate.events-ssl test failing because of expired cert. (Jon
Siwek)
- A large number of improvements and fixes for Bro's doc mode. (Jon
Siwek)
- Significant updates for RPC and NFS analyzers (Gregor Maier)
* Unify semantics for UDP and TCP connections.
* RPC can now log to a log file if desired.
* Portmapper can now log general activity to a log file and also log
actual port mappings.
* NFS analyzer now supports significantly more procedure calls as
as file name tracking and file content extraction.
- NetBIOS fixes. (Jon Siwek)
- A number of unit tests are more robust and portable. (Jon Siwek)
- A new BiF unique_id() that returns a string that's unique across Bro
instaces with high probablity. (Robin Sommer)
- Complete rewrite of the BinPAC SSL analyzer. (Seth Hall)
* DER certificates are extracted as strings to be used with
corresponding BiFs.
* x509_verify function to verify single certs and/or full
certificate chains.
* Removed hand written SSL analyzer.
* The ssl.bro script is just a place-holder for now. New version
will come with the other new scripts.
- New syslog analyzer. (Seth Hall)
- @load now supports loading a directory. With a directory "foo"
somewhere in BROPATH, "@load foo" now checks if there's a file
"foo/__load__.bro". If so, it reads that file in. (Robin Sommer)
- ASCII logger now escapes non-printable characters. Closes #450.
(Robin Sommer)
- Packaging tweaks and rewrite of 'dist' target. (Jon Siwek)
- Changes to allow DEB packaging via CPack, addresses #458. (Jon
Siwek)
- An extension to the ICMP analyzer to handle redirects. Julien
Sentier
- Removing old istate test-suite. (Robin Sommer)
- A hack to report missing GeoIP support only once. This closes #357,
but #455 captures the need for a more general solution. (Robin
Sommer)
- Bugfix: vectors in records were not initalized. Closes #421. (Robin
Sommer)
- If IPv6 default is not compiled in, the default BPF filters now
excludes IPv6 packets. (Robin Sommer)
- New bif bro_has_ipv6() to check whether IPv6 support is compiled in.
(Robin Sommer)
- Updating btests and a Makefile. "make" now runs all the tests.
(Robin Sommer)
- Moving the test-scripts from the old test-suite over to btest.
(Robin Sommer)
- Fix for major bug in POP3 analyzer, which didn't recognize '.'
terminators in multi-line replies if the terminator was bare (no
newline). This caused it to ignore the rest of the session that it's
analyzing. (Vern Paxson)
- Fix compiler warning with gcc-4.4.4 (Gregor Maier)
- Adding example documentation for a script's use of logging features.
(Jon Siwek)
- Adding &log attribute to static attr_names array. (Jon Siwek)
- Bro can now track packet and byte counts per connection. (Gregor
Maier)
* If 'use_conn_size_analyzer' is true, the event engine tracks
number of packets and raw IP bytes per connection. If
report_conn_size_analyzer is true, these values are included as
four new columns into conn.log
* I changed conn.bro so that the value of
report_conn_size_analyzer follows that of
use_conn_size_analyzer. For the new conn.log, we probably want
to get rid of report_conn_size_analyzer anyway.
- Fixing numerous compiler warnings and portability issues. (All)
- Switching vectors from being 1-based to 0-based. Note that this is a
change that break backwards-compatibility. (Robin Sommer)
- Increasing serialization format version for the recent 64-bit
changes. (Robin Sommer)
- Support for (mixed) MPLS and VLAN traffic, and a new default BPF
filter. (Seth Hall and Robin Sommer)
* Merging in the patch from #264, which provides support for mixed
VLAN and MPLS traffic.
* Changing Bro's default filter from being built dynamically to
being a static "ip or not ip". To get the old behaviour back
(i.e., the dynamically built filter), redef "all_packets" to
false.
* print-filter.bro now always prints the filter that Bro is
actually using, even if overriden from the command line. (Robin
Sommer)
- Changing the HTTP's analyzers internals to use 64-bit integers.
(Gregor Maier).
- Fixing bug with deleting still unset record fields of table type.
(Robin Sommer)
1.6-dev.99 Fri Apr 22 22:10:03 PDT 2011
- Extending the connection record with a unique identifier. (Robin
Sommer)
type connection: record {
[...]
id: string;
};
These identifiers very likely unique even across independent Bro
runs.
- Delete operator for record fields. (Robin Sommer)
"delete x$y" now resets record field "x" back to its original state
if it is either &optional or has a &default. "delete" may not be
used with non-optional/default fields.
- Fixing bug with nested record coercions. (Robin Sommer)
- Fixing a do_split() bug. (Seth Hall)
1.6-dev.94 Thu Apr 21 19:51:38 PDT 2011
- Fixing generation of config.h. (Jon Siwek)
- Updates and tests for NetBIOS name BiF. (Seth Hall)
- Fixing do_split bug(), and adding a test. (Seth Hall)
- When Bro is given a PRNG seed, it now uses its own internal random
number generator that produces consistent results across sytems.
Note that this internal generator isn't very good, so it should only
be used for testing purpses. (Robin Sommer)
- The BTest configuration now sets the environemnt variables TZ=UTC
and LANG=C to ensure consistent results. (Robin Sommer)
- Logging fixes. (Robin Sommer)
1.6-dev.88 Wed Apr 20 20:43:48 PDT 2011
- Implementation of Bro's new logging framework. We will document this
separately. (Robin Sommer)
- Already defined record types can now be further extended via the
'+=' operator. The added fields must be either &optional or have a
&default value. (Robin Sommer)
Example:
type Foo: record {
a: count;
b: count &optional;
};
redef record Foo += {
c: count &default=42;
d: count &optional;
};
global f: Foo = [$a=21];
print f;
Output:
[a=21, b=<uninitialized>, c=42, d=<uninitialized>]
- Enabling assignment of empty vectors ("vector()"). (Robin Sommer)
- Fixing attributes to allow &default attributes to be associated with
records fields of type tables/sets/vector. (Robin Sommer)
- '[]' is now a valid record constructor. (Robin Sommer)
- A instance of a record type A is now coercable into one of type B if
the fields of type A are a subset of those of type B. (Robin Sommer)
- A number of bug fixes and enhancements for record/set/table/vector
coercion. (Robin Sommer)
- Fixing a problem with records that have optional fields when used as
table/set indices. Addresses #367. (Robin Sommer)
- Fixing an off-by-one error in join_string_vec(). (Seth Hall)
- Updating to_count() to cope with 64bit ints. (Seth Hall)
- A new BiF count_to_v4_addr() to turn a count into an IPv4 address.
(Seth Hall)
1.6-dev.80 Mon Apr 18 14:50:54 PDT 2011
- New framework for generating documentation from Bro scripts. (Jon
Siwek)
This includes:
* Changes to Bro's scanner/parser to facilitate automatic
generation of Bro policy script documentation in
reStructuredText format.
* New command line flags -Z/--doc-scripts to enable the new doc
generation mode.
* Changes to bifcl to pass comments starting with "##" through
into the generated .bro script.
* A "doc" build target for the top-level Makefile to first
generate reStructuredText for a defined set of Bro policy
scripts, and then run that through Sphinx to create HTML
documentation.
1.6-dev.78 Mon Apr 18 12:52:55 PDT 2011
- Adding files to CMake build targets so they show up in generated IDE
projects. This addresses #413. (Jon Siwek)
- Fix unnecessary config.h preprocessor (re)definitions. This
addresses #414. (Jon Siwek)
- Updating istate tests. (Robin Sommer)
- Adding files to CMake build targets so they show up in generated IDE
projects.
- Adding new environment variable BRO_SEED_FILE to set the seed file
for the random number generator. (Robin Sommer)
1.6-dev.71 Fri Apr 1 16:06:33 PDT 2011
- Removing code for the following no longer supported functionality.
* Trace rewriting.
* DFA state expiration in regexp engine.
* Active mapping.
* Unused hash functions.
(Robin Sommer)
- Fixing crashes when SSL is not configured correctly. (Robin Sommer)
1.6-dev.66 Tue Mar 29 21:52:01 PDT 2011
- Initial btest setup (Don Appleman and Robin Sommer)
- Porting the istate tests to btest (not finished) (Robin Sommer)
1.6-dev.63 Mon Mar 21 16:31:15 PDT 2011
- Changes to the way user-modifiable config files are installed (Jon Siwek)
* Duplicates of the distribution's configuration files are now
always installed with a .example suffix
* Added --binary-package configure option to toggle configure
logic specific to the creation of binary packages.
* When not in binary packaging mode, `make install` never
overwrites existing configure files in case they've been
modified. The previous behavior (CMake's default) would only
avoid overwriting modified files if one consistently uses the
same build directory and doesn't reconfigure.
- Fixed an issue with Mac package's pre-install script not preserving
ACLs. (Jon Siwek)
- Minor cleanup/refactor of the make-mac/rpm-packages scripts. (Jon
Siwek)
- Add explicit CMake check for compiler. (Jon Siwek)
- Add alternative way to set BROPATH for running bro from build/ dir.
(Jon Siwek)
- Fixing compiler warnings (Gregor Maier)
- Remvoing leftover local variables that caused compile error on Mac
OS X. (Gregor Maier)
1.6-dev.53 Fri Feb 25 17:03:05 PST 2011
- Fixing file detector leak in remote communication module. (Scott
Campbell)
- Updating independent-state tests to work with new setup. (Robin
Sommer)
1.6-dev.49 Fri Feb 25 15:37:28 PST 2011
- Enum IDs can have explicitly defined values. (Gregor Maier)
- Extensions for the built-in function compiler, bifcl. (Gregor Maier)
* Support for policy-layer namespaces.
* Support for type declarations in bif files (with access them
from C++)
* Extended const declarations in bif files.
See http://bro.icir.org/devel/bif-doc for more information.
1.6-dev.48 Fri Feb 25 10:53:04 PST 2011
- Preliminary TCP Reassembler fix: deliver data after 2GB by disabling
the unused seq_to_skip feature. (Gregor Maier)
1.6-dev.47 Fri Feb 25 10:40:22 PST 2011
- Fixing endianess error in XDR when data is not 4-byte aligned.
(Gregor Maier)
- Fix for Val constructor with new int64 typedefs. (Gregor Maier)
- Updated fix for OS X 10.5 compile error wrt llabs(). (Gregor Maier)
- Fix more compiler warning wrt printf format strings. (Gregor Maier)
1.6-dev.45 Tue Feb 8 21:28:01 PST 2011
- Fixing a number of compiler warnings. (Seth Hall and Robin Sommer)
1.6-dev.44 Tue Feb 8 20:11:44 PST 2011
- A number of updates to the SSL analyzer, including support for new
ciphers; SSL extensions; and bug fixes. The analyzer does not longer
throw weird for exceeding a predefined cipherspec_size anymore.
(Seth Hall and Rmkml).
- The various split*() BiFs now handle strings containing null bytes
correctly. (Seth Hall)
- Adding new aux/btest submodule. This is a framework we will use in
the future for doing unit tests. (Robin Sommer)
1.6-dev.41 Mon Feb 7 13:43:56 PST 2011
- Smarter way to increase the parent/child pipe's socket buffer.
(Craig Leres).
- Fixing bug with defining bro_int_t and bro_uint_t to be 64 bits wide
on some platforms. (Robin Sommer)
1.6-dev.39 Mon Jan 31 16:42:23 PST 2011
- Login's confused messages now go through weird.bro. (Robin Sommer)
1.6-dev.36 Mon Jan 31 08:45:35 PST 2011
- Adding more configure options for finding dependencies, (Jon Siwek)
--with-flex=PATH path to flex executable
--with-bison=PATH path to bison executable
--with-perl=PATH path to perl executable
--with-python=PATH path to Python interpreter
--with-python-lib=PATH path to libpython
--with-python-inc=PATH path to Python headers
--with-swig=PATH path to SWIG executable
- Fixing typo in PCAPTests.cmake (Jon Siwek)
1.6-dev.33 Mon Jan 24 15:29:04 PST 2011
- Fixing bug in SMB analyzer. (Robin Sommer)
- Configure wrapper now deletes previous CMake cache (Jon Siwek)
- Fix for the --with-binpac configure option. (Jon Siwek)
1.6-dev.30 Thu Jan 20 16:32:43 PST 2011
- Changed configure wrapper to create config.status. (Jon Siwek)
1.6-dev.29 Thu Jan 20 16:29:56 PST 2011
- Fixing little problem with initialization of Bro-to-Bro event
communication. (Christian Kreibich)
1.6-dev.27 Thu Jan 20 13:52:25 PST 2011
- Fine-tuning of the HTTP analyzer in terms of raising protocol
violations and interrupted transfers. (Gregor Maier)
1.6-dev.21 Wed Jan 19 17:36:02 PST 2011
- Added 4 new BiFs and a new record type for testing the entropy of
strings. (Seth Hall)
find_entropy(data: string): entropy_test_result
This is a one shot function that accepts a string and
returns the result of the entropy calculations.
entropy_test_init(index: any): bool
This and the next two functions are for calculating entropy
piece-wise. It only needs an index which can be any type of
variable. It needs to be something that uniquely identifies
the data stream that is currently having it's entropy
calculated.
entropy_test_add(index: any, data: string): bool
This function is used to add data into the entropy
calculation. It takes the index used in the function above
and the data that you are adding and returns true if
everything seemed to work, false otherwise.
entropy_test_finish(index: any): entropy_test_result
Calling this function indicates that all of the desired data
has been inserted into the entropy_test_add function and the
entropy should be calculated. This function *must* be called
in order to clean up an internal state tracking variable.
If this is never called on an index, it will result in a
memory leak.
The entropy_test_result values have several measures of the
entropy, but a good one to work with is the "entropy" attribute.
It's a double and as the value approaches 8.0 it can be considered
more and more random. For example, a value of 7.832 would be
quite random but a value of 4.671 is not very random.
1.6-dev.20 Wed Jan 19 17:30:11 PST 2011
- BRO_DNS_FAKE is now listed in the --help output. (Seth Hall)
1.6-dev.18 Wed Jan 19 16:37:13 PST 2011
- Removing unnecessary expire timer from http_sessions. (Gregor
Maier)
1.6-dev.16 Sat Jan 15 14:14:21 PST 2011
- Updates to the build system. (Jonathan Siwek)
* ``make dist`` is now available to be used with the top-level
Makefile for creating source packages according to #344.
* ``make-rpm-packages`` and ``make-mac-packages`` scripts can
now generate binary packages according to #295.
* Additional configure options to change packaging behavior.
* OS X builds will now prefer to link static libraries of
optional dependencies that don't come with the vanilla
operating system.
* Fix for OS X 10.5 compile error dealing with the llabs()
function from stdlib.
* Installing as a different user than the one that
configured/built now works (although, a harmless error message
about not being able to write the install manifest may occur).
1.6-dev.3 Wed Dec 8 04:09:38 PST 2010
- Merge with Subversion repository as of r7137. Incorporated change:
* Fix for packet processing resumption when a remote Bro dies
during state synchronization (Robin Sommer).
1.6-dev.2 Wed Dec 8 03:57:03 PST 2010
- Compatibility fix for OpenSSL 1.0.0 (Christian Kreibich, Gregor
Maier).
1.6-dev.1 Sat Nov 27 12:19:47 PST 2010
- Merge with Subversion repository as of r7098. Incorporated changes:
* Rotation post-processors are now passed an additional argument
indicating whether Bro is terminating (Robin Sommer).
* Bro now consistently generates a file_opened event for all
fopen() calls. (Robin Sommer).
* You can now redefine the email_notice_to function (Robin
Sommer).
1.6-dev.0 Fri Nov 26 13:48:11 PST 2010
- The Bro source code is now developed in the new git repositories.
See the developer pages at http://www.bro-ids.org for more
information on the new development process.
- Bro's build and installation setup has been moved from GNU
autotools to CMake. As a result of that, layout and specifics of
the distribution has changed significantly.
- Lots of pieces have been removed from the distribution that are
either now unnecessary or are no longer maintained.
- As part of the cleanup, a numbef of Bro configure options and
their corresponding functionality have been removed, including:
* --disable-select-loop
* --with-dag
* --disable-nbdns
* --enable-activemapping
* --enable-activemapping
* --enable-shippedpcap
- The previous configure option --enable-int64 is now enabled by default,
and can no longer be disabled.
- ClamAV support has been removed, which has been non-functional for
a while already.
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1.5.3 Thu Mar 3 08:55:11 PST 2011
- Removing aux/broctl/policy/cluster-addrs.hot.bro from the
distribution. The script is no longer needed and could in fact break
an installation because it redefines an old variable that has went
away. (Robin Sommer)
- Smarter way to increase the communication module's pipe's socket
buffer size, resulting in a value closer to the allowed maximum.
(Craig Leres)
- BroControl now also maintains links from the log archive to the
current set of logs when running in standalone mode. (Robin Sommer)
- Bug fix for a file descriptor leak in the remote communication
module. (Scott Campbell)
- Bug fix for BroControl to now activate trace-summary's sampling in
cluster mode, but not anymore in standalone mode. (Robin Sommer)
- Broccoli updates:
* Accept empty strings ("") as values in the configuration file.
(Craig Leres)
* Support for specifying a separate host key for SSL-enabled
operation, with documentation update. (Craig Leres)
1.5.2 Wed Jan 12 17:34:55 PST 2011
- Portability fixes for --enable-int64 (Vern Paxson).
- Bug fix for Active Mapping support (Kevin Lo).
- Broccoli compiler warning fixes (Kevin Lo).
- Bug fixes for --enable-int64 and for avoiding bogus statistics /
bad memory references when generating profiling information upon
exit (Vern Paxson).
- Bug fixes for terminating connections (Tyler Schoenke and Vern Paxson).
- Removed now-quite-stale SSHv1 overflow detection, as it's more prone
to false positives than useful detection (Vern Paxson).
- The SWIG file now explicitly lists those pieces from broccoli.h which it
wants to wrap, rather than just including all of broccoli.h (Robin Sommer).
- http-header.bro now includes a global "include_header: set[string]" If it
contains any strings, then only those headers will be processed. If left
empty, then you continue to get the current behavior of processing all
headers. (Robin Sommer).
- Several changes to drop.bro (Robin Sommer):
* If True, the new flag Drop::dont_drop_locals indicates that
local hosts should never be dropped. On by default.
* If True, the new flag Drop::debugging activates extensive debugging
output for the catch-and-release logic. Off by default.
* The timeout for tracking dropping information is now 1 day
rather than 7 days, to better echo the one-restart-a-day semantics
used in the past.
* Bug fix for hosts once dropped by later cleared; some state
for them persisted.
- Portability fix for Broccoli Python bindings on 64-bit platforms (Robin
Sommer).
- The HTTP analyzer no longer attempts to track Server/User-Agent
versions, as these are hugely voluminous (Seth Hall).
- HTTP and SMTP no longer have extra-short inactivity timeouts, as
these were too often leading to premature expiration of a connection
(Robin Sommer).
- Tracking of HTTP refer[r]er's by setting log_referrer. (Vern Paxson).
- The "rst" tool (aux/rst/) now takes an optional "-I <text>" argument
that instructs it to inject <text> as payload rather than sending a RST
packet (Vern Paxson). <text> must be NUL-terminated, and the NUL is not
included.
- Bug fix for crashes in the DNS analyzer when processing replies for
which no request was seen (Robin Sommer).
- Addressed a number of lint nits (Vern Paxson).
- Rotation post-processors are now passed an additional argument
indicating whether Bro is terminating (Robin Sommer).
- Bro now consistently generates a file_opened event for all fopen() calls.
(Robin Sommer).
- The "cf" utility now ignores a leading "t=" prefix, for compatibility
with Bro's "tagged" logging format (Robin Sommer).
- You can now redefine the email_notice_to function (Robin Sommer).
- Fix for packet processing resumption when a remote Bro dies during
state synchronization (Robin Sommer).
- OpenSSL/X509 portability fix, at long last (Gregor Maier & Christian
Kreibich).
- Fix for compatibility with newer versions of autoconf (Gregor Maier).
- A larger BroControl update (Robin Sommer, if not marked otherwise):
o Increasing default timeouts for scan detector significantly.
o Increasing the manager's max_remote_events_processed to
something large, as it would slow down the process too much
otherwise and there's no other work to be interleaved with it
anyway.
o Adding debug output to cluster's part of catch-and-release
(extends the debugging already present in policy/debug.bro)
o Fixing typo in util.py. Closes #223.
o Added note to README pointing to HTML version.
o Disabling print_hook for proxies' remote.log.
o broctl's capstats now reports a total as well, and stats.log
tracks these totals. Closes #160.
o Avoiding spurious "waiting for lock" messages in cron mode.
Closes #206.
o Bug fixes for installation on NFS.
o Bug fix for top command on FreeBSD 8.
o crash-diag now checks whether gdb is available.
o trace-summary reports the sample factor in use in its output,
and now also applies it to the top-local-networks output (not
doing the latter was a bug).
o Removed the default twice-a-day rotation for conn.log. The
default rotation for conn.log now is now once every 24h, just
like for all other logs with the exception of mail.log (which is
still rotated twice a day, and thus the alarms are still mailed
out twice a day).
o Fixed the problem of logs sometimes being filed into the wrong
directory (see the (now gone) FAQ entry in the README).
o One can now customize the archive naming scheme. See the
corresponding FAQ entry in the README.
o Cleaned up, and extended, collection of cluster statistics.
${logdir}/stats now looks like this:
drwxr-xr-x 4 bro wheel 59392 Apr 5 17:55 .
drwxr-xr-x 96 bro wheel 2560 Apr 6 12:00 ..
-rw-r--r-- 1 bro wheel 576 Apr 6 16:40 meta.dat
drwxr-xr-x 2 bro wheel 2048 Apr 6 16:40 profiling
-rw-r--r-- 1 bro wheel 771834825 Apr 6 16:40 stats.log
drwxr-xr-x 2 bro wheel 2048 Apr 6 16:25 www
stats.log accumulates cluster statistics collected every time
"cron" is called.
- profiling/ keeps the nodes' prof.logs.
- www/ keeps a subset of stats.log in CSV format for easy plotting.
- meta.dat contains meta information about the current cluster
state (in particular which nodes we have, and when the last
stats update was done).
Note that there is no Web setup yet to actually visualize the data in
www/.
o BroControl now automatically maintains links inside today's log
archive directory pointing to the current live version of the
corresponding log file (if Bro is running). For example:
smtp.log.11:52:18-current -> /usr/local/cluster/spool/manager/smtp.log
o Alarms mailed out by BroControl now (1) have the notice msg in the
subject; and (2) come with the full mail.log entry in the body.
o Fixing broctl's top output. (Seth Hall).
o Fixing broctl's df output in certain situations.
o BroControl fix for dealing with large vsize values reported by
"top" (Craig Leres).
1.5.1 Fri Dec 18 15:17:12 PST 2009
- Due to a Python configuration problem, the original 1.5 distribution
did not include the BroControl component, which also introduced a
portability problem for CentOS. These issues have now been fixed (Robin
Sommer and Vern Paxson).
1.5 Wed Dec 16 21:28:47 PST 2009
- Bro now comes with a new framework, BroControl, for managing an
operational Bro setup, including support for installation, configuration,
and maintainance tasks such a log archival and mail notification. The
framework transparently supports both traditional standalone setups as
well as cluster installations in which multiple Bro boxes coordinate to
analyze a high-volume network link.
See aux/broctl/README for more information about BroControl.
Note, BroControl supersedes the older BroLite system, which is no longer
supported and has been deprecated for a while now.
- Numerous adjustments to DPD = dynamic protocol detection (Robin Sommer):
o The Analyzer::ProtocolViolation?() method can now be passed the
offending data (which POP3, SMTP, and FTP now do). This information
is added to the "reason" string passed to the script level.
o SMTP now more accurately reports violations.
o FTP stops processing when client & server successfully negotiate
an AUTH scheme (leading to subsequent encryption).
o Analyzer::ProtocolViolation() is virtual, and
TCP_ApplicationAnalyzer() overrides it to not report violations
for any partial connections, because very likely these arise just
due to the analyzer getting confused.
o TCP::IsPartial() returns true if any side did not start with
a SYN packet (used to be just be for the originator).
o The connection_state_remove handler in conn.bro now has a higher
&priority so that other handlers for the same event can use
determine_service() and see any changes it performs.
o DynDisable:max_volume specifies a volume limit (default 10K).
Once a connection exceeds this limit, further protocol
limitations will neither raise ProtocolViolation notices nor
cause the analyzer to be disabled.
o The event engine no longer raises protocol_violation events for
TCP connections which had gaps, as these have proven too unreliable.
(Note that, ideally, the *analyzers* should avoid reporting
protocol_violations when they can't reliably parse a connection
anymore after a gap; but many don't.)
- A set of new script functions provide support for incrementally computing
MD5 checksums (Seth Hall).
md5_hash_init(index: any): bool
Initializes an incremental hashing instance. "index" is
a value of arbitrary type, used to identify this particular
instance (you can have multiple concurrent instances by
using different index values). Returns T on success,
F on failure (such as the index is already in use).
md5_hash_update(index: any, data: string): bool
For the given hashing instance, updates the hash
based on the given data. Returns T on success, F on
failure (such as the index has not been initialized).
md5_hash_finish(index: any): string
Returns the MD5-printable hash for the given index
and terminates the instance, or the string "" if the
index was not active.
- Bro now supports a believed-to-be-robust mechanism for estimating the
proportion of traffic that it failed to capture ("measurement drops"),
which can arise due to overload in either Bro itself, the kernel's
packet filter, or problems with the link tapping mechanism (Vern Paxson).
The event engine can generate estimates for either live traffic or what
was previously recorded in a trace file, though traces subject to some
forms of selective omission (such as skipping over parts of a connection
to reduce storage) can lead to erroneous values.
The estimates are based on observing gaps in TCP data streams, and
come in two forms: the rate at which such gaps appear, and the relative
volume of data missing due to the gaps. (We've found however that the
volume-based estimator is not robust due to occasional packets with
incorrect sequence numbers, so this estimator is off by default.)
The easy way to get the estimates is to load capture-loss.bro.
By default, it generates a CaptureLossSummary notice upon Bro's exit,
which can look like:
1130222759.344066 CaptureLossSummary estimated rate = 0.00089124 / 0.000970997 (events/bytes)
If the estimated loss is none, however, it suppresses this notice,
unless you redef CaptureLoss::summary_if_none to T.
You can also get finer-grained access by defining a "gap_report"
event handler and redef'ing gap_report_freq to a non-zero interval
(such as "10 sec"). This event allows you to pinpoint regions in
time that exhibit significant capture loss. See capture-loss.bro
for an example of a handler for this event.
Finally, these changes include a number of fixes to Bro's
ack_above_hole/content_gap analysis, which is now significantly
more robust.
- GeoIP support now supports ASN lookups via the built-in
function lookup_asn(a: addr): count (Scott Campbell and Seth Hall).
- The GeoIP built-in's lookup_location() and lookup_asn() now
support IPv6 (Seth Hall). Note, the current GeoIP distribution
doesn't include any IPv6 databases, so for now these won't succeed,
but the hooks are in place for when databases become available.
- lookup_location() now falls back back to the country database if
the city database isn't available (Seth Hall).
- The new SuccessfulPasswordGuessing Notice is generated when a host
has been seen attempting password guessing (currently only for FTP
sessions) and then successfully logs in (Royal Chan). You can control the
threshold for such reports in terms of how many attempts the host must
have made by redef'ing the variable password_guessing_success_threshhold,
which defaults to 20.
- The new script http-detect-passwd.bro analyzes the Web items returned
for fetches that appear to be accessing the passwd file (Akhil Dhar).
It generates a PasswordFullFetch Notice if it appears that the item
includes a full password file, and PasswordShadowFetch if it looks like
a shadowed password file.
- The new built-in
system_env(cmd: string, env: table[string] of string)
works like system(), but puts the table entries into the environment
before invoking the command (Robin Sommer). Each <index> in the table
creates an environment variable of the form "BRO_ARG_<index>", whose
value is the corresponding table entry.
- The new script function
execute_with_notice(cmd: string, notice_info)
executes "cmd" with an environment containing the fields of the
notice_info, i.e., the information associated with a Notice (Robin Sommer).
Per the new system_env() function above, the environment variables appear
as "BRO_ARG_<tag>", where <tag> is the field tag as it appears in
notice.log when you enable use_tagging.
- The new built-in enable_raw_output(file) acts the same as
the attribute &raw_output (Seth Hall).
- The new built-in file_opened(f: file) event is generated any time Bro
opens a script-level file (Justin Azoff). You can use this, for example,
if you want to ensure that a given file has a prelude in it such as
human-readable headers, even when the file is rotated.
- The notice_info record has a new field
aux: table[string] of string &optional
which you can use for information specific to a given type of notice
(Robin Sommer). Entries in $aux appear as "aux_<index>" tags in notice.log.
- Another new notice_info record field is the boolean do_alarm (default=T),
which, if set to F, overides a notice action otherwise specifying to
generate an alarm (Robin Sommer). In other words, if do_alarm is F, no
alarm will be generated independent of the notice action.
This is a work-around for the fact that we can't specify more than one
action. In particular, we couldn't NOTICE_DROP but then *not* alarm,
which we now can by returning NOTICE_DROP yet setting do_alarm to F.
- The notice_info record field $dropped now appears in the tagged output
format if true (Robin Sommer).
- NOTICEs relating to scan detection now no longer include the connection
that triggered the notice, as it really doesn't contain any useful
information, given that the particular trigger simply depends on the
detection algorithm and its parameters (Robin Sommer). However, we do
explicitly set $p (port number) in the notice, and also $n with the
number of attempts.
- drop.bro now hardwires a Catch-and-Release redrop after seeing one
connection from a previously-dropped-but-already-released host
(Robin Sommer).
- drop.bro now provides some new hooks (Robin Sommer):
event address_dropped(a: addr)
Generated when an address has been dropped.
event address_restored(a: addr)
Generated when connectivity to an address has been restored,
such as using the Catch-and-Release mechanism.
event address_cleared(a: addr)
Generated when an address that was dropped in the past is
no longer being monitored looking for new connections
(as part of the Catch-and-Release mechanism).
- The new built-in function
hexdump(data_str: string) : string
returns a hex dump representation of the given input data (Christian
Kreibich). The dump renders 16 bytes per line, with hex on the left and
ASCII (where printable) on the right.
- Bro's notion of when a TCP connection begins now dastes to the first
instance of an initial SYN packet seen, rather than the last (Gregor Maier).
- The Time Machine script tm-contents.bro now generates
event contents_saved: event(c: connection, orig_file: string,
resp_file: string)
when the content of a connection has been completely saved to disk
(Robin Sommer).
- The mime.bro script now exports the MIME header callback table, and also
marks it as &redef'able so you can modify its entries (Matthias Vallentin).
The mime_log file is also now exported.
- A new signature file, policy/sigs/http-bots.sig, contains signatures
to detect some of the current HTTP based controlled bot families (Seth Hall).
- The signature engine's HTTP pattern matching has been fixed (Seth Hall)
to align with the documentation at:
http://www.bro-ids.org/wiki/index.php/Reference_Manual:_Signatures#Content_conditions
In particular, the content condition "http" is now referred to as
"http-request" (though "http" still works for backward compatibility),
"http-request-header" and "http-reply-header" now provide access to
headers seen in only one direction, and similarly for "http-request-body"
and "http-reply-body". (This latter is still accessible as "http-body"
for backwards compatibility.)
- The new script variable max_remote_events_processed: count (default 10)
sets a limit on the number of remote events processed in each round,
before tending to other inputs (Robin Sommer).
- If you set the new script variable dump_used_event_handlers to T,
then on startup Bro dumps out all of the event handlers that the
loaded set of scripts can invoke (Matthias Vallenti).
- Summaries for DNS PTR scanning now use a separate Notice,
DNS_PTR_Scan_Summary, rather than overloading DNS_PTR_Scan (Robin Sommer).
- scan.bro now provides a table skip_dest_server_ports: set[addr, port]
which lists servers (defined as an address and a port) excluded from
scan detection computations (Craig Leres and Jay Krous).
- When redefining values on the command line directly (using var=value),
quotation marks are now implicit only if "var" is a variable of type
string (Christian Kreibich). This allows other string-like values
(such as enum's) to be passed as well.
- scan.bro now explicitly loads conn.bro so that it can itself
be loaded independently (Robin Sommer).
- login.bro depends on scan.bro (because of tracking authentication
"scans"), so now it explicitly loads it (Vern Paxson).
- UDP_datagram_length_mismatch is now by default flagged just once per
originating host rather than once per connection, as it can generate
tons of messages (Vern Paxson).
- Removed now-long-boring flagging of access to Solaris "listen"
service as "hot" (Vern Paxson).
- Removal of libedit, since libreadline provides similar functionality
(Christian Kreibich).
- Added scripts missing from distribution: dce.bro, ncp.bro, and smb.bro
(Vern Paxson).
- ssh.bro now exports ssh_ports (Seth Hall)
- A number of improvements to inter-Bro communication (Robin Sommer).
(1) Remote communication now no longer includes location information for
serialized objects; that removes quite a bit of redundacy from the network
traffic.
(2) The new option 'remote_check_sync_consistency" disables the cross-check
on the receiving side of &synchronized state of whether the current value
of a variable has the value expected by the sender. Transmitting the
original values in addition to the updates generates quite a bit CPU &
network load in some cases (in particular, a table of tables). The default
for remote_check_sync_consistency is off, and so far that in particular
seems to reduce the proxy's load quite a bit.
(3) Complete overhaul of the internal caching of serialized objects. The
objective of the caching is avoid retransmitting already sent values over
and over again. It turns out, however, that some objects are very stable
and hardly change or get replaced (e.g., Bro types); while other change
all the time and are hardly reused some time later (e.g., Vals). Now
we maintain *two* caches independently for these types of objects; one
with a low turn-over one and another with a high one. This should reduce
CPU load on both sender and receiver sides.
The new scheme is only used if both communicating Bros support it; with
older Bros, as well as with Broccoli, we continue using the old scheme.
- Some reworking of remote printing (Robin Sommer), as follows. Bro now
uses a new interprocess message rather than print_hook events, to better
manage buffering and associated load (these can produce failures depending
on system configuration; see remote.log). A number of timeouts and
buffer sizes have been tuned. Internally, EINTR errors are now treated
separately from EAGAIN. Finally, even with remote_check_sync_consistency=F,
one type of consistency check was still being done; this is no longer
the case.
- The DNS analyzer now generates events (dns_query_reply/dns_rejected)
for replies with zero questions (Robin Sommer).
- Perftools support for incompatible changes in the 1.0 API (Robin Sommer).
- Rearranged (generally reducing, though not always) some state timeouts
associated with scan detection (Robin Sommer). In addition, when a
scanning address crosses ignore_scanners_threshold (meaning that it will
be ignored from now on anyway), it gets discarded from all state-tracking
tables. Finally, the ignore_scanners_threshold now applies all kinds
of scans, not just address scans.
- Substantial Broccoli updates, including a new initialization requirement
that breaks backward compatibility, support for enqueueing serialized
event data for transmission, and OpenSSL threadsafe initialization.
See aux/broccoli/ChangeLog for details (Christian Kreibich, Robin
Sommer, and Matthias Vallentin).
- Broccoli hashtable optimisation. See aux/broccoli/ChangeLog for
details (Christian Kreibich & Matthias Vallentin).
- Broccoli memory leak fixed, see aux/broccoli/ChangeLog for details
(Christian Kreibich).
- Broccoli: updates to bropipe tool (Steve Chan and Robin Sommer).
- Bug fixes for Broccoli Python bindings (Robin Sommer and Matthias Vallentin).
- Fixed nasty bug due to module scoping that completely kept stepping-stone
detection from working (Vern Paxson).
- A serious bug in the packet sorter has been fixed (Robin Sommer).
- Bug fix for extra NULs getting embedded in escaped strings (Seth Hall).
- Bug fix for HTTP messages that use "Connection: close" rather than length
headers, which yielded erroneous reassembled messages with \r\n's when
only \n's were present (Bernhard Ager).
- Fix for reporting on ICMP flows that are expired from the flow table
(Vern Paxson). Previously there was a race condition if the flow
was flushed prior to its summary timer expiring.
- The -l option (list the scripts that Bro loads) now correctly prints
scripts loaded by the prefix mechanism, and uses indentation to indicate
the load hierarchy (Robin Sommer).
- A bug has been fixed (really, worked around) in drop.bro that prevented
dropped addresses from being properly restored (Robin Sommer).
- Fixes for deadlocking problems in the Broccoli protocol. See
aux/broccoli/ChangeLog for details (Christian Kreibich & Robin Sommer).
- Bug fix for DNS analyzer on 64-bit machines (Gregor Maier).
- Bug fix for asynchronous DNS lookups to prevent some successful lookups
being reported as timed out (Robin Sommer).
- Bug fix for tracking line numbers associated with compound statements
(Po-Ching Lin).
- Fix for a rare condition in which the main Bro process couldn't kill
its child process (Robin Sommer).
- Fix for file rotation when the underlying file is deleted before the
timer expires (Robin Sommer).
- Fix for potential crash when communication connections break down,
and also for releasing cached objects (Robin Sommer).
- Fix for default table entries computed by function invocation to not
cache previous results (Robin Sommer).
- Fix for Bro's internal DNS resolution (Scott Campbell and Robin Sommer).
- Portability fix for DAG packet capture (Gregor Maier).
- Portability fix for --enable-brov6 (Robin Sommer).
- Portability fixes for FreeBSD (Vern Paxson).
- A work around for new_packet() crashing on IPv6 packets (Vern Paxson).
For now, IPv6 packets are skipped. Also, for fragments the event handler
is now only called for the fully reassembled packet.
- The new configuration option --disable-nbdns supports disabling non-blocking
DNS at configure time (Sean McCreary). Note, there are some known problems
with it in some environments.
- A number of configuration fixes and enhancements (Christian Kreibich
and Robin Sommer).
- Consistency nit for the configuration process (Seth Hall).
- A number of reference-counting and other memory management fixes
(Robin Sommer).
- Bug fix for inter-Bro communication lockup (Seth Hall and Robin Sommer).
- Bug fix for computing TCP payload length in new_packet event (Lothar Braun).
- Bug fix for sending boolean True values via Broccoli (Seth Hall).
- make distcheck fix to clean up .bif.bro files (Christian Kreibich).
- Bug fix for DPD's recognition of SSLv2 connections (Seth Hall).
- Bug fix for &default for tables indexed by subnets (Seth Hall).
- A bug has been fixed that could crash Bro when you called get_event_peer()
after a remote connection had already disppeared (Robin Sommer).
- Introduced a work-around for crashes that occur when Bro exits
due to handling a signal (Robin Sommer).
- Bug fix for checkpoint.bro - don't schedule timers for times that
aren't actually in the future (Robin Sommer).
- Hostname formatting fix for anon.bro (Fabian Schneider).
- Bug fix for redundant .log extension in Time Machine log file
(reported by CS Lee).
- Removed now-outdated special-casing of Linux reporting of packet filter
statistics (Peter Wurzinger and Robin Sommer).
- A number of memory leaks fixed (Robin Sommer).
- Addressed warnings from newer versions of g++ (Robin Sommer and Vern Paxson).
- Fixed an invocation issue in the ca-create script that prevented it from
working with recent OpenSSL versions (Craig Leres & Christian Kreibich).
- Comment fixed in drop-adapt (Justin Azoff).
- Duplicate code removed from Val (Seth Hall).
1.4 Fri Oct 17 11:08:52 PDT 2008
- We are no longer supporting a previous Bro release as the "stable"
version. Rather, the model now is that the current public release will
aim for increasing stability (occasionally updated with fixes), and those
who wish to use a "bleeding-edge" snapshot can do so via access to the
public SVN source code repository, as explained at
http://bro-ids.org/wiki/index.php/Subversion#Public_Access
Note that all previous releases remain available from the download page;
what is changing is that we no longer commit to support for the most
recent of these.
- We have clarified the copyright statement that covers most of the
code to remove the "advertising clause" that derived from older
BSD licenses, and we have removed copyright wording from most source
code files. See COPYING for the current wording and a list of
files that retain their own copyright notices.
- Bro now supports analyzing NetFlow v5 data, i.e., from Cisco routers
(Bernhard Ager). NetFlow can be useful for intrusion detection as it
allows analysis of traffic from many different points in the network.
Bro can now read NetFlow data from a UDP socket, as well as (mostly
for debugging purposes) from a file in a specialized format. You can
create these files with the programs given in aux/nftools.
Command line switches:
-Y|--netflow <ip>:<prt>[=<id>] | read flow from socket
This is the usual way of getting NetFlow data into Bro by
opening a UDP socket on <ip>:<prt> and reading all incoming
packets. Setting the <ip> to 0.0.0.0 should work on most
platforms. Optionally you may set an identifier <id> for the
source - useful if there are many different sources you want
to analyze in parallel. This might also be necessary if you
want to use this feature with a clustered Bro.
Examples:
bro -Y 0.0.0.0:5555 netflow
bro -i eth0 -Y 10.0.0.1:1234=src1 brolite netflow
-y|--flowfile <file>[=<ident>]
Used to read from a file. You can optionally include an
identifier for the source.
Examples:
bro -y myflowfile netflow
bro -y myflowfile=src1 otherflowfile=src2 netflow
Netflow Events:
event netflow_v5_header(h: nf_v5_header)
Generated upon reading a new NetFlow PDU, as summarized in the
argument. The field h_id gives the flow source identifier and
a serial number. You can use this field to associate subsequent
netflow_v5_record events with their header.
event netflow_v5_record (r: nf_v5_record)
Every record within a NFv5 PDU generates a corresponding
netflow_v5_record() event. The relatively complex timestamp
format of NFv5 is already converted to Bro's time type, and
the TCP header flags are separated into bools.
The distribution includes an example analysis script, netflow.bro.
It simply dumps received NetFlow records. If netflow_restitch is T
(the default), then Bro performs flow restitching as well, and two
script variables become relevant:
global netflow_finished_conn_expire = 310 sec &redef;
specifies how long to wait for additional flow records after
a RST or FIN for
const netflow_table_expire = 31 min;
Its setting only affects table declarations, and therefore
cannot be usefully redef'd.
Auxiliary programs:
Bro uses a custom format for flow data stored in files,
to enable preserving timestamps of the PDU arrivals and the
exporter's IP address. The tools nfcollector and ftwire2bro
in aux/nftools/ provide ways to manipulate the Bro NF file
format. The first dumps NetFlow data from a UDP socket to
stdout or to a file in Bro format. The second converts NetFlow
data in "wire" format to Bro format, and, while doing so,
fakes up the exporter's IP address and timestamp. You can get
"wire" format from normal flow-tools files, e.g., by using
'flow-export -f 4'. Please note that the Bro format is just
a hack to allow for easier debugging. Therefore the format
is not in fact platform independent, and not suitable for data
storage.
- A new DHCP analyzer generates the following events (Po-Ching Lin):
event dhcp_discover(c: connection, msg: dhcp_msg, req_addr: addr)
event dhcp_offer(c: connection, msg: dhcp_msg, mask: addr,
event dhcp_request(c: connection, msg: dhcp_msg,
event dhcp_decline(c: connection, msg: dhcp_msg)
event dhcp_ack(c: connection, msg: dhcp_msg, mask: addr,
event dhcp_nak(c: connection, msg: dhcp_msg)
event dhcp_release(c: connection, msg: dhcp_msg)
event dhcp_inform(c: connection, msg: dhcp_msg)
where dhcp_msg values look like:
type dhcp_msg: record {
op: count; # 1 = BOOTREQUEST, 2 = BOOTREPLY
m_type: count; # the type of DHCP message
xid: count; # transaction ID of a DHCP session
h_addr: string; # hardware address of the client
ciaddr: addr; # original IP address of the client
yiaddr: addr; # IP address assigned to the client
};
See dhcp.bro for the corresponding analysis script (which could
probably use some refinements).
Note, this analyzer is implemented using BinPAC, so you will need
to specify --use-binpac to activate it.
- A BitTorrent analyzer is now available (Nadi Sarrar). See the policy
scripts bittorrent.bro and bt-tracker.bro for the events generated for
analyzing transfers and tracker dialogs, respectively.
- The "Bro Lite" configuration is now deprecated and will not in
general be supported (Robin Sommer & Vern Paxson).
- "make install" now only installs a core set of files (Robin Sommer).
Policy files are now installed in <prefix>/share/bro/* (or whatever
configure determines $datadir to be), which is now in Bro's default
search path. It creates a directory <prefix>/share/bro/site for local
policy files, and the default BROPATH is extended to include this. The
default path no longer includes policy/local. You can install the
additional files used by the (now deprecated) "Bro Lite" configuration
using "make install-brolite".
- Substantial updates to Broccoli, including support for container
types (tables and sets) as well as a new metadata structure for event
callbacks, facilitating truly generic event handler implementations
(Christian Kreibich, Seth Hall and Robin Sommer). See aux/broccoli/ChangeLog
for details.
- Extensive changes to allow Bro to process packets captured in the
past intermingled with those captured in real-time (Matthias Vallentin
and Robin Sommer). This operation reflects combining Bro with use of
"Time Machine" functionality for packet capture.
- We have unfortunately had to disable support for configuring Bro
to use ClamAV, since it turns out that the key interface we need
for processing blocks of memory directly rather than whole files
is no longer supported by the package, and in fact was buggy even
when it was (Robin Sommer).
- The new signature option "http-body /<regexp>/" matches <regexp>
on the body data of HTTP entities (Robin Sommer). The matching is
done after decompressing the body, if necessary.
- The new built-in function identify_data(data: string, return_mime: bool)
analyzes the string "data" and returns its type according to libmagic,
if installed (Seth Hall). The second argument controls whether it should
be returned as a MIME-type or just an identifying string. For example,
identify_data("MZpofigu", F) returns the string "MS-DOS executable", and
print identify_data("MZpofigu", T) returns "application/x-dosexec".
- The new analysis script http-identified-files.bro identifies the
type of items returned by Web servers using libMagic (if available)
and generates notices for interesting types and mismatches between
URLs and types (Seth Hall).
You configure it using two variables. watched_mime_types is a pattern
(default /application\/x-dosexec/ | /application\/x-executable/ ) for
which any MIME type matching the pattern generates a HTTP_WatchedMIMEType
notice.
mime_types_extensions is a table mapping strings to patterns specifying
how URLs for the given MIME type should appear. (Ideally, this would
be a table mapping patterns to patterns, but Bro doesn't currently support
that.) It defaults to:
["application/x-dosexec"] = /\.([eE][xX][eE]|[dD][lL][lL])/
i.e., do Windows executables end in .exe or .dll.
You can also redef the pattern ignored_urls to specify URLs that should
not generate complaints. It defaults to matching Windows Update.
- The new script http-extract-items.bro extracts the items from HTTP
traffic into individual files (Vern Paxson). Files are named:
<prefix>.<n>.<orig-addr>_<orig-port>.<resp-addr>_<resp-port>.<is-orig>
where <prefix> is a redef'able prefix (default: "http-item"), <n> is a
number uniquely identifying the item, the next four are describe the
connection tuple, and <is-orig> is "orig" if the item was transferred
from the originator to the responder, "resp" otherwise.
- The workings of how Bro interfaces to external programs for dropping/
restoring connectivity of misbehaving hosts has been significantly
reworked (Brian Tierney and Robin Sommer).
First, dropping decisions used to be made directly by analyzer scripts,
such as scan.bro directly calling drop_address(). Now instead the
scripts generate Notices and then the notice policy can have an
action of NOTICE_DROP to codify that the response to the given Notice
is to drop the source. The new notice_action_filter of drop_source
drops the source of notices, and drop_source_and_terminate both
drops the source and terminates the corresponding connection.
So, to drop all sources triggering a specific notice, one can now, e.g.,
write:
redef notice_action_filters += { [Hot::SSH_Overflow] = drop_source };
Related to this change, notice_info has a new field $dropped, set to
true if the Notice triggered a (successful) drop.
Second, by redef'ing Drop::use_catch_release to T (default F) you can
activate "catch-and-release" logic. You use this mode when you need to
manage a limited number of possible blocks, or to build in automatic
"forgiveness" in situations where blocked sources might become benign
(such as due to dynamic IP addresses). If a source has been idle for
Drop::drop_time, then it is unblocked. However, if it is again seen as
block-worthy, then it is blocked for an interval of Drop::long_drop_time.
Third, ICMP scanning is now reported by its own notice, ICMPAddressScan,
rather than Scan::AddressScan.
- Google's perftools have replaced mpatrol for leak-checking and
heap-profiling (Robin Sommer). If Bro is compiled with --enable-perftools
and configure finds the perftools, there are two command-line options
available:
-m turns on leak checking of the main packet loop, with some
uninteresting leaks are suppressed. Currently, with one
exception (the RPC analyzer; problem not yet found), it reports
no leaks when running the test suite.
-M turns on heap profiling: Bro will take a snapshot of the heap
before starting the main packet loop and another one when
finished. These snapshots can then be analyzed with pprof.
For more information about the perftools see
http://code.google.com/p/google-perftools
- Notice tags are now generated in a pseudo-unique fashion that, with high
probability, ensures that tags generated by separate Bro processes don't
clash when logged to a common location, such as for a Bro cluster (Robin
Sommer). Tags are now string's rather than count's, and are associated
with all notices, not just that are connection-related. You can however
redef the string notice_tag_prefix or the function new_notice_tag to
further control how such tags are generated.
- Four new built-ins for type conversion (Robin Sommer):
function double_to_interval(d: double): interval
function addr_to_count(a: addr): count
function port_to_count(p: port): count
function count_to_port(c: count, t: transport_proto): port
- Many policy scripts have been modified to use modules & scoping
(Robin Sommer and Matthias Vallentin), which may require updates to
existing scripts/refinements.
- The new script variable dpd_conn_logs (default F), if true, changes the
semantics of the service field in connection logs written to conn.log,
as follows (Robin Sommer). It becomes a comma-separated list of analyzers
confirmed by DPD to parse the connection's payload. If no analyzer could
confirm its protocol, but the connection uses a well-known port, the
service is the name of the port with "?" appended (e.g., "http?"), as
long as the corresponding analyzer has not declined the connection.
In addition, ftp-data sessions are labeled "ftp-data" and portmapper
connections are labeled with the specific method-call (just as before).
dpd_conn_logs defaults to F because the change in semantics may break
scripts that parse conn.logs; but it will likely change to the default
in the future. With dpd_conn_logs turned off, conn logs are generated
as they used to be, with a few rare exceptions (with previous versions,
the service field was sometimes determined while the connection was still
alive; now it's always determined at the time when the conn.log entry
is written out).
- The SSL analyzer has been rewritten using BinPAC, with a number of
robustness improvements (Tobias Kiesling). It currently is only used
if you execute with --use-binpac.
- Python bindings for Broccoli are now available in
aux/broccoli/bindings/python/ (Robin Sommer). See README/README.html
in that director for details.
- The new "auth" option in remote.bro indicates whether a given side is
considered "authoritative" for shared state, in which case it sends its
initial state to &sync'ed peers (Robin Sommer). When two peers synchronize
their state, one side sends its current set of state to the other as
soon as the remote connection is established. The one sending the state
used to be the one who has been running longer; now it can also be
explicitly set via the "auth" flag in the Remote::Destination.
- Two new tuning parameters for scan.bro (Robin Sommer):
ignore_scanners_threshold (default 0):
If a host has scanned more than this many hosts, it is completely
excluded from further scan detection. 0 disables.
addr_scan_trigger (default 0):
A host is only tracked for address scanning once it has contacted
this many different hosts. Primarily intended for using a two-stage
scan detection with a Bro cluster: first, each node searches locally
for scanners by looking for hosts contacting more than
addr_scan_trigger destinations. Those hosts which do are then
globally tracked throughout the cluster by &synchronizing the scan
detector tables.
- When Bro serializes functions, it now does so by default using only
their name, rather than their full value (Robin Sommer). This prevents
propagation of expiration functions associated with tables and sets.
Note, currently there is no mechanism provided to switch from the
default behavior, but the internal hooks are in place to do so.
- The new built-in variable trace_output_file gives the name of the -w
output trace file (Robin Sommer).
- Bro no longer installs new file rotation timers when shutting down
(Robin Sommer).
- The new policy scripts remote-print-id{,-reply}.bro support convenient
access to printing the identifiers of a remote Bro (Robin Sommer).
You use the script remote-print-id.bro to request and receive the
printing; the remote Bro must have loaded remote-print-id-reply.bro
in order to process the request.
Example use:
bro -e 'redef PrintID::dst="<dst>" PrintID::id="<name-of-id>"'
<other scripts> remote-print-id
- scan.bro has been heavily modified to better support distributed scan
analysis (Matthias Vallentin and Robin Sommer).
- The check for unused event handlers is now turned off by default
(Robin Sommer). To enable, use "redef check_for_unused_event_handlers = T".
- The new script drop.bro has been split off from scan.bro to isolate
the logic concerning dropping addresses to block scans (Robin Sommer).
- The new -l flag lists each script as it is loaded (Robin Sommer).
- Textual descriptions of identifiers now include their attributes
(Robin Sommer).
- The new predefined function prefixed_id() returns a session identifier with
its peer-ID prepended if it's associated with a remote Bro (Robin Sommer).
This is now used when generating writing log files.
- remote.bro now assigns a priority of -10 to its bro_init() event handler
to allow others a chance to modify destinations (Robin Sommer).
- A large number of BinPAC updates (Ruoming Pang and Robin Sommer).
- The new built-in type_name(v): string returns the name of the type
of the value v (Vern Paxson). For example, "typename(5.2)" returns
"double". This function is mainly for internal debugging (i.e.,
finding mismatches between values generated by the event engine
versus how their type is expected by the script layer).
- The new built-in str_shell_escape() does some basic escaping on strings
that will be passed to system() (Christian Kreibich). Note, this function
isn't ready (robust enough) for routine use, however.
- The new built-in disable_print_hook(file) acts the same as
the attribute &disable_print_hook (Robin Sommer).
- The new script terminate-connection.bro factors out the terminate_connection()
functionality that used to be in conn.bro (Robin Sommer).
- The new attribute &group=<tag> can be associated with event handlers
to group them together into a set that can be manipulated as a whole
(Robin Sommer). <tag> is a string reflecting the name given to the group.
The built-in enable_event_group(group: string) turns on all the analyzers
in a given group, and disable_event_group(group: string) deactivates them.
- The new attribute &raw_output applies to variables of type file, disabling
escaping of non-printable characters (Seth Hall).
- You can now iterate over the characters in a string value using
a "for" loop, e.g., "for ( c in str ) ..." (Robin Sommer).
- The new built-in
function cat_sep%(sep: string, def: string, ...%): string
works similarly to cat(), except that it (a) separates the values
by "sep" and (b) substitutes "def" for empty strings (Seth Hall).
- The function string_escape() now takes a string of characters to escape
rather than a single character (Robin Sommer). Each character in the
string is preceded by '\' in the return value (also any embedded '\'s,
as before).
- The new built-in function global_ids() returns a table of all global
identifiers along with associated information (Robin Sommer). The
return value has type table[string] of script_id, indexed by the name
of the identifier and yielding records with the following fields:
type script_id: record {
type_name: string;
exported: bool;
constant: bool;
enum_constant: bool;
redefinable: bool;
value: any &optional;
};
- The new script function find_last(str: string, re: pattern) returns
the last occurrence of the given pattern in the given string, or
an empty string if no match (Robin Sommer). Note that this function
returns the match that starts at the largest index in the string, which
is not necessarily the longest match. For example, a pattern of /.*/
will return just the final character in the string.
- The new script variable record_all_packets, if redef'd to T (default F),
instructs Bro to record every packet it processes (Robin Sommer).
Prior to introducing this variable, Bro applied a few heuristics to
reduce recording volume. Setting this variable also causes packets
to be recorded very early in processing, which can be helpful for
debugging crashes.
- If the new script flag ssl_log_ciphers is set to T (default), ssl.bro
logs the ciphers seen (Robin Sommer).
- Much more expanded Time Machine support, now located in
policy/time-machine/ (Robin Sommer),
- The new command line option --status-file <file> (alias -U) specifies
the name of a file into which Bro will write an indicator of its current
processing status (Robin Sommer). Possible values include "INITIALIZING",
"RUNNING", "TERMINATING", "TERMINATED".
- The new policy script targeted-scan.bro looks for repeated access from
the same source to the same server, to detect things like SSH
password-guessing attacks (Jim Mellander).
- The "alternative" style for printing strings (i.e., a fmt() argument
of "%As") now renders the raw string, other than escape-expanding
embedded NULs (Vern Paxson). This change may be temporary, pending
development of more fine-grained control over string rendering.
- For now we have removed the %S functionality for fmt() (Robin Sommer).
%S was meant to print "raw" strings, but later processing of such
printing still introduces artifacts.
- GeoIP information now includes latitude and longitude (Seth Hall).
- ssh.bro now supports the variable skip_processing_after_handshake
which directs the event engine to omit any further processing of an
SSH connection after its initial handshake (Seth Hall and Robin Sommer).
This can help with performance for large file transfers but precludes
some kinds of analyses (e.g., tracking connection size). This change
also adds a scope of "SSH".
- Email notification of notices now allows for separate destinations
depending on notice type (in particular, a regular mail destination
versus a pager destination), and also escapes the notice to prevent
injection attacks (Seth Hall and Robin Sommer).
- The new policy script conn-flood.bro is a simple connection-flooding
detector, mainly meant as a demonstration (Robin Sommer).
- A large number of additions to the TLS/SSL known-ciphers suite (Seth Hall).
- Serialization now uses 64-bit IDs to cache items rather than 32-bit,
for robustness during long-running execution (Robin Sommer).
- The new script variable tcp_max_initial_window specifies, for flows
for which ACKs have never been seen, the maximum volume of initial
data after which Bro will assume that it is seeing only one side
of the connection and will not buffer data for consistency checking
awaiting the later arrival of ACKs (Robin Sommer). It defaults to 4 KB.
(Note, this used to be an internal value, so the behavior is not new.)
Set to 0 to turn off this functionality and have Bro attempt to
track all such flows.
- The new script variable tcp_max_above_hole_without_any_acks specifies,
for flows for which ACKs have never been seen, the maximum volume of
data above a sequence hole that Bro will tolerate for a connection
before giving up on tracking the flow (Robin Sommer). It defaults to 4 KB.
(Note, this differs from tcp_max_initial_window in that this threshold
applies to sequence holes rather than the beginning of flows. Like
tcp_max_initial_window this used to be an internal value.) Set to 0 to
turn off this functionality.
- The new script variable tcp_excessive_data_without_further_acks specifies
a threshold similar to tcp_max_above_hole_without_any_acks, but for
flows for which Bro has seen ACKs (Robin Sommer). It defaults to 10 MB.
Set to 0 to turn off the functionality.
- Equal signs ("=") in text for notices are now escaped when using the
tagged format to keep them unambiguous from the "=" delimiters
(Robin Sommer).
- The final tallies for notices are now processed as NoticeTally
NOTICE's rather than directly alarm'd (Robin Sommer).
- WeirdActivity notices now include an associated connection when appropriate
(Robin Sommer).
- Support for large (> 2^32 bytes) pcap trace files (Po-Ching Lin).
- Scoped names ("...::...") are now allowed in signature "eval"
constructs (Christian Kreibich).
- scan.bro is now decoupled from conn.bro, i.e., you can @load the
latter without getting the former (Vern Paxson). As part of this
change, the logic to invoke TRW is now in scan.bro.
- weird.bro has been updated with a number of missing Weird's (Vern Paxson).
- If when using inter-Bro communication the child Bro process terminates,
it now also terminates the parent process (Robin Sommer).
- BinPAC analyzers now interoperate with DPD (Robin Sommer).
- Some http.bro processing options are now exported so they can be
accessed in other scripts (Robin Sommer).
- SMTP analysis now applies to port 587/tcp as well as 25/tcp (Robin Sommer).
- $conn is now set in ServerFound notices (Robin Sommer).
- You can now create empty sets and tables using set() and table(),
i.e., the usual set/table constructors with no arguments (Vern Paxson).
By themselves, these have an unspecified type - you can't use them
directly other than to assign them. For example,
local bad_guys: set[addr];
...
bad_guys = set(); # start over assuming no bad guys
- A number of scripts have been (slightly) simplified to use the
new empty set()/table() constructors (Vern Paxson). Note that
these still aren't usable for field assignments in record constructors,
nor for attributes like &default = ...
- Removed unused syntax for declaring sets based on a list of initial
values (Vern Paxson).
- set() and table() can now be used as arguments to function calls
(Vern Paxson).
- The vestigial &match attribute has been removed.
- POP3 is now recognized using Dynamic Protocol Detection (Seth Hall).
- The new event expected_connection_seen(c: connection, a: AnalyzerTag)
is generated whenever a connection is seen for which we have previously
scheduled an analyzer via expect_connection() (Robin Sommer).
- The new built-in capture_state_updates logs all changes applied to
&synchronized variables, in a fashion similar to the capture_events()
built-in (Robin Sommer). An accompanying policy script,
capture-state-updates.bro, turns this on to the file state-updates.bst.
- If the new script variable suppress_local_output is set (default: F),
Bro suppresses printing to local files if there's a receiver for
print_hook events (Robin Sommer). This option is however ignored
for files with a &disable_print_hook attribute.
- The new notice action filter function file_if_remote specifies
that notices from sent from remote source addresses should
have an action NOTICE_FILE (Robin Sommer).
- The new notice action filter function file_local_bro_notices specifies
that notices generated by the local Bro instance (as opposed to a
remote peer) should have an action NOTICE_FILE (Robin Sommer).
- An arbitrary tag can now be past to post-processors for log rotation
(Robin Sommer).
- Default inactivity timeouts for interactive services shortened to
1 hour (Robin Sommer).
- The scanning variables distinct_{peers,ports,low_ports} are now
redef'able (Robin Sommer).
- The new -S (--summary-only) option for site-report.pl directs to
only generate connection summaries (Brian Tierney)
- More useful default config file for edit-brorule.pl (Brian Tierney).
- Bro now includes a test suite in testing/istate/ for its "independent
state" functionality (Robin Sommer).
- Support for parallel builds via make -j (Christian Kreibich).
- Bro's default search path now includes includes policy/sigs/ and
policy/time-machine/ (Robin Sommer).
- Bro's internal processing of interprocess communication has been
significantly overhauled to prevent potentially fatal race conditions
(Robin Sommer).
- Bro now checks calls to fmt() at compile-time to ensure that the
correct number of arguments are present (Vern Paxson). This is useful
in addition to Bro's run-time checking for arguments matching their
corresponding format-specifiers in the case of rarely-executed statements
that might not generate such run-time checks in routine testing.
- The ports associated with Telnet and Rlogin are now redef'able (Robin Sommer).
- MIME processing now removes leading whitespace from MIME headers
(Sanmeet Bhatia and Robin Sommer).
- TCP "weird" events reported by the connection compressor now match
(other than a few rare corner-cases) those produced for normal TCP
processing (rmkml and Robin Sommer).
- Added Scan::suppress_UDP_scan_checks to control false positives
on scan detection in environments with P2P protocols that use UDP
(Vern Paxson).
- The internal analyzer interface now includes an EndOfData() method that
analyzers can use to report that all of a message has been delivered
(Robin Sommer).
- Fix for a significant memory leak in processing UDP when using -w
(Robin Sommer). Note: this change turns off by default trace rewriting
for generic UDP traffic.
- Two serious regular expression bugs fixed (Vern Paxson). In the
first, searching for a regular expression inside a string would
fail if the pattern occurred only after an embedded newline. In
the second, insufficient buffer was allocated when compiling regular
expressions, leading to memory corruption.
- Base64 decoding bug fixes (Christian Kreibich and Ruoming Pang).
- Automatic rotation of files is now disabled for contents files written
by the TCP reassembler, which otherwise leads to mangled files
(Robin Sommer).
- Bro now ships with an updated version of libpcap (0.9.8), which hopefully
fixes problems managing trace files > 4 GB in size.
- Significant bug fixes for gzip- and deflate-encoded Web items (Robin Sommer).
- Bug fix for secondary-filter.bro (Vern Paxson).
- Removed a naming ambiguity regarding TCP states (Vern Paxson).
- Bug fix for signature scanner not matching all of its input (Vern Paxson).
- Bug fix for using port values in signatures (Robin Sommer).
- Minor policy script tweaks: state management for weird's, processing
of Notice tags associated with connections, and dependencies for
irc-bot.bro (Robin Sommer).
- aux/ portability fixes (Vern Paxson).
- Workarounds added for a BinPAC deficiency, which is that code in %cleanup
clauses can also be executed during recovery from exceptions when parsing
new data. This means that any delete's or Unref()'s need to also set the
corresponding pointer to nil (Vern Paxson).
- Bug fix for crashes with the non-BinPAC SSL analyzer (Robin Sommer).
- Tweak to peer-status.bro since Bro now requires events to be
declared prior to reference in a "schedule" statement (Robin Sommer).
- The signature keyword "enable" now optionally accepts the syntax
"foo:bar" to specify "activate analyzer bar as a child of analyzer foo"
(Robin Sommer). This is used for example for an XML-over-HTTP analyzer
that's in the works.
- irc-bot-syslog.bro now uses open_log_file() for its log file (including
the logging suffix) rather than a direct open (Vern Paxson).
- Bug fix for tracking Blaster across a Bro Cluster (Robin Sommer).
- Bug fix for the HTTP BinPAC analyzer chopping the trailing character
off of HTTP headers when generating the http_all_headers event (Gregor Maier).
- Bug fix for HTTP chunked items for which the chunk size line was terminated
by CRLF but the CR and LF came in separate packets (Gregor Maier).
- A bug has been fixed that would cause partial lines (for line-oriented
protocols) to fail to be processed when a connection terminated
(Robin Sommer).
- Bro no longer treats a signal arriving before a previous signal has
been processed as fatal, nor does it attempt processing of a termination
signal if seemingly there are no race conditions to worry about
(Robin Sommer). Both of these changes are an attempt to improve
Bro's robustness.
- Fix for attributes such as &encrypt not working in initial declarations
but only in later redef's (Seth Hall and Robin Sommer).
- Fixes for memory leaks in SSL processing (Seth Hall and Robin Sommer).
- Fix for POP3 analyzer to not treat lines like "<space>." as message
terminators (Robin Sommer).
- Bug fix for crashes arising from nil pointers in list expressions
(Seth Hall and Robin Sommer).
- Bug fix: a signature's "enable" would activate the corresponding analyzer
even if no event handlers were defined for it (Robin Sommer).
- Bug fixes to prevent crashes when mixing set_contents_file() with
subsequent explicit close(), and to ensure all data written to
file upon connection tear-down (Gert Doering and Robin Sommer).
- Configuration support for MacPorts and Fink package management systems
(Christian Kreibich & Vern Paxson).
- Communication-only Bro's now send out email alarms (Robin Sommer).
- Writes to a file that fail due are now run-time errors rather than
fatal internal errors, since often these occur due to the disk
being full (Robin Sommer).
- Byte-order bug fix for lookup_location() (Robin Sommer).
- BinPAC portability fix for 64-bit machines (Bernhard Ager and Robin Sommer).
- Portability fixes for newer versions of gcc (Jan Gerrit Goebel and
Robin Sommer).
- Some support for porting to Solaris (Stephan Toggweiler).
- Connection compressor bug fix for source and destination having the
same IP address, such as when monitoring loopback (Robin Sommer).
- Connection compressor bug fix for connections with multiple SYNs
(Robin Sommer).
- Bug fix for using already-declared local variables for looping
over vectors in a "for" loop (Robin Sommer & Vern Paxson).
- Bug fix for not processing truncated UDP packets (Tom Kho and Robin Sommer).
- Bounds-check added to BinPAC-generated code (Tom Kho and Robin Sommer).
- Bug fix for checking whether an IPv6 address is part of a subnet
(Seth Hall).
- Bug fixes for crashes relating to asynchronous DNS lookups performed
at start-up (Robin Sommer). These changes also lowered the timeout
before assuming failure from 20 seconds down to 5 seconds.
- Portability and const-ness fixes (Kevin Lo and Robin Sommer).
- Suppression of some content-gap complaints when running on traces
that have been filtered down to only TCP control packets (Robin Sommer).
- Removed unnecessary dependency in notice-action-filters.bro
that led to errors when loading icmp.bro by itself (Vern Paxson).
- Bug fix for potential infinite loop in client communiation (Robin Sommer).
- Bug fix in reference counting that could eventually lead to roll-over
(Robin Sommer).
- Bug fix in communication initialization (Robin Sommer).
- Internal documentation fix: timers are specified using absolute time,
not relative (Robin Sommer).
- Performance improvement for built-in find_all() function when running
on large strings (Robin Sommer).
- Memory leak fixes (Robin Sommer, Bernhard Ager, Christian Kreibich).
- Bug fix for error recovery when encountering an unknown link layer
(Bernhard Ager).
- Bug fix for reversing client & server in a connection (Po-Ching Lin).
- Bug fix for packet_contents when capture length exceeds the IP payload
length due to Ethernet frame padding (Christian Kreibich).
- Bug fix for tcp_packet event erroneously including Ethernet padding
in its contents (Vern Paxson).
- Bug fix for lookup_connection built-in (Seth Hall).
- Portability nit for libedit tarball (Vern Paxson).
- Broccoli portability fix for NetBSD (Christoph Leuzinger).
- Type-checking for script-level event invocation was completedly broken -
now fixed (Vern Paxson).
- Portability fixes for different versions of g++/STL (Nicholas Weaver
and Vern Paxson).
- Fix for dynamic detection of SSL via DPD (Robin Sommer).
- IPv6 portability fix for BinPAC-based DNS analyzer (Vern Paxson).
Note, more portability work is needed for it.
- Bug fix for bifcl error messages (Vern Paxson).
- Minor bug fix for remote communication, plus some improved communication
logging (Robin Sommer).
- Bug fix for &printhook (Robin Sommer).
- Bug fix for error message output (Robin Sommer).
- Bug fix for termination cleanup (Robin Sommer).
- Bug fix for some Rlogin corner cases (Robin Sommer & Vern Paxson).
- Bug fix for bifcl generation of "interval" types (Vern Paxson).
- Bug fix for getting connection memory statistics when Bro is
exiting (Robin Sommer).
- Config fix: --enable-debug now turns off -O2 for gcc (Robin Sommer).
- Bug fixes for "heavy" analysis (Vern Paxson).
- Broccoli bug fixes for types net and port (Robin Sommer).
- Bug fixes for Telnet environment options (Robin Sommer).
- Bug fix for accessing remote peer description (Robin Sommer).
- A fix for the connection compressor generating new_connection too
late (Robin Sommer).
- Fixes for DAG support, including configuration and multiple
interfaces (Robin Sommer).
- Bug fix for serializing time-stamps of table entries (Robin Sommer).
- Bug fix for dealing with peer IDs for remote communication (Robin Sommer).
- Bug fix to avoid installing timers when timers have already
been canceled (Robin Sommer).
- Bug fix for interplay between serializing connections and
connection compressor (Robin Sommer).
- Memory leak fix for enum's (Robin Sommer).
- Bug fix for files being closed prior to bro_done() (Vern Paxson).
- aux/broccoli/contrib was not included in distribution (Robin Sommer).
- Auto-configuration bug fix for BinPAC (Craig Leres).
- Bug fix for dynamic protocol detection (Robin Sommer).
- A number of configuration fixes for installation and portability
(Christian Kreibich, Brian Tierney, Robin Sommer, Dan Kopecek).
1.3 Mon Jul 16 22:11:00 PDT 2007
- The Bro manual has been wikified at:
http://www.bro-ids.org/wiki/index.php/User_Manual
and this is the format in which it will evolve in the future
(Christian Kreibich).
- Much more extensive support for SMB, NetBIOS and NCP (Chris Grier).
- The new attribute &priority=n defines the order of execution for handlers
of the same event (Robin Sommer). Handlers with higher priority are
executed first. n is an integer expression that must evaluate to a
constant when the script is loaded.
Example:
> cat foo.bro
event bro_init() &priority = -5 { print -5; }
event bro_init() &priority = 5 { print 5; }
event bro_init() { print 0; } # default priority=0
> ./bro foo.bro
5
0
-5
The connection_state_remove() handler in conn.bro now has priority
-10 and therefore executes after all other handlers for this event.
This fixes a long-standing problem of sometimes $addl fields not showing
up in connection summaries.
- The new expressions record(...), table(...), set(...) and vector(...)
are constructors for the corresponding aggregate types (Vern Paxson).
For example,
record($foo = "hi", $bar = -6)
is the same as the existing constructor
[$foo = "hi", $bar = -6]
For tables, sets, and vectors, the "..." values within the ()'s have
the same syntax as those that you can list in variable initializations.
For example,
table([1, T] = "black", [4, F] = "red")
returns a table of type "table[count, bool] of string".
set(4, 3, -1)
is a value of type "set[int]".
- You can associate attributes with table() and set() constructors
(Robin Sommer). For example:
local s = set(1.2.3.4) &read_expire = 5 secs;
associates a 5-second read expiration with the set assigned to s.
- Bro now explicitly supports port numbers reflecting a transport protocol
type of "unknown" (Christian Kreibich). Currently, this means "not TCP,
UDP or ICMP". The numerical value of such a port is the IP protocol,
so ranges from 0..255. For example:
global p: port = 0/unknown;
print fmt("%s", p);
print fmt("p is TCP? %s", get_port_transport_proto(p) == tcp);
print fmt("p is unknown? %s",
get_port_transport_proto(p) == unknown_transport);
yields
0/unknown
p is TCP? F
p is unknown? T
In comparisons of different protocol types, the following holds:
unknown < TCP < UDP < ICMP.
- If your system supports "GeoIP" (see http://www.maxmind.com/app/geolitecity
for a corresponding city database), then the new script function
lookup_location(a: addr): geo_location
returns a record of geographic information associated with an address
(Seth Hall). The geo_location record has $country_code, $region and
$city fields. If no information is available, each of these will be
set to empty strings.
If Bro hasn't been configured with GeoIP support, or if the address is
IPv6 that cannot be directly converted to IPv4, then Bro produces a
run-time error and likewise returns empty strings.
- Signature-matching on HTTP components now processes the URI with
escape sequences expanded (Robin Sommer). Ideally, there would be
two signature keywords, one for decoded URIs (corresponding to this
case) and one that allows matching against the URI as originally
transmitted.
- The connection compressor is no longer considered experimental, and
is used by default (Robin Sommer).
- The new function lookup_hostname(host: string): addr_set asychronously
looks up the IPv4 address(es) of the given host via DNS (Robin Sommer).
Like lookup_addr(), this function can only be used within a "when"
statement.
- The new built-in
raw_bytes_to_v4_addr(s: string): addr
takes a string that points to at least 4 bytes, and returns an address
corresponding to interpreting these as being an IPv4 address in network
order (Vern Paxson; suggested by Mike Dopheide).
- Trace-rewriting support for DNS, SMB (Chris Grier).
- The new script function find_all(str: string, re: pattern): string_set
returns a string_set giving all occurrences of the pattern "re" in
the string "str" (Robin Sommer). (Note that string_set's are unordered.)
- The new policy script save-peer-status.bro generates a log
to peer_status.$BRO_LOG_SUFFIX of updates received from
communication peers (Robin Sommer).
- The policy script print-filter.bro now includes two (scoped) variables,
terminate_bro and to_file, which control whether to exit after printing
the filter (default T) and whether to write to the log file
pcap_filter.$BRO_LOG_SUFFIX or (default) to stdout (Robin Sommer).
- The new script variable check_for_unused_event_handlers controls whether
Bro checks for unused event handlers (Robin Sommer). It defaults to T,
which was the past behavior (always report).
- Bro now terminates if the only pending activity is future timers
(Robin Sommer). It used to wait for those timers to expire, but this
can cause fundamental problems if the timers are associated with table
management (since these might never completely drain).
- Tables and sets inside of records are now initialized to empty
values rather than uninitialized (Vern Paxson).
- A new variable allow_services_from (in hot.bro) complements the
existing allow_service_to variable (Brian Tierney). It specifies
that access to the given service from the given originator is
allowed.
- global_sizes() no longer reports internal variables (Robin Sommer).
- The IRC analyzer is now activated if any of the (many) IRC event
handlers are defined (Robin Sommer).
- The default value for tcp_close_delay is now 5 sec rather than 0 sec
(Robin Sommer). This prevents some spurious connection events.
- Improved logic for dealing with "reversed" connections such
as backscatter (Vern Paxson).
- You can now left-justify fields when using fmt() with "%-" like
in sprintf (Christian Kreibich).
- Updates to DNS query types (Larry Leviton).
- Added mechanism to http-header.bro to skip printing some HTTP headers
(Larry Leviton).
- The IrcHotWord notice now sets the associated connection (Robin Sommer).
- If a notice has a tag, it's no longer overridden (Robin Sommer).
- ServerFound notices now set the port field (Robin Sommer).
- The built-in lookup_ID() now returns the string "<unknown id>" if the
ID does not exist, rather than a run-time error (Robin Sommer).
- The new tuning option ProtocolDetector::suppress_servers specifies a
set of analyzers for which Bro generates ServerFound notices, but not
ProtocolFound (Robin Sommer). This both reduces log file size and
conserves memory.
- A new notice_action_filter, tally_notice_type_and_ignore, works the same
as tally_notice_type but returns IGNORE (Robin Sommer)
- Setting summary_interval == 0 disables the creation of irc-bots.summary.log
(Robin Sommer).
- If you @load foo and a directory "foo" is in your path, Bro no longer
tries to load it (Robin Sommer).
- A number of BinPAC fixes and enhancements (Ruoming Pang, Chris Grier
and Vern Paxson).
- BinPAC now resides in aux/binpac rather than src/binpac (Ruoming Pang
and Christian Kreibich). This reflects a decoupling of it from Bro so
that it can be used to generate protocol analyzers for other projects too.
- Removed example Inktomi entries from skip_scan_sources initialization,
since they no longer exist (Vern Paxson).
- The variable make notice_once_per_orig_tally_interval is now
redef'able (Brian Tierney).
- SIGPROF to the communication child process now logs resource stats to
remote.log (Matthias Vallentin).
- The new built-in getpid(): count returns Bro's process ID (Robin Sommer).
- Patterns for detecting IRC-based bots updated (Robin Sommer).
- irc-bot-syslog now logs just bots, not all IRC client/servers (Robin Sommer).
- The new variable suppress_notice_actions in notice.bro suppresses
notice_actions events for selected notice types (Robin Sommer).
- Files opened during operation now rotate just like those opened at
startup (Robin Sommer).
- ResourceStats now also logs elapsed time and the reported number of
packets-on-the-link (Mark Dedlow).
- Printing a "file" value now produces its name (Robin Sommer).
- Removed deliberate truncation of payload in port 80 FIN packets
(Vern Paxson).
- remote.log now includes received peer_descriptions (Robin Sommer).
- Significant POP3 analyzer speed-ups (Vern Paxson).
- Updated README (Vern Paxson).
- Fix for "@load a" followed by "@load a.bro" not loading the same file
twice (Robin Sommer).
- Bug fixes for propagating state operations to uninitialized variables
and for spurious state inconsistency messags (Robin Sommer).
- Bug fix for sending final sync-points during pseudo-realtime mode
(Robin Sommer).
- Fix for possible buffer overflow (Christian Kreibich).
- Bug fix for spurious end-of-file's during inter-Bro communication
(Robin Sommer).
- Bug fix for dpd_match_only_beginning=F (Robin Sommer).
- Bug fix for updating timestamps (Christian Kreibich).
- Bug fix for skipping ADU processing in adu.bro (Christian Kreibich
and Zhichun Li).
- Fix for ICMPs that carry ICMP headers (or non-TCP/UDP/ICMP headers)
within them (Vern Paxson).
- Fix for files being rotated after the timer queue has been deleted
(Vern Paxson).
- Bug fix for signature-matching with IPv6 subnets (Vern Paxson).
- Bug fix for connection compressor setting connection origin (Robin Sommer).
- Bug fix for interconn.bro when processing peculiar connections (Vern Paxson).
- Fix for off-by-one buffer in sscanf call (Christian Kreibich).
- Fixed inefficiency/warning flagged by g++ (Vern Paxson).
- Bug fix for NUL string termination in SMB processing (Zhichun Li).
- Fix for over-ref'ing of file Val's (Vern Paxson).
- Fixes for some g++ warnings (Christian Kreibich, Vern Paxson).
- gcc 3.4.2 portability fixes (Robin Sommer).
- Minor build fixes for Broccoli, including a version bump to match that
of Bro. See aux/broccoli/ChangeLog for details.
- distcheck fixes (Christian Kreibich).
- Configuration portability fixes (Matthias Vallentin, Jean-philippe Luiggi).
- OpenBSD portability fixes (Jean-philippe Luiggi, Christian Kreibich).
1.2.1 Mon Dec 11 16:22:58 PST 2006
- Fixed delayed triggering of new_connection events when using the
connection compressor.
- Fixed tracking of first packet in TCP analyzer. (Reported by Guohan Lu)
- The syslog built-in got lost during some previous merge.
- Fixed crash if local variable is given as timeout value for table.
(Reported by Mike Wood.)
- Fixed using "time" values as table indices.
- Added ssh to default brolite DPD configuration.
- Fixed catching up to real-time in case of lull.
- Fixed Broccoli "BRO_DATA_FORMAT_VERSION" to match version in Bro.
- Fixed Makefile problem in doc directory.
- Fixed Makefile dependency problem in binpac directory.
- Added Linux tuning to brolite install script.
- Modified Makefile to include broccoli/contrib.
- Adding missing initialization to remote serializer.
- Minor documentation updates for reference manual and Broccoli.
1.2 Tue Oct 17 12:09:49 PDT 2006
- Bro now supports DPD, dynamic protocol detection (Robin Sommer, Holger
Dreger, and Michael Mai). With DPD, Bro can analyze protocols regardless
of what port numbers they use: it infers the protocol based on which
application analyzers can parse it without error. Adding this functionality
involved extensive changes to Bro's internals, but also now enables
multiple Bro analyzers to work on the same connection, either concurrently
or one nested inside the other (we have not taken much advantage of this
latter capability yet, but see the FTP events discussed below).
There are a number of new policy scripts, events, and variables associated
with DPD processing, as follows.
Scripts:
You activate DPD by @load'ing dpd.bro. It in turn instructs Bro
to load the signature file policy/sigs/dpd.sig. Note that Bro
uses signatures to expedite deciding which analyzers to try on
a given connection; it does *not* simply use the signatures to
make the determination of which protocol is in use, as this is
insufficiently robust. (At this point, Bro provides signatures
for FTP, IRC, HTTP, SMTP, and SSH. In the future we plan to add
other protocols.)
Along with dpd.bro, you need to @load detect-protocols.bro or
detect-protocols-http.bro. The former enables general detection
of application-layer protocols, while the latter does further
inspection of HTTP sessions to characterize applications running
on top of HTTP such as Gnutella or SOAP. (Loading dpd.bro
is separate from loading one of these scripts because in principle
Bro could use a different means than signatures to activate
the analyzers, although currently it does not.)
If you @load dyn-disable.bro, then once an analyzer determines
that it does not match a given connection, it is deactivated
(and a Notice is generated). Otherwise, it still proceeds to try
its best to analyze the connection (to possibly be more robust
against evasion).
The scripts dce.bro and smb.bro enable DPD for the Windows DCE and
SMB protocols, respectively. (Note that analysis of these protocols
is undergoing a major expansion, not yet complete.)
Events:
event protocol_confirmation(c: connection, atype: count, aid: count)
Generated when the given connection has been confirmed as
conforming with the application type (protocol) specified
by atype. aid is a globally unique analyzer ID that identifies
a particular analyzer instance.
The values for atype are symbolic names associated with
each of Bro's analyzers, such as ANALYZER_IRC. See the
initialization at the beginning of Analyzer.cc for the
full set of names.
The function analyzer_name(atype: count): string translates
these symbolic names into text. For example,
analyzer_name(ANALYZER_IRC)
yields "IRC".
event protocol_violation(c: connection, atype: count, aid: count,
reason: string)
Generated when the given connection has been found to
violate the protocol of the given application type, with
"reason" giving details.
Variables:
dpd_buffer_size: count (default 1024)
Specifies how much pending data Bro keeps for connections
that have not been classified yet. Once this fills, the
data is deleted, though classification can still continue
(see below).
dpd_match_only_beginning: bool (default T)
If set, specifies that Bro should stop signature matching
if it has processed dpd_buffer_size bytes.
dpd_ignore_ports: bool (default F)
If set, then Bro does not take into consideration the port
numbers associated with connections when attempting to
classify them (which can otherwise help the process in
some cases).
dpd_reassemble_first_packets: bool (default T)
If set, then Bro does TCP stream reassembly before applying
signature-matching to detect protocols.
likely_server_ports: set[port]
Specifies a list of ports that Bro will consider as likely
used by servers. For example, if Bro sees a connection
that has already been established (so it does not know
which side sent the initial SYN), and one side uses a port
in this set, then it will assume that that side is the
server (connection responder). The set is empty unless
you populate it or @load server-ports.bro, which specifies
a large number of values.
dpd_config: table[AnalyzerTag] of dpd_protocol_config
Specifies the DPD configuration associated with each tag.
The type dpd_protocol_config is simply:
type dpd_protocol_config: record {
ports: set[port] &optional;
};
i.e., an optional $ports field specifying a set of ports
associatd with the tag. For example, ftp.bro now includes
the equivalent of:
redef dpd_config += {
[ANALYZER_FTP] = [$ports = 21/tcp]
};
Functions:
The function
expect_connection(orig: addr, resp: addr, resp_p: port,
analyzer: count, tout: interval)
is called to alert Bro that a new connection is expected, initiated
by orig to a server running on resp's port resp_p (note: orig's port
is not specified) which will correspond to the specified analyzer
(e.g., "FILE", which is used to analyze files transferred by FTP -
see next item). "tout" is a timeout to associate with the waiting.
The function
function disable_analyzer(cid: conn_id, aid: count)
instructs Bro to disable the analyzer that generated the current
event, assuming the analyzer is associated with the given connection
ID. This is used by the dyn-disable.bro script discussed above.
- A much more complete BinPAC compiler, along with new HTTP, DNS, and
RPC/Portmap analyzers in binpac (Ruoming Pang). The flag "--use-binpac"
activates the BinPAC-based analyzers (currently for HTTP and DNS).
See www.cs.princeton.edu/~rpang/binpac-paper.pdf for a description of
BinPAC, and let Ruoming know if you are interested in using BinPAC to build
new analyzers.
- A new type of analyzer, FILE, analyzes the contents of a connection as
though it were a data file (Robin Sommer). Currently, it can generate
two events:
event file_transferred(c: connection, prefix: string, descr: string,
mime_type: string)
Indicates that the connection transferred a file. "prefix"
is the beginning of the file's data; "descr" and "mime_type"
are indicators of the file's type, as reported by the
"libmagic" library.
descr/mime_type are only set if Bro is configured on a
system that includes the "libmagic" library.
event file_virus(c: connection, virname: string)
Indicates the connection transferred an executable
corresponding to a known virus of the given name.
This functionality is only available if Bro is configured
on a system that includes the "libclamav" library.
Note, this analyzer is enabled via a call to expect_connection by
the FTP analyzer.
- New events relating to IRC analysis (Robin Sommer):
event irc_client(c: connection, prefix: string, data: string)
Generated upon seing a client message sent over the given
IRC connection. "prefix" is the command's prefix as defined
by the IRC protocol. It is used by servers to indicate the
true origin of the message; it may be empty. "data" contains
the message.
event irc_server(c: connection, prefix: string, data: string)
Same for server messages.
event irc_user_message(c: connection, user: string, host: string,
server: string, real_name: string)
Generated upon seeing an IRC "USER" command.
event irc_password_message(c: connection, password: string)
Generated upon seeing an IRC "PASS" command.
event irc_channel_topic(c: connection, channel: string, topic: string)
Generated upon seeing an IRC server reply that includes
the channel topic.
event irc_global_users(c: connection, prefix: string, msg: string)
Generated upon seeing an IRC server reply that includes
a count of the number of IRC users.
- The new experimental script irc-bot.bro tracks IRC-based bots (Robin Sommer).
The accompanying script irc-bot-syslog.bro syslog's the state of the
bot analysis every IrcBot::summary_interval seconds (default 1 minute).
- The new script proxy.bro looks for open Web proxies by matching incoming
requests to a server with outgoing requests it makes (Robin Sommer). It
generates HTTPProxyFound Notices when it finds one.
- Changes to notices.bro (Robin Sommer):
- notice_policy_item's now have a default $result of
NOTICE_FILE and a default $priority of 1.
- The new notice_action_filter, notice_alarm_per_orig, alarms
on the first NoticeType from a specific source. Subsequent
instances are tallied.
- notice_action_filters now reside in the new script
notice-action-filter.bro (automatically loaded by notice.bro).
- The notice actions NOTICE_ALARM_PER_CONN, NOTICE_ALARM_PER_ORIG,
and NOTICE_ALARM_ONCE have been removed, as they were never
actually implemented.
- If the notice_policy returns IGNORE or FILE, the action_filters
filters are no longer consulted.
- A new attribute for tables and sets, &mergeable, changes the semantics
of assignments, as follows (Robin Sommer). Given two &mergeable tables/sets
A and B, an assignment "A = B" becomes actually a join "A = A \cup B"
(i.e., union). The envisoned use is to help avoid race conditions
when doing remote state synchronization.
- The semantics of &synchronized expire_funcs has changed (Robin Sommer).
Now, when a table entry is expired and the operation is propagated to a
a peer, the peer will call its expire_function.
- TRW analysis now skips UDP traffic because it currently treats
all UDP connections as failures (Robin Sommer).
- trw.bro has been split into trw-impl.bro (the algorithm) and
trw.bro (which simply activates the analysis), to facilitate writing
scripts that have hooks into TRW analysis but don't presume it's
active (Robin Sommer).
- The option report_remote_notices in remote.bro has been replaced
by a new script you include, remote-report-notices.bro (Robin Sommer).
- The new function connect_peer() explicitly connects to a remote host
(Robin Sommer).
- The new script remote-send-id.bro sends the current value of an ID
to a remote Bro and then terminates processing (Robin Sommer). It's
intended for use from the command-line, as in
bro -e "redef dst="<dst>" id="<name-of-id>" <scripts> remote-send-id
The other scripts must set up the connection. <dst> is an index into
Remote::destinations corresponding to the destination.
- New built-ins {suspend,resume}_state_updates() can be called to
temporarily avoid propagating updates to &sync'ed values (Robin Sommer).
This can avoid duplicated activity.
- The new function terminate_communication() instructs Bro to end its
communication with remote peers (Robin Sommer).
- The new event remote_state_access_performed is raised when remote state
access has been performed (Robin Sommer). This is primarily for debugging.
- The log() built-in has been renamed to ln() to avoid conflict (Vern Paxson).
- bifcl now generates event generation wrapper functions from event.bif
(Ruoming Pang). For example, to generate event http_reply, currently
one writes:
val_list* vl = new val_list;
vl->append(BuildConnVal());
vl->append(new StringVal(fmt("%.1f", reply_version)));
vl->append(new Val(reply_code, TYPE_COUNT));
if ( reply_reason_phrase )
vl->append(reply_reason_phrase);
else
vl->append(new StringVal("<empty>"));
ConnectionEvent(http_reply, vl);
In the future, one will be able to just call bro_event_http_reply(), and
the code generated by bifcl looks like:
void bro_event_http_reply(Connection* c, StringVal* version,
bro_uint_t code, StringVal* reason)
{
val_list* vl = new val_list;
vl->append(c->BuildConnVal());
vl->append(version);
vl->append(new Val(code, TYPE_COUNT));
vl->append(reason);
mgr.QueueEvent(http_reply, vl, SOURCE_LOCAL, c);
}
Accompanying this change is a semantic shift to types "string" and "port"
in .bif files. They used to be translated to C++ types BroString* and
uint32, respectively. Now they are translated to StringVal* and PortVal*.
The functions in bro.bif are changed accordingly, and please be aware
of this change when you write built-in functions in future.
Also for this change, the parameter 'new' for rsh_request has been renamed
'new_session', as 'new' is a reserved word for C++.
- Some ICMP "connections" now have services identified ("icmp-echo",
"icmp-unreach") rather than just listing the service as "other"
(Ruoming Pang).
- The new option remote_trace_sync_interval specifies an interval after
which each Bro will stop processing its trace and wait for all others
to signal that they have reached the same time (Robin Sommer). The
intent is support for operating Bro in a distributed cluster fashion
(and in particular for debugging such clusters when running off-line
on traces).
This option only works in pseudo-realtime mode, and requires the new
global remote_trace_sync_peers to give the total number of remote peers
(not including self). Signaling is done via a new communication message
type.
- Extensions for DNS transformation/anonymization, including introduction
of trace transformation for protocols other than TCP (Jason Lee).
Not yet fully developed/debugged.
- Extensions for HTTP transformation/anonymization (Martin Casado).
Not yet fully developed/debugged.
- The $conn field is now included in HTTPProxyFound notices (Robin Sommer).
- Changed service inference algorithm to favor lower-numbered
likely-servers over higher-numbered ones (Vern Paxson).
- In pseudo-realtime mode, Bro now uses real-time for deciding which
peer should send state (Robin Sommer).
- Time synchronization for Bro's running on traces in pseudo-realtime mode
added (Robin Sommer).
- Avoidance of false content gaps improved when sorting packets with
out-of-order timestamps (Ruoming Pang).
- Packets from the packet sorter are now more robustly drained upon
termination of input (Ruoming Pang).
- Documentation for deep-copy updated (Christian Kreibich).
- Nasty fragment reassembly bug fixed (Vern Paxson).
- Serious bugs in EDNS0 processing fixed (Vern Paxson).
- Fixed significant misfeature of interconn.bro that stopped all processing
of a connection once it makes a detection (Vern Paxson).
- Fixes for &read_expire operation across synchronizes tables (Robin Sommer).
- Fixes for multiple peers exchanging initial &sync state simultaneously
(Robin Sommer).
- Improvements to graceful termination of Bro when communicating with
remote peers (Robin Sommer).
- Fix for ICMP analyzer not always generating icmp_sent events
(Robin Sommer). This appears to still need some work, as now
it generates redundant events.
- Fix for initial exchange of &sync state which could lead to
referencing unknown IDs (Robin Sommer).
- Fix to scan detection for differing semantics of connection compressor
vs. non-compressor (Robin Sommer).
- Bug fix for distinguishing regular expression matches of length 0 from
those of length 1 (Ruoming Pang).
- Fix for SSH version parsing in the presence of content gaps (Robin Sommer).
- Bug fix for IRC that could lead to crashes (Robin Sommer).
- Bug fix to refrain from adding new timers when a connection has
already been removed from the connection table (Robin Sommer).
- Bug fix for packet_contents not including the transport-layer header
(Robin Sommer).
- Some memory leaks fixed (Robin Sommer).
- A bunch of portability and distribution problems fixed (Christian
Kreibich, Robin Sommer, Vern Paxson).
1.1 Mon May 15 10:50:33 PDT 2006
- Bro now supports a "when" statement for taking action upon something
becoming true asynchronously (Robin Sommer). This provides a powerful
new mechanism with numerous applications.
Syntax:
when '(' <expr> ')' <stmt> [timeout <interval> '{ <stmts> '}']
where the first <stmt> can be a single statement or a block enclosed
in {}'s, but the set associated with "timeout" must be enclosed in
{}'s (to reduce ambiguities in Bro's grammar).
Bro executes the first statement when <expr> becomes true. If you give
a timeout and the condition has not been satisfied before it expires, Bro
executes the second statement instead.
A simple example:
global t: table[addr] of count;
event connection_established(c: connection)
{
local orig = c$id$orig_h;
if ( orig !in t )
{
t[orig] = 1;
when ( t[orig] == 5 )
print fmt("%s has established 5 connections", orig);
timeout 1 hr
{
print fmt("%s has NOT established 5 connections", orig);
delete t[orig];
}
}
else
++t[orig];
}
Notes:
- The condition may be evaluated more than once, and at arbitrary
times.
- When the when-body is executed, the condition is guaranteed to be
still satisfied.
- Expression reevaluation is primarily triggered by modifications
to globals. However, reevaluations do not take place immediately
but potentially at a later point. This means that if we change a
global to a value which would execute the trigger but then change
it back, the change may go unnoticed.
- Inside the condition you may introduce new locals. For example,
when ( (local x = foo()) && x == 42 ) ...
Such an assignment always yields true as its expression value
(but the assignment might be delayed, for example if foo() is
a delayed function call - see below).
Delaying function calls
=======================
Functions called inside the condition of a when-clause may delay their
results until they're ready. This works for both script-level and built-in
functions.
For script-level functions, there is a new construct, "return <when-stmt>",
to delay a function's result. When used, the function returns at the
time the when-stmt's condition becomes true, and it yields the value
that the when-stmt's body then returns. Toy example:
global X: table[string] of count;
function a() : count
{
# This delays until condition becomes true.
return when ( "a" in X )
{
return X["a"];
}
timeout 5 min
{
return 0;
}
}
event bro_init()
{
# Installs a trigger which fires if a() returns 42.
when ( a() == 42 ) { print "Yippie!"; }
X["a"] = 42;
}
There's also a new built-in function which can delay
lookup_addr(host: addr)
performs asynchronous DNS address->hostname lookups. Example:
local h; addr;
[...]
when (local name = lookup_addr(h)) { print h, name; }
See the function gen_hot_notice_with_hostnames() in conn.bro for
a more worked-out example of using the "when" clause to translate the
local address in SensitiveConnection notices to a hostname (contributed
by Brian Tierney). This functionality is activated by redef'ing
xlate_hot_local_addr to T.
Here is the full evaluation model of a when's condition:
- The condition may be evaluated more than once, at arbitrary times.
- It is always fully evaluated, no matter whether some former
evaluation has been suspended by a delaying function call.
- All function calls which do not delay are always *fully* executed
each time the condition is evaluated.
- Function calls which delay are only executed *once*; their result is
cached and re-used in the case the condition is evaluated again.
- The condition is guaranteed to be true when the body is executed
(potentially using cached function results)
- By default Bro now uses a configuration similar to what used to be
activated using reduce-memory.bro, along with some additional state
timeouts that are new (Robin Sommer and Vern Paxson). This allows for
better state management out-of-the-box, at the cost of some precision
of analysis and resilience to evasion. In particular, the intent is to
move towards being able to run Bro continuously without inexorably growing
the amount of memory used until exhaustion.
You can access a configuration similar to the previous default state
management settings by loading heavy-analysis.bro. It turns on a
load-prefix of "heavy", so when you load XXX.bro, a file heavy.XXX.bro
will also be automatically loaded if present. Note that, as was the
case for reduce-memory, you need to load heavy-analysis prior to other
files for it to have effect.
- The new module clear-passwords.bro monitors login/FTP/IRC/POP traffic
for cleartext passwords (Jason Lee).
- The new script service-probe.bro looks for remote hosts that repeatedly
connect to the same service on local hosts (for a configurable set of
services and connection sizes) in order to detect brute-forcing attacks
such as password-guessing (Jim Mellander).
- A new ARP analyzer generates three events:
event arp_request(mac_src: string, mac_dst: string,
SPA: addr, SHA: string, TPA: addr, THA: string);
event arp_reply(mac_src: string, mac_dst: string,
SPA: addr, SHA: string, TPA: addr, THA: string);
event bad_arp(SPA: addr, SHA: string, TPA: addr, THA: string,
explanation: string);
with a corresponding policy script arp.bro (Chema Gonzalez and Vern Paxson).
It writes logs to arp.$BRO_LOG_SUFFIX. It has not been tested much yet.
- Bro Lite changes (Jason Lee):
- default user for is now user 'bro'
- now uses the correct sysctl on FreeBSD 6
- now uses the correct Perl path if site-report.pl not installed
into '/usr/local/bro'
- no longer prompts to encrypt email unless you pick to email reports
- The default Bro Lite install now only checkpoints Bro once a week
(Brian Tierney).
- Implicit Bro file extensions (such as .bro for policy scripts and .sig
for signatures) are now searched for first rather than only if the
non-extension-version of the file doesn't exist (Vern Paxson). For
example, running "bro -r trace mt" now first searches $BROPATH for
"mt.bro" before searching for "mt", whereas it used to do these in
the other order.
- There's now a simpler mechanism for redef'ing variables on the command-line
(Christian Kreibich). Any command line arguments of the form <var>=<val>
are now expanded into policy code of the form "redef var=val;", where
<val> is wrapped in quotation marks if the value appears to be a string
and doesn't have quotation marks already. This works with strings with
whitespace such as foo="Hello World"; however, note that it means you
can't use the mechanism to redef an enum value.
- The Bro distribution now includes (and builds by default) Christian
Kreibich's Broccoli library (Bro C Client Library), which enables programs
to communicate with running Bro's (Christian Kreibich and Jason Lee).
Configure with --disable-broccoli to turn this off.
- Built-in functions log(x: double): double and exp(x: double): double
which do natural logarithms and their inverses (Jaeyeon Jung).
- The new built-in function gethostname() returns the local host's name
(Jason Lee & Robin Sommer).
- The new built-in function reading_traces() returns true if Bro
is reading trace files (Robin Sommer).
- The new built-ins suspend_processing() and continue_processing() provide
script-level control for instructing the event engine to stop or resume
processing packets (Robin Sommer). This is useful for coordinating
simultaneous processing by multiple Bro's.
- Email notices are now by default sent via /bin/mail, with "[Bro Alarm]"
in the subject.
- redef'ing a function now replaces the existing body rather than
supplementing it (Robin Sommer), which was a bug.
- You can now configure Bro to process encapsulated IP packets either
by setting, as before, a fixed encap_hdr_size (for VLANs), or setting
parse_udp_tunnels to T (Ruoming Pang). For the latter, you specify a
UDP tunnel port using udp_tunnel_port (the previous variable "tunnel_port"
has gone away); or you can leave it set to its default of 0/udp, in which
case Bro will look for IP encapsulated in UDP packets on any port.
- Added a simple form of profiling based on sampling the work done
per-packet (Vern Paxson). The event engine generates a
event load_sample(samples: load_sample_info, CPU: interval, dmem: int)
event every load_sample_freq packets (roughly; it's randomized), where
load_sample_freq defaults to 20. "samples" is simply a set[string]; it
contains the names of the functions, event handlers, and their source
files that were accessed during the processing of the sampled packet,
along with an estimate of the CPU cost of processing the packet and
(currently broken) memory allocated/freed.
- Bro now includes experimental support for Endace DAG cards (Gregor Maier
and Robin Sommer). To activate, configure with
--with-DAG=/path/to/dagtool/installation
and use "dag0" as the network interface. You may need to configure the
card with the dagtools first. In general, if dagsnap works, Bro should
work as well.
- Log rotation has changed in a number of ways (Mark Dedlow & Robin Sommer):
* The new variable log_rotate_base_time: string, if defined,
specifies that logs should be rotated at log_rotate_base_time +
i * rotate_interval intervals. Format is as a string in
24-hour time, "%H:%M", e.g, "12:00". This format may change
in the future to instead be a Bro time type.
* RotateLogs::date_format can be redefined to change format of
timestamps in rotated files.
* RotateLogs::build_name() can be redefined to implement an
arbitrary naming scheme for rotated files.
Note, this code has not been extensively tested.
- Bro now by default builds a version of malloc bundled with its
distribution (Vern Paxson & Brian Tierney).
- The syntax for the clone operator now looks like a function call,
"copy(x)" (Vern Paxson).
- The new flag DNS::logging (default F), if T, disables generation of
dns.log (which is often uninteresting and very large), though it
still performs analysis leading to NOTICEs (Robin Sommer).
- A new global, hostile_domain_list, has been added to dns.bro which
lists domains to be flagged if A or MX records are queried (Scott Campbell).
- Added globals dns_skip_all_{auth,addl} to skip all DNS AUTH/ADDL processing
(Vern Paxson). Skipping these is on (true) by default, because such
processing is quite expensive.
- backdoor.bro now turns off by default some detectors that from experience
have too many false positives, or (such as for HTTP) too many uninteresting
true positives (Brian Tierney). In addition:
- the module now generates a BackdoorFound notice for each backdoor
- the new variable dump_backdoor_packets (default F) if set causes
the packet that triggered the backdoor detection to be written to
backdoor-packets/<tag>:<time>
- the new variable backdoor_ignore_host_port_pairs is a set[addr, port]
specify host/port combinations to ignore
- 587/tcp is now recognized as another SMTP port, and 7000/tcp as
a popular IRC port ignored by default
- brolite-backdoor.bro is a sample of using backdoor.bro
- A bunch of enhancements and fixes for the IRC backdoor detector
(Vern Paxson).
- The cf utility in aux/cf/ now gets the format to use (unless you specify
-f fmt) from $CFTIMEFMT in the environment. You can now specify -f
without a format to revert to the default format. This change also
includes a significant performance improvement when processing large
files (Mark Dedlow and Craig Leres).
- Cleanups for brolite.bro and brolite-backdoor.bro (Brian Tierney).
brolite.bro now uses rotate-logs by default.
- backdoor.bro now enables analysis of partial connections (Vern Paxson).
- brolite config cleanup: removed smtp.bro from default, increased
max_timer_expires, changed default BROPATH to look at site dir
first (Brian Tierney).
- The reference manual has been updated for the terminology changes
of log -> alarm, alert -> notice, and rule -> signature (Vern Paxson).
Some vestiges of the older terminology remain, in part because they're
still present in some facets of Bro.
- The new script function get_current_packet(): pcap_packet returns
the current packet as a "pcap_packet" record with fields $ts_sec,
$ts_usec, $caplen, $len (all of type count) and $data (a string)
reflecting the corresponding libpcap values (Christian Kreibich).
You can write this packet to a dump file using the new function
dump_packet(pkt: pcap_packet, file_name: string): bool, which writes
(or appends) the packet to a file of the given name, returning T
on success and F on error.
- The new fmt() specifier 'T' converts values of type "time" to ISO
format timestamps, analogous to how 'D' does this for ISO dates
(Mark Dedlow). fmt("%T", <time>) is equivalent to
fmt("%s", strftime("%F-%T.%N", <time>)), except that strftime
does not (yet) offer "%N" for nanoseconds (but see 'date +%F-%T.%N').
- The new %S format for fmt() inserts a "raw" version of the given string -
that is, embedded NULs, control characters, etc., are present without
any escaping (Christian Kreibich).
- Zero-padding and field widths now work for all fmt() formats rather than
just %e/%f/%g (Christian Kreibich). For example, you can now say:
local filename = fmt("log-%04.txt", ++counter);
and get logfiles log-0001.txt, log-0002.txt, ..., log-0999.txt, etc.
- The 'x' format specifier now supports values of type "addr", converting
them t hex (Mark Dedlow). For example,
fmt("str=%s hex=%x", 1.2.3.4, 1.2.3.4)
produces
str=1.2.3.4 hex=01020304
The field designation is either %08x (if compiled for IPv4 only) or
%08x%08x%08x%08x (if compiled with IPv6 support).
- firewall.bro has been extended to support multiple independent
rule-sets (by calling begin() for the start of the next one),
specifying sets of addresses, being FTP-aware, and with a more
streamlined Notice message (Robin Sommer).
- The HTTP script variables maintain_http_sessions and http_sessions
are now exported so they can be redefined or, for the latter, have
timeouts added/adjusted (Robin Sommer).
- You can load the new policy script log-append.bro to change Bro's
behavior so that when it runs appends to existing log files rather
than overwriting them (Mark Dedlow).
- New &disable_print_hook attribute for files (Robin Sommer). If set,
print statements to the file don't trigger the print_hook event. This
is useful to keep the output of certain files from being propagated to
peers.
- You can now associate "classes" with remote peers (Robin Sommer). When
connecting, a node may send a specific class to which it considers itself
belonging. The accepting side can then tune its configuration based on
the received class.
This is primarily for the having multiple unrelated Broccolis running on the
same host, all connecting to the same remote Bro (e.g., sshd and syslog
sensors).
To use this, on the Bro side the record Remote::Destination now has a
field "class: string" (default: unset). If set, the given config entry
only applies for connecting remote peers that send the given class.
If it is set and we're connecting to another peer, we propagate the class.
Example:
On the listening Bro:
redef Remote::destinations += {
["peer-1"] =
[$host = 127.0.0.1, $class="ftp", $events = /ftp.*/],
["peer-2"] =
[$host = 127.0.0.1, $class="http", $events = /http.*/]
};
On peer 1:
redef Remote::destinations += {
["master"] =
[$host = 127.0.0.1, $class="ftp",
$events = /.*/, $connect=T]
};
On peer 2:
redef Remote::destinations += {
["master"] =
[$host = 127.0.0.1, $class="http",
$events = /.*/, $connect=T]
};
All of these may run on the same host.
- A bunch of changes to adu.bro (Christian Kreibich):
- New ADU_MAX_DEPTH limits depth (at ADU granularity) into a
flow up to which ADUs are reported.
- Handles UDP.
- New event adu_done(c: connection) signals that no further ADUs
will be delivered for a connection. This is useful since adu.bro
relies on event connection_state_remove() to remove state, and
if a policy using adu.bro likewise uses this event type then
event sequencing can cause adu_tx/rx events to occur after
connection_state_remove() has been processed.
- Now correctly clips ADU to maximum allowed size. (Note, this
has been temporarily commented out because it relies on a new
string function that has not yet been integrated into the
main distribution.)
- Now can ignore specific connections dynamically.
- TCP content gaps are now recognized and ADU delivery is for now
stopped for such flows, unless explicitly requested.
- No longer logs to file in test mode.
- The new function add_notice_tag() explicitly adds a unique notice tag
to a connection's $addl field (Robin Sommer). This is sometimes necessary
to ensure that the tag appears in the connection summary.
- Bro now performs serialization (such as when checkpointing &persistent
tables or communicating them between Bro's) in an incremental fashion,
intermingling transfers of large tables with ongoing packet processing
(Robin Sommer). Doing so helps avoid packet drops for large items.
This has not yet been implemented for the initial handshake done
for &synchronized items.
- ssl.bro now stores certificates by default in the subdirectory "certs/"
(Robin Sommer).
- Analysis of weak/unknown ciphersuites in ssl.bro reworked (Holger Dreger).
- New cipher for SSL analysis, SSL_CK_RC4_64_WITH_MD5 (Holger Dreger).
- load-levels and cpu-adapt now log their adaptations to the log file
rather than generating alarms (Robin Sommer).
- The default adaptation levels in cpu-adapt have been tweaked for better
behavior (Robin Sommer).
- A new structure of the event loop (implemented by Robin Sommer) is now
enabled during configuration by default (Christian Kreibich). You can
revert to the previous structure using --disable-select-loop.
- When configuring Bro, the version of pcap that comes with the Bro
distribution is no longer used by default (Jason Lee). Instead,
the system one is used, or one at the same directory level as Bro.
To use the Bro distribution version, configure with --enable-shippedpcap.
- backdoor.bro now has comments clarifying that it does not itself
alter capture_filters (Vern Paxson).
- If you set backdoor_stat_period to 0 sec, then this now turns off
the periodic component of backdoor analysis (Holger Dreger).
- The filters specified in notice_action_filters now take an additional
argument specifying the action that has been determined so far (Robin
Sommer). This allows the filter to decide to not change the current
action, if it so wishes.
- The new event notice_alarm(n: notice_info, action: NoticeAction) is
generated for every notice that results in an alarm (Robin Sommer).
- Tallying of notices is now done using a notice, which has type NoticeTally
(Robin Sommer).
- The new notice action filter alarm_always_notice specifies an action
of NOTICE_ALARM_ALWAYS (Vern Paxson).
- If the watchdog expires and Bro isn't generating a packet trace file,
the current packet is saved to "watchdog-pkt.pcap" (Robin Sommer).
- New boolean globals tcp_contents_deliver_all_{orig,resp} allow easy
requesting of content delivery for all TCP traffic in orig/resp directions
(Christian Kreibich).
- The new event udp_contents(u: connection, is_orig: bool, contents: string)
delivers the contents of UDP packets analogous to tcp_contents (Christian
Kreibich). The boolean globals udp_content_deliver_all_{orig,resp} and
tables udp_content_delivery_ports_{orig,resp} control for which ports
content is delivered, analogous to the globals that control tcp_contents.
- New option --set-seed=n sets the random number seed to n (Vern Paxson).
- Notices now report current time for remotely-received notices rather
than network time (Brian Tierney).
- Notices now include a tag es=<peer_description> any time a peer
description is defined, not just for remote notices (Robin Sommer).
- The global log_as_connection has been removed from icmp.bro, which now
only logs ICMP flows via the usual connection logging (Vern Paxson).
- The Destination variable $accept_state has been renamed $accept_input
to better reflect its meaning (Vern Paxson).
- A remote destination's $sync field now indicates whether to accept
ongoing state changes from peers, rather than just upon start-up
(Robin Sommer). The variable $accept_state controls whether we
accept events.
- Logging of forms of Bro communication has been unified (Robin Sommer).
- Updates for packet filtering documentation (Christian Kreibich).
- A new global, stp_skip_src, lists sources that should be skipped for
stepping-stone analysis (Vern Paxson). ssh-stepping.bro adds sources to
this list if they've instantiated more than src_fanout_no_stp_analysis_thresh
connections, keeping them blocked until they've been idle for 15 seconds.
- Added a default notice-policy.bro as an example (Brian Tierney).
- Expanded on descriptive text in notice-policy.bro (Vern Paxson).
- ef removed from aux/hf/, as it's of little use and a headache to
maintain for portability (Vern Paxson).
- The version of libpcap bundled with the distribution has been
elevated to 0.8.3 (Jason Lee).
- Bro now compiles again if non-blocking DNS is not available (Robin Sommer).
- Resource statistics logging now differentiates between offline
processing vs. remote-communication-only (Mark Dedlow and Robin Sommer).
- The script variable ICMP::distinct_pairs now times out its state,
with a default of 15 minutes after creation (Robin Sommer).
- The Bro version reported now includes "-debug" if Bro was configured
with --enable-debug (Robin Sommer).
- scan.bro now defaults "shut_down_all_scans" to T, meaning it by
default detects scans on all ports, not just those in the set
shut_down_scans (Vern Paxson). Please note, this variable is
misnamed - it should be "detect_all_scans" - but that change is
waiting on reworking the basic structure of scan detection.
- Major bug fix for signature matcher missing matches on analyzer data
(Robin Sommer). For example, a condition "http /foo/" would only have
match with the first URL in a connection, not subsequent ones. Fixing
this changes the calling sequence of the match_signatures() built-in to
take an additional final parameter, "clear", which, if set, resets the
matcher to its starting state prior to matching.
- Serious bug in regular expression matching - and hence signature engine -
fixed (Robin Sommer).
- Bug fix for formatting (via fmt()) of very long strings (Vern Paxson).
- Fixed mail_reports.sh to correctly find sendmail binary on various systems
(Brian Tierney).
- Numerous changes to Bro's internal string representation, and more
flexibility in how strings are rendered for display (Christian Kreibich).
- Pseudo-real-time now can be initialized using an optional argument
that corresponds to the degree of time compression (Robin Sommer).
For example, --pseudo-realtime=0.5 causes time to advance half as fast
as it would in real-time. The default value is 1.0; any value > 0 is
allowed.
- The SSH analyzer now looks for just linefeeds as line terminators when
extracting version strings, rather than carriage-return-line-feeds, to
match actual implementations rather than the RFC (suggested by Chema
Gonzalez).
- Playing back events from files now working again (Robin Sommer).
- Bro now uses current_time() rather than network_time to track the
modification time of variables, since network_time doesn't advance
when only receiving events (Robin Sommer).
- Bug fixes for IPv6 support, including processing UDP traffic
(which had been completely broken) and subtle interactions (actually,
lack thereof) between the connection compressor and IPv6 that
could lead to crashes (Vern Paxson).
- Portability tweaks for NetBSD, 64-bit Linux SuSe and FreeBSD 5.4
(Christian Kreibich, Jason Lee and Vern Paxson).
- Bug fix for IPv6 "::" constants that start with hex digits specified
using 0x (Vern Paxson).
- Calling the built-in terminate() function twice now has no additional
effect (Christian Kreibich). It used to terminate Bro abruptly, without
cleanly shutting down.
- Removed active.bro; use active_connection() + connection_record() instead
(Vern Paxson).
- Bro lite reports now work with rotated logs files (Brian Tierney)
- Bug fix for conditions such as "payload /^user/", which now work equivalent
to "payload /user/" (Robin Sommer).
- Tweaks to sensitive patterns in HTTP request URIs to reduce false
positives (Brian Tierney).
- Bug fixes for strip() built-in function (Holger Dreger).
- Memory leak in built-in function to_addr() fixed (Ruoming Pang).
- Bug fix for "hot" connections sometimes not having their notice tag
appearing in connection summaries (Robin Sommer).
- Bug fixes for IRC analysis (Vern Paxson and Robin Sommer).
- Syslogging now works if Bro is running in communication-only mode
i.e., live, but not reading a network interface (Robin Sommer).
- Bug fix to allow tuning of TRW parameters (Vern Paxson).
- Bug fixes for SSL analysis (Holger Dreger).
- Removed logic that inverted orig/resp in some scans (Vern Paxson).
- Lint & memory allocation tweaks (Vern Paxson).
- Bug fixes for inactivity timeouts (Robin Sommer).
- Bug fix for Bro Lite cron job (Jason Lee).
- When binding to a listening port for remote communication fails,
the port number is now reported (Robin Sommer).
- Some spurious reporting removed from configure output (Jason Lee).
- Fix for "weird"'s generated by connection compressor but not
recognized at the policy script level (Vern Paxson).
- Fixes for detecting content gaps and not matching previously delivered
data (Ruoming Pang).
- Bug fixes for TCP rewriter (Ruoming Pang).
- Bug fixes for crashes in SSL analyzer (Vern Paxson).
- Bug fix for avoiding busy-waiting when a communication child dies
(Robin Sommer).
- Bug fix for BiF's that use 'T' and 'F' in character constants
(Vern Paxson).
- Memory leak fixes (Robin Sommer, Christian Kreibich, Vern Paxson and
Ruoming Pang).
- The peer table for inter-Bro communication is now correctly indexed by a
peer_id (Robin Sommer).
- Bug fix for exchange of initial &synchronized state which could
prevent communication from entering main phase (Robin Sommer).
- Bug fix for propagating incremented table values derived from
a table's &default (Robin Sommer).
- Bug fixes for the POP3 analyzer when analyzing non-NUL-terminated strings
or bad base64 encodings (Vern Paxson).
- Updates for Bro's internal hash functions (Ruoming Pang).
- The debug and communication log files now comply with $BRO_LOG_SUFFIX
(Robin Sommer).
- Some internal debugging additions (Ruoming Pang).
- Internal cleanup regarding "const" strings (Ruoming Pang).
- A number of casts changed to use modern C++-style pointer casting
such as reinterpret_cast and static_cast (Ruoming Pang).
- Bug fixes for inter-Bro communication on 64-bit systems (Robin Sommer).
- Bug fixes for detecting errors for SSL connections (Robin Sommer).
- Potential null pointer dereference fixed (Robin Sommer).
- Inter-Bro communication is now more reliable in the presence of errors
(Robin Sommer).
- Performance enhancement for tracking values whose elements might
change (Robin Sommer).
- Fixes for peers having differing enum lists (Robin Sommer). This can
occur because they're running different scripts and which do different
redef +='s to add enum values.
- += now works for interval types (Vern Paxson).
- Bug fix for exchanging peer descriptions (Robin Sommer).
- Bug fix for processing multipart-MIME HTTP messages with content-length
headers (Ruoming Pang).
- Bug fix for failing to escape "'s in HTTP server replies (Robin Sommer).
- Bug fix for propagating increment operations on tables (Robin Sommer).
- Bug fixes for files (Robin Sommer): set open time to current time if
network time is not initialized; when deserializing files, prevent them
from being closed immediately due to reference-counting interaction.
- Bug fix to prevent reporting some scans twice (Robin Sommer).
- Bug fix for printing enum's (Christian Kreibich).
- When not configured with --enable debug, Bro now still accepts (yet ignores)
option -B (Robin Sommer).
- Serialization enhancements and fixes, including a change of the
protocol version number (Robin Sommer).
- Bug fix for logging inter-Bro communication (Robin Sommer).
- Bug fixes for enumerating attributes and timers (Robin Sommer).
- Bug fix for signatures matching first on one side of the connection,
and then on the other, being reported twice (Robin Sommer).
- Inter-Bro communication now continues to work even when packet processing
has been suspended (Robin Sommer).
- Fix for running multiple Bro's together in pseudo-realtime (Robin Sommer).
- Tweak to print-resources.bro so it can be loaded standalone (Vern Paxson).
- Bug fix for &persistent state not being save if Bro wasn't running
with an input source (Robin Sommer).
- Bug fix for which process ID to check to see if children are still alive
(Robin Sommer).
- Bug fix for no longer crashing if the expiration function associated
with a table deletes the element from the table rather than returning
an interval of 0 secs to indicate it should be deleted (Chema Gonzalez).
- Bug fix for OutboundTFTP notice: now checks to ensure that not only is
the source local, but the destination is not local (Vern Paxson).
- Bug fix for a subtle interaction mediated by errno, which could cause a
failed read() to later confuse pcap_dispatch() (Chema Gonzalez).
- Bug fix for TCP contents assertion checking (Ruoming Pang).
- Bug fix for error output on small RPC fragments (Ruoming Pang).
- Fix for connection compressor bug in tracking connection history
(Robin Sommer).
- Bug fix for potential floating point exception in signature engine's
resource-profiling code (Robin Sommer).
- Bug fix for low-level List data structure when replacing a list element
beyond the end of a list (Robin Sommer).
- Bug fix in initializing capabilities when setting up communication between
Bro peers (Robin Sommer).
- A number of connection compressor bug fixes: weird's for spontaneous
FINs and RSTs, consistent processing of "connections" that begin with
RSTs, correct checksum computations, and weird's printed to stderr if
no event handler defined (Robin Sommer).
- load_sample_freq is now &redef (Vern Paxson).
- Bug fix for backdoor detector incorrectly matching substrings (Vern Paxson).
- Bug fix for canceling timers sometimes failing to cancel all of
them (Robin Sommer).
- Error handling during un-serialization now handled more robustly
(Robin Sommer).
- Bug fix for division by zero if expensive_profiling_multiple
set to zero (Robin Sommer).
- Bug fix for connection logs failing to track all of the annotation
($addl) associated with a connection (Vern Paxson).
- Portability fix for BinPAC (Ruoming Pang).
- Fix to NFS analyzer for missing values in events reporting failed requests
(Vern Paxson).
- autogen.sh now aborts as soon as one of the tools it invokes fails
(Christian Kreibich).
- Fixed bug where not having SSL would cause bro to not compile (Jason Lee).
- State-holding fix for adu.bro (Christian Kreibich).
- A number of configuration tweaks (Craig Leres & Christian Kreibich).
- Fix for sig-functions.bro: checks isApache* functions, which ensure
that Apache is indeed in the software set before accessing the index
(Brian Tierney and Robin Sommer).
- Smith-Waterman fixes and test suite script (Christian Kreibich).
1.0 Sun Oct 23 17:27:45 PDT 2005
- Bro now includes BinPAC (Binary Protocol Analyzer Compiler), a language
and compiler for automating the construction of analyzers for binary
protocols (Ruoming Pang).
- Ruoming has used BinPAC to rewrite the analyzers for DCE/RPC (with
significant enhancements, including adding the endpoing mapper) and SMB
(likewise enhanced and bug-fixed), and creating a new analyzer for NCP
(Netware Core Protocol). The NCP analyzer generates two events:
ncp_request(c: connection, frame_type: count, length: count,
func: count)
ncp_reply(c: connection, frame_type: count, length: count,
req_frame: count, req_func: count,
completion_code: count)
- The beginnings of an analyzer for NetBIOS name service (Ruoming Pang).
It generates the following events:
event nbns_standard_name_query(c: connection)
event nbns_nbstat_query(c: connection)
event nbns_name_reg_request(c: connection)
event nbns_nb_name_query(c: connection)
- New IRC analyzer (Roland Gruber). It generates a lot of events; see
policy/irc.bro. Note, the formatting of the log file will at some point
be changed to be more uniform and streamlined.
- ICMP events now include an initial parameter of type "connection",
the same as for TCP & UDP flows (Ruoming Pang). This facilitates
traffic analysis by associating generic connection events such as
connection_state_remove with ICMP events. This affects:
event icmp_sent(c: connection, icmp: icmp_conn)
event icmp_echo_request(c: connection, icmp: icmp_conn,
id: count, seq: count, payload: string)
event icmp_echo_reply(c: connection, icmp: icmp_conn, id: count,
seq: count, payload: string)
event icmp_unreachable(c: connection, icmp: icmp_conn,
code: count, context: icmp_context)
event icmp_time_exceeded(c: connection, icmp: icmp_conn,
code: count, context: icmp_context)
- New POP3 analyzer (Florian Schimandl, Hugh Dollman and Robin Sommer).
Loading pop3.bro analyzes the protocol messages, and loading mime-pop.bro
also extracts the email headers and content.
- New events (Ruoming Pang):
connection_first_ACK(c: connection)
generated upon the ACK completing a TCP handshake. Useful
in detecting "blink scans" (a FIN coming from the client
right after the ACK)
tcp_rexmit(c: connection, is_orig: bool, seq: count, len: count,
data_in_flight: count, window: count)
generated when a TCP sender retransmits data
rpc_call(c: connection, prog: count, ver: count, proc: count,
status: count, start_time: time,
call_len: count, reply_len: count)
can be used to process RPC calls in a generic fashion
nfs_reply_status(n: connection, status: count)
supplies the status of NFS server replies
netbios_session_raw_message(c: connection, is_orig: bool, msg: string)
access to a NetBIOS SSN message in raw terms
smb_get_dfs_referral(c: connection, max_referral_level: count,
file_name: string)
generated for SMB DFS referal requests
dns_message(c: connection, is_orig: bool, msg: dns_msg, len: count)
low-level event generated for each DNS request/reponse
dce_rpc_bind(c: connection, uuid: string)
generated for DCE RPC binds
dce_rpc_message(c: connection, is_orig: bool, ptype: dce_rpc_ptype,
msg: string)
low-level access to DCE RPC messages; see const.bif for
dce_rpc_ptype values
epm_map_response(c: connection, uuid: string, p: port, h: addr)
reply from DCE portmapper
- New operator |x|, a sizeof operator (Christian Kreibich). It yields
absolute values for numerical values, file size for files, the number
of enums for an enum type, the number of addresses in subnets, number
of fields in records, numeric equivalent for addresses, the number of
elements in vectors/sets/tables, and the length of strings.
- A new clone operator, "* <expr>", produces deep copies of aggregate
values and the usual duplicates for atomic ones (Christian Kreibich).
For example, while:
1 type foo: record {
2 c: count;
3 s: string;
4 };
5
6 f1$c = 10;
7 f1$s = "Hello";
8
9 f2 = f1;
10 f1$c = 20;
11 f1$s = "World";
12
13 print fmt("%d/%s %d/%s", f1$c, f1$s, f2$c, f2$s);
yields "20/World 20/World", changing line 9 to:
f2 = *f1;
yields "20/World 10/Hello".
- New operators "+=" and "-=", which work on both numerical values and
strings (Christian Kreibich).
- "+" now works on strings: s1 + s2 yields the concatenation of both
(Christian Kreibich).
- You can now express the equivalent of ICMP "port numbers" using
<number>/icmp, where <number> is the ICMP type (Ruoming Pang).
- Bro now accepts long versions of options, such as --readfile for -r
(Christian Kreibich).
- Bro now has a "pseudo-realtime" mode, activated by --pseudo-realtime,
that causes it to mimic real-time operation when executing against
a trace (Robin Sommer). This is useful for evaluating performance in
a controlled fashion.
- SMTP analyzer changes (Roger Winslow): support for 554 code in RCPT
responses; logging when the server refuses the argument to RCPT;
support for 502 code in response to a HELP command.
- Addition of two universal hash functions: H3 (from David Moore, based
on code by Ken Keys) and TwoWise (Dietzfelbinger, from Yin Zhang)
(Ruoming Pang). Use --enable-h3 and --enable-dietzfelbinger to enable
them (used as the hash function for short data). H3 is used by default.
- The "bif" compiler for compiling Bro built-in functions now supports
an "enum" type (Ruoming Pang). The syntax is:
enum dce_rpc_ptype
%{
DCE_RPC_REQUEST,
DCE_RPC_PING,
DCE_RPC_RESPONSE,
%}
which is translated to an enum declaration of "dce_rpc_ptype" in
Bro, an EnumType* enum_dce_rpc_ptype in NetVar.{h,cc} and a C++ enum
BroEnum::dce_rpc_ptype {...}.
One limitation is that redef's on enum types cannot be taken into
account because the bif is parsed at compile time.
- 64-bit integer support via --enable-int64 (Ruoming Pang).
- The new, experimental policy script adu.bro provides a generic way to
extract application-layer ADUs (Christian Kreibich). It heuristicly
groups blocks of content sent from one side to another, uninterrupted
by any data in the opposite direction, into an approximate ADU (request
or reply). These then generate adu_tx (originator -> responder) and
adu_rx (responder -> originator) events. You can control on which ports
it does this analysis, as well as the amount of data inspected nad
grouped, using variables documented in the script.
- The new built-in function
function str_smith_waterman(s1: string, s2: string, params: sw_params)
: sw_substring_vec
computes the Smith-Waterman overlap between two strings (Christian Kreibich).
The third parameter is a record with two fields, $min_toklen (minimum
length for common tokens) and $sw_variant, which takes a value of 0
for single-matching and 1 for multiple-matching.
The return value is a vector of sw_substring records, which hold the
following fields:
str: string; # the common subsequence
index1: count; # where it occurs in input string 1
index2: count; # where it occurs in input string 2
new: bool; # true if start of new alignment
- If you set the new control variable record_state_history to T, then
connections recorded to the conn.$BRO_LOG_SUFFIX log file will include
a field that shows the different states encountered during the connection
(Mark Allman):
Symbol State
------ -----
S Initial SYN seen for TCP connection.
H SYN-ACK seen for TCP connection.
D Data packet seen (TCP or UDP).
A Pure ACK seen for TCP connection.
F FIN seen for TCP connection.
R RST seen for TCP connection.
I TCP connection included a FIN+RST packet.
Q TCP connection included a packet with multiple connection
control flags other than FIN+RST (e.g., SYN+RST).
C Connection included one or more packets with failed checksums
(TCP or UDP).
The symbols are printed in upper-case for connection originators and
lower-case for responders. The S/H/F/R symbols are also repeated if
Bro sees the corresponding control packet subsequently with a different
sequence number.
For example, a TCP connection which Bro saw from the beginning, i.e.,
a normal establishment, followed by the client (originator) first sending
data, then the server responding, followed by the server initiating a
normal close which the client then completes, will be annotated as
"ShADadfF".
- The "for" looping construct now can be used to iterate over the non-empty
indices of a vector (Christian Kreibich).
- If you set the new variable skip_http_data to T (default: F), then the
HTTP analyzer will attempt to not reassemble the data portions of HTTP
request/responses (Ruoming Pang). This can be a performance benefit
in environments with high volumes of HTTP traffic, though it may not be
a large win if the processing is dominated by executing the policy script.
- The new built-in
remask_addr(a1: addr, a2: addr, top_bits_from_a1: count): addr
take some top bits (e.g. subnet address) from a1 and remaining bits
(intra-subnet part) from a2 and merge them to get a new address (Ruoming
Pang). Useful for anonymizing at the subnet level while preserving
serial scans.
- The new built-in
decode_netbios_name(name: string): string
takes a string in NetBIOS encoding and returns its original form
(Ruoming Pang).
- The new variable ignore_keep_alive_rexmit controls whether to
include keep-alives when counting retransmitted packets (Ruoming Pang).
It defaults to F (i.e., do count them).
- The calling sequence of dce_rpc_request and dce_rpc_reply have changed to:
event dce_rpc_request(c: connection, opnum: count, stub: string)
event dce_rpc_response(c: connection, opnum: count, stub: string)
(Ruoming Pang). Use dce_rpc_message to get access to the RPC type and
the raw message.
- The calling sequence of the netbios_session_message event has changed to:
netbios_ssn_message(c: connection, is_orig: bool,
msg_type: count, data_len: count)
(Ruoming Pang). Previously it was parameterized with the connection
and the raw message (now available via netbios_session_raw_message).
- The calling sequences of smb_com_{read,write}_andx have changed to
no longer include the is_orig parameter because it is in fact fixed for
these events (Ruoming Pang).
- The calling sequence of smb_message has changed (Ruoming Pang) to:
smb_message(c: connection, is_orig: bool, cmd: string,
body_length: count)
- Bug fix specifying the &default value for tables that yield function
values (Ruoming Pang). For example:
type tcp_content_handler_func:
function (c: connection, is_orig: bool, seq: count,
contents: string);
function default_tcp_content_handler(c: connection, is_orig: bool,
seq: count, contents: string)
{
# do something ...
}
const tcp_content_orig_handlers: table[port] of
tcp_content_handler_func = {} &redef &default =
default_tcp_content_handler;
Previously, Bro would take the function given with &default as the default
function to call when accessing a missing element, rather than a default
*value* to directly return. Bro now checks the value type against the
function type to see if they match in type.
- The new variables forward_remote_events and forward_remote_state_changes
specify whether to broadcast events/state received from one peer to other
peers (Robin Sommer). Both default to F. Note, these options are temporary;
they will disappear when we add a more sophisticated script-level
communication framework.
- Vectors can now be initialized using the syntax such as
global foo: vector of string = ["foo","bar"];
(Robin Sommer).
- Bug fixes for &synchronize'ing vectors (Robin Sommer).
- The internal implementation of strings in the policy language has
been heavily revamped (Christian Kreibich).
- String built-in functions are now in strings.bif rather than bro.bif
(Christian Kreibich). This includes two new built-ins:
str_split(s: string, idx: index_vec): string_vec
strstr(big: string, little: string): count
string_vec is a new policy script type that is an alias for
"vector of string".
- The new options --load-seeds <file> and --save-seeds <file> let you
record Bro's seeds to a file and then re-use these seeds in a later
invocation (Christian Kreibich). The primary intended usage is to
provide determinism in hash table iterations etc. for debugging purposes.
- Communication protocol changes (Robin Sommer):
* Internal PING/PONG messages to measure round-trip times. The new script
remote-ping.bro issues PINGs every second and logs to remote.log.
* Optional data compression if libz is available. Remote::Peer$compression
specifies compression level, with no compression being the default.
* Inter-Bro communication is now performed in four explicit phases:
// Setup:
// Initial phase.
// VERSION messages must be exchanged.
// Ends when both peers have sent VERSION.
// Handshake:
// REQUEST_EVENTS/REQUEST_SYNC/CAPTURE_FILTER/CAPS/selected SERIALs
// may be exchanged.
// Phase ends when both peers have sent PHASE_DONE.
// State synchronization:
// Entered iff at least one of the peers has sent REQUEST_SYNC.
// The peer with the smallest runtime (incl. in VERSION msg) sends
// SERIAL messages comprising all of its state.
// Phase ends when peer sends another PHASE_DONE.
// Running:
// Peers exchange SERIAL (and PING/PONG) messages.
// Phase ends with connection tear-down by one of the peers.
* Serializing network packets includes textual tags for identification.
* Serializing files includes the state of buffering.
- Pending events for remote peers are now flushed when Bro terminates,
and the net_done event is *not* propagated to peers (Robin Sommer).
- Makefile.am cleanups (Christian Kreibich).
- libpcap portability fix for OpenBSD (Gordon Willem Klok).
- Performance bug fix for SMTP relay detection (Vern Paxson).
- sprintf -> snprintf tweak (Vern Paxson).
- Bug fix for serializer regular-expression matchers (Robin Sommer).
- Some fixes for access to uninitialized variables/state (Christian Kreibich
and Vern Paxson).
- More informative messages for some internal errors (Christian Kreibich).
- Bug fixes for implementation of vectors (Christian Kreibich).
- Fixes for FreeBSD 5 installs (Jason Lee).
- gcc 4.0 compatibility (Christian Kreibich).
- Bug fix for correctly propagating libpcap failures (Chema Gonzalez).
- Bug fixes for prefix-preserving IP address anonymization (Chema Gonzalez).
- The MIME analyzer in mime.bro is now in "module MIME" (Vern Paxson).
- Bug fix for the IRC backdoor detector (Scott Campbell).
- The capture filter used for NFS traffic now includes UDP fragments,
since NFS UDP traffic is often fragmented (Ruoming Pang).
- New internal mechanisms to suspend/resume processing to enable a Bro
receiving synchronized state to put its own packet processing on hold
(Robin Sommer).
- A bug with the serialization cache not being used for modified objects
has been fixed (Robin Sommer).
- A number of enhancements to inter-Bro communication performance and error
handling improved (Robin Sommer).
- Internal restructuring to fix problems with dispatching packets when
using the packet sorter (Ruoming Pang).
- Christian Kreibich has contributed a number of fixes for code flaws
such as potentially unsafe library calls.
0.9a10 Tue Sep 6 10:41:53 PDT 2005
- Fixes for portability to 64-bit architectures (Christian Kreibich).
- Bug fix for broken syslog'ing of alarms (Scott Campbell).
- The manual has been updated to clarify that aggregate values in events
are passed as shallow copies, so that modifications to elements of the
values after posting the event but before it's handled will be visible
to the handlers for the events (Christian Kreibich).
- HTTP logging now includes the host from the Host header in the request
(Craig Leres). Note, currenty this only is done when using http-reply.bro,
not if you only analyze requests.
- You can now specify a passphrase for the SSL cert used for inter-Bro
communication by redef'ing the variaable "ssl_passphrase" (Christian
Kreibich). Leaving it unchanged causes the passphrase to be read
interactively.
- Certificates created using ca-issue now have 2-year lifetimes rather
than the default of 30 days (Christian Kreibich).
- A problem with handshaking between Bro peers has been fixed (Christian
Kreibich).
- A bug has been fixed in scanning false positives due to backscatter
in the form of SYN ACKs (Vern Paxson).
- Alerts sent via email now use a From address configured from bro.cfg
(Randy Mcclelland-Bane). Also, if sending an alert via gpg fails,
it's sent instead as plaintext.
- Scan notices now include information about the connection that
triggered the scan detection decision (Vern Paxson).
- Exported some TRW variables so the user can adjust their associated
timers (Vern Paxson).
- The new script variable dns_max_queries sets a maximum on the number of
queries that can appear in a DNS request (Scott Campbell & Vern Paxson).
If more queries appear, the request is treated as non-DNS traffic and
ignored. The variable defaults to a value of 5. Setting it to 0 turns
off this functionality, so Bro processes all apparent requests.
- The "weird" messages generated by the DNS analyzer now have a more
regular naming structure and processing (Scott Campbell and Vern Paxson).
- Tweaked bif_arg.cc to pass gcc4.0 and bro.bif to not collide with
uuid in OSX 10.4 (Jason Lee). Now works on OSX 10.4, though use
--disable-localpcap when compiling.
- Bro now compiles cleanly under OpenBSD (Jason Lee).
- NOTE: the connection compressor has a known serious bug and should
not be used at present. Since it is an experimental feature, fixing it
is deferred to the next release.
- Some bugs fixed in the management of hash keys when using the
connection compressor (Robin Sommer).
- Tweak for the connection compressor to generate truncated_header weird's
(Robin Sommer).
- Temporary bug fix for type clash in SSL version numbers (Vern Paxson)
by making them consistently of type int. The correct fix is probably
for them to be consistently of type count, depending on how Bro's notion
of general version processing, and its SSL analyzer, both evolve.
- Bug fix for trace rewriting failing if Bro was not compiled to check
assertions (Martin Casado).
- Fixed logic bug in signal handling regarding whether we're currently
idle waiting for input vs. processing a packet or the event queue
(Vern Paxson). Note, this change has not been heavily tested.
- Some bug fixes for correct operation when DNS names fail to resolve
(Vern Paxson). It's not clear that these fixes are complete, however.
- Fixed to not compile libpcap when --disable-localpcap is given to configure
(Jason Lee).
- Fixed configuration of local pcap for IPv6 if --enable-brov6 is specified
(Jason Lee).
- A problem with "make install" when building from the libpcap included
in the sources has been fixed (Christian Kreibich).
0.9a9 Thu May 19 23:31:33 PDT 2005
- First cut at analyzer for NFS (Vern Paxson). It generates the following
events:
event nfs_request_null(n: connection)
event nfs_request_getattr(n: connection, fh: string, attrs: nfs3_attrs)
event nfs_request_lookup(n: connection, req: nfs3_lookup_args,
rep: nfs3_lookup_reply)
event nfs_request_fsstat(n: connection, root_fh: string,
stat: nfs3_fsstat)
event nfs_attempt_null(n: connection, status: count)
event nfs_attempt_getattr(n: connection, status: count, fh: string)
event nfs_attempt_lookup(n: connection, status: count,
req: nfs3_lookup_args,
dir_attrs: nfs3_opt_attrs)
event nfs_attempt_fsstat(n: connection, status: count,
root_fh: string, obj_attrs: nfs3_opt_attrs)
- The new script OS-fingerprint.bro integrates Bro's new passive OS
fingerprinting mechanism with the software.bro framework (Vern Paxson).
- You can now operate on patterns using && and || (Vern Paxson).
If p1 and p2 are patterns, then p1 && p2 yields a pattern that matches
their concatenation and p1 || p2 yields a pattern that matches either.
Note that the syntax for this may change in the future to a single '&'
or '|', which would be more consistent with the use of '|' in
constructing pattern constants.
- An experimental "connection compressor" tracks not-yet-established
connections using much less memory than Bro normally does (Robin Sommer).
This is potentially a major win during flooding attacks and high-speed
scans. You activate it by setting use_connection_compressor to T. You
can then control the granularity of its processing using the variables
cc_handle_resets, cc_handle_only_syns, and cc_instantiate_on_data. See
bro.init for brief discussion of these.
- The experimental new script firewall.bro supports firewall-rule-like
processing of connections in terms of allow/deny (Robin Sommer). It is
not particularly efficient.
- sensor-sshd.bro provides an experimental interface for receiving
events from instrumented SSH servers that communicate with Bro via
the Broccoli client library (Christian Kreibich and Robin Sommer).
Supporting this also entailed extensions to login.bro so it can
process the events even though they don't correspond to a connection
known to Bro's event engine.
- The new built-in function match_signatures() can be used in a policy
script to send text directly into the signature engine (Robin Sommer).
- Correction: the 0.9a8 CHANGES states that the mail_script variable used
for NOTICE_EMAIL defaults to mail_script.sh. The correct value is instead
"mail_notice.sh".
- The scripts rsh.bro and passwords.bro, and the passive-fingerprinting
signatures policy/sigs/p0fsyn.osf were inadvertantly left out of the
0.9a8 distribution.
- Added s2b (snort to bro) files into the distribution. (Jason Lee)
- Non-blocking packet capture under Linux has been fixed (Robin Sommer).
- Fixed printing of DNS replies, which used to work but was broken
a number of months ago (Vern Paxson).
- The new script brolite-sigs separates out how signatures are configured
in Bro Lite so the functionality can be enabled/disabled with a simple
load statement (Roger Winslow). That is, to use signatures with Bro
lite, simply add "@load brolite-sigs".
- The new script variable enable_syslog (default T) controls whether
alarm's are syslog'd (Robin Sommer). As before, syslogs can only happen
when Bro is reading from live network traffic (this should be changed
at some point, to accommodate real-time Bro's that don't read the network
but collect events from other sensors). Previously, in that case syslog's
always happened; now, you can turn them off using this variable.
- The new script variable expensive_profiling_multiple controls how
often, when doing profiling, to perform more expensive forms of
profiling, in particular, memory consumption profiling (Robin Sommer).
If profiling_interval is set to 15 sec and expensive_profiling_multiple
is set to 20, then expensive profiling will be done every 5 minutes
(these are the defaults now in profiling.bro). Also, the profiling_update
event now includes a second argument, expensive: bool, which indicates
whether the update corresponds to one of these expensive profiling
intervals.
- First cut at parsing DNS AAAA replies (Scott Campbell). This is quite
incomplete - currently, the replies are turned into fake A record replies,
due to the difficulty of dealing with IPv6 addresses if Bro wasn't built
to analyze IPv6 traffic.
- software.bro has been tweaked to have a new control variable,
"only_report_local" (default F). If true, then only software versions
for local addresses (as determined by is_local_addr()) will be
reported.
- synflood.bro now has a script variable max_sources (default 100) that
specifies the maximum number of sources to track for a given victim
(Robin Sommer).
- Remote peers now negotiate their versions of the serialization format
(Robin Sommer). If they don't agree then the connection is terminated.
- Generic UDP request/response processing has been moved into the new
policy script udp-common.bro, which, unlike udp.bro, does *not* set the
packet filter to capture all UDP traffic (Robin Sommer). A number
of UDP-based policy scripts have been modified to use udp-common.bro
rather than udp.bro.
- When printing serialized/independent state, access times are now
again included (Robin Sommer).
- Bro's implementation of timers has been switched (reverted) to using
priority queues (Vern Paxson).
- The http-request.bro script variables skip_remote_sensitive_URIs and
const sensitive_post_URIs are now exported so they can be accessed
externally (Robin Sommer).
- Some new rootkit filenames have been added to ftp.bro and
http-request.bro (Brian Tierney). The plan is to eventually
merge these lists so there's only one main list.
- trw.bro is now scoped as a module "TRW" (Brian Tierney).
- Better support of the '--disable-localpcap' flag to configure, and
consolidated all the pcap checks in configure.in (Jason Lee).
- A bug in processing bare carriage-returns in Telnet input/output
has been fixed (Vern Paxson).
- The Bro Lite bro.rc script has been tweaked to use the 'ax' flags
instead of '-ax' (Jason Lee).
- A bug with reporting ICMP "ports" (i.e., type + code) has been fixed
(Vern Paxson).
- Bug fix for excessively large RPC messages (Ruoming Pang).
- A bug with /0 subnet prefixes has been fixed (Robin Sommer).
- The function record_connection() now takes the file to write to
as its first argument (Robin Sommer).
- remote.bro now tracks whether a given Destination is connected
(Robin Sommer).
- mail_notice.sh is now installed as part of installing a distribution
(Jason Lee).
- Fixed bug where the sort order for the test suite changed depending
on locale. (Jason Lee)
- Bug fix for email_notice() when notice_action_filters not defined for
given notice (Vern Paxson).
- The test suite test for rare-events fixed to not give false positives
(Jason Lee).
- Date added for 0.9a8 release.
0.9a8 Wed Feb 16 17:09:34 PST 2005
- aux/rst/ contains the source for the "rst" tool used by Bro (via the
policy script function terminate_connection() in conn.bro) to tear
down established connections by forging RST packets.
- Bro's main event loop has been reworked (Robin Sommer). This should
(1) not cause any visible differences in most cases, (2) improve
performance in some cases, (3) fixed problems running Bro without
a network input (but still receiving asynchronous input from remote
event sources). There are some more changes coming to this soon.
- Passive OS fingerprinting has been added, based on Michal Zalewski's
"p0f" tool (Holger Dreger). Currently, it's limited to fingerprinting
clients based on the initial SYNs they send. To use it, define
an event handler:
OS_version_found(c: connection, host: addr, OS: OS_version)
OS_version is a record containing a string $genre (e.g., "Solaris"),
a string $detail (e.g., "2.0.27"), a count $dist (hop-count distance
from monitor to host), and $match_type, which specifies via an
enumerated type whether the match was direct from a signature,
generic to the genre, or "fuzzy".
The match is done against a passive fingerprinting signature file,
which is specified by the variable passive_fingerprint_file.
It defaults to "sigs/p0fsyn", which is found using $BROPATH
and has an "osf" suffix added.
You can restrict the matching to only be performed for hosts from
particular subnets by adding those subnets to the variable
generate_OS_version_event. If it's empty (default), then all subnets
are analyzed.
Note, the passive fingerprinting should be integrated with the
version-tracking in software.bro, but this hasn't been done yet.
- Support for IPv6 has been repaired and brought up to date. Note, however,
that inter-Bro communication currently only works over IPv4.
- Signature-matching is now off by default in brolite.bro. If you want
to use it, define use_signatures = T prior to @load'ing it.
- Notices are now tied to their corresponding connections (Scott Campbell).
- New backdoor detectors for IRC, SMTP, Gaobot (Scott Campbell).
- Signature matches now have a connection associated with them (Scott Campbell).
- Bro scripts that set initial timers (via "schedule" statements in a
bro_init handler) but don't have any source of network input (trace
files or live interfaces) now execute in real-time, with network_time
set to the current time, rather than having their timers expire immediately.
- Default timeouts have been added to tables in trw.bro and http.bro, which
have been found operationally to potentially grow very large (Scott Campbell).
- The new policy script large-conns.bro can be included in order to
track the size of TCP connections (each direction is referred to
as a "flow") using a secondary packet filter (Chema Gonzalez).
This method is completely separate from Bro's usual size accounting,
and offers the advantages that it tracks sizes even for connections
that don't terminate (or for which Bro misses their establishment)
and for connections with sizes > 4 GB.
The interface is via the function:
function estimate_flow_size_and_remove(cid: conn_id, orig: bool)
If $orig=T, then an estimate of the size of the forward (originator)
direction is returned. If $orig=F, then the reverse (responder) direction
is returned. In both cases, what's returned is a "flow_size_est" record,
which includes a flag, $have_est, indicating whether there was any
estimate formed. If $have_est is T, then the record also includes
an estimate in bounded by $lower (lower bound) and $upper (upper bound).
The estimate also includes $num_inconsistent, which, if > 0, means that
the estimates came from sequence numbers that were inconsistent, and
thus something is wrong - perhaps packet drops by the secondary filter).
Finally, calling the function causes the flow's record to be deleted.
- An RSH analyzer has been contributed by Manu (ManuX@rstack.org).
It generates rsh_request and rsh_reply events, and the following notices:
DifferentRSH_Usernames
Client and server username differ.
FailedRSH_Authentication
Attempt to authenticate via RSH failed.
InteractiveRSH
The RSH session appears to be interactive (multiple
lines of user commands).
SensitiveRSH_Input
SensitiveRSH_Output
RSH client input or server output match input_trouble/
full_input_trouble or output_trouble/full_output_trouble.
- The new notice action NOTICE_EMAIL indicates that in addition to
logging an alarm, it should also be sent out as email (Scott Campbell).
By default, email is only sent if Bro is running on live traffic;
you can override this via redef'ing the script variable mail_notification.
Mail is sent using the script specified by the mail_script variable
(default: "mail_script.sh", which is now included in the distribution,
but at present is not installed), which must be in $PATH. The mail
is sent to the username specified in mail_dest (default: the local
"bro" user, though you can change this to name@domain).
Note that specifying email as a separate notice action may change
in the future, to instead be an attribute that's associated with
other notice actions. For example, it may make sense to want
to specify both NOTICE_ALARM_PER_CONN and NOTICE_EMAIL; currently,
however, you can't do this.
- A similar notice action NOTICE_PAGE does the same thing as NOTICE_EMAIL
except it send the mail to mail_page_dest (Scott Campbell).
- You can now use the attribute &rotate_size for file objects to
specify the maximum file size in bytes (Robin Sommer). If the limit
is reached, the file is rotated similiarly as is already done with
&rotate (which, for consistency, has been renamed to &rotate_interval).
For both &rotate_size and &rotate_interval, when they trigger they
now generate events (rotate_size and rotate_interval, respectively;
each takes the file as the sole argument) rather than invoking
&postprocessor, which has been removed.
There's also a new variable log_rotate_size to set a global size maximum.
Related to log rotation are the following new built-in functions:
rotate_file(f: file) closes the file, moves it to a temporary
name, and opens a new one. It returns the new "rotate_info"
record, which gives the temporary name and the open/close times.
rotate_file_by_name(s: string): similar, but call by the name
of the file rather than a Bro script value. This is needed
because some files are not represented by file objects but need
to be rotated nevertheless (most importantly, the tcpdump save
file and the dump files for dump_current_packet()). This function
rotates the file with the given name.
Finally, you can load the new policy script rotate-logs.bro to
get default behavior of rotating all log files every hour.
- The new "@unload <script>" directive specifies that future @load's of
<script> should be skipped. This is useful for overriding analyzers
loaded by scripts that pull in a bunch of analysis. For example,
@unload ntp
@load mt
would load all of the "mt" analyzers *except* ntp.bro.
- The new built-in function get_file_name(f: file): string returns
the filename associated with a file (John McNicholas).
- The new built-in function get_contents_file(id: conn_id, direction: count)
returns the contents file (set using set_contents_file()) for the given
direction (John McNicholas).
- The new built-ins time_to_double() and double_to_time() convert between
double values and time values (Robin Sommer). The new built-in floor()
returns the floor of a double value; this returned value is also a double.
Thus, floor(-3.4) returns -4.0.
- Support for sending packets between Bro's (Robin Sommer).
- Bro now has a geneal mechanism internal for traversing policy scripts
(Umesh Shankar). Various script analyses can be specified using the
new -z flag.
Currently, the one supported form of analysis is "-z notice", which
prints all of the different types of notices that the script you've
loaded can generate. For example, "bro -z notice ftp" will generate:
Found NOTICE: BackscatterSeen
Found NOTICE: FTP_PrivPort
Found NOTICE: FTP_BadPort
Found NOTICE: PortScan
Found NOTICE: FTP_ExcessiveFilename
Found NOTICE: ScanSummary
Found NOTICE: AddressDropped
Found NOTICE: DroppedPackets
Found NOTICE: SensitiveConnection
Found NOTICE: FTP_UnexpectedConn
Found NOTICE: SSH_Overflow
Found NOTICE: FTP_Sensitive
Found NOTICE: TerminatingConnection
Found NOTICE: PasswordGuessing
Found NOTICE: AddressDropIgnored
Found NOTICE: AddressScan
- The signature rule language now supports an "active" keyword,
which can be set to "true" or "false", with the latter turning
off the rule (Roger Winslow). If set to false the signature will
not be loaded into the rule matcher, otherwise it is.
- The signature rule language now supports meta data of the form
".MMM<whitespace>XXX", where MMM is arbitrary text which makes up the name
of the meta data option and where XXX is arbitrary text up to the end
of the current line (Roger Winslow). The intent is that some forms of
meta-data will be regularized/standardized in the future - information
such as date modifed, category/class, weighting, etc. For now, it
provides a way to annotate rules with nominally more structure than just
using comments (though it is currently treated the same, i.e., everything
is ignored).
- The following meta data option names are now reserved: .version, .revision,
.version-date, .revision-date, .date-created, .location
- The new enumerated type "transport_proto" is used to specify different
types of transport protocols: "tcp", "udp", "icmp", and "unknown_transport".
Associated with it are new built-in functions: is_udp_prot(), is_icmp_port(),
get_conn_transport_proto, and get_port_transport_proto. The latter two
map a given connection and a given "port" value to their corresponding
transport_proto value.
- A bunch of tuning (regular expressions for sensitive login sessions,
scan detection thresholds, forbidden/hot usernames and filenames,
sensitive URIs, "weird" actions) have been incorporated from
operational configurations.
- Serious bugs in managing large numbers of files fixed.
- A serious bug with negative DNS TTL settings (and, more generally,
with negative timer values) fixed.
- The traditional connection logging format is no longer supported.
- The SMTP analyzer's state machine processing has been modified to
correctly deal with clients that (incorrectly) pipeline their commands
(Ruoming Pang).
- A bug fixed in detecting SMTP relays for connections w/o message bodies
(Ruoming Pang).
- A bunch of bugs in recording connection summaries for UDP flows
have been fixed.
- A new script module, passwords.bro, generates PasswordExposed notices
for activity (currently just rlogin/telnet logins) that expose passwords.
- A new script module, file-flush.bro, can be loaded to cause all log
files to be flushed every file_flush_interval (default: 10) seconds.
This is handy if you like to watch the files in real time.
- Zone transfers now generate a ZoneTransfer notice, unless the host
making the request is in DNS::zone_transfers_okay.
- Bro's DNS cache (generated using -P and accessed using -F) is
now kept in the .state/ subdirectory rather than in the user's
home directory (Roger Winslow).
- Some changes to remote propagation of events/values and detection of
state inconsistencies (Robin Sommer).
- A fix for avoiding delays on low-volume links for some systems for
which it can take a long time to fill up the pcap buffer, and pcap doesn't
return partial buffers (Robin Sommer).
- A bug in table expiration timers has been fixed (Robin Sommer).
- A bug in comparing subnets has been fixed.
- A bug in using a non-constant value for a &write_expire attribute
has been fixed.
- A bug in using CONTENTS_BOTH for writing reassembled streams to
files has been fixed (John McNicholas).
- A subtle but potentially damaging bug in fragment reassembly has
been fixed.
- A bug with using local variables of vector types has been fixed.
- A bug with comparing strings has been fixed.
- Bro no longer generates the RST_with_data "weird", as with modern
stacks it's no longer any sort of strange occurrence.
- Related to this, the signature rule matcher no longer matches
against the payload of RST packets. (Note, this is an imcompatibility
with Snort.)
- Portmapper mappings are now written in the connection log in
alphabetical order.
- The event engine variable frag_timeout now defaults to 5 minutes if you
load frag.bro, and is accessed via redef rather than by defining the
global directly.
- The interval that signatures.bro waits for until generating a signature
summary can now be set using the new script variable sig_summary_interval,
and a bug in generating the summaries has been fixed (Robin Sommer).
- The new script peer-status.bro generates periodic "update" events regarding
a remote peer's status (Robin Sommer). These take the form:
type peer_status: record {
res: bro_resources;
stats: net_stats;
current_time: time;
cpu: double; # average CPU load since last update
default_filter: string; # default capture filter
};
- The bro_resources record returned by resource_usage() now includes
three additional fields, $version (the version of Bro), $debug
(T if Bro was compiled with debugging information), and $start_time
(the time Bro began executing - clock time, not network time).
- The new built-in function same_object(o1: any, o2: any): bool
returns true if its arguments refer to the same object, false
otherwise. This can be useful for comparing tables, for example
in calls to table element expiration functions.
- The new built-in function bro_is_terminating(): bool returns true if
Bro is done reading from its network input source(s) and is now
in its final termination cleanup (Robin Sommer).
- A new built-in strftime() formats a timestamp, returning a string
(Robin Sommer).
- A new built-in file_size() returns the size of the file with a
given name (Robin Sommer - note: *not* a Bro file value).
- A potential deadlock with inter-Bro communication has been fixed
(Robin Sommer).
- Bro now always forks a copy of itself when executing, as this
can save considerable memory when using inter-Bro communication
(Robin Sommer).
- The Bro interconnection protocol now includes explicit handshaking
during session establishment to mark that a peer is ready (Robin Sommer).
Implementing this includes a change in the wire protocol that is
incompatible with the protocol used in the past.
- The TCP inactivity timer is now started whenever a connection
transitions from a pre-establishment state (including "inactive")
to some sort of established state (Robin Sommer). Prior to this
fix, connections for which a proper SYN handshake was not seen would
not be timed out as inactive.
- The --disable-openssl configure option has been removed; now
the only option is --with-openssl, and --with-openssl=no disables
use of OpenSSL (Gregor Maier).
- A bug in invoking &expire_func functions has been fixed (Robin Sommer).
- A bug in logfile rotation has been fixed (Robin Sommer).
- A bug in recognizing negative floating point values has been fixed.
- worm.bro now suppresses the default signature action for worms
it knows about, since it generates events for them (Robin Sommer).
The list of worms detected via signatures now includes Bagle-BC.
- Signatures for known worms are now skipped when doing signature
summaries and scan detection, if worm.bro is loaded (Robin Sommer).
- request_remote_events and request_remote_sync now implicitly
do set_accept_state, too.
- Better error handling for SSL connections (Robin Sommer).
- Bug fixed which caused diagnostic messages to be lost when using
inter-Bro communication (Robin Sommer).
- gcc 3.4 portability fixes (Brian Lindauer).
- Solaris portability fixes (Robin Sommer).
- The Bro distribution now includes and uses its own version of libpcap
for portability reasons (Jason Lee).
- Some minor bug fixes to handling of tcpdump save files (Robin Sommer).
- Detection added for a (now quite old) SSHv1 overflow attack.
- A bug in skipping processing of connections for large chunks of
data has been fixed (Chema Gonzalez).
- Some memory leaks fixed (Robin Sommer).
- fmt()'s "%d" format now accepts values of enum types.
0.9a7 Mon Nov 1 13:21:05 PST 2004
- New terminology:
o We've found that the term "log" has been too overloaded,
sometimes meaning "something to record for audit purposes" and
other times meaning "something worthy of getting the operator's
attention right now, for example via syslog".
We are now using "log" to only refer to the first of these,
and refer to the second as "alarm".
o We've found that "alert" (and the ALERT() function, etc) is
likewise confusing. Some expect it to mean something alarm-worthy
(to use the new name from the previous item), while others
expect it to mean the output from a sensor, which might not
be worth getting the operator's attention.
We are now using "notice" to refer to what had previously
been called "alert". So, for example, rather than call ALERT()
you call NOTICE() to enter something into the I've-seen-something-
maybe-it's-worth-an-alarm framework. The first field associated
with such a call was $alert, which is now $note. These notifications
are logged to notice.$BRO_LOG_SUFFIX rather than "alert."...
These changes show up in many places in the policy scripts. Some
globals log_XXX are now alarm_XXX or notice_XXX. However, uses of
"log" that refer to "something to record for audit purposes" remain
using that name.
- Bro now uses the "automake" suite of tools for its configuration
(Jason Lee). This includes major reworking of its Makefile's
and configuration/installation scripts.
- Bro now flags event handlers that are never invoked (Umesh Shankar).
This catches typos like:
event bro_initt() { ... }
- The scripts directory now includes scripts for generating nightly
reports (Roger Winslow). Note that these are not presently a supported
part of the public Bro distribution, but will become so in the
not-too-distant future.
- policy/brolite.bro contains a first version of the "Bro Lite" configuration
(Scott Campbell, Roger Winslow, et al). This also is not presently
supported for the public Bro distribution, but will become so.
- Major overhaul of the internal serialization framework (Robin Sommer).
Some more changes are forthcoming, though not on this scale.
- Changed packet capture/drop statistics to (1) explicitly track received
packets, (2) work better under Linux, (3) report link counts when available
(Robin Sommer). The net_stats record field "interface_drops" has been
renamed "pkts_link". It reports the number of packets captured by the
NIC (if available), rather than the number dropped by the NIC (which never
actually worked, anyway).
- The DNS analysis has been extended to deal with TSIG and to better deal
with EDNS0 records, and its overall structure reworked somewhat
(Scott Campbell & Roger Winslow). The scripts in dns.bro are now
inside "module DNS" scope.
- Improved logging of ICMP flows (Scott Campbell). This includes the
addition of a "len" field in icmp_conn. Some significant additional
changes/improvements to ICMP processing will be coming very soon.
- The Bro README and "quick start" documentation has been updated
(Brian Tierney and Jim Rothfuss).
- Some significant state-holding problems in the presence of packet filtering
or packet drops have been fixed.
- You can now instruct the signature engine to constrain the number of
regular expressions it groups together into a single matcher using the
new script variable "sig_max_group_size" (Robin Sommer). This can result
in significant memory savings for large sets of signatures. It can
increase the CPU processing required; however, in our testing so far
this does not appear to be the case.
- The signature engine now provides better location information in its
error messages (Robin Sommer).
- statistics.bro has been renamed profiling.bro. Along with this change,
the following script variables have been renamed:
do_statistics => do_profiling
statistics_file => profiling_file
statistics_interval => profiling_interval
- A new script, stats.bro, can be included to generate light-weight
running statistics on memory and CPU use.
- A new script, print-sig-states.bro, can be included to generate
periodic dumps of signature-matching statistics (Robin Sommer).
- The connect_clear()/connect_ssl() built-ins have been replaced
by a single connect() function (Robin Sommer).
- Remote Bro's (more generally, remote event sources) are now represented
in policy scripts using the new "event_peer" type rather than as an
addr/port pair, and get_event_source() has been renamed get_event_peer()
(Robin Sommer). The new function get_local_event_peer() returns an
event_peer corresponding to the local Bro.
- The new script remote-print.bro can be used to print all events
received from remote sources (Robin Sommer).
- When you call send_state(), an event finished_send_state() is now
generated when all of the state has been sent (Robin Sommer).
- If you define a handler for the new print_hook() event, then it
will be invoked every time a "print" statement executes (Robin Sommer).
- The -g flag no longer takes a directory as an argument, but always
uses the .state directory (Robin Sommer).
- The new -I flag prints the value of a given identifier and exits
(Robin Sommer). This flag may go away in the future. For now,
its role is that it prints timestamps of things like table elements,
which are no longer printed by a regular "print" statement.
- If a connection proceeds as SYN ->, <- RST, RST ->, this is now
reported as a regular rejected ("REJ") connection, rather than
one reset by the originator ("RSTO").
- The TCP analyzer now attempts to detect connections for which
one side's packets are being filtered out. It does this using
two C++ variables (not presently accessible at the script level),
const int max_initial_window = 4096;
const int max_above_hole_without_any_acks = 4096;
If more than max_initial_window data has been sent by one side and no
ACKs have been seen by the other side, then it's assumed that no ACKs
will ever show up, and the buffer for the data should be reclaimed
immediately after it's reassembled. max_above_hole_without_any_acks
plays a similar role, but for the case when measurement drops have
lead to Bro holding data for a sender for which it will never see
an ACK (which would be an "ACK above hole"), so it should give up
on buffering it.
- A portability bug in Bro's parser code has been fixed (Jason Lee).
- STARTTLS for SMTP now causes the SMTP session to be ignored, rather
than generating a slew of error messages because the session becomes
unparseable.
- Bro's "watchdog" handler is now less prone to crashing due to
using non-reentrant library calls (Jason Lee).
- Better error message when reading an input file encounters an error.
- adtrace now prints its usage when invoked without the required
arguments (Jason Lee).
- A bug in using recursive script functions has been fixed.
- A bunch of whitespace/code layout tweaks.
0.9a4 Wed Sep 8 17:33:54 PDT 2004
- The directory structure of the Bro distribution has changed (Jason Lee).
The source code is now in a subdirectory, src/, and the scripts
snort2bro (and snort2bro.cfg) and make-ftp-safe-vocabulary.awk have
been moved into scripts/.
- "make install" has been revamped (Jason Lee).
- The format of the alert log file has changed. Fields in it are
colon-separated. THIS WILL LIKELY CHANGE SOON.
- The policy for formatting signature matches has been revamped,
including colon-separated fields in the signature log file
(Roger Winslow). THIS WILL LIKELY CHANGE SOON.
- The BRO_ID environment variable has been renamed BRO_LOG_SUFFIX.
- A new flag, -e, lets you specify Bro code to execute via the command
line (Christian Kreibich). So, for example,
bro -r mytrace.tcpdump -e 'redef traditional_conn_format = T' tcp
will run tcp.bro on the trace "mytrace.tcpdump", but with
traditional_conn_format redefined to be true. Note that statements
have an implicit ';' added to them for convenience.
- A new signature alert, "MultipleSigResponders", is generated if a
host triggers the same signature on multiple responders.
- Bro now supports "packet profiling", which provides fairly fine-grained
statistics on number of packets processed, volume, elapsed real/user/system
time, and change in memory consumption (Holger Dreger). Three variables
control the output. The double pkt_profile_freq controls the frequency
of output. The units in which it's interpreted depends on the setting
of the pkt_profile_mode variable (which is of type pkt_profile_modes,
an enum). A value of PKT_PROFILE_MODE_SECS means that statistics
are generated every pkt_profile_freq seconds; PKT_PROFILE_MODE_PKTS
means every pkt_profile_freq packets; and PKT_PROFILE_MODE_BYTES, every
pkt_profile_freq bytes. The default (PKT_PROFILE_MODE_NONE) means
to not generate packet profiling.
Packet profiling is written to the new log file, pkt_profile_file.
If you "@load pkt-profile", you can turn on packet profiling using
some handy defaults.
- statistics.bro now reports on how many TCP connections are in
<originator-state, responder-state> for the different TCP endpoint
states (SYN sent, SYN ack'd, connection established, etc.).
Contributed by Holger Dreger.
- tcp_content_delivery_ports_{orig,resp} are now table's of bool rather
than set's (Ruoming Pang). The semantics are that if you have a
tcp_contents event handler, then if the orig/resp port is in the given
table *and the yield value is T*, then the event will be invoked. This
allows you to now explicitly skip over some ports.
- The processing of default values in tables has been changed internally
(Ruoming Pang). It's possible this has introduced some subtle bugs
(as some of these came up during testing).
- A serious bug in Base64 processing has been fixed (Ruoming Pang).
- The NetBIOS and SMB analyzers have been updated in minor ways
(Ruoming Pang).
- statistics.bro now reports a "lag" figure indicating the elasped
time between the last expired timer's target expiration time and
the current packet timestamp (Robin Sommer). Lag can grow if Bro
is getting behind in timer expiration due to the setting of
max_timer_expires.
- Bro's default filter is now "tcp or udp or icmp" rather than
"tcp or udp".
- alert_info records now have an optional port associated with them
(for example, to be used to describe scan activity).
- A bug has been fixed in which deleting a table element with an
associated timer could crash Bro (Robin Sommer).
- A bug that would cause a crash for malformed EPASV directives
has been fixed (Robin Sommer).
- A bug with inactivity timeouts not being generated for partial
connections has been fixed (Robin Sommer).
- A bug in synflood.bro has been fixed (Robin Sommer).
- Some tuning adjustments to incremental expiration of table entries
(Robin Sommer).
- Improved portability to Darwin (Christian Kreibich).
- alert_info records now have additional optional fields, "iconn"
(associated ICMP connection), "dst" (destination address), and
"p" (associated port). The source_is_responder fields has been
removed.
- The default packet filter now includes "icmp".
- Some memory allocation/free mismatches & minor leaks (Robin Sommer).
- Minor tweaks to ssl.bro (Robin Sommer).
- Bro now supports "null" link layers (Christian Kreibich).
- aux/adtrace contains a program that spits out MAC/IP information
from traces (Holger Dreger).
- A bug in handling malformed RPCs has been fixed (Scott Campbell).
- The formatting of "weird" messages that have additional parameters
has been changed to be more regularized with other "weird" messages.
- The new "weird" type "base64_illegal_encoding" takes the place of
some previously unstructured Base64 "weird" errors.
- A tweak to ftp.bro will give it slightly more consistent results
for some forms of unusual traffic.
0.9a3 Wed Jul 7 22:06:26 PDT 2004
- Improved ICMP processing, including scan detection (Scott Campbell).
- ICMP "connections" are now considered unidirectional.
- Fixed broken VLAN support (integration of original patch was incomplete).
- Fixed a bug in erroneously generating additional "ContentGap"
alerts after an initial one.
- Connection durations are now always reported as floating-point decimal,
never in exponential notation.
- Removed unused time parameter from a bunch of internal calls.
- Fixed some compilation warnings.
- "make clean" now removes generated policy/*.bif.bro files (Christian
Kreibich).
0.9a2 Fri Jun 11 00:07:04 PDT 2004
- NetBIOS analysis has been extended with a CIFS/SMB analyzer (Ruoming Pang).
While this is incomplete, it has many important elements. The corresponding
events:
smb_message(c: connection, is_orig: bool, cmd: string, msg: string)
smb_com_tree_connect_andx(c: connection, path: string, service: string)
smb_com_nt_create_andx(c: connection, name: string)
smb_com_transaction(c: connection, is_orig: bool, subcmd: count,
name: string, data: string)
smb_com_transaction2(c: connection, is_orig: bool, subcmd: count,
name: string, data: string)
smb_com_read_andx(c: connection, is_orig: bool, data: string)
smb_com_write_andx(c: connection, is_orig: bool, data: string)
This analyzer is still experimental.
- Greater support for vectors (Umesh Shankar), much of it taken from
the 'S' language.
You can use a boolean vector as an index into another vector (providing
both are the same length) and each 'T' value in the index extracts the
corresponding element from the indexed vector. For example, "x[x > 3]"
returns a vector whose elements are those elements of x that are greater
than, while if y is a vector of the same length as x then "y[x > 3]"
extracts those elements of y that have the same position as the elements
in x that are greater than 3.
You can also use an arithmetic vector to index another vector. Each
element present in the index is extracted. So, for example:
global a: vector of count;
global b: vector of string;
a[1] = 3;
a[2] = 3;
a[3] = 1;
b[1] = "foo";
b[2] = "bar";
b[3] = "bletch";
print b[a];
prints:
[bletch, bletch, foo]
- The new built-ins any_set() and all_set() return true if for a given
boolean vector any element is true or all of the elements is true
(Umesh Shankar). So, for example, "any_set(x < 0)" returns T if
an element of x is less than zero.
- The new built-in sort() takes a vector as an argument and sorts it
*in place* (Umesh Shankar). (The in-place semantics may change in the
future.) An optional second argument can be used to specify a
function to call for comparing elements, and is required for non-arithmetic
vectors. For example, the following could be used to sort a vector
of strings based solely on the length of the strings:
function string_compare(a : string, b: string): int
{
local la = byte_len(a);
local lb = byte_len(b);
return (la < lb) ? -1 : ((lb > la) ? 1 : 0);
}
- The new function order() has the same calling sequence as sort(),
but instead of returning (and altering in place) the sorted vector,
it returns a "vector of count" giving the *indices* that if used
to index the vector will return it sorted. So, for example,
given two vectors x and y of the same length (but not necessarily
of the same type),
local x_sort_indices = order(x);
x = x[x_sort_indices];
y = y[x_sort_indices];
will assign x to a sorted version of itself and also rearrange y such
that elements of y that were paired with elements of x originally
remain paired after the sorting.
- The ICMP analyzer now has a general notion of "context", i.e., the packet
associated with ICMP status messages such as Unreachable or Time Exceeded
(Ruoming Pang). This changes the parameters to the icmp_unreachable
event. A new event, icmp_time_exceed, is now also available.
- The tcp_segment even has been replaced by a pair of new events (Ruoming Pang):
tcp_packet(c: connection, is_orig: bool, flags: string, seq: count,
ack: count, len: count, payload: string)
is invoked for each TCP packet. "flags" is a string containing "SFAPU"
for the SYN/FIN/etc TCP flags.
tcp_contents(c: connection, is_orig: bool, seq: count,
contents: string)
is invoked for each chunk of the byte-stream that has been reassembled
in sequence, providing it satisfies tcp_content_delivery_ports_{orig,resp},
per the next item.
- You can specify the set of ports for which contents should be reassembled
for the originator (responder, respectively) stream using the new sets
tcp_content_delivery_ports_{orig,resp} (Ruoming Pang). This can be
useful for user-level stream analysis for protocols not known to Bro's
event engine. These controls may change to a "table of bool" in the future,
in order to support an &default attribute.
- New built-in functions (Ruoming Pang):
function interval_to_double(i: interval): double
Converts a value of type "interval" to "double".
function write_file(f: file, data: string): bool
Writes the given string to the given file, returning
F on error.
function is_ascii(str: string): bool
Returns T if the given string consists entirely of
ASCII characters (i.e., in the range 0..127).
function sqrt(x: double): double
Returns the square-root of x, or -1 and a run-time error
if x is < 0.
function uuid_to_string(uuid: string): string
Takes a UUID and returns its string representation, where
UUID = Universal Unique Identifier as defined per
http://www.opengroup.org/onlinepubs/9629399/apdxa.htm#tagcjh_20
function string_to_ascii_hex(s: string): string
Returns the ASCII hex representation of the given string.
For example, string_to_ascii_hex("foo") returns "666f6f".
function match_pattern(s: string, p:pattern): pattern_match_result
Matches the given pattern against the given string, returning
a record with three fields:
matched: bool; # T if a match was found, F otherwise
str: string; # portion of string that first matched
off: count; # 1-based offset where match starts
For example,
match_pattern("foobar", /o*[a-k]/)
returns
[matched=T, str=f, off=1]
because the *first* match is for zero o's followed by an [a-k],
while
match_pattern("foobar", /o+[a-k]/)
returns
[matched=T, str=oob, off=2]
- Functions that terminate without returning a value when they were declared
to do so now generate a run-time warning (Christian Kreibich). Functions
in the standard set of policy scripts that did this have been fixed.
- The new event non_dns_request(c: connection, msg: string) is generate
to make the contents of malformed DNS requests available for analysis,
with the assumption that these are actually some other protocol entirely
(Ruoming Pang).
- If you redef truncate_http_URI to have a value >= 0, then any HTTP
URIs generated by the event engine will be truncated to this length
(Ruoming Pang). This can be convenient when analyzing traffic that
generates huge URIs (as do some automated attacks).
- "SEARCH" is now recognized as a standard HTTP method (Ruoming Pang).
- The new event connection_EOF(c: connection, is_orig: bool) is generated
when one side of a connection closes (Ruoming Pang).
- synflood.bro and the corresponding event engine internals now works
in terms of probabilities (0.0-1.0) instead of percentages (0-100)
(Robin Sommer). The script has had several tweaks, including using
new_connection() rather than connection_attempt(), which gives it
quicker response and broader coverage (it'll detect non-TCP flooding,
too, so "synflood" is now a bit of a misnomer), at the cost of perhaps
more CPU load.
- A signature for Witty has been added to policy/sigs/worm.sig (Ruoming Pang).
- Makefile now has a "test" target. Currently this only works for internal
development (we haven't put together a public test suite yet; that will
take some time due to the need to make sure no sensitive information leaks).
- The built-in function generator now knows about "double" as a built-in type
(Ruoming Pang).
- Some generated files have been removed from the Bro distribution since
they're redundant (Ruoming Pang).
- A bug has been fixed in which contents files might not be correctly
written upon termination of Bro (Ruoming Pang).
- A bug has been fixed in which UDP connections didn't generate
new_connection events (Ruoming Pang).
- Support for the Linux "cooked capture" link layer (Ruoming Pang).
- BPF support has been factored into a separate class, BPF_Program, which
makes for easier portability (Christian Kreibich).
- A serious low-level Dictionary bug has been fixed (Christian Kreibich).
- A bug that could cause Bro to crash if it receives an event from another
Bro that it isn't able to process has been fixed (Christian Kreibich).
- A bug in set file descriptors non-blocking has been fixed
(Christian Kreibich).
- A bug that could cause some error messages to generate crashes has
been fixed.
- The global skip_event_tcp_option has been removed.
0.9a1 Mon Jun 7 01:33:00 PDT 2004
- 0.8a85 is the new STABLE release, to be updated only for bug (and
portability) fixes. 0.9a1 is the new CURRENT release, which will have
both bug fixes and new functionality.
- Support for FTP EPRT, EPSV (Holger Dreger).
- Change to timer management to recover memory more quickly (Robin Sommer).
- Tweaks to eliminate a number of compiler warnings (Robin Sommer).
- Statistics now report number of connections terminated due
to inactivity (Robin Sommer).
- New Makefile target, pub-tar-no-doc builds a public tarball but
without the (large) doc/ subdirectory.
- Bug fix for identd requests with illegal port numbers.
- The example of a scanning source to skip in scan.bro has been changed
from an AltaVista robot (now obsolete) to a Google robot.
- Some previously fatal internal errors have now been turned into
"internal warnings", which Bro is able to continue operating
in their presence.
0.8a84 Wed May 26 23:33:39 PDT 2004
- autoconf tweaks for portability to Darwin (Christian Kreibich).
- Fixed subtle bug in chunked-IO reads (Christian Kreibich).
- Fixed bug for the "discarder" framework in which specifying an
IP discarder would cause other TCP/UDP/ICMP discarders not being
invoked (Christian Kreibich).
- Fatal bug in signature matching fixed (Robin Sommer).
- Missing member variable initialization fixed (Robin Sommer). (Needed for
compilation with new versions of gcc.)
- Makefile bug for "make install" fixed.
- Fixed bug that could lead to Bro crashing if an SMTP session
had data sent after a RST.
- Removed some out-of-date SMTP analysis warning messages.
0.8a82 Tue Apr 27 11:53:24 PDT 2004
- Fixed inactivity timer loop when a packet arrives exactly when
the timer is set to expire.
0.8a81 Mon Apr 26 22:46:37 PDT 2004
- A bunch of memory leaks fixed (Chema Gonzalez).
- A new HTTP analyzer variable, content_truncation_limit, controls
how much of an HTTP request/reply contents element (i.e., what's
passed to http_entity_data - this is *not* the entire content, but
the next chunk's worth) is logged in the log file. It defaults
to 40 bytes. Setting it to 0 means "log all of it".
- Fix to avoid crashing for malformed RPC requests.
- Improved OpenSSL auto-configuration (Robin Sommer).
- Fix for compiling without OpenSSL.
- A new built-in, double_to_count(), converts a value of type "double"
to the corresponding "count" (Chema Gonzalez). We should probably
add floor(), ceil(), etc.
- Parameterization of trw.bro tweaked (Jaeyeon Jung).
0.8a79 Wed Mar 24 22:02:53 PST 2004
- Bro now has an SSL analyzer, written by Michael Kuhn and Benedikt Ostermaier,
with further development by Scott Campbell. It generates the following
events:
event process_X509_extensions(c: connection, ex: X509_extension)
event ssl_X509_error(c: connection, err: int, err_string: string)
event ssl_certificate(c: connection, cert: X509, is_server: bool)
event ssl_certificate_seen(c: connection, is_server: bool)
event ssl_conn_alert(c: connection, version: count, level: count,
description: count)
event ssl_conn_attempt(c: connection, version: int)
event ssl_conn_established(c: connection, version: int,
cipher_suite: count)
event ssl_conn_reused(c: connection, session_id: sessionID)
event ssl_conn_server_reply(c: connection, version: int)
event ssl_conn_weak(name: string, c: connection)
event ssl_session_insertion(c: connection, id: sessionID)
Note, it still has a lot of rough edges; particularly, handling
non-conformant input. It also generates unnecessary ContentGap alerts
due to the way it runs multiple analyzers (SSLv2 and SSLv3) on a single
connection. This will be fixed in the fairly near-term future.
- The manual has been updated with chapters on signatures (Robin Sommer)
and using the interactive debugger (Umesh Shankar), along with a partial
description of the new SSL analyzer (Michael Kuhn and Benedikt Ostermaier)
and a number of updates to the documentation of built-in functions (Umesh
Shankar), though this latter is still not complete since Umesh actually
contributed this quite a while ago.
- Ruoming Pang has contributed a crude analyzer for DCE/RPC (used for Windows).
It generates simple dce_rpc_request and dce_rpc_reply events. It should
not be considered stable.
- The traditional connection logging format (traditional_conn_format)
is no longer the default. The 0.8a70 release notes stated that this
was the case but this time it really is :-).
- An experimental "vector" type has been added (Umesh Shankar). A vector
is an aggregate type. For example:
local suspect_hosts: vector of addr;
You can randomly access elements starting with the first as 1, e.g.,
suspect_hosts[1] = 10.0.0.8;
and can also add elements at later postions even if there are gaps:
suspect_hosts[31337] = 10.0.0.9;
*The semantics and capabilities of vectors will be changing considerably.*
- Umesh Shankar has developed a framework for generating IDMEF messages.
Currently it needs a modified version of libidmef, which is not included
in this distribution. Contact me or Umesh if you want a copy.
- A new attribute &synchronized causes the given global variable to
be *synchronized* across concurrent instances of Bro (which are
intercommunicating via remote.bro). Any change made by one of them
to the variable will be reflected (soon after) in the copy at
the others. A new event remote_state_inconsistency is generated
if two instances both change the value before they're synchronized.
(Contributed by Robin Sommer.)
- trw.bro implements a new scan detection algorithm, Threshold Random Walk
(Jaeyeon Jung). It's described in an upcoming IEEE S&P symposium paper.
The analyzer generates two events:
TRWAddressScan, # source flagged as scanner by TRW algorithm
TRWScanSummary, # summary of scanning activities reported by TRW
TRW is generally much more sensitive than Bro's regular detection algorithm.
- vlan.bro provides support for VLAN encapsulation. More generally, Bro
now has support for any sort of constant-offset encapsulation (Vinod
Yegneswaran). You specify the header size by redef'ing encap_hdr_size.
You can also redef tunnel_port to be a UDP port which Bro treats as being
the encapsulation (in the packet stream - not addressed to it) rather
than all traffic.
- If you turn on statistics (i.e., via @load statistics) and also redef
segment_profiling to T, then Bro will generate to the statistics file
a trace of its "segment" processing. A segment is a unit of internal
execution. Profiles look like:
1058517499.615430 segment-processing-packet dt=0.000013 dmem=0
1058517499.615430 segment-draining-events dt=0.000012 dmem=0
1058517499.615671 segment-expiring-timers dt=0.000010 dmem=0
1058517499.615671 segment-processing-packet dt=0.000010 dmem=0
1058517499.615671 segment-draining-events dt=0.000012 dmem=0
1058517499.615671 segment-policy/conn.bro:282 dt=0.000011 dmem=0
1058517499.615671 segment-policy/conn.bro:253 dt=0.000012 dmem=0
The first line states that at the given (packet) timestamp, the event
engine processed a packet, taking 13 usec of CPU time to do so, and
not consuming any memory (from the kernel's perspective; this is *not*
fine-grained memory consumption). The next lines indicate 12 usec were
spent draining events and 10 usec expiring timers. The last two lines
indicate that the functions at lines 282 and 253 in conn.bro were
executed, requiring 11 usec and 12 usec, respectively.
Note #1: timings are just what rusage() reports, so not necessarily
very accurate for small times.
Note #2: there's a bug in tracking function line numbers that hasn't
been ferreted out yet, so they're only roughly correct.
- The inactivity_timeout global has been split into tcp_inactivity_timeout/
udp_inactivity_timeout/icmp_inactivity_timeout (Robin Sommer). Using
this, the default inactivity timeouts for UDP and ICMP have been changed
from "no timeout" to 10 seconds. This is needed because otherwise
analyzing a stream of UDP or ICMP traffic generally gobbles up memory
quickly and never recovers it; and there's seems little point in trying
to consolidate long-lived-but-often-inactive UDP/ICMP streams.
- The new policy script cpu-adapt.bro is an extension to load-levels.bro
(see CHANGES for 0.8a37 below) to adapt the packet filter based on the
current CPU load. If the load is below cpu_lower_limit (default 40%),
the load-level is decreased. If it's above cpu_upper_limit (default
90%), it's increased. (Robin Sommer)
- The new policy script hand-over.bro can be used for a new running
instance of Bro to smoothly take over operation from an old instance,
i.e., it implements hand-over of state between two Bro instances when
checkpointing (Robin Sommer). First, all persistent data (i.e. variables
declared &persistent and connections for which make_connection_persistent()
has been called) is transferred from the old instance to the new instance.
Then the old instance terminates itself and the new one starts processing.
The host from which we want to take over the state has to be added to
remote_peers_clear (or remote_peers_ssl), setting hand_over to T. The
host which is allowed to perform a hand-over with us has to be added
with a port of 0/tcp and hand_over=T. An example for a handover between
two instances on the same machine:
@load hand-over
redef remote_peers_clear += {
[127.0.0.1, 47756/tcp] = [$hand_over = T],
[127.0.0.1, 0/tcp] = [$hand_over = T]
};
(This interface may be subject to change in the future.)
- New script functions (Robin Sommer):
function terminate()
Terminates Bro via SIGTERM.
function dump_config()
Dumps Bro's full configuration into state_dir (one file per
variable/type/function, etc.)
function send_state(ip: addr, p: port)
Send all of persistent state to the remote host.
function set_accept_state(ip: addr, p: port, accept: bool)
If accept is true, state sent by the given host will be
accepted (default: false)
function make_connection_persistent(c: connection)
Declare the given connection state to be persistent (i.e.
to be saved upon termination and exchanged by send_state).
checkpoint.bro uses this to declare some services to be
persistent by default.
function is_local_interface(ip: addr): bool
Returns true if the given address is assigned to a local interface.
- Printing of sets and tables now includes timestamps indicating when the
element was added (Robin Sommer):
ID did_ssh_version = {
[129.187.20.9, F] = 1 @11/01-15:55,
[212.144.77.26, T] = 2 @11/01-15:55,
[141.84.116.26, T] = 10 @11/01-15:55,
[217.232.245.249, T] = 1 @11/01-15:55,
[217.235.217.149, T] = 1 @11/01-15:55,
[129.187.39.13, F] = 2 @11/01-15:55,
[129.187.208.139, F] = 1 @11/01-15:55,
}
The format may change in the future, and will probably be made an option.
- Similarly, you can print functions to get both a timestamp of the last
time the given block was executed and a count of how often (Robin Sommer):
ID record_connection = record_connection
(@11/01-16:03 #6549)
{
id = c$id;
local_init = is_local_addr(id$orig_h);
local_addr = local_init ? id$orig_h : id$resp_h;
remote_addr = local_init ? id$resp_h : id$orig_h;
flags = local_init ? "L" : "";
if (remote_addr in neighbor_nets)
(@<never> #0)
flags = cat(flags, "U");
if ("" == flags)
(@11/01-16:03 #2110)
flags = "X";
is_tcp = is_tcp_port(id$orig_p);
;
if (is_tcp)
(@11/01-16:03 #6549)
{
if (c$orig$state in conn_closed || c$resp$state in conn_closed
)
(@11/01-16:03 #4739)
duration = fmt("%.6g", c$duration);
else
(@11/01-16:03 #1810)
duration = "?";
[...]
- You can now specify numbers using hex constants, e.g., 0xabcd = 43981
(Michael Kuhn and Benedikt Ostermaier).
- A new function, join_string_array(sep: string, a: string_array) concatenates
strings in 'a' and inserts 'sep' between every two adjacent elements
(Ruoming Pang). E.g., join_string_array("", {"a", "b", "c"}) returns
"a b c", and join_string_array("", a) is the same as cat_string_array(a).
- checkpoint.bro now makes some services persistent by default
(Robin Sommer).
- The new_packet event now includes both the associated connection
and a pkt_hdr describing the packet (Robin Sommer).
- The functions functions connect_ssl() and connect_clear() have been replaced
by a single connect() function taking an additional parameter to
differentiate the types (Robin Sommer).
- The new function stop_listening() unbinds the listen port (Robin Sommer).
- A new flag packet_filter_default says whether the Bro-level packet-filter
will by default accept all or reject everything (Robin Sommer).
- Function calls can now be sent to remote Bro's, though there isn't yet
an interface for accessing this from the script level (Robin Sommer).
- Bro now has an generalized internal framework for serializing objects
and monitoring access to state (Robin Sommer).
- Better memory allocation accounting (Robin Sommer).
- A minor tweak to the output generated by statistics.bro.
- Improved localization of source code for functions in messages (but
there are still some bug).
- Autoconf looks for -ltermcap (Robin Sommer).
- Fixes for bugs in the management of table expiration values (Chema Gonzalez).
- A bug in printing "void" values has been fixed (Chema Gonzalez).
- -t bug fixed (Chema Gonzalez).
- A bug has been fixed in which sometimes "expression value ignored"
was erroneously generated.
- A bug with packet_contents and UDP packets with checksum errors
has been fixed (Ruoming Pang).
- A memory leak in packet timestamp sorting via packet_sort_window
has been fixed (Ruoming Pang).
- A bug has been fixed in expanding NULs when printing strings (Ruoming Pang).
- Bug fixes for extracting connection contents via contents.bro (Ruoming Pang).
- Bogus error message "Can't install default pcap filter" when using -F
removed.
0.8a70 Sun Feb 8 14:19:45 PST 2004
- Bro has a new home page at
http://www-nrg.ee.lbl.gov/bro.html
It includes a "wish list" of Bro development projects:
http://www-nrg.ee.lbl.gov/bro-wishlist.html
- The "match" expression has been completely overhauled (Umesh Shankar).
It now has the syntax:
match EXPR1 using EXPR2
Its semantics are complicated, but it's very powerful (see its use for
alert filtering below). EXPR1 can have any type T. EXPR2 must be of
type "set[R]", where R is a record type. R must have the following fields:
$pred - type is "function(T): bool". This is the predicate
associated with the record. It is passed in EXPR1's
value and returns true or false.
$result - can have any type T'. This is the value to use when
if $pred returns true for EXPR1.
$priority - type must be arithmetic (count, int, double). This
is the priority associated with the match of EXPR1
if $pred returns true.
The way the expression works is that EXPR1 is evaluated yielding a
value V. EXPR2 is then evaluated yielding a set of records whose
type includes the above fields. Bro then spins through each of the
records in the set and tests whether its $pred predicate holds for V.
If so, it records the given $result and the associated $priority.
It then returns for the value of the entire expression the $result
with the highest $priority.
Here's an example. The following script:
global match_stuff = {
[$pred = function(a: count): bool { return a > 5; },
$result = "it's big",
$priority = 2],
[$pred = function(a: count): bool { return a > 15; },
$result = "it's really big",
$priority = 3],
[$pred = function(a: count): bool { return T; },
$result = "default",
$priority = 0],
};
print match 0 using match_stuff;
print match 10 using match_stuff;
print match 20 using match_stuff;
when executed will print:
default
it's big
it's really big
(Note that Bro actually will first evalute $priority before evaluating
$pred, and if it already has a better (higher) priority result, it
will not bother calling $pred.)
- There's a new syntax for designating function values (Umesh Shankar).
It currently only works when associating a function value with a
record field in a record constructor:
[$foo(bad_guy: addr) = { launch_counter_DDOS(bad_guy); return 3.14; }]
is equivalent to:
[$foo = function(bad_guy: addr): double = {
launch_counter_DDOS(bad_guy);
return 3.14;
}]
The goal is to make such functions easier on the eye to express.
The changes are (1) no "function" keywork necessary, (2) no function
return type necessary (note, it is inferred from the "return" statement
in the function body; eventually this will work for all functions, not
just those in record constructors), (3) the '=' sign comes after the ')'
rather than before the keyword "function".
Given this syntax, we can rewrite the initialization of match_stuff
in the example above as:
global match_stuff = {
[$pred(a: count) = { return a > 5; },
$result = "it's big",
$priority = 2],
[$pred(a: count) = { return a > 15; },
$result = "it's really big",
$priority = 3],
[$pred(a: count) = { return T; },
$result = "default",
$priority = 0],
};
- The motivation behind these elaborate new mechanisms is to provide a
powerful and streamlined way to filter alerts. According, alert.bro
now processes any alerts generated via ALERT() through a new global,
alert_policy. alert_policy's type is set[alert_policy_item], where
alert_policy_item is:
type alert_policy_item: record {
result: AlertAction;
pred: function(a: alert_info): bool;
priority: count;
};
The idea is that you specify your alert post-filtering by redef'ing
new elements into alert_policy. For example, here are two post-filtering
rules used at LBL to weed out uninteresting alerts:
# Ignore connections marked as sensitive because they're
# NTP to otherwise-sensitive hosts (first clause) or they happen
# to involve 2766/tcp (Solaris Listen), which happens frequently
# to ftp.ee.lbl.gov if Bro misses the PORT negotiation.
[$pred(a: alert_info) =
{
return a$alert == SensitiveConnection &&
(a$conn$id$resp_p == 123/udp || # NTP
a$msg == /Solaris listen service/);
},
$result = ALERT_FILE,
$priority = 1],
# Ignore sensitive URIs if the request was unsuccessful (code 404,
# or not answered.)
[$pred(a: alert_info) =
{
return a$alert == HTTP::HTTP_SensitiveURI &&
a$msg == /.*((\(404)|(no reply)).*/;
},
$result = ALERT_FILE,
$priority = 1],
These rules are part of:
redef alert_policy += {
... these records and others ...
};
The idea behind them is to demote certain alerts that would ordinarily
be syslog'd (i.e., the associated action is ALERT_LOG_ALWAYS) to instead
just be recorded in the alert.$BRO_ID file. Naturally, there are
many other possibilities. For example:
[$pred(a: alert_info) = {
if ( a$alert == FTP::FTP_Sensitive &&
a$msg == /.*crown_jewels.*/ )
{
system("page_the_duty_officer \"crown jewels theft!\"");
return T;
}
else
return F;
},
$result = ALERT_LOG_ALWAYS,
$priority = 1000],
would run the program page_the_duty_officer with the argument "crown
jewels theft!" if an FTP_Sensitive alert was generated and the log message
included the text "crown_jewels". More generally, post-filtering needn't
just be about deciding on how the alert is logged; the processing can
run programs, update tables, etc., just like any other function call might.
- You can use the new function tally_alert_type in an alert_action_filters
initialization in order to suppress immediate logging of an alert and
instead have Bro generate a summary of how many times the given alert
was seen when it exits. You can use another new function, file_alert,
to specify an alert_action_filters initialization that causes the alerts
to just be written to the alert.$BRO_ID file but not otherwise logged.
For example:
redef alert_action_filters += {
# Just summarize various packet capture glitches.
[[RetransmissionInconsistency, ContentGap, DroppedPackets,
AckAboveHole]] =
tally_alert_type,
[RemoteWorm] = file_alert,
};
would specify that RetransmissionInconsistency (etc.) alerts should just
be reported in the log file (log.$BRO_ID) as a total count, and
RemoteWorm should only be put in the alert.$BRO_ID file, but not
otherwise logged or counted.
You could get the same functionality by writing alert_policy records,
but they're quite a bit bulkier than the above. Note that
alert_action_filters entries take precedence over alert_policy
records, but are evaluated *after* the "match" on alert_policy,
so if you have predicates in alert_policy with side-effects (like the
invocation of page_the_duty_officer in the example above), those
will still happen.
- The alert_info record (which is used in calls to ALERT) now has
slots for some more additional information:
user: string; # can hold an assocaited username
filename: string; # an associated filename
method: string; # associated HTTP method
URL: string; # associated URL
n: count; # any associated count/number/status code
(These are all &optional, so you don't need to specify them if they're
not appropriate.) A number of ALERT's in the default policy scripts
have been changed to include these. The intent is to add more such
information in the future. Ideally, alert_policy records shouldn't
be doing checks like "a$msg == /.*((\(404)|(no reply)).*/" but instead
extracting the status code directly from a field of 'a' (which is an
alert_info record).
- ALERT now fills in the '$id' field in the alert_info record with
the $id of the associated connection, if the caller didn't suppy
a $id but did supply a $conn. Likewise, it will fill in $src with
the $orig_h from $id (after first filling in $id). The net result
is that you can rely on $id and $src being set for any alert that
has an associated connection.
- The HTTP analysis scripts (policy/http*.bro) have been converted to
use the "module" facility, similar to how ftp.bro was converted for
0.8a48. This may require changing some of your own scripts, generally
just to add "HTTP::" qualifiers.
- Now that the variables associated with FTP analysis are part of an
"FTP" module, the "ftp_" prefix associated with:
ftp_excessive_filename_len
ftp_excessive_filename_trunc_len
ftp_guest_ids
ftp_hot_cmds
ftp_hot_files
ftp_hot_guest_files
ftp_ignore_invalid_PORT
ftp_ignore_privileged_PASVs
ftp_log
ftp_skip_hot
has been removed, and these are now called:
excessive_filename_len
excessive_filename_trunc_len
guest_ids
hot_cmds
hot_files
hot_guest_files
ignore_invalid_PORT
ignore_privileged_PASVs
log_file
skip_hot
To get to them from other scripts, you specify, for example,
redef FTP::guest_ids = { .... };
whereas before you had to use:
redef FTP::ftp_guest_ids = { .... };
- The new connection logging format introduced in 0.8a57 is now the
default, unless you redef the new variable "traditional_conn_format"
to be T (Robin Sommer). Connections using unidentified ephemeral
ports now have a service of simply "other" rather than other-XXXX.
The 'U' connection status flag has been removed (unless you're using
traditional_conn_format).
- Tables can now be directly indexed by records, and indexing using records
is no longer interchangeable with using a corresponding list of indices
(Umesh Shankar). This may require adjustments to existing policy
scripts.
- Hostnames such as www.google.com now have type set[addr] rather than
a funky internal list type.
- The new function dump_current_packet(file_name: string) dumps a copy of
the current packet to the file with the given name, appending it if the
file already exists (Robin Sommer). The file is in tcpdump format.
A handy use for this is in an event handler for signature_match(),
to record packets that match given signatures.
- The event new_packet() is invoked for each new packet (Robin Sommer).
It currently doesn't provide the packet contents but soon will in
a fashion similar to secondary-filter.bro.
- "cf -f fmt" specifies a strtime() format. -u specifics UTC time rather
than local time (Mark Delow and Craig Leres). cf now has a man page
(Craig Leres).
- Two new variables, backdoor_ignore_local and backdoor_ignore_remote,
can be used to specify backdoor signatures that should be ignored
if the server is local/remote.
- A bug has been fixed in which a "next" executed in the final iteration
of a for loop would mangle the subsequent processing of the outer
statements (Chema Gonzalez).
- Bug fixes for MIME and Base64 processing (Ruoming Pang).
- pcap.bro now builds its filter in the opposite order (restrict_filters
first), which can improve packet filtering performance (Robin Sommer).
- A bug in &default has been fixed.
- More SSL autoconf tweaks (Robin Sommer).
- Portability for different pcap_compile_nopcap() calling sequences
(Chema Gonzalez).
- Some tweaks for a minor reduction in memory consumption.
- A memory leak for secondary packet filters has been fixed.
- The localization of error messages (what script line they correspond to)
has been improved.
0.8a58 Tue Dec 16 08:55:47 PST 2003
- Compatibility with older versions of libpcap (Chema Gonzalez).
0.8a57 Tue Dec 9 10:14:30 PST 2003
- The format of Bro's connection summaries is changing. The new format
looks like
1069437569.904605 0.230644 1.2.3.4 5.6.7.8 http 59377 80 tcp 610 275 S3 L
That is, <timestamp>, <duration>, <originator address>, <responder address>,
<service>, <originator port>, <responder port>, <originator bytes>,
<responder bytes>, <connection state>, <flags>. (Robin Sommer)
The script variable traditional_conn_format=T specifies to use the old
format rather than this new one. This is *currently* the default, but
will change soon to default to F instead. If you have comments on this
new format, we'd like to hear them.
- The SigAction's available in signatures.bro have been extended (Robin Sommer).
SIG_FILE_BUT_NO_SCAN is like SIG_FILE but without any horizontal/vertical
processing; SIG_LOG_ONCE logs only an alert only the first time it occurs;
SIG_LOG_PER_ORIG logs only the first instance of an alert generated by a
particular originator; SIG_COUNT has been renamed SIG_COUNT_PER_RESP; and
SIG_SUMMARY suppresses logging of individual alerts but generates a
per-originator summary.
- A new -p option for snort2bro tells it to only process signatures that
include matching on payload (Robin Sommer).
- You can now explicitly include or exclude particular SIDs when
running snort2bro by specifying a configuration file via -c (Robin
Sommer). The format is simple, just "include" or "ignore" followed
by the SID number:
# sid-526 BAD TRAFFIC data in TCP SYN packet
ignore 526
# sid-623 matches a null-flags stealth scan. Include it even
# if we build with -p, since it doesn't tend to generate any
# false positives.
include 623
The new "snort2bro.cfg" file gives examples (i.e., the above).
- Bro can now serialize functions and event handlers, meaning that these
can be passed as values between Bro's and dumped using -g (Robin Sommer).
One of the main goals in supporting this is to allow in situ alteration
of the Bro's configuration (e.g., you can edit a function and change its
functioning and have a running Bro pick up the change without having to
stop and be restarted). Such dynamic reconfiguration is experimentally
supported via -g <dir> (see below).
- &persistent state is now stored in the *directory* given by state_dir
(default: "./.state"), one file per variable, rather than a single file
(Robin Sommer).
- Storing &persistent state to disk is now done incrementally: after writing
each file, there's a delay of state_write_delay (default: 0.1 secs) before
working on the next file (Robin Sommer). This may introduce small
inconsistencies, but prevents load spikes that can lead to packet drops.
Currently, there is no mechanism to incrementally store a single variable
(like a large table), although there is already some framework in place
to eventually support this.
- The *experimental* new -g <dir> option dumps the script-level configuration
(excluding things defined in internal default scripts like bro.init)
into the directory <dir>. These files may be printed with "bro -x <file>",
or copied into the state_dir of a running Bro, which will then pick up
the change if it has loaded checkpoint.bro. (When picking up changes,
event handlers are always added, while functions, types, and variables
replace the current ones).
- Table values are now incrementally expired rather than all at once
(Robin Sommer). That is, if the expiration timer goes off and hundreds
of values should now be expired, the work of doing so is spread over
chunks of table_expire_size (default: 50) at a time, separated by a
delay of table_expire_delay (default: 0.1 secs). This change aims to
prevent large processing spikes that can lead to packet drops.
- New built-ins sub() and gsub() act like awk's functions of the same
name, changing substrings (either first, or all) that match a given
regular expression to a given target string. (Note, the calling sequence
differs from the order used by awk.)
- The new auxiliary script aux/scripts/mvlog is a handy way to manage
checkpointed logs. See the script for documentation.
- The &expire_func function now takes two arguments. The second is
of type "any" and corresponds to the index(es) of the element being
expired. To access the individual indices, you use a new assignment form:
[a, b, c] = index_val;
(where index_val is the second argument of type "any"). This assigns
a to the first index, b to the second, and c to the third. NOTE: the
use of "any" types here is *temporary* and will be changing in the
future to a general "tuple" notion. (Robin Sommer)
- scan.bro and conn.bro have been reworked to consume less memory and to
support more flexible state expiration (Robin Sommer).
- The new builtin rescan_state() causes Bro to re-read any persistent
data values (Robin Sommer).
- snort2bro now supports continued lines ("\<newline>") (Robin Sommer).
- The calling sequences of the software_version_found() and
software_parse_error() events has changed, and a new event,
software_unparsed_version_found(), is invoked for raw version
strings (i.e., the version string prior to the event engine
attempting to parse it into version/major/minor) (Robin Sommer).
- Software version tracking for clients now tracks all versions, not just
the latest version (Robin Sommer).
- alert_info records now include an optional field event_src, which is the
source of the event if it was received from an external Bro (Robin Sommer).
- Regular expressions now support {} iteration values of 0, and generate
better error messages.
- Output generated by icmp.bro is now redirected into an "icmp" log file
(Robin Sommer).
- autoconf tweaks for configuring OpenSSL on Linux (Ruoming Pang, Robin Sommer).
Tested on RedHat (thanks to Anton Chuvakin), Debian, FreeBSD, Solaris.
- You can now turn off using OpenSSL even if the OS supports it, via
configuring with --disable-openssl (Robin Sommer).
- Variable size computations (per global_sizes()) are now more accurate
(Robin Sommer).
- A bug with combining file encryption and log rotation has been
fixed (Robin Sommer).
- A problem tracking directionality in signatures fixed (Robin Sommer).
- Bro now continues running if DNS is not functioning (Robin Sommer).
- Rewriter memory use has been significantly reduced (Ruoming Pang).
- Some bugs with -A/-w interaction have been fixed (Ruoming Pang).
0.8a48 Tue Oct 21 15:56:13 PDT 2003
- There is now a mechanism in place for multiple Bro's to communicate with
one another via sockets (Robin Sommer). *This is quite experimental at
this point* and may have significant bugs and/or need significant
enhancements.
By loading listen-clear.bro or listen-ssl.bro, an instance of Bro starts
to listen on a TCP port. The first of these listens for unencrypted
connections and the second for connections encrypted via OpenSSL. To
connect to a listening Bro, you load remote-clear.bro or remote-ssl.bro.
For this connection, you specify which events you want to receive by
giving a regular expression (e.g. "/http_*/" for all HTTP events),
although only those events for which you have defined a local handler
are actually requested. Communication is uni-directional in the sense
that for a certain connection the events go only from Bro-A to Bro-B but
not from B to A (you could set up a second connection for the other
direction, though).
The OpenSSL certificate used to authorize remote Bro's is specified in
the script variable "ssl_ca_certificate" and the private key and certificate
for the local Bro via "ssl_private_key".
If Bro A connects to Bro B, by default it sends over its capture_filter.
But Bro B uses it only if it has loaded remote-pcap.bro. This is the
beginning of structuring inter-Bro trust mechanisms. It is done via two
new events, remote_capture_filter(ip: addr, p: port, filter: string) and
send_capture_filter(ip: addr, p: port, s: string) : bool.
The implementation forks a second process which does the socket
communication, so that the main process should not be affected too much.
The two processes communicate via a pipe.
You can call is_remote_event() to determine whether the event currently
being handled was originated by a remote Bro (if T) or the local Bro
(if F).
If a connection with a remote Bro terminates (for whatever reason), Bro
may try to reconnect automatically.
A new function, get_event_source(), returns a record event_source
describing the source that raised the last event.
See doc/ssl.txt for an explanation of how to create the keys/certificates.
- A fledgling Gnutella analyzer has been contributed (Mark Allman).
It generates the following events:
event gnutella_text_msg(c: connection, orig: bool, headers: string)
event gnutella_binary_msg(c: connection, orig: bool, msg_type: count,
ttl: count, hops: count, msg_len: count,
payload: string, payload_len: count,
trunc: bool, complete: bool)
event gnutella_partial_binary_msg(c: connection, orig: bool,
msg: string, len: count)
event gnutella_establish(c: connection)
event gnutella_not_establish(c: connection)
event gnutella_http_notify(c: connection)
- Bro now supports a secondary channel for acquiring packets (Chema Gonzalez).
You access it by redef'ing the new global "secondary_filters", adding
table[string] of event(filter: string, pkt: pkt_hdr). The string
specifies a tcpdump filter; anytime a packet matches the filter
(including packets that would *not* otherwise be matched by
capture_filter), then the given event handler is invoked.
For example,
redef secondary_filters += {
["tcp[13] & 7 != 0"] = rst_syn_fin_flag,
}
will invoke rst_syn_fin_flag() anytime a TCP packet is seen for
which the SYN/FIN/RST bits are non-zero. The event handler will
be passed the string "tcp[13] & 7 != 0" (so it can tell which
of possibly multiple filters matched) and a pkt_hdr value, which
includes the IP header and, if present, the TCP, UDP or ICMP header.
Another example, and what motivated the addition, is:
redef secondary_filters += {
["ip[10:2] & 0xffc == 0x398"] = sampled_1_in_1024_packet,
}
which will invoke sampled_1_in_1024_packet() any time the given
10 bits in the IP checksum match the pattern 0x398. If the checksum
field is uniformly distributed then this roughly corresponds to
1-in-1024 random sampling. (Chema has also developed BPF additions
to support true random sampling.)
See policy/secondary-filter.bro for examples.
- Bro now does a much better job of keeping track of how much memory
has been allocated for different structures (Robin Sommer).
This includes more accurate computations for global_size().
In addition, if you redef "statistics_interval" to be a non-zero time
interval, then with that periodicity a summary of memory usage (including
memory used by event engine components) is dumped to the file
"statistics_file". In addition, at this point a "do_statistics" event
is generated. You can also call the new built-in statistics_update()
to generate memory statistics on demand.
The above structure is likely to change in the future. statistics_interval
will probably go away, to be replaced by either explicit calls to
statistics_update() (which you can do on a timer if you like by using
"schedule"), or by a built-in function that returns a record of all
the statistics, that you can then format however you want.
- A major memory leak in HTTP analysis has been fixed (Ruoming Pang).
- New attributes &rotate = <interval expression> and
&postprocessor = <string expression> can be associated with a file
variable in order to specify how often the file should be rotated to a
new filename on disk, and, when rotation occurs, the name of a shell
program to run on the now-older version as a postprocessor (Robin Sommer).
- Similarly, log_postprocessor and log_rotate_interval specify the default
values for files. Unless redef'd, these themselves default to the empty
string (= no postprocessing) and 0.0 seconds (no rotation). (Robin Sommer)
- A new attribute, &encrypt, applies to variables of "file" type. It specifies
that the version on disk should be encrypted, using either the key specified
as the value of the attribute, or, if no value is specified, using the
value of the new script variable log_encryption_key. The key is an OpenSSL
public key; it's used to then embed a Blowfish session key. (Robin Sommer)
A new utility, aux/bdcat/bdcat ("Bro decrypt cat") can be used to decrypt
the files.
- The internal structure of TCP analysis has been significantly altered.
Previously, TCP_Endpoint tracked endpoint state and TCP_EndpointContents
(derived from it) did stream reassembly. These have now been separated;
TCP_Endpoint still tracks endpoint state, but TCP_EndointContents has
become TCP_Contents, and is only loosely coupled with TCP_Endpoint.
The reason for this change is to lay the groundwork for (1) applying
an application analyzer to a connection after several packets for
the connection have already been read, and (2) applying *multiple*
application analyzers to a single connection.
- Bro now supports the universal hashing if USE_UHASH is defined
(Ruoming Pang). Universal hashing is a lighter-weight alternative
to MD5 hashing that retains the property of being very difficult
for an attacker to guess. It comes in two flavors, a 32-bit
version (which you get with USE_UHASH) and a faster 16-bit version
(which you get if you also define USE_NH). Bro will likely switch
to using these by default in the near future, as their performance
gain over MD5 is significant.
- New built-in functions srand() and rand() provide access to the
usual C-style random number seeding & generation (Chema Gonzalez).
- You can now specify server/client addresses to leave in the clear in
IP address anonymization (via the built-in variables preserve_orig_addr
and preserve_resp_addr). Correspondingly, the built-in function for
IP anonymization now takes a parameter to specify the type of the address
(orig, resp, or other), instead of the method of anonymization
(seq, random, prefix-preserving). (Ruoming Pang)
- Trace anonymization now has prelminary support for handling TCP options
via the new event "tcp_option" (Ruoming Pang). It is only invoked
if skip_event_tcp_option is F (it defaults to T).
- A new event, tcp_segment, is similar to the event "packet_content"
but provides more information: is_orig (directionality), sequence
number, and acknowledgement number (Ruoming Pang).
- ./configure finds OpenSSL if it's in some standard location. Otherwise,
you may specify it --with-openssl=<path>. If OpenSSL is not available,
Bro compiles cleanly without and gives warnings if a script tries use SSL.
(Robin Sommer)
- The internal links in manual/entire.html have been fixed so they
now work (Chema Gonzalez).
- A new policy script, blaster.bro, detects instances of the W32.Blaster
worm (Robin Sommer).
- Signature files (for input to the signature engine) now reside in
policy/sigs/*.sig. This directory is now on the default $BROPATH.
- sig.ex.ssl-worm.bro and sig.ex.web-rules.bro have been updated
to reflect changes in keywords (Robin Sommer). They've been
renamed to ex.ssl-worm.sig and ex.web-rules.sig and reside
in policy/sigs/, per the above.
- The module facility has been changed to have its scope limited to
the current file plus any additional files that are automatically
processed based on its name plus $BRO_PREFIXES.
- As an experiment, ftp.bro has been modified to a style that includes
using "module FTP". Likely other policy scripts will be converted
in the near future, and their variable names changed accordingly
(e.g., "ftp_hot_files" will become "FTP::hot_files").
- The new "match" expression has been modified to allow any yield type
rather than just strings. It is likely to change significantly again
soon.
- Iterating over multi-dimensional tables/sets now works (Robin Sommer).
For example:
const remote_peers_ssl : table[addr, port] of Peer &redef;
[...]
for ( [ip, p] in remote_peers_ssl )
connect_ssl(ip, p, remote_peers_ssl[ip, p]$retry);
- Checkpointing of persistent state on SIGHUP now happens via bro.init
(Robin Sommer). Not tested.
- fmt() now supports %x for hexadecimal formatting (Chema Gonzalez).
- alert.bro logs the source for remote alerts; by redefining the new
"event_source_description: string" locally, you can specify how a
Bro is to refered to on the remote side. (Robin Sommer)
- software.bro now tracks HTTP clients, too (Robin Sommer). This
will be extended in the near future.
- Some FreeBSD 5.X porting tweaks (Sergey Osokin).
0.8a37 Wed Sep 3 23:20:21 PDT 2003
- A new attribute, "&persistent", indicates that a global variable's
state should persist on disk (Robin Sommer). Currently, they
reside in the file "state.bst".
For example, given the following script:
global a = 0 &persistent;
event bro_init()
{
print ++a;
}
then every time you run it, a increases by one.
You can dump the state file using "bro -x state.bst <scriptname>".
(There's also a partially-implemented XML dumper which you invoke via
"bro -X state.bst <scriptname>".)
If you send Bro a HUP signal, then it will synchronize all persistent
state to disk (checkpoint) at that point. More generally, the policy
script can do so at any time by calling the new built-in checkpoint_state().
By including the new policy script "checkpoint", you can have Bro
automatically checkpoint every time checkpoint_interval elapses
(default 15 minutes).
- You can also record events to disk by loading the "capture-events"
policy script, or calling the new capture_events() built-in (Robin Sommer).
The events are written to the file events.bst. You can dump these
using bro -x (or -X), as above. You can also replay them using
"bro -R <file>". By replaying them using the same policy script as
generated them, you should get the same output. For example:
bro -r tracefile myscript capture-events
...
bro -R events.log myscript
- An experimental module facility has been added (Umesh Shankar).
The module facility implements namespaces. Everything is in some namespace
or other. The default namespace is called "GLOBAL" and is searched by
default when doing name resolution. The scoping operator is "::" as in
C++. You can only access things in the current namespace, things in the
GLOBAL namespace, or things that have been explicitly exported from a
different namespace. Exported variables and functions still require
fully-qualified names. The syntax is as follows:
module foo; # Sets the current namespace to "foo"
export {
int i;
int j;
}
int k;
module bar;
int i;
foo::i = 1;
bar::i = 2;
print i; # bar::i (since we're currently in module bar)
j = 3; # ERROR: j is exported, but the fully qualified name
# foo::j is required
foo::k = 4; # ERROR: k is not exported
The same goes for calling functions.
One restriction currently in place is that variables not in the "GLOBAL"
namespace can't shadow those in GLOBAL, so you can't have
module GLOBAL;
global i: int;
module other_module;
global i: int;
It is a little confusing that the "global" declaration really only means
that the variable i is global to the current module, not that it is truly
global and thus visible everywhere (that would require that it be in
GLOBAL, or if using the full name is ok, that it be exported). Perhaps
there will be a change to the syntax in the future to address this.
The "module" statement cuts across @load commands, so that if you say:
module foo;
@load other_script;
then other_script will be in module foo. Likewise if other_script changes
to module bar, then the current module will be module bar even after
other_script is done. However, this functionality may change in the future
if it proves problematic.
The policy scripts in the Bro distribution have not yet been updated to
use it, but there is a backward-compatibility feature so that existing
scripts should work without modification. In particular, everything is
put in GLOBAL by default.
- The hooks are now in place for communicating events between running
Bro's. An experimental implementation of doing so (written by Robin
Sommer) will be integrated shortly.
- A side-effect of getting those hooks in place is that event handlers must
now be directly specified (by naming them) rather than indirectly
(for example, by indexing a table whose value yields an event handler).
This may be fixed soon.
- An experimental "match" expression scans a list of predicates to find
the first one that's both true and has the highest priority (Umesh Shankar).
The general form is:
match some_record on {
[ priority, class_name, predicate ],
...
}
where "predicate" is evaluated in the context of the value (and type) of
some_record. For example, if some_record has a field "num_alerts"
then predicate could be "$num_alerts > 5".
"priority" is a non-negative integer (i.e., of type "count"), and,
for now, "class_name" is a string.
For example,
global c: conn_id;
c = [ $orig_h = 0.0.0.0, $orig_p = 0/tcp,
$resp_h = 1.1.1.1, $resp_p = 1/tcp ];
print match c on {
[ 2, "emptyweb", $orig_h == 0.0.0.0 && $resp_p == 80/tcp ],
[ 1, "emptyhost", $orig_h == 0.0.0.0 ],
[ 0, "should not match", 1 == 0 ]
};
will print "emptyhost".
The initial intent behind this is to eventually provide more flexible
customization of alert processing, though it clearly has broader
applicable. *It is very likely that the semantics and perhaps the syntax
of "match" will change in the near future.*
- Bro's packet filter is now computed via pcap.bro (which is automatically
included via bro.init). It uses two new built-ins:
precompile_pcap_filter(id: PcapFilterID, s: string): bool
install_pcap_filter(id: PcapFilterID): bool
The first is for precompiling a pcap filter so it can be installed
or turned off dynamically. Associating an ID with the filter, you can
then activate the filter by calling the second function (installing a
new filter replaces the current one). (Robin Sommer)
Be default, pcap.bro is responsible for building a pcap string based on
the capture/restrict_filters defined by the various analyzers. It compiles
and installs this filter, so there is no observable difference in usage
to the old implementation, except capture/restrict_filter are now *tables*
rather than strings, and are written as plural rather than singular.
So the analyzers need to define something like this:
[finger.bro]
redef capture_filters += { ["finger"] = "port finger" };
This then allows "finger" to be used as the name for the corresponding
filter element (see the next item).
- load-level.bro is an experimental policy script for allowing Bro to
shed or add on load (in terms of which protocols it analyzes). It
provides three interface functions for setting the current loadlevel:
# level is LoadLevel1, ..., LoadLevel10
function set_load_level(level: PcapFilterID): bool
function increase_load_level()
function decrease_load_level()
load-levels.bro defines ten different load levels (from 1 to 10, with 10
being the one with the most load imposed on the system), which are
configured by defining capture_load_level/restrict_load_levels: Example:
redef capture_load_levels += {
["dns"] = LoadLevel1,
["smtp"] = LoadLevel2,
["http-reply"] = LoadLevel3,
["http-request"] = LoadLevel8,
};
This means for example: "include the capture_filter associated with
'http-reply' if the current load level is 3 or below". There's a similar
mechanism for restrict_filters:
redef restrict_filters += {
["cs-only"] = "net 131.159.0.0/16",
};
redef restrict_load_levels += {
["cs-only"] = LoadLevel7,
};
This applies the given restrict_filter if the current load level is 7
or *above*.
The pcap filters belonging to the ten load levels are built and pre-compiled
on startup. The three functions shown above just call install_pcap_filter()
then. (Robin Sommer)
- drop-adapt.bro tries to adjust the load level based on the current
packet drop rate (Robin Sommer).
- synflood.bro is an experimental policy script for detecting SYN floods.
It is not yet documented, other than brief comments in the script.
(Robin Sommer)
- Subnet lookups now use Patricia trees instead of brute-force, which should
make them significantly more efficient for large lists, e.g., local nets.
(Robin Sommer)
- Due to the addition of modules, which use a C++-style "::" scope
designator, you now need to use "0x" to introduce IPv6 address constants
that use "::" to specify a series of empty octets. For example, you
used to be able to specify "deadbeef::cafe", but now this needs to be
"0xdeadbeef::cafe". Note that "1::2" can still be written without needing
a "0x" prefix; it's just hex constants that start with letters that need
the prefix.
- A new built-in, escape_string(), takes a string and returns a copy of
it that uses escape sequences for any problematic characters (Robin Sommer).
- A number of low-level bug fixes and portability tweaks (Robin Sommer,
Ruoming Pang, Christian Kreibich, Chema Gonzalez).
- A new timer, status_update_timer, fires for each ongoing connection
every connection_status_update_interval seconds (default = 0 seconds,
which means "don't fire). (Robin Sommer)
- An additional Bro-level packet filter can filter/sample packets
based on their src/dest ip/subnet (using a Patricia tree for
efficiency; Robin Sommer). install_src_addr_filter(ip, flags, p) drops
packets originating from ip with probability p (0..100) if none of the
given TCP flags is set. install_src_net_filter, install_dst_addr_filter
and install_dst_net_filter" work similarly. The corresponding "uninstall_*"
functions remove the filters again.
- The @if/@else/@endif mechanisms have been made more powerful (Robin Sommer).
- New configure option --enable-debug to compile without optimization
(Robin Sommer).
- Small tweaks to the mpatrol support (Robin Sommer).
- SMTP is now one of the services which can use a small inactivity timeout
in inactivity.bro (Robin Sommer).
- Alerts for signatures which are triggered by worms may be suppressed if
we already know that the triggering host is indeed infected by a worm
(Robin Sommer).
- Matches of a signature can now be counted per destination host
(Robin Sommer).
- snort2bro now ignores sid-526 ("BAD TRAFFIC data in TCP SYN packet";
Robin Sommer). Due to different semantics of Bro (matching stream-wise)
and Snort (matching packet-wise) this signature generates a lot of
false positives.
0.8a34 Sun Jul 13 09:11:32 PDT 2003
- The new "subnet" type corresponds to a CIDR prefix (Robin Sommer).
You can use the '/' operator to convert an address to a subnet. The
"in" operator tests whether an address matches a subnet (e.g., "1.2.3.4
in 1.2.255.255/16" yields T).
You can index tables with index type "subnet" using addresses to retrieve
the table entry with the longest-matching prefix for that address, and
a number of tables/sets in the default scripts have been converted to
this form. So for example the local_16_nets and local_24_nets variables
have been replaced by local_nets, which has type "set[subnet]",
is_local_addr() now refers to it, and skip_scan_nets_{16,24} have
likewise been consolidated into skip_scan_nets.
One present deficiency is that subnets can't be used in tables/sets
with multiple indices. Fixing this is going to take some time ...
Another deficiency is that the longest-match lookup is not very
efficient.
Caution: this feature has been only briefly tested with IPv6.
- ALERT now generates an event "alert_action" which includes the alert_info
and AlertAction associated with an alert. You can define your own handler
to further customize alert processing.
- The "snort2bro" conversion utility has been extended to recognize some
of the new Snort options (depth, distance, within; Robin Sommer). For
example:
alert .... ( ... content:"|00 01 86 B8|";
content:"|00 00 00 02|"; distance:4; within:4;
content:"%x %x"; distance:16; within:256;
... )
is converted to:
signature sid-1891 {
...
payload /.*\x00\x01\x86\xB8.{4}\x00\x00\x00\x02.{16}.{0,251}%x %x/
}
Note that not all of the new Snort options are supported yet.
- You can refer to script variables within signatures (Robin Sommer).
For example, the following is now possible:
signature sid-547 {
ip-proto == tcp
src-ip != local_nets
dst-ip == local_nets
dst-port == 21
event "FTP \"MKD \" possible warez site"
tcp-state established
payload /[mM][kK][dD] /
}
This makes the signatures independent of the environment ("local_nets"
in the above examle). snort2bro now converts some well-known Snort
variables into corresponding Bro variables, a number of which are listed
in policy/site.bro.
- The default action for signature matches is now logging insted of only
writing it to a file (Robin Sommer).
- You can now use the '^' and '$' regular expression operators inside
subpatterns. For example, /foo$/ | /^bar/ works now.
- You can now use "msec" for milliseconds (in addition to "usec" for
microseconds, "sec" for seconds, etc).
- The log_file_name and open_log_file functions are now redef'able
if you want to change their behavior.
- Bro now exits more cleanly upon exhaustion of memory (Robin Sommer).
- A bug was fixed for the case of getrlimit() returning RLIM_INFINITY for
the maximum number of open files.
- Numerous additions of std:: scoping to address porting issues (Robin Sommer).
- gcc 3.X and Solaris portability fixes.
- A new event RemoteWorm is the complement of LocalWorm (Robin Sommer).
- A bug in which the FTP analyzer would complain about failing to be able
to look up a connection has been fixed (Robin Sommer).
- You can now configure Bro using "--with-mpatrol" to activate MPatrol
debugging hooks. When built in this mode, -m dumps the leak table
and -M shows unfreed parts of the heap. These dumps can also be
triggered using SIGUSR1 / SIGUSR2, respectively. (Robin Sommmer)
- A script function get_contents_type() which returns the type of reassembling
being done for a connection (none/orig-side/resp-side/both; Robin Sommer).
- A minor bug fix for the regular expression matcher (Robin Sommer).
0.8a32 Thu Jun 12 23:33:21 PDT 2003
- The low-level hash functions have been reimplemented to use HMAC-MD5 to
counter the "algorithmic complexity attacks" discussed in the USENIX
Security 2003 paper by Scott Crosby & Dan Wallach
(http://www.cs.rice.edu/~scrosby/hash/) (Ruoming Pang and Vern Paxson)
- Ruoming Pang has made extensive changes to the anonymization/transformation
framework. A paper describing it will appear in SIGCOMM 2003, and will
be included with the Bro distribution once the final copy is ready.
- Internal hash tables now resize incrementally, and more quickly.
This makes a big difference in avoiding lengthy processing pauses when
processing large volumes of traffic. (Craig Leres)
- gcc 3.1 and Linux portability tweaks.
- The calling sequence of http_request() has changed. It now includes both
the original URI and the escaped URI. This was made explicit because
we found that if only the original was passed in, it was too easy to forget
to expand the escapes in it; and if only the escaped was passed in,
some attacks could be missed. (Ruoming Pang)
- Signature rules can now refer to "finger" payload as well as HTTP and FTP
(Robin Sommer).
- The signature engine now includes keywords "dst-ip", "dst-port",
"ip-proto", "src-ip", "src-port". (Robin Sommer)
- Packet sorting now defaults to off.
- The FTP analysis now attempts to track the current directory (Ruoming Pang).
- A number of scan detection additions have been added (Scott Campbell):
activate_landmine_check = F
if T, then access to more than landmine_thresh_trigger
addresses in landmine_address (a set of addresses)
constitutes a scan
activate_priv_port_check = T
if T, then inbound access to more than priv_scan_trigger
privileged ports (i.e., ports < 1024) is considered a port
scan. You can exclude particular services from the count
via troll_skip_service, which defaults to { smtp, ftp,
ssh, 20/tcp, http }.
- The SMTP analysis now includes a new alert, HotEmailRecipient, which
is triggered by mail sent to any of the addresses specified in
the pattern hot_recipients (which defaults to a bare /@/).
- The new built-in cat_string_array_n() is like cat_string_array() except
it takes starting and ending indices as additional arguments (Ruoming Pang).
- The new built-in sort_string_array() takes a string array and returns
a sorted version (Ruoming Pang). It currently uses an N^2 algorithm
so shouldn't be used on large arrays.
- The new built-in subst() substitutes all instances of a given pattern
in a given string with another string (Ruoming Pang). For example,
subst("fooboo", /oo/, "xxx") returns "fxxxbxxx".
- The new built-in cut_tail() trims characters from the end of a string
(Ruoming Pang). For example, cut_tail("fooboo", 2) returns "foob".
- sub_bytes() can now take a negative argument, which is with respect to
the end of the string rather than the beginning (Ruoming Pang).
- The new built-in md5_hmac() returns an HMAC-MD5 hash of the given string
(Ruoming Pang). The HMAC secret key is generated from available entropy
when Bro starts up, or it can be specified for repeatability using
the new -K flag.
- The new built-in split_all() is like split() except that the returned
array also includes the parts of the string that match the pattern
used for the splitting (Ruoming Pang).
- The new built-in split_n() splits up to a given number of instances,
optionally returning the parts matching the split pattern (Ruoming Pang).
- The new built-in split_complete() is the most general of the split functions
(the others are written in terms of it). It splits a given string
with separators that appear in either a given pattern or a given set
of strings, optionally including the text matching the separators in
its return value, and optionally limiting the number of matches to
a specified maximum. (Ruoming Pang)
- The new built-in to_string_literal() returns an escaped version of a string
suitable to feeding into Bro's parser. For example, calling it on
"foo\nbar" (where '\n' is an embedded newline) returns "foo\x0abar".
(Ruoming Pang)
- subst_substring() has been renamed subst_string (Ruoming Pang).
- unescape_URI() no longer takes the connection as an argument (Ruoming Pang).
- config.guess and config.sub updated
- String escape expansion has been refined (Ruoming Pang) and some bugs
fixed. It now supports a format that's consistent with Bro string literals.
- Scanning of octal and hex escape sequences now stops after 3 or 2
characters, respectively. For example, "\0007" now expands to a
NUL followed by the character '7'. (Ruoming Pang)
- Bug fixes for handling of bare <CR> and <LF>'s when <CRLF> expected
(Ruoming Pang), and associated "weird" events.
- A bug in signature matching reassembly has been fixed (Robin Sommer).
- A bug in reporting "bad connection size" for connection sizes > 2GB
has been fixed.
- A bug in computing sizes for large partial connections has been fixed.
- A bug in delayed generation of connection closed events has been fixed.
- A framework has been added for compression of some "weird" events
to only be generated once-per-connection (Ruoming Pang).
- Some of the "weird"'s generated by the HTTP analyzer have been regularized.
- Some memory management fixes.
- A performance problem in allocating List objects was fixed.
- The copyright dates have been updated (Craig Leres).
0.8a22 Wed Jan 15 16:47:18 PST 2003
- There is now a "conditional compilation" feature similar to C's preprocessor
(contributed by Robin Sommer). "@if (expr)" will include the remaining
input lines up to "@endif" only if the given expression evaluates to true.
"@ifdef (id)" will do so only if the given identifier has been defined,
and "@ifndef (id)" only if it has not been defined. There's currently
no "@else" directive, and conditionals cannot be nested. Both of these
will be added in the future.
- New built-in functions (contributed by Ruoming Pang): parse_dotted_addr
takes a string in A1.A2.A3.A4 form an returns a corresponding "addr" value.
unescape_URI takes a URI that possibly has embedded escape sequences
("%61" for 'a') and expands the sequences.
- The URIs in HTTP requests are no longer automaticaly escaped. Instead,
they need to be manually escaped using unescape_URI. As this is likely
error-prone (users forgetting to do so), the plan is to change the
calling sequence of http_request in the near future so that *both* the
escaped and the unescaped URI are passed in.
- A number of g++ 3.0 porting tweaks have been added (thanks to Sean Irvine).
- The term "rule" has been systematically changed to "signature" (Robin Sommer).
- The functionality of monitoring packet drops has been moved into its
own policy script, netstats.bro (Robin Sommer).
- A number of rewriter and location bugs have been fixed.
- Some missing HTTP "weird"'s have now been included in weird.bro.
0.8a21 Thu Nov 28 23:31:38 PST 2002
- A new mechanism will reorder packets within a window of packet_sort_window
if due to their sequence numbers they appear to be out of order. This
can help a great deal when reading from multiple NICs. (Contributed by
Ruoming Pang.)
- A bug in regular-expression matching in which for example
"2a1" == /[0-9]+/ would evaluate as true was fixed (Ruoming Pang).
- There's now a rewriter/anonymizer for FTP (Ruoming Pang).
- The rewriter/transformation framework now supports a notion of
delaying transformation decisions until later (Ruoming Pang).
- An incompatibility with bison 1.75 has been identified. The problem
is that bison 1.75 generates incomplete location information for empty
grammar productions (those for which the RHS is empty). This will
be fixed soon.
- Some bugs in the signature engine have been fixed (Robin Sommer).
- The sources no longer use the not-fully-portable hash_map template
(Umesh Shankar).
- Some bugs with the debugger getting confused about source line number
locations, and also with the -t option, have been fixed (Umesh Shankar).
- If a content gap occurs wholly inside an HTTP entity, then the analyzer
will skip over the entity and continue processing any subseqeuent HTTP
items, rather than giving up on processing the connection (Ruoming Pang).
- The following new built-in functions have been contributed by
Ruoming Pang:
function cat_string_array(a: string_array): string
function split_all(str: string, re: pattern): string_array
function strstr(big: string, little: string): count
function subst_substring(s: string, from: string, to: string): string
function int_to_count(n: int): count
function fmt_ftp_port(a: addr, p: port): string
0.8a20 Sun Nov 17 20:09:31 PST 2002
- This is the first "public" Bro release (one accessible directly via
the Web). The README has been correspondingly updated.
- The user manual has been significantly expanded, and source for it is
now included in the release.
- Some "active mapping" bug fixes (contributed by Umesh Shankar).
- The configuration script now checks for the presence of the necessary
BIND libraries/headers, and also for bogus --enable options (contributed
by Craig Leres).
- backdoor.bro now includes a tcpdump filter for detecting (some) KaZaA
traffic.
- http-reply.bro now tracks the sizes of request/response entities.
- http-request.bro now treats an URL accessing variants of /etc/netconfig
as sensitive.
0.8a18 Sun Oct 27 15:28:23 PST 2002
- Improvements to the performance of the SMTP analyzer (Ruoming Pang).
A new function, skip_smtp_data, skips over the content of an SMTP
message.
- If you're doing trace rewriting and specify -w but don't specify -A,
then the rewritten traffic goes to the -w save file, along with the
usual non-transformed traffic (Ruoming Pang). If a connection is
being transformed (which is specified by the new rewriting_smtp_trace
and rewriting_http_trace globals), then only its transformed packets
are written to the -w file; not the original packets. This allows
a potentially large degree of trace compression. There's also a
new variable omit_rewrite_place_holder, which directs that when rewriting
a trace, packets that are placeholders for now-missing packets (because
you've excised content) are *not* generated. This diminishes the
timing fidelity of the transformed trace, but also can save a large
amount of space.
- SMTP relay analysis is now standalone from regular SMTP analysis
(Ruoming Pang).
- Some memory management and error propagation fixes (Ruoming Pang and
Vern Paxson).
0.8a16 Wed Oct 23 23:48:40 PDT 2002
- "--enable-brov6" was broken by some recent changes. It now works again.
- Some "make distclean" tweaks.
- Error checking for "redef enum".
0.8a15 Tue Oct 22 00:02:51 PDT 2002
- Fixed Makefile bug which left out wide-spread dependencies on
event.bif.netvar_h.
0.8a14 Mon Oct 21 01:16:46 PDT 2002
- The "add" statement has been implemented. This allows you to add
elements to sets. A bunch of policy scripts that used to use
"table [] of bool"'s for this purpose have been modified to instead
use sets.
- You can now extend the elements of an enum using "redef enum foo += { ... }"
where the additional names are listed in the braces (contributed by
Umesh Shankar). A number of policy scripts have been tweaked to use
this for the (fairly) new Alert enum. This allows you to create new
Alert's without alert.bro having to know about them.
- Some bugs in identifying error locations have been fixed.
- -A now supports anonymizing IP addresses (contributed by Ruoming Pang).
This includes four new functions: preserve_prefix, preserve_subnet, and
preserve_net (which direct that the corresponding prefix/net/subnet
not be anonymized) and anonymize_addr (which returns the anonymized
version of the given address.
- Some bugs in HTTP -A have been fixed (thanks to Ruoming Pang).
- The beginnings of support for CIDR prefixes has been added by
Ruoming Pang.
- Porting tweaks (use of map's rather than hash_map's, courtesy of
Umesh Shankar; libedit comments fixed).
- http-detail.bro has gone away.
- Some more copyright and $Header -> $Id fixes.
- There is now a function string_cat() which concatenates two strings
and returns the result (contributed by Ruoming Pang). This function
is useful in place of the regular cat() because the latter will
expand escape sequences etc. It probably shouldn't, and that may
change in the future.
0.8a11 Sun Oct 13 10:53:07 PDT 2002
- The framework for defining built-in functions has been extended
so it's also now the way to specify the interface to event handlers.
See event.bif. (Contributed by Ruoming Pang)
- A new policy script, http-abstract.bro, can be loaded when doing HTTP
transformation (via http-rewriter.bro), contributed by Ruoming Pang.
It trims HTTP text responses to the first "http_abstract_max_length" bytes.
- A new built-in, skip_http_entity_data, specifies that the entity
associated with the current HTTP request/response should be skipped
(Ruoming Pang).
- More changes have been made to the mechanisms for tracking source
file locations in order to pinpoint error messages. If you encounter
problems, please let me know.
- If you try to configure but your resolve library lacks res_mkquery(),
this is now a fatal error (Craig Leres). Ideally, someone will modify
DNS_Mgr.cc to work even if only blocking DNS is available ...
- In most (not all, yet) of the sources, the copyright notices have
been updated, $Header has been changed to $Id, and config.h is included
(Craig Leres).
0.8a10 Tue Oct 8 16:05:42 PDT 2002
- The way that Bro tracks the source-file location associated with
different objects has been tweaked, with a resulting savings of
about 10% in Bro's memory use and a smidgen of CPU time, too.
- Built-in functions now are better about identifying source-file locations
associated with errors.
- The http.$BRO_ID log file format has changed to no longer track
individual request streams. (So it is now closer to how it used
to operate in 0.7.)
- The autoconf setup has been tweaked so that you use:
--enable-brov6 enable IPV6 processing
--enable-activemapping enable active mapping processing
--enable-expire-dfa-states enable DFA state expiration
rather than --with-XXX as was the case in 0.8a9.
0.8a9 Mon Oct 7 10:15:12 PDT 2002
- A bunch of configuration/autoconf/portability tweaks (Craig Leres).
These include compiling under Solaris.
When running ../configure, you can now specify:
--with-brov6 enable IPV6 processing
--with-activemapping enable active mapping processing
--with-expire-dfa-states enable DFA state expiration
You no longer set these via editing the Makefile.
- Some bugs with HTTP 1.0 keep-alive connections fixed (Ruoming Pang).
- The "hf" suite of utilities is no longer distributed in aux/. Instead,
get it directly from ftp://ftp.ee.lbl.gov/hf.tar.gz.
- bro_logchk has been renamed bro-logchk.pl and is now distributed in aux/
(it was supposed to be before, but was inadvertantly left out of the list
of distribution files).
0.8a7 Fri Oct 4 22:24:30 PDT 2002
- HTTP reply analysis has been split into a number of policy script files:
http-body.bro
http-detail.bro
http-entity.bro
http-event.bro
http-header.bro
http-reply.bro
http-rewriter.bro
so you can readily trade off how detailed the HTTP processing is vs.
the CPU/memory it consumes (contributed by Ruoming Pang).
- Bro now generates login_prompt events when the $TTYPROMPT environment
variable is passed during a login session, in order to detect the
recently publicized Solaris remote /bin/login exploit.
- Ruoming Pang has extended the framework for defining "rewriter" functions
to now also serve as the way to define general built-in functions.
- bro.init has been rearranged to have a more logical structure (courtesy
of Ruoming Pang).
- Craig Leres contributed a number of portability & autoconf tweaks.
- Craig Leres has extended nb_dns.{h,c} to support IPv6 DNS lookups.
Bro does not yet take advantage of these extensions.
- The beginnings of portability to gcc 3.2 were added. There unfortunately
is more work to do here!
- The README has finally been updated to have more correspondence with
the 0.8 release.
0.8a6 Wed Oct 2 18:58:12 PDT 2002
- Upgrade to autoconf 2.53 (Craig Leres).
0.8a5 Tue Oct 1 19:04:53 PDT 2002
- The regular expression matcher how has a mechanism to stop scanning when
no further match is possible (Robin Sommer). If you find problems with
regular expression matching, especially if you're using EXPIRE_DFA_STATES,
please let us know!
- Rule/signature files are now searched for using $BROPATH (Robin Sommer).
In addition, you can define a list of signature files to incorporate
using the new global "signature_files". For example,
redef signature_files += "web-stuff"
will look for web-stuff and web-stuff.sig along $BROPATH.
- The tcp_deliver_undelivered variable is now tcp_match_undelivered and
only applies to delivering otherwise-undelivered data to the signature
matcher; such data is *not* delivered to any associated analyzer.
(Robin Sommer)
- The framework for tracking version numbers now allows negative as
well as positive versions (Robin Sommer).
0.8a4 Tue Oct 1 15:54:58 PDT 2002
- Support for extracting the contents of HTTP replies (and POST's),
and for transforming/anonymizing HTTP traffic, contributed by Ruoming Pang.
- Some minor internal tweaks to the timer management to help track patterns
of timer expiration.
0.8a3 Mon Sep 23 22:48:07 PDT 2002
- HTTP reply handling refined in policy scripts.
- New built-in functions to_int(), to_count() convert strings to
integers/counts.
- Bug fixes for DNS_Mgr and IPv6.
- AckAboveHole alerts now ignored (just written to alert.$BRO_ID) by default.
- Packets belong to ignored connections (for example, partial connections if
partial_connection_ok is false) are no longer recorded to the save file.
- Some minor formatting/naming tweaks.
0.8a1 Sat Sep 21 22:09:23 PDT 2002
- IPv6 support enabled if you build using -DBROv6. Deficiencies: Bro
doesn't yet look up hostnames for AAAA records; no handling of extension
headers (if you have traces of these, please send them to me!); no
handling of FTP PORT/PASV w/ IPv6 addresses (again, if you have traces,
please send them!); DNS analyzer doesn't understand AAAA yet (again,
please send me traces!); you have to change the capture_filter line
in tcp.bro (as indicated in the script) in order to process TCP traffic,
due to deficiencies in libpcap's support for IPv6.
- Bro is migrating towards a more structured way of handling log messages /
alerts. Analyzers now @load alert.bro, which has a function ALERT()
for processing alerts. Soon this function will provide a variety of
filtering/processing hooks; expect changes.
- Bro now has an HTTP response analyzer (contributed by Ruoming Pang).
The HTTP policy scripts have been split up into http.bro (just general
definitions), http-request.bro (handles requests; loaded by http.bro),
http-reply.bro (handles replies; you need to explicitly load this), and
http-detail.bro (handles individual headers). http-reply.bro will be
undergoing some significant reworking in the near future; probably the
scripts will be merged back into a single http.bro plus http-detail.bro.
- ssl-worm.bro contains a prototype policy script for detecting the
Scalper SSL worm (contributed by Robin Sommer). It uses the signature
file sig.ex.ssl-worm.bro. If someone has traces of Scalper in action
to send us, that would be great.
- A new policy script, contents.bro, extracts the contents of each
Bro connection into its own pair of files (one file for each
direction). Use in conjunction with -f or discarder_XXX() to
extract specific connections.
- A new built-in function, strcmp(), returns the usual comparison between
two strings (contributed by Robin Sommer).
- A new event, content_gap(), is generated when Bro detects that it is
forced to skip over data in a reconstructed TCP stream because it is
missing from the packet input.
- BIND8 is no longer included with the distribution. If this causes you
problems, let me know.
- aux/scripts/bro_logchk is a Perl script for parsing Bro HTTP & FTP logs
(contributed by Jim Barlow).
- You can now compare addresses to see which is larger. a < b means
that in network order, the octets making up 'a' are ordered before
those for 'b'. E.g., 128.2.3.4 < 128.2.3.5 < 129.0.0.1. Note that
IPv4 addresses are all < IPv6 addresses (other than IPv4 addresses
that are embedded in IPv6 addresses, e.g., ::128.2.3.4 < 128.2.3.5).
- Serious bug in TCP option handling fixed.
- Some bugs in CRLF handling fixed (courtesy Ruoming Pang).
- Bug in the implementation of &optional fixed.
- Bug in computing memory statistics when not reading packets (from
an interface or the trace file) fixed.
- You can now include a trailing comma after the last item in an
"enum" enumeration list.
- port-name.bro now maps 389/tcp to "ldap".
- A bug has been fixed in loading files multiple times
v0.7a175 Thu Aug 29 21:14:34 PDT 2002
- bro -s <file> reads in a *signature* file to search for regular expressions
in packet payloads or TCP byte streams (written by Robin Sommer).
See policy/rules.bro for an example of a policy script for processing
the matches.
Note that this feature is experimental, and will be evolving in the
near future.
- The python script "snort2bro" reads in Snort signatures and translates
them into Bro signature rules, suitable for processing using -s.
An example of its operation is seen by running
python snort2bro < sig.ex.web-rules.snort
which, after reading in sig.ex.classification.config and
sig.ex.reference.config, generates the output given in
sig.ex.web-rules.bro, which is suitable to use as input to
bro -s.
- bro -d invokes a gdb-like debugger (written by Umesh Shankar). You can
set breakpoints and watchpoints, examine tracebacks, print Bro expressions,
and the like. Type "help" for on-line help.
- bro -t <tracefile> turns on tracing of the policy script execution,
written to the given file.
- Bro now includes an SMTP analyzer, which includes processing MIME
message bodies (written by Ruoming Pang). See smtp.bro and mime.bro
for related policy scripts. smtp.bro includes several experimental
techniques for detecting mail relaying.
- You can now define enumerated types, such as
type scanner_type: enum {
SCANNER_STEALTH, SCANNER_HIGH_SPEED, SCANNER_AMBIGUOUS,
};
Enumerated types can be compared for equality with one another, and used
as table indices, but cannot be converted to/from integers.
- bro -A <file> invokes an experimental, general trace transformation/
anonymization framework (written by Ruoming Pang) which writes a modified
tcpdump trace file from the input (which can be the network or another
trace file) with potentially extensive modifications to the recorded
packets.
Transformers are built from .rw files (currently, {finger,ftp,ident,smtp}.rw),
which are processed by the utility "rwcl" to generate both event engine
analyzer components and rewriter policy scripts (for example, after
configuring and building Bro, you'll find the scripts
policy/{finger,ftp,ident,smtp}.rw.bro).
See policy/smtp-rewriter.bro for an example of a policy script that
performs transformation/anonymization.
- New built-ins:
split(s: string, p: pattern): string_array;
takes a string and splits it into pieces at each occurrence of
the regular expression pattern p. (The functionality is like
that in awk.) It returns a string_array, which is a table[count]
of string that is indexed starting at 1, giving the different
pieces.
For example,
split("foobar", /o/)
returns a 3-element table, for which [1] is the string "f",
[2] is the string "" (empty), and [3] is the string "bar".
split1(s: string, p: pattern): string_array;
split1() does the same thing as split(), but only performs splitting
at the first occurrence, so it returns either a one-element table
(if the pattern doesn't appear in the string) or a two-element
table. split1("foobar", /o/) returns a 2-element table for which
[1] is "f" and [2] is "obar".
md5_hash(s: string): string
returns (in human-readable form) the MD5 hash of a given string.
So, for example,
md5_hash("foobar")
yields "3858f62230ac3c915f300c664312c63f".
to_addr(s: string): addr
takes a string representing an address in "dotted quad" format
and returns the correponding "addr" value.
set_buf(f: file, buffered: bool)
sets the given file to have its writes buffered or unbuffered
depending on the value of "buffered". It does not return a value.
connection_exists: function(c: conn_id): bool
returns T if the given connection identifier corresponds to a
currently instantiated connection (one for which the event engine
has state), F otherwise.
lookup_connection(c: conn_id): connection
returns the "connection" record associated with the given
connection identifier, or a fatal run-time error if there
isn't one.
set_inactivity_timeout(c: conn_id, timeout: interval): interval
sets the inactivity timeout for the given connection to the
given interval, returning the old interval.
If the interval is non-zero, then when no packets have been
processed for a connection after that much time has elapsed,
the connection is deleted, and an "inactivity_timeout" event
generated.
get_matcher_stats(): matcher_stats
used for gathering statistics about the signature matcher
rewriting_trace(): bool
returns T if -A was specified (anonymize/rewrite a trace),
F otherwise.
- New events:
connection_state_remove(c: connection);
Invoked when the event engine has removed the connection from
its state.
connection_SYN_packet(c: connection, pkt: SYN_packet);
Invoked for each SYN/SYN-ACK packet.
connection_timeout(c: connection);
Invoked when the event engine times out a connection - for
example, because the originator sent a SYN that was never
answered, so the connection was never established.
connection_reused: event(c: connection);
Invoked when the event engine decides that a new SYN for
an existing connection reflects a reuse of the connection
four-tuple, rather than belonging to the existing connection.
- New globals:
const ignore_checksums = F &redef;
If true, then the event engine does not verify checksums (and
hence will not discard packets with bad checksums).
const tcp_deliver_undelivered = F &redef;
If true, then when the event engine closes a connection, if
that connection has a chunk of data not yet delivered to its
analyzer (which only happens if the data is above a sequence
hole, indicating either a packet filter glitch or a protocol
botch), then the undelivered data will at that point be delivered
to the connection's analyzer.
const tcp_reassembler_ports_orig: set[port] = {} &redef;
const tcp_reassembler_ports_resp: set[port] = {} &redef;
Sets of ports for which, if a connection has the corresponding
originator/responder port, then the event engine will reassemble
the byte stream of the connection.
Normally, the event engine reassembles byte streams for any
connection for which there's an analyzer, and otherwise doesn't.
These variables can be used to force reassembly for the originator
or responder side (respectively) of connections for which there
isn't otherwise an analyzer. This is useful when doing signature
matching on reassembled byte streams, for protocols that are
not otherwise analyzed by the event engine.
const table_expire_interval = 1 min &redef;
How often to check table entries to see whether they've expired
(see &read_expire, etc., below).
const requires_trace_commitment = F;
If true, then when rewriting/anonymizing traces, nothing will
actually be written to the edited trace file unless you call:
rewrite_commit_trace(c: connection, commit: bool, future: bool)
If "future" is true, then future rewritten packets will be
automatically commited; otherwise, writing them to the trace
file requires another explicit rewrite_commit_trace() call.
const inactivity_timeout = 0 secs &redef;
As noted above, when a connection becomes inactive, time it out
after this interval. If 0 secs, then don't time it out.
- An SSH analyzer extracts SSH client/server version information. See
ssh.bro for the related policy script.
- There's now a (very) simple TFTP analyzer available in tftp.bro.
- You can now set the global "frag_timeout" to an interval which controls
how long fragments are kept before discarding them (contributed by Ashley
Thomas). If you don't set the global, or set it to 0.0 sec, then fragments
are kept around indefinitely.
- An implementation of an experimental anti-evasion technique, "active
mapping", has been written by Umesh Shankar. It is not yet ready for
general use, and isn't compiled in unless -DACTIVE_MAPPING.
- Four new attributes can now be associated with tables (implemented
by Robin Sommer): &read_expire, &write_expire, and &create_expire
will delete table entries after a given interval has elapsed since
the table entry was last read, written, or created. For example:
global a: table[addr] of count &default=0 &create_expire = 5 sec;
will delete each entry added to it 5 seconds after the entry was added,
regardless of subsequent read/write activity to the element.
&expire_func allows you to associate a function with the table such that
whenever an entry expires, the function is invoked. It's passed the
value of the table entry (not the index - perhaps this should be changed),
and returns an "interval" value. If the interval is <= 0.0 seconds, then
the table entry is immediately deleted. Otherwise, it is deleted after
the given interval has elapsed.
- When listing multiple attributes, you no longer separate them with
commas. For example, if you used to have:
global a: table[string] of count &default=0, &redef;
you now need to use:
global a: table[string] of count &default=0 &redef;
- You can now construct records using
[$field1 = <expression>, $field2 = <expression>, ...]
Such record values can be assigned to other records providing that the
target value's type includes all of the fields (same name and type)
present in the record value, and that any missing fields have the
&optional or &default attribute (see next item).
You can also include a record value inside the record constructor, and
all of its fields will be included in the constructed record value.
- Record fields can now be annotated with &optional, to indicate
that the field needn't be present, or &default, which indicates
a default value to provide if the field is missing.
- You can query as to whether a record has a value for a given field
using the new "?$" operator. So for example:
type my_rec: record {
num: count &default = 0;
msg: string; # mandatory, since no &optional/&default
};
global r: my_rec;
r = [$msg = "hello"];
print r?$num, r?$msg, r$num;
will print "F, T, 0," because even though 'r' has a default value
for $num (which shows up when printing r$num), that field is missing,
hence r?$num is F.
- An experimental scheme has been added (by Umesh Shankar) for managing
general attributes associated either with all values ("global attributes")
or particular particular values. This scheme is likely to change in
the near future, and hence isn't explained here further.
- The DNS analysis now includes ADDL and AUTH records, and much richer
policy script analysis (per policy/dns.bro).
- You can now "redef" a function or event handler to override its
previous definition. For a function, this looks like:
redef log_hook = function(msg: string): bool
{
...
}
For an event handler, it's just the usual definition preceded by "redef.
For example,
redef event ack_above_hole(c: connection) { }
would replace the default ack_above_hole handler with one that does nothing.
- HTTP server and HTTP proxy backdoor detectors have been added,
generating http_signature_found and http_proxy_signature_found,
respectively (contributed by Ruoming Pang).
- A KaZaA backdoor detector has been added, which generates
kazaa_signature_found for likely KaZaA connections.
- The new policy scripts flag-irc.bro and flag-warez.bro provide
hooks for defining site policies for detecting IRC and access
to warez.
- portmapper.bro now tracks the services it sees, and the names are
used in connection summaries rather than generic services like port-656.
- bro -C (or redef'ing the "ignore_checksums" global to T) instructs
Bro to ignore any checksum errors and go ahead and analyze such packets.
- The (trivial) policy script print-globals.bro dumps out all of the policy
script global variables and the amount of memory they consume.
- The policy script code-red.bro has been renamed worm.bro and generalized
to detect Nimda as well as Code Red 1 & 2.
- A bunch of additional default sensitive URIs have been added to http.bro.
http.bro also now doesn't report worm-related URIs.
- A bunch of less common portnames were removed from port-names.bro.
- Empty regular expressions are now allowed.
- The finger_request event now has a third parameter, the additional
text after the username.
- More systematic handling of NULs and CRLF by the event engine.
- Hex escape sequences now must have exactly two hexadecimal characters.
- FYI - work has begun on significantly altering the way that policy
scripts generate alerts.
- Work has begun (by Robin Sommer) on a general framework for tracking
client/server versions. See software.bro.
- Work has begun on a NETBIOS analyzer (see NetbiosSSN.cc). Contributions
(e.g., finishing it :-) welcome.
- Work has begun on migrating the internals to process IPv6 in addition
to IPv4.
- A number of bug fixes, leaks, and memory allocation lint tweaks.
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
v0.7a90 Thu Sep 06 00:50:43 PDT 2001
- Better state management, especially for use in high-volume sites. There
are now a number of ways to track the resources consumed by Bro while it
runs, and to reduce the amount of memory Bro consumes. Adding
@load print-resources
will log a summary of the resources consumed by Bro when it exits, and
@load reduce-memory
will change a number of default values in order to significantly diminish
the amount of memory Bro requires.
Other ways to lower the required memory discussed in the next three items.
- The global "maintain_http_sessions" now controls whether http.bro tracks
multiple HTTP connections between the same hosts as a single HTTP session.
Doing so costs a considerable amount of state (so set to F for reducing
the memory impact of HTTP analysis).
- The global "suppress_scan_checks", if true, turns off address and port
scan detection. (You can't achieve this by simply not @load'ing scan.bro,
because it's loaded by some of the default policy scripts.) Turning it
off can save a lot of memory.
- Note, the ident.bro is also expensive in terms of state-holding.
- New library functions:
- resource_usage() returns a record detailing real time,
CPU time, memory, other getrusage info, and the current
and maximum number of TCP/UDP/ICMP connections, and
timers and fragments.
- val_size() returns the size in bytes needed to represent
a given value (which can be a record, a table, or a
simple constant, for example). It's not fully accurate
but is in the right ballpark.
- global_sizes() return a table mapping every global
variable to its size (per val_size()). Useful for
tracking which ones are growing large over time.
- You can now control a number of timers related to Bro's connection
management. Setting them to lower values generally decreases the
amount of state Bro has to keep (see reduce-memory.bro), though
this can also make it easier for an attacker to evade detection:
tcp_SYN_timeout: interval
Controls how long Bro waits after seeing the
beginning of a connection (whether due to a SYN
or not; the timer is misnamed) before checking
whether it elicited any reply.
tcp_session_timer: interval
After a connection has closed, wait this long for
further activity before checking whether to time
out its state.
tcp_connection_linger: interval
When checking a closed connection for further
activity, Bro should consider it inactive if there
hasn't been any for this long. It also complains
if the connection is reused before this much time
has elapsed.
tcp_attempt_delay: interval
Bro waits this long upon seeing an initial SYN
before timing out the connection attempt.
tcp_close_delay: interval
Upon seeing a normal connection close, Bro flushes
state after this much time.
tcp_reset_delay: interval
Upon seeing a RST, Bro flushes state after this
much time.
tcp_partial_close_delay: interval
Bro generates a connection_partial_close event this
much time after one half of a partial connection
closes, assuming there has been no subsequent
activity.
non_analyzed_lifetime: interval
If a connection belongs to an application that you
aren't analyzing, Bro times it out after this
interval. If 0 secs (default), then it doesn't
time it out.
dns_session_timeout: interval
ntp_session_timeout: interval
rpc_timeout: interval
Bro waits this long before timing out a DNS/NTP/RPC
request.
max_timer_expires: count
The maximum number of timers to expire after
processing each new packet. The value trades off
spreading out the timer expiration load with
possibly having to hold state longer. A value of 0
means "process all expired timers with each new
packet".
- Two new flags control other facets of Bro's connection management,
and thus state-holding:
partial_connection_ok: bool
Whether Bro should analyze connections for which
it doesn't see the beginning, only the middle.
This can be very expensive to do in the face of
stealth-scanning, which looks like a bunch of
partial connections.
Note, the HTTP analyzer has been modified to
now always skip partial connections. This should
instead be user controllable.
tcp_SYN_ack_ok: bool
If true, Bro instantiates connection state when
it sees a SYN ack but not the initial SYN (even
if partial_connection_ok is false). The intent
behind this knob (which is not well tested) is
to allow you to filter out initial SYNs and only
react to SYN acks. This keeps Bro from holding
state during SYN scans and SYN flooding, except
for when the destination responds.
- Some other miscellaneous thresholds that you can now modify from your
policy script:
heartbeat_interval: count
How often to generate net_stats_update() events.
This timer really isn't needed any more, since
you can use "schedule" to achieve the same effect.
tcp_storm_thresh: count
If Bro sees this many FINs/RSTs in a row, it
flags them as a "storm".
tcp_storm_interarrival_thresh: interval
The FINs/RSTs must come with this much time or less
between them.
- The state management for line-oriented applications like HTTP requests
has been improved.
- The HTTP analyzer now expands %hex sequences. If anyone has a Unicode
expander to contribute, that'd be terrific.
- The Code Red detection is more robust (fewer false positives).
- A new redefinable variable, skip_services, lists applications that should
not be analyzed for purposes of detecting scans. (Default: ident)
The point of having this (poorly named) hook is so that code-red.bro
can add "http" to it, to avoid reporting the zillions of Code Red scans
that a site can see.
- Bro now matches regular expressions using lazy DFA evaluation. The upshot
of this is (1) it no longer maintains .bro-RE-cache.v1, (2) it starts
up fast regardless of whether you've added new regular expressions, (3)
you can afford to add lots of regular expressions, and (4) it's actually
a bit faster.
- The list of "hot_ids" has been expanded with a number of other common
root-privileged accounts.
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
v0.7a61 Fri Apr 06 14:51:47 PDT 2001
- An NTP analyzer has been added. See policy/ntp.bro for a corresponding
policy file that detects the newly discovered NTP remote buffer overflow
attack.
- example-attacks/ is a new directory that contains trace files of attacks.
Currently, there are just two to play with:
bro -r example-attacks/ftp-site-exec.trace mt
will run on a trace of a "site exec" overflow attack, and
bro -r example-attacks/ntp-attack.trace mt ntp
will run on an example of the NTP overflow.
- The doc/ directory includes the postscript and HTML versions of the
first draft of the Bro manual.
- A new policy file, icmp.bro, has preliminary (and only partially developed)
policy for analyzing ICMP.
- The file libpcap.bufsize.patch includes the patch necessary on some systems
to increase the maximum libpcap buffer size.
- You can now use anonymous functions in &default expressions, so for
example you can do:
global foo: table[count] of string = {
[1] = "1st", [2] = "2nd", [3] = "3rd",
} &default = function(n: count): string { return fmt("%dth", n); };
and then referring to foo[5] will yield "5th".
- There's now a "for" statement to iterate over the indices of a table
or the members of a set:
for ( i in foo )
for the above "foo" will iterate with i assigned to 1, 2, and 3; *but
not in general in that order*.
- The function contains_string() has been removed, and now you can instead
use an expression like
"bar" in "foobar"
which will yield T.
- The scan detection now has a mechanism for attempting to detect SYN flooding
backscatter and flagging it as different from a stealth scan.
- New event handlers:
new_connection_contents()
like new_connection(), but reassembles the
stream so you can use set_content_files() to
write it to a file
udp_session_done()
invoked when a UDP session (which is defined on
a per-protocol basis; currently only for NTP)
finishes.
ntp_message()
invoked for each NTP message
- UDP processing now does accounting for detecting scans.
- UDP processing now tracks numbers of requests/replies for sessions that
support that notion. The connections are annotated by udp_session_done()
with "[m,n]" for "m" requests and "n" replies, providing either m or n > 1.
- New variable accessible from policy:
watchdog_interval
how often the watchdog should check for whether
Bro is making internal progress
- A bunch of functions no longer have a first argument of the current time;
get it instead from network_time() if you need it:
authentication_accepted
authentication_rejected
conn_weird
conn_weird_addl
flow_weird
net_weird
- A bunch of functions now return bool rather than int values:
set_contents_file
set_login_state
set_record_packets
skip_further_processing
- The variable "hot_dests" has been renamed to "hot_dsts".
- 111/tcp is now identified as "portmap" rather than "rpc".
- Connections flagged as hot for some types of characteristics are now
annotated with the reason associated with the decision. (I think a lot
more of this is needed.)
- Portmapper dumps are annotated with the results of the mapping. This
will be streamlined in the future.
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
v0.7a48 Wed Sep 13 14:37:30 PDT 2000
- Changes between this release and v0.6 missing :-(
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
v0.6 Wed Jul 21 17:02:50 PDT 1999
- Support for regular expressions added. You specify lex-style regular
expressions between '/'s, for example "/\/etc\/(passwd|shadow)/" has
the type "pattern" and matches /etc/passwd or /etc/shadow (the slashes
in the pattern need to be escaped or else they'd delimit the end of the
pattern). Pattern-matching is via the "in" operator, so for example:
if ( filename in /\/etc\/(passwd|shadow)/ )
sensitive_file_access(filename);
or
const sensitive_files = /\/etc\/(passwd|shadow)/;
...
if ( filename in sensitive_files )
sensitive_file_access(filename);
Presently the "in" operator requires that the entire left-hand side
be matched by the pattern. So, for example, if you want to find the
string "eggdrop" anywhere inside the string "line", you would use
if ( line in /.*eggdrop.*/ )
If you leave off either of the .*'s, then eggdrop will only be matched
at the beginning or end of the line.
In the future, there will be mechanisms for specifying whether you
want to match anywhere in a line, or anchored; accordingly, *the above
syntax is subject to change*.
Bro compiles regular expressions into DFAs for fast matching. This can take
quite a bit of time for complicated patterns. Consequently, it maintains a
cache of compiled regular expressions in $HOME/.bro-RE-cache-v1. You can
always safely remove this file; Bro will recreate/repopulate it as needed.
It does not clean up unused entries from it, so if you change your patterns
frequently, you will accumulated lots of old ones and should delete the
file to garbage collect them.
- An rlogin analysis module has been added and the telnet analysis
generalized to generic "login" analysis, with the following events:
login_failure(c: connection, user: string, client_user: string,
password: string, line: string)
Generated on a failed attempt to log in. client_user is
the remote user name, if the login is via the rlogin
protocol.
login_success(c: connection, user: string, client_user: string,
password: string, line: string)
Generated on a successful attempt to log in.
login_input_line(c: connection, line: string)
Generated per line of input typed by the user.
login_output_line(c: connection, line: string)
Generated per line of output generated by the server.
login_confused(c: connection, msg: string, line: string)
Generated when a login dialog confuses the heuristic
analyzer. msg is a tag for the state mismatch that
was unexpected, line is the corresponding dialog text.
login_confused_text(c: connection, line: string)
Once a connection is in the confused state, then this
is generated for each subsequent line.
login_terminal(c: connection, terminal: string)
Generated if the terminal type associated with the
connection is seen.
login_display(c: connection, display: string)
Generated if the display associated with the connection
is seen.
excessive_line(c: connection)
Generated when the connection has produced an excessively
long line.
login_input_line() and login_output_line() are very powerful for
detecting intrusions, when coupled with regular-expression matching.
login_terminal() is used to detect backdoors that are triggered
by the terminal environment variable.
- An ident analysis module has been added (port 113). It generates
ident_request, ident_reply, and ident_error events. Port 113 used
to be referred to as "auth"; now it's referred to as "ident".
- A new type of scan detection has been added, which is triggered
by a remote host trying a large number of username/password
combinations. See the account_tried() function in scan.bro.
- The default search path for .bro files is now
.:priv-policy:policy:pub-policy:/usr/local/lib/bro
where priv-policy/ is intended for private policy and pub-policy/
for public policy. The Bro alpha distribution ships with a
sample set of pub-policy scripts.
- New built-ins:
system(s: string): int
executes the given shell command using system()
and returns its status.
set_contents_file(c: conn_id, direction: count, f: file)
copies connection c's reassembled byte stream in
either the originator-to-responder direction (if
direction is CONTENTS_ORIG) or the responder-to-
originator direction (CONTENTS_RESP) to the file f.
reading_live_traffic(): bool
returns true if Bro is running on live traffic (read
from a network interface), false if it's reading from
a save file.
mkdir(f: string): bool
creates the given directory, returning true if it
was able to, false if not.
get_orig_seq(c: conn_id): count;
returns the highest sequence number sent by the
originator of connection c.
get_resp_seq(c: conn_id): count;
same for c's responder.
- Additional new events (other than those related to the new analyzers):
new_connection(c: connection)
is generated whenever a new connection is seen.
partial_connection(c: connection)
is generated whenever a new partial connection (one
that doesn't begin with a SYN handshake) is seen.
pm_bad_port(r: connection, bad_p: count)
is generated when a portmapper response contains
a bad port number.
- Functions, tables and sets can now be assigned. Assignment is
made by reference to the underlying object.
- Bro no longer looks up identifiers using getservbyname() to see if they
should be interpreted as port numbers, since this led to portability
problems. Instead, a number of constants are defined in bro.init:
bgp, domain, finger, ftp, gopher, http, ident, rlogin, smtp, ssh and telnet,
- Bro now supports an arbitrary number of open files (not bound by
the system's limit on file descriptors).
- There's now a finger_reply event to go with finger_request.
- A bunch more RPC service names have been added, thanks to Job de Haas
and others.
- A bug has been fixed in the watchdog handling that caused it to
sometimes expire after a period of network inactivity.
- The Bro paper in doc/ has been revised (it isn't quite up-to-date,
but considerably closer than the USENIX version).
- There has been a large amount of reworking of the internals, both
to Bro itself and in the policy scripts. If you find something you're
wondering about, feel free to send me mail asking about it.
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
v0.5 Sun Oct 4 00:19:35 PDT 1998
- Added Linux support.
- Major autoconf changes.
- Some tweaks to suppress g++ warnings.
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
v0.4 Fri Sep 11 00:08:04 PDT 1998
- The new aux/ directory has some utility programs and scripts.
See aux/README for details.
- Documentation (though exceedingly limited) describing the connection
log summaries generated by policy/tcp.bro now available in doc/conn-logs.
- The Telnet username recognition code has been heavily reworked and is
now significantly more robust.
- A new policy file, policy/scan.bro, detects port and address scanning.
- Bro now detects the "Land" attack (a trivial addition to its spoof
detection).
- The distribution now comes with BIND 8.1.2.
- A new flavor of "weird" event has been added, flow_weird, for unusual
events associated with <src IP addr, dst IP addr> flows (i.e., a coarser
granularity than TCP or UDP connections).
- Bro now reassembles fragments, checking for overlaps and consistency.
Associated with fragments are the following flow_weird's:
excessively_large_fragment
excessively_small_fragment
fragment_inconsistency
fragment_overlap
fragment_protocol_inconsistency
fragment_size_inconsistency
fragment_with_DF
incompletely_captured_fragment
- The TCP stream reassembly code was rewritten to share functionality
with the new fragment reassembly code.
- If a handler for it is present, then Bro will generate "packet_contents"
events containing the contents of each packet it receives. This is just
for experimenting with, and, in particular, this event does *not* reflect
TCP stream reassembly.
- The handling of "conn_weird" events in tcp.bro now has more options.
In the weird_action table, WEIRD_LOG_ALWAYS means that the weird event
should always be logged; WEIRD_LOG_PER_CONN means that it should be
logged once per connection; and WEIRD_LOG_PER_ORIG that it should be
launched once per originator IP address.
- The example hot.bro now includes provisions for flagging sensitive
inbound and outbound services.
- Bro now limits the number of events processed when draining the event
queue after processing a packet. This makes Bro less prone to packet
loss during high-speed scans.
- The line-oriented TCP endpoint class was split into TCP_EndpointLine,
which supports the line-oriented functionality, and TCP_NVT (derived
from TCP_EndpointLine), which implements the Network Virtual Terminal
used by Telnet and FTP.
- The TCP_NVT class now understands the Telnet Environment option.
- Escape sequences are now '\' followed by 1 or more octal digits,
instead of excatly three octal digits (which is error prone).
- If the watchdog timer expires, it now reports the number of events
processed in the current batch of packets, as well as other timing
information.
- Bro now should not report packet drops that occur after it has begun
to exit (these can occur when draining the pending event queue takes
a while).
- Bro now detects TCP acknowledgements that occur above a sequence
hole, generating an ack_above_hole event. Nominally, this indicates
packet filter drops, but in fact some buggy TCPs manage to do this :-(.
- Fledgling HTTP support added. An http_request event is generated when
a new HTTP request is seen, and http_stats is generated when an HTTP
connection terminates, giving (uninteresting to most people) statistics
concerning the request(s). A lot more is needed: parsing persistent
connections and HTTP replies, for one. This is just a start. A stub
for http_reply exists but these events are not presently generated.
- Ported to Linux (thanks to Pascal Bouchareine).
- A bug in to_lower() and to_upper() was fixed.
- The reporting for unexpected FTP connections now more directly
identifies the corresponding FTP session.
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
v0.3 Fri Feb 13 19:59:39 PST 1998
- Timers are now implemented using calendar queues rather than priority
queues (with thanks to Craig Leres).
- The new byte_len() built-in returns the length of a string interpreted
as a set of bytes (including any NUL bytes, especially the final one
for a typical string).
- The new sub_bytes() built-in extracts a subset of a string interpreted
as a set of bytes (i.e., immune to any embedded NULs).
- Fixed bad interaction with the latest version of libpcap that on some
BPF systems would cause Bro to exit any time a little bit of time went by
without any traffic matching its filter.
- A bug with constant-folding of the ?: operator has been fixed.
- A new "conn_stats" event delivers statistical analysis (number of packets
transmitted, retransmitted, out-of-order, replicated) of each connection.
If you define a handler for it, then *no* other TCP processing is done.
This was added for off-line analysis of traces with large numbers of
connections in them.
- Some minor portability tweaks.
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
v0.2 Sat Oct 25 11:22:37 PDT 1997
- Added autoconf support, thanks to Scott Denton.
- Ported to FreeBSD, Solaris.
- Fixed a bug in which partial connections were ignored even for
protocols (e.g., FTP) that can make use of partial dialogs.
- Included BIND version 8 sources directly in the distribution.
- Better usage() information (again thanks to Scott), -h and -v flags.
- README, CHANGES files created.
BroControl
1.7 | 2017-06-26 15:55:09 -0700
* Release 1.7.
1.6-3 | 2017-06-26 10:52:27 -0400
* Set a value for the global_hash_seed global constan
Upon broctl install or deploy, broctl now sets a random value for the Bro
script constant global_hash_seed. This value is stored in the broctl
state database so that we can use the same value every time after it is
created.
This is needed for bloom filters to be mergeable across a bro cluster.
Addresses BIT-1819. (Daniel Thayer)
1.6 | 2017-06-06 17:43:14 -0500
* Release 1.6
* Pruning CHANGES a bit (Daniel Thayer)
1.5-49 | 2017-04-30 12:53:44 -0400
* Allow more than one logger to be defined.
This adds initial support for running a Bro cluster with multiple logger
processes. This is primarily useful for installations that use something
like Kafka or Logstash to aggregate logs. (Daniel Thayer)
* Add a "--version" option to show broctl version (Daniel Thayer)
* Added a new option MailReceivingPackets to allow users to disable
broctl cron mail that no packets were seen on an interface. (Daniel Thayer)
* A large number of unused code removal and code cleanups (Daniel Thayer)
* Fix some failing tests when using python 3 (Daniel Thayer)
* The "start" helper script now reports error if PID string is empty (Daniel
Thayer)
* Fixed the sorting of node names in command output (e.g. "worker-10"
should be output after "worker-2"). Now the order of names is based on
the "count" node attribute instead of the name. (Daniel Thayer)
* Fixed some bugs in stats-to-csv script (proxies were being handled like
workers, and it was assuming that the manager is named "manager").
Also added more error checking. (Daniel Thayer)
* Fix potential cases of unhandled IndexError and ValueError. (Daniel Thayer)
* Fixed a few cases where the ps plugin didn't return non-zero when an
error occurred. (Daniel Thayer)
* Fix shell scripts to no longer depend on bash (Daniel Thayer)
* Improve the run_cmds() and run_localcmd() functions by returning output
as a string (instead of list of strings) and check and handle output
string correctly in all cases. (Daniel Thayer)
1.5-21 | 2017-03-17 13:18:58 -0400
* Fix some tests to make sure the test tmp dir is removed (Daniel Thayer)
* Update crash-diag script due to recent change where "bro -v" now outputs
the version to stdout. Also fixed crash-diag to not show stderr output
from running "bro -N". (Daniel Thayer)
* Add a new broctl option to expire crash directories
Added functionality to broctl cron to remove crash directories older than
the number of days specified in the new option CrashExpireInterval (the
default value is 0, which means crash directories never expire). (Daniel
Thayer)
* Add a test for expiration of crash directories (Daniel Thayer)
* Reduce disk usage of post-terminate and crash-diag
Changed post-terminate and crash-diag so that the bro binary is not
copied when there is no core file. Also, the crash report is now
saved to disk only when crash-diag is run from post-terminate (i.e.,
the "diag" command will no longer create any files). (Daniel Thayer)
* Change archive-log to use "mv" instead of "cp"
Changed archive-log to "mv" (rather than "cp") logs when not using gzip
for better efficiency. This means we will not have the logs in the tmp
directory when Bro crashes, so the scripts have now been simplified to
never attempt to keep a copy of archived logs in the tmp dir (previously,
logs >100MB were always being deleted anyway). (Daniel Thayer)
1.5-12 | 2017-03-13 13:43:43 -0400
* Prevent the broctl check and scripts commands from hanging
Changed the check-config script to run bro with the "-a" option
when running "broctl check" in the hope that this will prevent broctl
from hanging for any reason. The "-a" option prevents bro
from running any bro script statements (previously, "check" would
cause bro to exit after handling the bro_init event) but should still
be able to identify the same bro scripting errors as before.
Also, to prevent "broctl scripts" from hanging, set the value
of "exit_only_after_terminate" to False (in broctl/check.bro) in case
another script sets the value of that constant to True. Since "bro -a"
prevents bro from creating the loaded_scripts.log file, that option
cannot be used with "broctl scripts". (Daniel Thayer)
1.5-9 | 2017-01-26 16:38:17 -0500
* Fix some failing tests
Added a new broctl option, called StopWait, to force the stop command
to wait for the post-terminate script to finish. This is needed
because some tests were failing due to background log-archive processes
creating logs after "broctl stop" finished, which was preventing the
test directory from being deleted. (Daniel Thayer)
* Fix post-terminate to not generate invalid timestamps
Fixed the code that tries to extract the base name and timestamp
from a log filename, because it wasn't extracting them correctly
when the base name contained a period (this doesn't happen for any
of the standard Bro logs) or if the timestamp in the filename wasn't in
the format YYYY-MM-DD-HH-MM-SS (this could happen if Bro terminates
but for some reason doesn't execute the code in the writers/ascii.bro
script that renames the log, or if someone uses a different forma
by redefining Log::default_rotation_date_format). The fix involves
first removing the log suffix, then trying to extract the timestamp
in one of the two default timestamp formats. This procedure is more
reliable than the previous method of making assumptions about how many
period characters should be in a log filename.
Also, when Bro terminates normally, post-terminate now just tries to
archive all log files, instead of only those that were rotated. This
is to avoid missing any logs. This also means that the
stderr.log/stdout.log files are now archived when Bro terminates
normally (instead of only when Bro crashes), which is useful to
capture any error messages from archive-log or Bro.
Also fixed an issue that could occasionally occur when post-terminate
archives an unrotated log file (i.e., no timestamp in the filename)
and a different log with the same base name was archived after
post-terminate started, then the computed start time of the unrotated
log would be later than the end time. Fixed by setting the start time
to equal the end time.
Also added the node name to the subject line in the email sent when
post-terminate fails to archive a log. (Daniel Thayer)
* Add error checking of archive-log timestamp parameters
Check if the format of the timestamp command-line parameters matches
the required format. If not, exit with an error message. This will
prevent archive-log from creating an archived log file with a corrup
filename or in a directory with a corrupt name.
Also simplified the code that gets the current century. (Daniel Thayer)
1.5-5 | 2017-01-26 13:34:37 -0500
* Fix crash-diag script to use the correct debugger, because on some systems
the correct debugger to use is not called "gdb" (currently, this
affects OS X and OpenBSD). (Daniel Thayer)
1.5-2 | 2016-12-06 12:35:40 -0800
* Don't show output of "ulimit -v" in crash reports on OpenBSD; adjusting
it always fails and showing the value only creates confusion. (Daniel Thayer)
1.5 | 2016-11-16 14:51:05 -0800
* Pruning CHANGES a bit. (Daniel Thayer)
* Update broctl.rst using "make doc". (Daniel Thayer)
1.5-beta2 | 2016-11-02 11:08:45 -0700
* Release 1.5-beta2.
1.5-beta-56 | 2016-11-02 13:44:41 -0400
* A number of portability fixes, mostly related to OpenBSD. (Daniel Thayer)
1.5-beta-48 | 2016-11-02 13:38:34 -0400
* Fix bug where standalone bro port isn't recorded to state.db, and
add more test cases. (Daniel Thayer)
1.5-beta-41 | 2016-11-01 09:34:19 -0700
* Add support for local-logger.bro site policy script. (Daniel Thayer)
* Add a few clarifications to broctl documentation. (Daniel Thayer)
1.5-beta-31 | 2016-10-07 14:55:07 -0400
* Improve diag command output. (Daniel Thayer)
* Add new option SitePolicyScripts to replace SitePolicyStandalone.
Also marked SitePolicyManager, SitePolicyWorker, and SitePolicyStandalone
as deprecated in the documentation. (Daniel Thayer)
* Fix a couple of failing tests. (Daniel Thayer)
* Fix a failing test on FreeBSD. (Daniel Thayer)
* Improved the documentation, especially documentation of node attributes,
documentation of broctl commands, and added a section about Bro/BroControl
communication. (Daniel Thayer)
1.5-beta-24 | 2016-09-26 16:24:21 -0400
* Define all BroControl exceptions in the new exceptions.py file.
The broctl client will now handle only those exceptions, showing a
useful error message instead of a stack trace. As before, if a
standard Python exception is raised (this is not expected to occur),
then broctl will terminate with a stack trace, which is useful to
help debug the problem. (Daniel Thayer)
1.5-beta-22 | 2016-09-26 16:11:21 -0400
* Fix crash-diag script to not confuse log files with core files
Fixed the crash-diag script to not include any log filenames that
contain the word "core" in the list of core files. (Daniel Thayer)
* Improve crash-diag script's handling of core filenames
Fixed the script to handle filenames that contain a space. (Daniel Thayer)
1.5-beta-19 | 2016-09-26 15:50:22 -0400
* Fix a bug where broctl loses state of running Bro nodes
If a node name contains uppercase letters, then restarting broctl while
that node is running results in a confusing warning about that node still
running, and broctl discards the PID of that node. Fixed by converting
the node name to lowercase before checking the state database (where all
keys are converted to lowercase).
Addresses BIT-1676. (Daniel Thayer)
* Report an error if a user defines node names differing only by case (such
as "worker-1" and "Worker-1"). This check is needed because keys
in the state db are converted to lowercase. (Daniel Thayer)
* Improve error messages for plugin API functions (Daniel Thayer)
* Removed the restriction that plugin state variables must be string
type, because normal state variables have no such restriction. (Daniel Thayer)
* Fixed the getGlobalOption() function in the plugin API. It did not
convert its argument to lowercase, and could return the value of a
state variable.
Also simplified some code by replacing the config has_attr() function
with a new function get_option(), which helps reduce the number of places
in the code where keys are converted to lowercase. (Daniel Thayer)
* Do not set a plugin state var. with invalid name (Daniel Thayer)
* Improve code that sets plugin option values
Improved error messages to include the name of the plugin, and fixed a
problem where any option with an invalid name was being set (now such
options are skipped). (Daniel Thayer)
* Simplify some broctl cron-related code by using get_state() (Daniel Thayer)
* Fix the subst() function for non-string data types (Daniel Thayer)
* Code simplification involving the config get_state() function
Added an optional default parameter to the config get_state() function,
and changed that function to convert the key to lowercase. These changes
help simplify some code by reducing the number of conversions to lowercase. (Daniel Thayer)
* Simplify code by not converting option values to lowercase (Daniel Thayer)
* Fix problem with custom node keys that are not lowercase (Daniel Thayer)
* Improve documentation of case-sensitive issues in broctl (Daniel Thayer)
* Remove redundant lowercase conversions of state var. names (Daniel Thayer)
1.5-beta-2 | 2016-09-01 12:03:46 -0400
* Improve crash reports by showing Bro plugin info (Daniel Thayer)
1.5-beta | 2016-08-12 13:20:27 -0700
* Release 1.5-beta.
* Fix rsync error message to not show ssh login banner. (Daniel Thayer)
* Run "make doc" to update broctl.rst (Daniel Thayer)
* Pruning CHANGES a bit (Daniel Thayer)
1.4-150 | 2016-08-09 13:38:17 -0400
* Show python stack trace if unexpected exception is raised.
(Daniel Thayer)
* Improve broctl error messages and error handling across the board.
(Daniel Thayer)
* Add a new optional node type "logger" that will handle logging
instead of the manager. (Daniel Thayer)
1.4-132 | 2016-07-14 18:23:27 -0400
* Don't run capstats on interfaces with packet source prefix. (Daniel Thayer)
1.4-130 | 2016-07-13 14:36:34 -0400
* Improve the text of crash reports with instructions on how to
get a backtrace, which should reduce the amount of useless crash
reports mailed to the Bro team. (Daniel Thayer)
1.4-127 | 2016-07-06 08:58:18 -0500
* Ignore packet source prefix of interface name when using capstats. (Jan Grashoefer)
1.4-125 | 2016-07-02 17:53:42 -0500
* New plugin function "broctl_config" so plugin authors can add their own
script code to the autogenerated broctl-config.bro script. (Seth Hall)
1.4-122 | 2016-07-02 12:05:23 -0500
* Follow symlinks to directories when searching for plugins. (Jon Siwek)
1.4-119 | 2016-06-28 11:11:19 -0400
* Fix race condition in reading/writing broctl-config.sh (Daniel Thayer)
1.4-117 | 2016-06-22 12:14:37 -0400
* Improve broctl behavior when unable to stop a node. (Daniel Thayer)
1.4-112 | 2016-06-14 16:14:52 -0700
* Fix a failing test on some platforms and improve its error
message. (Daniel Thayer)
* Add Bro plugin directory to broctl plugin search path. (Daniel Thayer)
* Update test baselines. (Daniel Thayer)
* Changed the default value of the StatusCmdShowAll option so that
the broctl status command runs faster. (Daniel Thayer)
* Changed the status-timefmt test so that it can be run in parallel
with the other tests. (Daniel Thayer)
* Remove dead code and update docs. (Daniel Thayer)
* Rename serialization set for cluster tests. (Daniel Thayer)
* Change node hostname resolution to be more consistent. (Daniel Thayer)
* Add another test for broctl start command. (Daniel Thayer)
* Prevent start helper from getting in infinite loop. (Daniel Thayer)
1.4-100 | 2016-05-17 16:22:25 -0700
* Updating baseline for Bro control framework change. (Robin Sommer)
* Fix for running broctl tests on OS X 10.11 (Daniel Thayer)
1.4-96 | 2016-04-28 13:43:22 -0400
* Fix inconsistent return value data type for some commands, so that
they always return a CmdResult. (Daniel Thayer)
1.4-94 | 2016-04-28 13:29:34 -0400
* Fix the top command on OS X 10.10 or newer. (Daniel Thayer)
* Fix build-bro script for running broctl tests on FreeBSD. (Daniel Thayer)
1.4-91 | 2016-03-31 15:08:24 -0500
* Explicitly close the Broccoli connection to avoid resource leak. (Aaron Eppert)
1.4-89 | 2016-03-31 12:02:19 -0500
* Prevent ssh login banners from appearing in broctl output. (Jon Schipp)
1.4-87 | 2016-03-31 10:35:47 -0400
* Eliminate unnecessary writes to the state db. (Daniel Thayer)
1.4-84 | 2016-03-11 16:32:46 -0600
* Support ip command for getting local IP addrs. (Jon Schipp)
1.4-77 | 2016-01-20 14:44:36 -0500
* Changed LogExpireInterval to allow users to specify a more
granular log expire interval, which is a number followed by
a unit: "day", "hr", or "min". An integer value with no unit
is still allowed and interpreted the same as before. (Daniel Thayer)
* More verbose error message for logexpireinterval value. (Daniel Thayer)
* Prevent log expire interval from being less than rotation interval. (Daniel Thayer)
* Improve the ps test diff canonifier. (Daniel Thayer)
* Improve the cron-expire test script. (Daniel Thayer)
1.4-70 | 2016-01-19 22:42:10 -0600
* Fix custom plugin commands to behave more like built-in commands. (Aaron Eppert/Daniel Thayer)
* Add README.rst -> doc/broctl.rst symlink. Addresses BIT-1413 (Johanna Amann)
1.4-61 | 2015-12-19 13:39:47 -0800
* Add broctl.cfg options PcapSnaplen and PcapBuflen to set pcap's
packet snap length and buffer size, respectively. (Jan Grashoefer)
1.4-57 | 2015-12-11 12:00:07 -0500
* Simplify some code and fix a test that can fail on OS X. (Daniel Thayer)
* Improvements to broctl documentation. (Daniel Thayer)
* Improve diagnostic and error messages. (Daniel Thayer)
* Add more private IP space to etc/networks.cfg (Daniel Thayer)
* Add a new broctl option, MailArchiveLogFail, to control sending
log archive mail. (Daniel Thayer)
* Check for invalid option names and values more carefully. (Daniel Thayer)
* Fix use of ssh to always use IP address to avoid host key verification
failures, and use BatchMode consistently to avoid a misleading
error message when rsync fails. (Daniel Thayer)
* Changed post-terminate to attempt to archive logs that have already
been rotated. Also changed crash-diag output file extension to no
longer use ".log" in order to avoid post-terminate trying to
archive it. (Daniel Thayer)
* Send email if post-terminate fails to archive logs, and changed
the post-terminate script to run archive-log serially instead
of multiple instances simultaneously in the background.
(Daniel Thayer)
* Rename logs in the spool/tmp/post-terminate directory to indicate
they were successfully archived when archive-log is run with the "-c"
option. (Daniel Thayer)
* Capture output of background post-terminate script to file
"post-terminate.out" which might be helpful for debugging
problems with log archival. (Daniel Thayer)
* Add bro node type to post-terminate dir name (Daniel Thayer)
1.4-36 | 2015-12-08 13:21:05 -0500
* Fix problem of unexpected ifconfig output with some locales (Daniel Thayer)
1.4-34 | 2015-10-27 21:13:15 -0500
* Added plugin for custom load balancing (Jan Grashoefer)
1.4-30 | 2015-08-21 17:23:39 -0700
* Updating submodule(s).
1.4-28 | 2015-07-29 15:33:37 -0500
* Handle a missing broctl-config.sh symlink (Justin Azoff)
1.4-26 | 2015-07-27 14:13:43 -0400
* Create broctl-config.sh automatically (Daniel Thayer)
* Undo a previous change for lb_procs error checking (Daniel Thayer)
* Update broctl.rst by running "make doc" (Daniel Thayer)
* Convert boolean config values to python bool type (Daniel Thayer)
1.4-20 | 2015-07-27 09:12:44 -0400
* Merge remote-tracking branch 'origin/topic/dnthayer/ticket1434' (Justin Azoff)
* Improve the broctl top helper script for FreeBSD (Daniel Thayer)
1.4-18 | 2015-07-27 09:03:22 -0400
* Improve error message for invalid broctl plugin config values (Daniel Thayer)
* Improve error message for invalid broctl config values (Daniel Thayer)
* Improve error checking for local IP addresses (Daniel Thayer)
* Cleanup some error msgs and source code comments (Daniel Thayer)
* Close ssh connections upon config reload (Daniel Thayer)
* Check for dangling Bro nodes every time node.cfg is loaded (Daniel Thayer)
* Improve check for dangling Bro nodes (Daniel Thayer)
* Remove unnecessary state variable type conversions (Daniel Thayer)
* Convert config option values to correct data type (Daniel Thayer)
* Check config file contents rather than timestamp (Daniel Thayer)
* Add ability for broctl to reload its configuration, which the
deploy command will do if a config file change is detected. (Daniel Thayer)
* Avoid caching config values because config might change (Daniel Thayer)
* Update a broctl test file (Daniel Thayer)
* Keep track of both loaded plugins and active plugins (Daniel Thayer)
* Reorganize some code (no changes in functionality) (Daniel Thayer)
* Remove some config options and add a new one (Daniel Thayer)
1.4-1 | 2015-07-22 13:20:49 -0500
* Fix test setup script to not overwrite LD_LIBRARY_PATH (Jon Siwek)
1.4 | 2015-06-09 09:19:56 -0500
* Release 1.4.
1.4-beta-22 | 2015-06-02 10:34:44 -0500
* Update broctl man page for deploy command (Daniel Thayer)
* Updating baselines. (Robin Sommer)
1.4-beta-20 | 2015-05-28 12:15:28 -0700
* Slight output tweaks. (Robin Sommer)
1.4-beta-19 | 2015-05-28 11:59:39 -0700
* Improve documentation on site-specific customization. (Daniel
Thayer)
* Don't use daemon threads in ssh_runner. (Daniel Thayer)
* Improve broctl documentation. (Daniel Thayer)
* Fix minor error with restart clean. (Daniel Thayer)
* Improve and extend tests. (Daniel Thayer)
* Improve error messages related to the env_vars option. (Daniel Thayer)
* Remove code that was automatically removing quoted values of the
env_vars option. (Daniel Thayer)
* Show help when user runs broctl with unknown command. (Daniel
Thayer)
* Improve visibility of archive-log error messages. (Daniel Thayer)
* Add sanity checks on broctl options. (Daniel Thayer)
* Improve error messages involving the state database file.
Addresses BIT-1397 (Daniel Thayer)
* Fixed error when a broctl command outputs binary data. (Daniel
Thayer)
* Fix the config change warnings on Python 3. (Daniel Thayer)
* Fix an issue with the ps plugin where the "run-bro" script would
appear in the output on some systems. (Daniel Thayer)
* Inform user to run broctl deploy to get started. (Daniel Thayer)
* Fix communication with muxer for newer Python versions. (Daniel
Thayer)
* Set correct Python path in Python scripts. (Daniel Thayer)
1.4-beta | 2015-05-07 20:26:22 -0700
* Release 1.4-beta.
1.3-221 | 2015-04-22 15:20:20 -0500
* Improve the test build script to show build error output. (Daniel Thayer)
1.3-220 | 2015-04-21 14:54:49 -0400
* Fix problem where use of broargs causes error message (Daniel Thayer)
* Avoid unnecessary string building in logging functions (Daniel Thayer)
* Handle broctl output messages more consistently (Daniel Thayer)
* Don't show certain warnings when they're not useful (Daniel Thayer)
* Fix the interactive command tab completion feature (Daniel Thayer)
* Simplify some SQL and remove unused code in the state database (Daniel Thayer)
1.3-212 | 2015-04-17 15:27:14 -0500
* Fix the use of the "first-line" helper script (Daniel Thayer)
* Added a new broctl option "CommandTimeout" that specifies the number
of seconds to wait for a command to return results. This value is
passed to ssh_runner. (Daniel Thayer)
* Improve error reporting for ssh_runner (Daniel Thayer)
* Changed the status command to run only one helper script so that the
status command takes half as long to run in the worst-case scenario.
This involved replacing the "cat-file" helper with a new one that
can handle multiple files, and only outputs the first line of each file.
(Daniel Thayer)
* Remove unused default timeout values in ssh_runner. Also changed the
ping timeout and changed the code to actually use it. (Daniel Thayer)
* Fix response handling (Justin Azoff)
* Enable json serialization of CmdResult objects (Justin Azoff)
* Enable BatchMode for ssh
From the ssh manual:
If set to ``yes'', passphrase/password querying will be disabled.
This option is useful in scripts and other batch jobs where no user
is present to supply the password. (Justin Azoff)
* Improve some error messages (Daniel Thayer)
* Fix to prevent broctl from hanging when an exception occurs.
Make sure that the finish method is called (to signal that we're done
to the ssh_runner worker threads). (Daniel Thayer)
1.3-197 | 2015-04-16 16:15:25 -0500
* Use daemon threads only for remote hosts (Daniel Thayer)
* Fix to prevent the broctl stop command from hanging (Daniel Thayer)
* Remove the run-cmd helper script (Daniel Thayer)
1.3-185 | 2015-04-03 14:54:06 -0400
* Update test baselines. (Daniel Thayer)
* Improved error reporting in several cases. (Daniel Thayer)
* Added checks if there are any nodes to start or stop to avoid
executing code unnecessarily. (Daniel Thayer)
* Preserve order of hosts in command lists to be executed. (Daniel
Thayer)
* Catch the KeyboardInterrupt exception. (Daniel Thayer)
* Reorganize code for the df command. (Daniel Thayer)
* Python 3 compatibility fixes. (Daniel Thayer)
* Make sure "broctl deploy" error messages are visible. (Daniel Thayer)
* Speedup the deploy command by checking only one node of each node
type. (Daniel Thayer)
* Fix a race condition that results in data loss on the SSH control
channels. (Daniel Thayer)
* While waiting for lock, show owning PID of lock. (Daniel Thayer)
* Make sure broctl always closes any file that it opens. (Daniel Thayer)
* Update broctl install requirements list. (Daniel Thayer)
* Don't show log header lines in "broctl scripts" output. (Daniel
Thayer)
* Added functions to cleanup before broctl terminates (Daniel
Thayer)
1.3-165 | 2015-03-30 13:46:23 -0500
* BIT-1326: Add configure-time check for required sqlite3 python
module. (Jon Siwek)
1.3-162 | 2015-03-17 09:36:26 -0700
* Update the documentation. (Daniel Thayer)
* Add a new command "deploy" which does a "check", "install", and
"restart". The intention of this command is to reduce the chance
that users will forget to install after modifying their
configuration. (Daniel Thayer)
* Sort broctl command output for easy readability.
* Remove duplicate nodes from input so that broctl can't run a
command twice for the same Bro node. (Daniel Thayer)
* Improve error output. (Daniel Thayer)
* Allow specifying alternate Bro script directory via "--scriptdir"
option of the configure script when building Bro. (Daniel Thayer)
* Allow specifying alternate location for etc/ directory via the
"--conf-files-dir" option of the configure script when building
Bro. (Daniel Thayer)
* Simplify internals of the main broctl script. (Daniel Thayer)
* Removed the use of BROCTL_INSTALL_PREFIX for modifying the install
prefix at run-time. This was only intended for use by the test
scripts. Now the test setup scripts just modify all the files
where the install prefix is hard-coded. (Daniel Thayer)
1.3-150 | 2015-03-04 12:17:42 -0800
* Significant improvements (mostly internal), reorganization, and
cleanup across the whole code base. (Justin Azoff and Daniel
Thayer)
This includes:
- Refactor broctl to make it usable as a library (reduce global
state, module-level setup code, and functions return results
instead of printing).
- Integrate ssh_runner code into broctl to fix current problems
(use only one connection per host instead of one per Bro node;
broctl shouldn't hang when a host goes down or if we forgot to
run "broctl install"),
- Write state info using SQLite state storage instead of writing
to a plain text file (broctl.dat).
- When the node config changes, we now do additional checks if
there are any Bro nodes running that are no longer in our node
config and warn user if any are detected.
- Keep track of the expected state (running or stopped) of each
Bro node, and have broctl cron start or stop nodes as needed.
- Improved broctl cron by adding two new options (MailHostUpDown
and StatsLogEnable) to enable users the option to turn off
unwanted functionality to speed up broctl cron and reduce the
chance of errors.
- When broctl cron tries to send email but fails, now it will
output a message that includes the text it was trying to mail.
- Silence warning messages that are intended for interactive use
of broctl when broctl cron runs to reduce unwanted emails from
cron.
- Added new broctl option StatusCmdShowAll to enable users to
speed up "broctl status" significantly.
- Fixed the stats-to-csv script to not create files that can
never include any data.
- Fixed archive-log script to detect exit status of gzip or cp
command, so that we don't delete log file when the archival
fails.
- Improved post-terminate script to process log files more
consistently.
- Made all broctl command output go to stdout (previously, some
output would go to stderr, which made grepping or redirecting
the output more difficult),
- Improved the default broctl.cfg file to show more of the
useful options.
- Added more error checks to help catch errors earlier.
- Some error message output is more specific and helpful now.
1.3-12 | 2014-12-08 13:53:23 -0800
* Add man page for broctl. (Raúl Benencia)
1.3-9 | 2014-12-01 12:03:53 -0600
* Remove execute permission on scripts not needing it. (Raúl Benencia)
1.3-8 | 2014-10-31 09:17:27 -0500
* BIT-1166: Add configure options to fine tune local state dirs.
(Jon Siwek)
1.3 | 2014-06-02 08:59:01 -0700
* Fix for capstats to display correct interface name when using
PF_RING+DNA with pfdnacluster_master. (Daniel Thayer)
* Fix for capstats with PF_RING+DNA pfdnacluster_master.
(Daniel Thayer)
1.3-beta | 2014-05-19 16:29:36 -0500
* Improve documentation of PFRINGFirstAppInstance option (Daniel Thayer)
* Update broctl.rst with "make doc" (no other changes) (Daniel Thayer)
* Move some content into the main Bro docs in a new section "Cluster
Configuration". (Daniel Thayer)
* Rename the broctl option pfringdnafirstappinstance to
pfringfirstappinstance. (Daniel Thayer)
* Remove references to the now unused BROMAGIC (Daniel Thayer)
1.2-129 | 2014-05-01 20:58:28 -0700
* A bug fix and feature add for PF_Ring support. (Seth Hall)
- Reset the app_instance for the case where there
are multiple dnaclusters on a single host.
- Add naming support for zerocopy (zc) clusters.
* Use a hash to determine if a config change occurred. (Daniel Thayer)
* Change hosts() function in the plugin API to return a list of
nodes instead of just hostnames. (Daniel Thayer)
* Add warnings when node config or broctl.cfg has changed. (Daniel Thayer)
* Code simplification, remove the unused broctl "home" option, and
improved a couple warning messages. (Daniel Thayer)
* Fixed a bug where broctl cron could email about the "$total"
pseudo-node not receiving any packets. (Daniel Thayer)
* Code reorganization for the getDf function to avoid direct output
and thereby reporting the same error message multiple times for
the same host. (Daniel Thayer)
* Cleanup some code for style consistency, reformat some comments to
fit on an 80-column display, and remove some dead code. (Daniel
Thayer)
* Replace the update-stats script with Python code. (Daniel Thayer)
* Gather disk usage by host rather than by node. The output now also
shows both node and host names and is now sorted by node type.
* Adjust column widths for top, netstats, peerstatus commands.
(Daniel Thayer)
* Change the broctl exec command to run only once per host. (Daniel
Thayer)
* Changed the hosts() function so that it preserves the order of the
returned node list as it was sorted by the nodes() function.
(Daniel Thayer)
1.2-106 | 2014-04-10 08:32:18 -0700
* Update test baselines, and minor code cleanup. (Daniel Thayer)
1.2-104 | 2014-04-05 01:01:29 -0400
* Updated PF_Ring plugin now supports PF_Ring+DNA. (Seth Hall)
1.2-99 | 2014-03-30 22:21:20 +0200
* Update documentation with better install/setup instructions.
Addresses BIT-1160 (Daniel Thayer)
1.2-97 | 2014-03-16 07:40:31 -0700
* Minor doc update for a broctl option. (Daniel Thayer)
* Adjust broctl status output to avoid bad column alignment. (Daniel
Thayer)
* Do not ping when checking if a host is alive. Removed the ping
from the host alive check because the ping might be blocked by a
firewall, and neither Bro nor broctl needs the ability to ping
hosts. (Daniel Thayer)
* If the current version of Bro doesn't match the version when
broctl install was previously run, then a warning message (to run
broctl install) is displayed when broctl starts. Addresses
BIT-1152. (Daniel Thayer)
* Reduce the risk of losing track of state info. Changed the way
broctl updates PIDs and crash flags by writing the new values to
disk immediately, one at a time, as soon as each new value is
available. Also changed the way that the state file is updated
when each command finishes by doing the update as an atomic
operation. (Daniel Thayer)
* Better error handling for a number of broctl commands. (Daniel Thayer)
* Improve error output when broctl install has not been run yet.
(Daniel Thayer)
* Fix a failing test on FreeBSD 10. (Daniel Thayer)
* Changed the output of the check command to be more specific about
what it is actually checking. (Daniel Thayer)
* Improve handling of dead hosts and closed/hanging connections.
(Daniel Thayer)
* Fixed a typo in the run-bro script that was causing the memlimit
option to be ignored. Added added a test to verify that memlimit
is used. (Daniel Thayer)
* Simplify code that execs commands locally. (Daniel Thayer)
* Prevent infinite loop in start helper script if it cannot execute
the run-bro script. (Daniel Thayer)
* pf_ring plugin: Show error if lb_procs is needed but not given,
and disable plugin if not used. (Daniel Thayer)
* Catch an exception that is raised when loading a plugin that does
not override all required methods, and output an error message.
(Daniel Thayer)
* Fix start helper script to return nonzero on error. (Daniel
Thayer)
* Improve start/stop command output for crashed nodes.
* Added a test for stopping a node that crashes during shutdown.
(Daniel Thayer)
1.2-73 | 2014-02-28 14:44:51 -0800
* Added ability of broctl cron to expire entries in stats.log that
are older than the number of days specified in the new broctl
option StatsLogExpireInterval. Addresses BIT-123. (Daniel Thayer)
* Add broctl option BroPort to change the starting Bro port.
Addresses BIT-1117. (Daniel Thayer)
1.2-66 | 2014-02-06 20:29:20 -0800
* Make sure logs are archived after broctl kills Bro. Addresses
BIT-1126. (Daniel Thayer)
1.2-63 | 2014-02-04 09:10:39 -0800
* Fix a few sporadic test failures. (Daniel Thayer)
1.2-61 | 2014-01-31 11:11:39 -0800
* Fix error handling for process command. (Daniel Thayer)
* Update and improve the tests of broctl process. (Daniel Thayer)
* Improve broctl help message for the process command. (Daniel
Thayer)
* Reorder the broctl process command Bro arguments. Addresses
BIT-1124. (Daniel Thayer)
1.2-56 | 2014-01-28 15:54:14 -0800
* A large set of improvements to the test build scripts to address
error scenarios, fix failures to report problems, and provide
convenience features. (Daniel Thayer)
Includes:
- New Makefile target "rerun" to more easily re-run failed
tests.
- Two new environment variables recognized by test scripts:
* If Bro fails to build, you can define an environment
variable BROCTL_TEST_BUILDARGS which specifies
additional options that will be passed to Bro's
"configure" script.
* Defining BROCTL_TEST_USEBUILD will use the Bro default
build directory (instead of a custom build directory for
the broctl tests).
* Add lots of new tests. (Daniel Thayer)
1.2-28 | 2014-01-22 10:47:49 -0800
* Fix bug with timemachineport broctl option. (Daniel Thayer)
* Improved formatting of cluster-layout.bro for readability. (Daniel
Thayer)
1.2-26 | 2014-01-21 07:12:38 -0800
* Update the docs. (Daniel Thayer)
1.2-23 | 2014-01-20 12:22:42 -0800
* Move some output about slow nodes to debug.log. (Daniel Thayer)
* Improve broctl output formatting. (Daniel Thayer)
* Fix redundant emails from broctl cron when dead host found.
(Daniel Thayer)
* Fix broctl top on OS X Mavericks. (Daniel Thayer)
* Fix plugin init return values. This also fixes the myricom plugin,
which wasn't explicitly returning a value from its init method and
therefore was being disabled as a result. (Daniel Thayer)
* Enable dead hosts caching while in cron mode. (Justin Azoff)
* Use getattr for looking up plugin methods for simplifying the
plugin code. (Justin Azoff)
* Remove redundant plugin initialization. (Justin Azoff)
1.2-12 | 2014-01-20 11:23:23 -0800
* Fix bug with IPv6Comm broctl option, which had no effect. (Daniel Thayer)
1.2-10 | 2014-01-13 01:57:53 -0800
* Add a new option "PFRINGClusterType" that allows a user to specify
a PF_RING cluster type; it defaults to 4-tuple (which is different
from the 6-tuple that previous versions used). The PF_RING plugin
uses this information to set the corrresponding environment
variable for a PF_RING-aware libpcap. Addresses BIT-1108. (Daniel
Thayer)
* Minor reorganization of the README to avoid redundancy. (Daniel
Thayer)
1.2-3 | 2013-12-09 13:24:28 -0800
* Remove unused Broxygen-style script comments. (Jon Siwek)
1.2 | 2013-11-07 07:04:54 -0800
* Release 1.2.
1.2-beta-28 | 2013-11-06 00:22:24 -0800
* Improve check-pid helper script. (Daniel Thayer)
1.2-beta-26 | 2013-11-01 04:51:57 -0700
* Add another warning message when a host is not alive. (Daniel
Thayer)
1.2-beta-24 | 2013-10-31 00:19:41 -0700
* Do not check if the local host is "alive". (Daniel Thayer)
1.2-beta-22 | 2013-10-26 19:19:31 -0700
* Document which broctl options override Bro script variables.
(Daniel Thayer)
* Updates and clarifications to docs. (Daniel Thayer)
1.2-beta-17 | 2013-10-18 13:22:16 -0700
* Fix internal lookup of nodes, which would fail to return the right
items in some cases when node naming didn't match standard
terminology. Addresses BIT-1091. (Daniel Thayer)
1.2-beta-13 | 2013-10-10 13:38:58 -0700
* Updating copyright notice. (Robin Sommer)
* Fix the broctl "top" command output on Linux. (Daniel Thayer)
* Fix a race condition when sendmail option is empty string. (Daniel
Thayer)
* Fix a deadlock when capturing output from local command. (Daniel
Thayer)
* Improve portability of shell scripts used by broctl. (Daniel
Thayer)
* Fix for setting REPO in Makefile. (Robin Sommer)
1.2-beta | 2013-09-23 20:30:31 -0700
* Update 'make dist' target. (Jon Siwek)
* Fix problem with the "broargs" options that would occur when a
command-line argument in broargs contained a space. (Daniel
Thayer)
* Change submodules to fixed URL. (Jon Siwek)
1.1-190 | 2013-09-20 14:26:41 -0700
* Add more links in BroControl documentation. (Daniel Thayer)
1.1-188 | 2013-09-18 14:46:10 -0700
* Add tests for new BroControl features (CPU pinning, PF_RING
multiple cluster IDs, "env_vars") (Daniel Thayer)
* Fix link to git repo to be consistent with other links. (Daniel
Thayer)
* Fix broken doc links. (Jon Siwek)
1.1-182 | 2013-08-27 13:32:35 -0700
* Improve CPU pinning documentation and error message. Addresses
BIT-1068 (Daniel Thayer)
* Switching to relative submodule paths. (Robin Sommer)
* Documentation fixes. (Daniel Thayer)
* Minor fixes for broctl tests. (Daniel Thayer)
* Fix bug with usage of cmd_restart_pre method. (Daniel Thayer)
* Remove unused subdirectory "spool/scripts". (Daniel Thayer)
* Remove unused imports, variables, and semicolons. (Daniel Thayer)
1.1-171 | 2013-08-16 15:36:14 -0700
* Changed and document the behavior of the SitePolicyPath broctl
option to not clobber existing files/directories when copying, in
order to match the expected behavior (directories earlier in the
list take precedence over directories later in the list when
duplicate filenames are encountered). Addresses BIT-714. (Daniel
Thayer)
* A series of changes to make broctl return useful exit codes. (Vlad
Grigorescu, Daniel Thayer).
Generally, broctl now returns 0 if everything went ok with regards
to what the documentation says should have happened, and 1
otherwise. We keep the following exceptions for now though:
- "cron" always returns 0.
- "status" and "top" return 0 if all bro nodes are
running, and returns 1 otherwise.
- commands provides by plugins always return 0.
1.1-158 | 2013-08-02 17:06:57 -0700
* Add ability to set environment variables in node.cfg and
broctl.cfg via new "env_vars" options taking a comma-separated
list (e.g., "env_vars=VAR1=1,VAR2=2"). Variables in node.cfg take
prioroty over broctl.cfg. Addresses BIT-1010. (Daniel Thayer)
1.1-150 | 2013-07-14 08:00:44 -0700
* Fix broken link in README. (Johanna Amann, thanks kraigu)
1.1-148 | 2013-07-03 17:06:44 -0700
* Updates to test infrastructure. (Daniel Thayer)
- Fix canonifier script for handling missing gdb.
- Update baselines for recent changes to crash-diag.
- Remove "make quick" from the README.
- Minor cleanup of the build script.
- Remove unused Makefile variable.
- Remove the "-j" option to make as it can cause lock-ups on
some machines.
- Replace realpath command with more portable Python equivalent.
1.1-140 | 2013-06-07 16:35:08 -0700
* Adding OS to crash output. (Robin Sommer)
* Giving the broctl test suite its own build directory. (Robin Sommer)
1.1-137 | 2013-05-31 17:16:14 -0700
* New regression test suite for BroControl. "make test" runs it. See
testing/README for more information. (Daniel Thayer)
1.1-101 | 2013-05-24 17:55:41 -0700
* Add support for CPU pinning. To use CPU pinning, a new per-node
option "pin_cpus" can be specified in node.cfg, and the OS must be
either Linux or FreeBSD (if such a node.cfg is used on another OS,
then the "pin_cpus" option is ignored). Addresses #996. (Daniel
Thayer)
1.1-99 | 2013-05-24 17:34:44 -0700
* Allow multiple conn-summary.log files to be processed to avoid
conflicts when stopping Bro shortly after a log rotation. (Daniel
Thayer)
* Prevent deletion of unarchived logs during "broctl stop" when
archiving takes a while. (Daniel Thayer)
1.1-94 | 2013-05-17 13:29:04 -0700
* Don't import readline, it's loaded implicitly already. (Daniel
Thayer)
1.1-92 | 2013-05-17 07:37:13 -0700
* Removing uncessary directory check. (Robin Sommer)
1.1-91 | 2013-05-16 20:25:00 -0700
* Stop trying to create the stats/www directory if it already
exists. Addresses #1007. (Seth Hall)
* Another batch of fixes. (Daniel Thayer)
This includes:
- Fix usage of PF_RING interface containing semicolons.
- Fix broctl exec command to check for errors.
- Fix a race condition during broctl start.
- Remove some dead code.
- Fix exit status output in debug log.
* Add support for the "--scriptdir" configure option. Adresses
#993. (Daniel Thayer)
1.1-79 | 2013-05-10 19:39:55 -0700
* A set of bug fixes and robustness improvements. (Daniel Thayer)
This includes:
- Add more error checking and reporting to cron command.
- Improve error checking of top helper output.
- Improve error checking of capstats output.
- Fix a bug when the time command is not found.
- Fix the broctl top and cron commands on OS X.
- Fix a couple of bugs in the broctl ps plugin.
- Remove unused broctl scripts.
- Improve the check-pid helper script.
1.1-63 | 2013-04-25 16:14:51 -0400
* Add support for multiple PF_RING cluster IDs
Instead of assigning the same PF_RING cluster ID to every worker
in a Bro cluster, the pf_ring broctl plugin has been modified to
automatically assign a different PF_RING cluster ID for each se
of workers on a host that all sniff the same interface. The firs
such set of workers on a host are assigned the globally-configured
PF_RING cluster ID (this is the "pfringclusterid" broctl option in
broctl.cfg). Each subsequent set of workers on a host that sniff
another interface are assigned a different value (incremented by
one from previous value). Addresses #943. (Daniel Thayer)
1.1-61 | 2013-03-22 12:25:22 -0700
* Fix problem with the cron command hanging sometimes. Addresses
#591. (Seth Hall)
1.1-59 | 2013-03-17 13:36:04 -0700
* Lots of small fixes, cleanup, and documentation improvemets (in
particular, but not only, to the plugin API). (Daniel Thayer).
This includes:
- Check for plugins with same prefix
- Prevent capstats from being run with invalid args
- Fix plugin inconsistency for certain broctl commands
- Document the broctl user option KeepLogs?
- Add a note in documentation about editing crontab
- Fix broctl plugin option names to be case-insensitive
- Remove reserved word "cluster" from node args
- Fix documentation of broctl commands
- Add calls to plugin cmd_restart_pre/post methods
- Fix instructions for adding plugin directories
- Fix the broctl check command to report results
- Fix handling of cmd_diag_pre for diag command
- Changed return value of plugin API "execute" method
- Add return value to some cmd_<cmd>_pre methods
- Add a check for state variables in broctl.cfg
- Changed "hosts" method to return list of hosts
- Call "done" method from plugin API
- Call hostStatusChanged with correct arg type
- Fix the parseNodes method in plugin API
- Fix the "error" method in broctl plugin API
- Fixed tab-completion of commands with node args
- Fix broctl plugin API documentation errors
- Fix typos in TestPlugin? output messages
- Add cron "--no-watch" option to broctl "help" output
- Fix the "execute" method of the Plugin class
- Fix various bugs and remove some unused code
1.1-26 | 2012-12-20 17:53:52 -0800
* Add Bro version to crash reports. (Robin Sommer)
* Add a new broctl option "MailConnectionSummary" that specifies
whether or not to mail the connection summary reports. (Daniel
Thayer)
1.1-23 | 2012-12-06 15:52:20 -0800
* Update documentation for recent MailFrom change. (Daniel Thayer)
1.1-21 | 2012-12-06 08:34:14 -0800
* MailFrom broctl.cfg option now adds a redef for Notice::mail_from.
(Jon Siwek)
* Bump CPack RPM package requirement to python >= 2.6.0. (Jon Siwek)
1.1-18 | 2012-10-31 14:24:27 -0700
* Add new broctl.cfg option "MailAlarmsInterval" to allow user to
specify alarm mail interval. Default is once per day. (Daniel
Thayer)
1.1-12 | 2012-10-24 15:53:48 -0700
* Add a message at the top of broctl-generated crash report emails
that explains how to submit the crash report to a mailing list
address. Addresses #876. (Daniel Thayer)
1.1-10 | 2012-10-19 15:10:20 -0700
* Fix `broctl install` to now also copy subdirs in SitePolicyPath.
Addresses #902. (Jon Siwek)
1.1-8 | 2012-10-19 14:52:23 -0700
* Add options CompressCmd and CompressExtension to customize log
compressions scheme. (Justin Azoff)
1.1-3 | 2012-09-25 06:23:34 -0700
* Updates to documentation. (Daniel Thayer)
1.1 | 2012-08-24 15:09:04 -0700
* Fix MailAlarmsTo broctl config option. Addresses #814. (Daniel
Thayer)
* Fix configure script to exit with non-zero status on error. (Jon
Siwek)
1.1-beta-2 | 2012-08-10 12:29:56 -0700
* Updates to disable STDERR printing from the reporter framework.
(Seth Hall)
1.1-beta | 2012-07-20 07:03:21 -0700
* Fix broctl startup when using custom config file dirs. (Jon Siwek)
* Change crash report info to include stack traces from all threads.
(Jon Siwek)
* Changed the invocation of gdb that produces the crash report. (Jon
Siwek)
1.0-64 | 2012-07-10 16:07:50 -0700
* Remove automatic override of config file directory with /usr prefix.
* Small updates to BroControl docs. (Daniel Thayer)
1.0-58 | 2012-07-02 15:55:06 -0700
* Improvements to built-in load-balancing support. Instead of adding
a separate worker entry in node.cfg for each Bro worker process on
each worker host, it is now possible to just specify the number of
worker processes on each host. (Daniel Thayer)
This change adds three new keywords to the node.cfg file (to be
used with worker entries): lb_procs (specifies number of workers
on a host), lb_method (specifies what type of load balancing to
use: pf_ring, myricom, or interfaces), and lb_interfaces (used
only with "lb_method=interfaces" to specify which interfaces to
load-balance on).
Two new broctl plugins (which operate automatically and the user
doesn't need to be aware of them) are added to set the appropriate
environment variables when either PF_RING or myricom
load-balancing is being used.
1.0-43 | 2012-07-02 15:40:01 -0700
* Improve README. Rewrote the section on site-specific customization
so that it is more clear about the load order of scripts relevant
to site-specific customization. Removed the description of
several features that don't seem to work: "worker-1.local.bro" is
not automatically loaded, there is no example policy in
local-manager.bro, local-manager.bro and local-worker.bro do not
automatically load local.bro, and proxies do not automatically
load local-worker.bro. (Daniel Thayer)
1.0-40 | 2012-06-06 11:52:06 -0700
* Fix the "cron disable" command, which didn't work. This also
removes the config option CronEnabled. The command is now the
only way to turn off cron operation. (Daniel Thayer)
1.0-38 | 2012-05-24 17:42:37 -0700
* Improvements to IPv6 support. (Jon Siwek)
- Add ability to manage a cluster over non-global IPv6 scope (e.g.
link-local), by specifying "zone_id" keys per node in node.cfg
and "ZoneID" option in broctl.cfg.
- Replace socket.gethostbyname lookups with socket.getaddrinfo to
support IPv6.
- ::1 is now recognized as the IPv6 loopback and a "local" address
where before 127.0.0.1 was expected.
- Update usages of ping, ssh, rsync, and ifconfig to work with IPv6
addresses.
- New "IPv6Comm" option in broctl.cfg can be set to 0 to turn off
IPv6-based communication capabilities (on by default).
1.0-35 | 2012-05-17 11:57:30 -0700
* BroControl tweaks to support non-ASCII logs. (Robin)
- The main change is that we give another argument to
post-processors that indicates the writer type that produced
the log. That comes with an incompatible part: the
make-archive-name script now receives the writer as its
2nd(!) argument. Customized versions need be adapted.
- The standard postprocessors now check whether they are
processing something else than ASCII logs and adapt their
behaviour accordingly (e.g., by not compressing, and or not
running trace-summary).
1.0-32 | 2012-05-14 17:20:17 -0700
* Fix typos in broctl docs. (Daniel Thayer)
1.0-29 | 2012-05-03 11:34:29 -0700
* Added an option to specify 'etc' directory. Addresses #801.
(Daniel Thayer)
* Fix typos. (Daniel Thayer)
1.0-24 | 2012-04-24 14:37:49 -0700
* Update some broctl option descriptions. (Daniel Thayer)
1.0-22 | 2012-04-19 09:52:44 -0700
* Options SitePolicyStandalone, SitePolicyManager, and
SitePolicyWorker were unused. Now they are, and they replace the
hard-coded defaults if defined. Addresses #797. (Daniel Thayer)
1.0-20 | 2012-04-19 09:08:32 -0700
* Remove unused broctl options and fixed a couple of typos in the
option names. (Daniel Thayer)
1.0-17 | 2012-04-16 18:06:28 -0700
* Fixed lots of documentation typos and broken links. (Daniel
Thayer)
* Update broctl help information. (Daniel Thayer)
1.0-13 | 2012-04-09 15:59:17 -0700
* Remove "-p" option from broctl "scripts" command help. (Daniel
Thayer)
* Updating helper script to work with conn.log in Bro 2.0. (Daniel
Thayer)
1.0-9 | 2012-03-28 15:46:02 -0700
* Improve error message when failing to update broctl-config.sh
symlink (Jon Siwek)
* Raise minimum required CMake version to 2.6.3. (Jon Siwek)
* Remove the unused "PolicyDirBroCtl" option. (Daniel Thayer)
* Rename the spool/policy directory so it is less visible. Addresses
#767. (Daniel Thayer)
1.0 | 2012-01-10 18:57:50 -0800
* Tweaks for OpenBSD support. (Jon Siwek)
0.5-beta-43 | 2012-01-03 14:45:40 -0800
* broctl now creates spool directories it finds missing. Addresses
#716. (Edward Groenendaal)
0.5-beta-39 | 2011-12-16 02:49:28 -0800
* Add StopTimeout option to broctl.cfg that sets the number of
seconds to wait after issuing the 'stop' command before sending a
SIGKILL to Bro instances. Adresses #608. (Jon Siwek)
* Add CommTimeout option to broctl.cfg that sets the number of
seconds to timeout Broccoli connnections. Addresses #608. (Jon
Siwek)
* Re-order the way local.bro and local-<node>.bro scripts are
loaded. Node-specific local scripts now load after local.bro so
tha identifiers defined by the loading of local.bro can be used in
them. Addresses #663 (Jon Siwek)
0.5-beta-34 | 2011-12-02 17:17:14 -0800
* Make BroControl more robust when a node dies. (Robin Sommer)
* Disable collecting of prof.logs. The logs can get huge, which lets
cron take a while. (Robin Sommer)
* Fix standalone->cluster upgrade failing to update logs/current
symlink. Fixes #676. (Jon Siwek)
* Fix broctl 'scripts' command in cluster mode. Fixes #655. (Jon
Siwek)
* Teach 'check' command to generate temporary versions of autogen.
files. Addresses #658. (Jon Siwek)
* Submodule README conformity changes. (Jon Siwek)
0.5-beta-20 | 2011-11-14 20:04:21 -0800
* Fixing some platforms behaving poorly during configure-time checks
when a superproject's languages didn't encompass a subproject's.
(Jon Siwek)
* Configure sendmail option in options.py instead of broctl.cfg.
Fixed #645. (Jon Siwek)
* Fix extraneous installation of BroControl plugins. (Jon Siwek)
* Apply patch for BroControl Python 2.3/2.4 compatibility. Closes
#662. (William Jones)
* Avoid rerunning the previous command when hitting just enter in
broctl. (Justin Azoff)
0.5-beta-12 | 2011-11-06 19:23:43 -0800
* broctl.cfg now determines sendmail location at configure-time.
Addreses #645 (Jon Siwek)
* Disable log expiration by default. Addresses #613. (Jon Siwek)
* Make symlink to broctl-config.sh update with `broctl install`.
Addresses #648 (Jon Siwek)
* Fixed a problem when host= in standalone is not 127.0.0.1 or
localhost. (Seth Hall)
0.5-beta | 2011-10-27 17:45:15 -0700
* Updating submodule(s). (Robin Sommer)
0.41-143 | 2011-10-26 10:15:16 -0500
* Update submodules. (Jon Siwek)
0.41-142 | 2011-10-25 20:17:25 -0700
* Updating submodule(s). (Robin Sommer)
0.41-137 | 2011-10-25 15:44:18 -0700
* Updating CHANGES and VERSION. (Robin Sommer)
* Make dist now cleans the copied source. (Jon Siwek)
0.41-130 | 2011-10-18 08:03:35 -0700
* Distribution cleanup and some README fixes. (Robin Sommer)
* Fixed a bug caused by communication framework API update. Reported
by Daniel. (Seth Hall)
0.41-128 | 2011-10-06 17:23:03 -0700
* Change broctl.cfg LogRotationInterval to be specificed in seconds. (Jon Siwek)
* Force broctl 'process' command to enable local logging. Addresses
#632 (Jon Siwek)
0.41-124 | 2011-10-05 16:58:10 -0700
* New broctl.cfg option for log rotation interval. Addresses #630.
(Jon Siwek)
* Removed some of the broct/nodes/* scripts and instead
consolidated their functionality into the node-specific scripts
that come with Bro's cluster framework. (Jon Siwek)
* Within the cluster framework, local-<node>.bro scripts should now
be loaded after the distributions <node>.bro script so things can
be overrided. (Jon Siwek)
* Auto-generated broctl scripts are loaded after all node-specific
scripts and can override their options. (Jon Siwek)
* Move configuration of PFRINGClusterID from broctl.cfg.in to
options.py. Addresses #621. (Jon Siwek)
* Add configure-time check for libpcap PF_RING support. Addresses
#621 (Jon Siwek)
* Fixing typo with process command. (Robin Sommer)
* Script cleanup. (Seth Hall)
- Reshuffling "check" functionality into check.bro.
- Removing some code to deal with the non-existent react framework.
* Give check command its own script for tuning options. Addresses
#618). (Jon Siwek)
* Stop and restart command now stop worker nodes first. Addresses
#596. (Jon Siwek)
* broctl check no longer rotates logs. Addresses #618. (Jon Siwek)
0.41-101 | 2011-09-08 02:20:28 -0400
* Implementing PF_RING environment variables. (Seth Hall)
0.41-99 | 2011-09-04 09:08:59 -0700
* Added --with-pcap configure option. (Jon Siwek)
* Various smaller tweaks to CMake setup. (Jon Siwek)
* Removed alarm log mailing postprocessing script from BroControl.
(Jon Siwek)
* Log rotation is disabled when using the 'process' command to
analyze trace files. (Jon Siwek)
* Fixed 'scripts' command. (Jon Siwek)
* Fixed inconsistent rotated-log naming. (Jon Siwek)
* Changed the 'mail-log' postprocessor to mail alarm.log's. (Jon
Siwek)
* Fix Config.state key capitalization inconsistencies. (Jon Siwek)
* Fixes for broctl 'check' command. Addresses #548. (Seth Hall and
Jon Siwek)
* Updated README. (Jon Siwek)
* Copy bro binary only in NFS mode (fixes #361). (Jon Siwek)
* Fix install command failing because of missing parent dirs. (Jon Siwek)
* Removing the analysis.dat file since it's not used anymore. (Seth Hall)
* Better informational output if attempt to remove old scripts
before installing new ones failes. Addresses #470. (Craig Leres)
* Updating log rotation support for the new logging rotation code.
(Seth Hall)
* Updates for cleanup and meshing with Bro reorg. (Seth Hall)
0.41-73 | 2011-08-13 12:14:28 -0700
* Moving README*. into subdir doc. The top-level README is now
auto-generated. (Robin Sommer)
0.41-68 | 2011-08-05 12:49:30 -0700
* Install example config files dynamically when the distribution
version differs from existing version on disk. (Jon Siwek)
0.41-63 | 2011-08-03 22:10:40 -0700
* Revamped how the work is split between Bro and BroControl. Much of
functionality previously found in BroControl policy scripts has
moved over to Bro. (Seth Hall)
* Adapted BroControl to Bro 2.0 policy scripts.
* A new plugin interface allows external Python code to hook into
BroControl processing. See README for more information. (Robin
Sommer)
Two example plugins are shipped: (1) "ps.bro" shows all Bro
processes currently running on any cluster node, even if not
managed by BroControl; (2) "TestPlugin" is a demo plugin
demonstrating all the functionality a plugin can use (but doesn't
do anything sensible with it).
* A new offline mode for processing a trace. The new command
"process <trace>" runs Bro offline on the given trace, using the
current BroControl configuration. One can optionally give give
further Bro command line options and scripts. In cluster mode the
the Bro process loads both manager and worker configurations
simultaniously.
Addresses #273. (Robin Sommer)
* Removed the "analysis" command. (Seth Hall)
* Installation does no longer differentiate between standalone and
cluster mode. node.cfg now fully controls this. (Seth Hall)
* Tons of little fixes, improvements, and polishing (Seth Hall, Jon
Siwek, and Robin Sommer)
0.41-9 | 2011-06-01 11:35:36 -0700
* Standardize shell script hashbang on install. (Jon Siwek)
* Fix binary package broctl-config.sh symlink installation
regression. (Jon Siwek)
* Changes to allow DEB packaging via CPack, addresses #458. (Jon Siwek)
* Fixed a problem with the "update" command, which could delete data
from many global state tables unintentionally. (Seth Hall)
0.41-2 | 2011-05-02 11:29:07 -0700
* Symlink install scripted at install time for CMake 2.6
compatibility. (Jon Siwek)
0.41 | 2011-04-07 21:14:53 -0700
* Tweaks to the documentation generation. (Robin Sommer)
* CMake tweaks. (Jon Siwek)
* Bugfix: trace-summary sampled in standalone mode rather than cluster
mode. (Robin Sommer)
* Bugfix: Creating links from the log directory to the current log files
didn't work in standalone mode. (Robin Sommer)
0.4-19 | 2011-01-31 15:26:48 -0800
* A new option CompressLogs (default on), indicating whether
archived logs are to be gzipped. (Robin Sommer)
* A lot of configure/cmake/install/package tuning. (Jon Siwek)
* Adding /sbin and /usr/sbin to path local-interfaces script
searches for ifconfig. Closes #293. (Robin Sommer)
* Fixing uncaught exception in lock file handling. (Seth Hall).
* Making cluster event specifications redefinable. (Seth Hall).
* Fixing for pretty printing numerical values. (Seth Hall).
* Fixing "netstats" command distinction between cluster and
standalone mode. (Justin Azoff)
0.4-10 | 2011-01-15 14:14:05 -0800
* Changes for CPack binary packaging (Jon Siwek)
* Fix package configuration macro returning from sub-project too early (Jon Siwek)
* Add warning when building and installing are done by different users (Jon Siwek)
* Changes to broctl's "make install" process (Jon Siwek)
- Simplify install by not compiling python code.
- The broctl-config.sh symlink needs to be made at configure time
and install()'ed in order for CPack packaging to correctly bundle it
- Reverted a change in (90ddc4d) to that caused spool/ and logs/
directories to not be installed in the case that they existed at
configure time.
* Fix for PackageMaker not accepting non-numeric versions (Jon Siwek)
0.4-9 | 2011-01-12 08:51:11 -0800
* Making df portably deal with long lines in the OS's df output.
(Robin Sommer)
0.4-8 | 2011-01-04 20:30:41 -0800
* Changing some installation paths. "broctl install" copied a
number of files to share/bro/*, which violates the common
assumption that things there are static. It can also create
permission problems if the user running "broctl install" is not
the one installing Bro. So now the pieces copied/generated by
"broctl install" are moved to spool/*. (Robin Sommer)
* The CMake install does no longer recreate some of the top-level
directories when they already exist. That makes it possible to
now symlink them somewhere else after the first install. (Robin
Sommer)
* When broctl doesn't find spool/broctl.dat it no longer aborts
but just warns. That allows CMake to skip installing an empty
one. (Robin Sommer)
* Deleting an unused policy file. (Robin Sommer)
* Updating update-changes script. (Robin Sommer)
0.4-5 | 2010-12-20 14:10:25 -0800 | 768a9e550c3554de2e0bf9e3af2ae99400203046
* New helper script for maintaing CHANGES file. (Robin Sommer)
0.4-1 | 2010-12-20 12:03:34 -0800 | a05be1242b4e06dca1bb1a38ed871e7e2d78181b
* Fix for dealing with large vsize values reported by "top" (Craig
Leres)
* Fixed the top helper script to assign the command variable
appropriately. (Seth Hall)
* Escape commands given to CMake's execute_process (Jon Siwek)
0.4 | Fri Dec 10 01:35:36 2010 -0800 | df922e8a64a631aadb485b5044fe9ae1046d47ca
- Moving BroControl to its own git repository.
- Converting README to reST format.
- Renamed "Capstats" config option to "CapstatsPath".
- Merge with Subversion repository as of r7098. Incorporated changes:
o Increasing default timeouts for scan detector significantly.
o Increasing the manager's max_remote_events_processed to
something large, as it would slow down the process too much
otherwise and there's no other work to be interleaved with it
anyway.
o Adding debug output to cluster's part of catch-and-release
(extends the debugging already present in policy/debug.bro)
o Fixing typo in util.py. Closes #223.
o Added note to README pointing to HTML version.
o Disabling print_hook for proxies' remote.log.
o broctl's capstats now reports a total as well, and stats.log
tracks these totals. Closes #160.
o Avoiding spurious "waiting for lock" messages in cron mode.
Closes #206.
o Bug fixes for installation on NFS.
o Bug fix for top command on FreeBSD 8.
o crash-diag now checks whether gdb is available.
o trace-summary reports the sample factor in use in its output,
and now also applies it to the top-local-networks output (not
doing the latter was a bug).
o Removed the default twice-a-day rotation for conn.log. The
default rotation for conn.log now is now once every 24h, just
like for all other logs with the exception of mail.log (which is
still rotated twice a day, and thus the alarms are still mailed
out twice a day).
o Fixed the problem of logs sometimes being filed into the wrong
directory (see the (now gone) FAQ entry in the README).
o One can now customize the archive naming scheme. See the
corresponding FAQ entry in the README.
o Cleaned up, and extended, collection of cluster statistics.
${logdir}/stats now looks like this:
drwxr-xr-x 4 bro wheel 59392 Apr 5 17:55 .
drwxr-xr-x 96 bro wheel 2560 Apr 6 12:00 ..
-rw-r--r-- 1 bro wheel 576 Apr 6 16:40 meta.dat
drwxr-xr-x 2 bro wheel 2048 Apr 6 16:40 profiling
-rw-r--r-- 1 bro wheel 771834825 Apr 6 16:40 stats.log
drwxr-xr-x 2 bro wheel 2048 Apr 6 16:25 www
stats.log accumulates cluster statistics collected every time
"cron" is called.
- profiling/ keeps the nodes' prof.logs.
- www/ keeps a subset of stats.log in CSV format for easy plotting.
- meta.dat contains meta information about the current cluster
state (in particular which nodes we have, and when the last
stats update was done).
Note that there is not Web setup yet to actually plot the data
in www/.
o BroControl now automatically maintains links inside today's log
archive directory pointing to the current live version of the
corresponding log file (if Bro is running). For example:
smtp.log.11:52:18-current -> /usr/local/cluster/spool/manager/smtp.log
o Alarms mailed out by BroControl now (1) have the notice msg in the
subject; and (2) come with the full mail.log entry in the body.
Broccoli
1.100 | 2017-06-05 10:21:11 -0700
* Release 1.100.
1.99 | 2016-11-02 10:54:30 -0700
* Release 1.99.
1.98 | 2016-08-12 13:16:00 -0700
* Release 1.98.
* Increasing data format version for Bro change. (Robin Sommer)
1.97-14 | 2016-04-07 13:31:52 -0700
* Fix some typos in the Broccoli user manual. (Daniel Thayer)
1.97-9 | 2016-03-04 12:37:43 -0800
* Update for new CMake OpenSSL script (Johanna Amann)
* Add README.rst -> README symlink. Addresses BIT-1413 (Johanna
Amann)
1.97-3 | 2015-11-10 13:31:13 -0800
* Fix to compile with OpenSSL that has SSLv3 disalbed. (Christoph
Pietsch)
1.97 | 2015-05-07 11:48:17 -0700
* Use @rpath in broccoli.dylib's install_name on OS X.
* Fix a memory leak: table attributes weren't freed. (Jon Siwek)
1.96 | 2014-05-19 16:17:14 -0500
* Remove code corresponding w/ Bro's unused Val::attribs. (Jon Siwek)
1.95-13 | 2013-12-09 13:23:48 -0800
* Remove unused code in bro_vector_set_nth_val(). (Jon Siwek)
* Fix memory leaks in relation to freeing BroVectors. (Jon Siwek)
1.95-10 | 2013-12-04 09:34:59 -0800
* Update type serialization format/process to align with Bro's
changes to preserve type name info and remove old compatibility
stuff. (Jon Siwek)
1.95-3 | 2013-12-03 10:53:34 -0800
* Add support for consuming events w/ vector args. (Jon Siwek)
Producing events w/ vector args is still unsupported, and bindings
are still missing support as well.
1.95 | 2013-11-06 00:23:50 -0800
* Don't build ruby bindings by default, use --enable-ruby to do so.
(Jon Siwek)
1.94 | 2013-10-24 16:49:30 -0700
* Release.
1.93-17 | 2013-10-15 11:19:19 -0700
* Fix a minor memory leak recently introduced. (Jon Siwek)
1.93-15 | 2013-10-14 14:20:20 -0700
* Fix misc. issues reported by Coverity (Return value checks,
time-of-check-time-of-use, null ptr checking, and a
use-after-free). (Jon Siwek)
* Fixed __bro_list_val_pop_front() to not erase the entire list but
remove only the first element. (Jon Siwek)
* Updating copyright notice. (Robin Sommer)
1.93-10 | 2013-10-02 10:38:27 -0700
* Remove dead code. (Jon Siwek)
* Fix mem leaks. (Jon Siwek)
* Updated specfile and configure script for libdir. (Derek Ditch)
Package maintainers and those that would otherwise compile from
source were unable to specify the installation directory of
architecture dependent libraries. Namely, many distributions use
lib64/ versus lib/ for the installation of architecture dependent
library archives.
* Add new 'configure' option, --libdir
* Defaults to old behavior of "$prefix/lib"
* Follows Kitware example for ProjectConfig.cmake on wiki
See https://github.com/bro/broccoli/pull/1
* Added back config-file bits to CMakeLists.txt (Derek Ditch)
* Fix for setting REPO in Makefile. (Robin Sommer)
1.93 | 2013-09-23 20:21:20 -0700
* Update 'make dist' target. (Jon Siwek)
* Change submodules to fixed URL. (Jon Siwek)
* Fix a compiler warning. (Daniel Thayer)
* Fix a broken link in documentation. (Daniel Thayer)
* Switching to relative submodule paths. (Robin Sommer)
* s/bro-ids.org/bro.org/g. (Robin Sommer)
1.92-9 | 2013-01-31 12:17:39 -0800
* A test program for sending packets through Broccoli, moved over
from the Time Machine repository. (Seth Hall)
1.92-7 | 2012-12-20 12:13:39 -0800
* Sync up with attribute definitions in Bro. (Daniel Thayer)
* Rebuild only necessary files for new prefix. (Daniel Thayer)
1.92-4 | 2012-12-05 15:37:54 -0800
* Improved error checking/reporting in case of out of memory
situations. (Bill Parker)
1.92-3 | 2012-11-23 19:51:14 -0800
* Bump data serialization format version for Bro's new "hook"
function. (Jon Siwek)
1.92 | 2012-08-22 16:15:18 -0700
* Fix configure script to exit with non-zero status on error (Jon Siwek)
1.91 | 2012-07-10 16:08:50 -0700
* Add --conf-files-dir option to configure wrapper script. (Jon Siwek)
1.9 | 2012-07-05 12:59:54 -0700
* Fix a warning, and fix other typos in documentation. (Daniel Thayer)
1.8-28 | 2012-05-24 17:37:35 -0700
* Tweak a test script to register events with both IPv4 & IPv6
loopback. (Jon Siwek)
* BROCCOLI_CONFIG_FILE env. variable can now specify config file
path. (Jon Siwek)
* Add ability to connect to Bro peers over IPv6. (Jon Siwek)
1.8-23 | 2012-05-03 11:33:03 -0700
* Fix typos and a few reST formatting problems. (Daniel Thayer)
1.8-21 | 2012-04-24 14:48:44 -0700
* Add option to set 'etc' directory. Addresses #801. (Daniel Thayer)
* Change BroAddr to use standard IPv4 in IPv6 mapping. (Jon Siwek)
The size field is now removed and the bro_util_is_v4_addr()
function can be used instead to check whether the BroAddr is IPv4
or not. Addresses #800.
* Add timeout to broccoli-v6addrs.c test. Addresses #793. (Jon
Siwek)
* Update IPv6 literal syntax in test scripts. (Jon Siwek)
1.8-8 | 2012-03-09 15:13:14 -0800
* Bump data format version corresponding to Bro's removal of match
expression. (Jon Siwek)
* Adding missing include needed on FreeBSD. (Robin Sommer)
* Update Broccoli library to handle IPv6 addrs/subnets. Addresses
#448. Addresses now use a new BroAddr struct to hold the address
data and BroSubnet changed to use a BroAddr member instead of a
single uint32 to represent the address. (Jon Siwek)
* Raise minimum required CMake version to 2.6.3. (Jon Siwek)
1.8 | 2012-01-10 19:33:08 -0800
* Tweaks for OpenBSD support. (Jon Siwek)
1.71-26 | 2012-01-03 15:41:37 -0800
* Remove record base type list since it's been removed from Bro.
(Jon Siwek)
1.71-22 | 2011-12-03 15:58:34 -0800
* Support for more types (not exposed at the API-level yet) to allow
exchanging more complex record types. Adresses #606. (Christian
Kreibich)
* Broccoli now identifies itself as such when connecting to a peer.
This allows Bro to adapt its serialization format based on what's
supported by Broccoli. Adresses #606. (Christian Kreibich)
1.71-11 | 2011-11-07 05:44:16 -0800
* Fixing compiler warnings. Addresses #388. (Jon Siwek)
* Update broccoli-ruby submodule. (Jon Siwek)
* Fix CMake warning when python bindings are disabled. Fixes #605.
(Jon Siwek)
1.71 | 2011-10-27 17:42:45 -0700
* Update submodules. (Jon Siwek)
1.7 | 2011-10-25 20:18:58 -0700
* Make dist now cleans the copied source. (Jon Siwek)
* Distribution cleanup. (John Siwek and Robin Sommer)
* Changed communications protocol option to listen_ssl from
listen_encrypted. (Seth Hall)
* Bug fix for a Bro test. (Seth Hall)
* Updates to make broccoli work with communication API updates.
1.6-35 | 2011-09-15 16:48:01 -0700
* Adding Ruby bindings for Broccoli. (Seth Hall)
* Broccoli API docs are now generated via Doxygen. Addresses #563.
(Jon Siwek)
* Converting manual to reST-format. (Don Appleman and Jon Siwek)
1.6-26 | 2011-09-04 09:26:47 -0700
* FindPCAP now links against thread library when necessary (e.g.
PF_RING's libpcap). (Jon Siwek)
* Install binaries with an RPATH. (Jon Siwek)
* Remove the 'net' type from Broccoli. Addresses #535. (Jon Siwek)
* Workaround for FreeBSD CMake port missing debug flags. (Jon Siwek)
1.6-13 | 2011-08-08 16:18:24 -0700
* Update broping.c test to use 64-bit int width for Bro counts (Jon Siwek)
* Install example config files dynamically when the distribution
version differs from existing version on disk. (Jon Siwek)
1.6-6 | 2011-07-19 17:54:57 -0700
* Update broccoli tests scripts to use new Bro policy organization
(Jon Siwek and Robin Sommer)
1.6 | 2011-05-05 20:32:42 -0700
* Moving ChangeLog to CHANGES for consistency. (Robin Sommer)
* Fixing write/read functionality for Bro's values that are now
64-bit. (Jon Siwek)
* Converting build process to CMake (Jon Siwek).
* Import of Bro's aux/broccoli subdir from SVN r7107 (Jon Siwek)
===== Old Subversion ChangeLog starts here.
Wed Mar 2 15:38:02 PST 2011 Christian <christian@whoop.org>
- Accept empty strings ("") as values in the configuration file
(Craig Leres).
- Support for specifying a separate host key for SSL-enabled operation,
with documentation update (Craig Leres).
- Version bump to 1.5.3.
------------------------------------------------------------------------
Fri Oct 9 18:42:05 PDT 2009 Christian <christian@whoop.org>
- Version bump to 1.5.
------------------------------------------------------------------------
Fri Sep 25 10:09:03 PDT 2009 Christian <christian@whoop.org>
- Bropipe fixes: set a connection class for robustness reasons;
removes some C/C++ confusion (Seth Hall).
------------------------------------------------------------------------
Mon Jun 29 17:56:00 PDT 2009 Christian <christian@whoop.org>
- SWIG bindings update.
------------------------------------------------------------------------
Mon Jun 29 15:29:35 PDT 2009 Christian <christian@whoop.org>
- Support for sending raw serialized events via the new API function
bro_event_send_raw(), with much help from Matthias Vallentin.
------------------------------------------------------------------------
Mon Jun 29 15:20:58 PDT 2009 Christian <christian@whoop.org>
- Fix for buffered data remaining in transmit buffer when calling
for_event_queue_flush().
- Added bro_conn_get_connstats() which reports statistical information
about a connection in a new dedicated structure BroConnStats. For now
this is only the amount of data buffered in the rx/tx buffers.
------------------------------------------------------------------------
Mon Jun 29 15:18:10 PDT 2009 Christian <christian@whoop.org>
- All multiprocess/-threading synchronization code has been removed.
------------------------------------------------------------------------
Mon Jun 29 15:10:59 PDT 2009 Christian <christian@whoop.org>
- Broccoli now requires initialization before any connections may be
created. The reason is twofold: (i) it provides a clean method for
initializing relevant parts of Broccoli in multithreaded environments,
and (ii) it allows configuration of parts of Broccoli where the
normal approach via configuration files is insufficient.
For details on the initialization process, refer to the manual, but
generally speaking, a call to
bro_init(NULL);
at the beginning of your application is all that is required. For the
time being, a number of high-level API calls double-check whether you
have called bro_init() previously.
- Broccoli now supports the callback functions OpenSSL requires for
thread-safe operation. Implement those callbacks as required by your
threading library, hook them into a BroCtx structure previously
initialized using bro_ctx_init(), and pass the structure to
bro_init(). This will hook the callbacks into OpenSSL for you.
O'Reilly's book "Network Security with OpenSSL" provides an example
of how to implement the callbacks.
------------------------------------------------------------------------
Thu Jun 25 16:46:37 PDT 2009 Christian <christian@whoop.org>
- Fix to Python bindings: added required bro_init() call (Matthias
Vallentin).
------------------------------------------------------------------------
Thu May 28 10:27:30 PDT 2009 Christian <christian@whoop.org>
- The BroEvMeta structure used in compact event callbacks now allows
access to the timestamp of event creation.
------------------------------------------------------------------------
Fri Mar 27 23:39:10 CET 2009 Christian <christian@whoop.org>
- Fixed a memory leak triggered by bro_event_send() but actually caused
by lack of cleanup after an underlying string duplication. Thanks to
Steve Chan and Matthias Vallentin for helpful feedback.
------------------------------------------------------------------------
Wed Mar 25 11:26:16 CET 2009 Christian <christian@whoop.org>
Formatting robustness fixes to bropipe (Steve Chan).
------------------------------------------------------------------------
Thu Feb 12 19:28:24 PST 2009 Christian <christian@whoop.org>
- Updates to contributed bropipe command (Steve Chan):
- Proper parsing of default host/port.
- Support for "urlstring" type, which urlencodes spaces in strings
and other special characters.
------------------------------------------------------------------------
Thu Dec 11 09:37:12 PST 2008 Christian <christian@whoop.org>
- Optimization: the internal slots vector of hashtables is now lazily
allocated when the first actual insertion happens. Since hashtables
are used in various places in the BroVal structures but frequently
remain empty, the savings are substantial. Thanks to Matthias
Vallentin for pointing this out.
------------------------------------------------------------------------
Mon Nov 3 11:07:49 PST 2008 Christian <christian@whoop.org>
- Fixes for I/O deadlocking problems:
- A bug in the implementation of BRO_CFLAG_YIELD has been
fixed. Input processing now only yields after the
handshake is complete on *both* endpoints.
- When events arrive during bro_conn_connect(), it could happen
that deadlock ensues if no additional data are sent and
__bro_io_process_input() can not read new input data. It no
longer returns immediately in that case, and instead attempts
to process any available input data.
------------------------------------------------------------------------
Sat Oct 4 15:05:07 CEST 2008 Christian <christian@whoop.org>
- Added bro_record_get_nth_name() to the API (Seth Hall).
- make install no longer worked for documentation, apparently as part
of Bro's make install cleanup. This isn't quite right since gtk-doc
documentation is normally installed in a well-known place and
Broccoli will normally need to be installed via "make install", but
for now I'm leaving it uninstalled and instead provide a specific
"install-docs" target for people who want documentation installed.
- Documentation updated where missing, and rebuilt.
- Copyright years updated.
------------------------------------------------------------------------
Mon Sep 22 21:34:13 CEST 2008 Christian <christian@whoop.org>
- Updated broping.bro (and broping-record.bro, slightly) to explicitly
declare the used event types ahead of their use.
------------------------------------------------------------------------
Mon Sep 8 11:30:35 CEST 2008 Christian <christian@whoop.org>
- Use of caching on received objects is now disabled by default, but can
be enabled using the new connection flag BRO_CFLAG_CACHE. The old
BRO_CFLAG_DONTCACHE is kept for backward compatibility but no longer
does anything. Keeping the caches between Bro instances and Broccoli
synchronized still needs to be implemented completely, and in the
meantime no caching is the safer default. Thanks to Stephen Chan for
helping track this down.
------------------------------------------------------------------------
Wed Jul 16 01:47:16 PDT 2008 Christian <christian@whoop.org>
- Python bindings for Broccoli are now provided in the bindings/python
subdirectory (Robin Sommer). They are not built automatically. See
the instructions in bindings/python/README for details.
- Minor documentation setup tweaks.
------------------------------------------------------------------------
Thu May 15 14:05:10 PDT 2008 Christian <christian@whoop.org>
Event callbacks of the "compact" type are now able to obtain start- and
end pointers of the currently processed event in serialized form, from
the receive buffer stored with the connection handle.
------------------------------------------------------------------------
Wed Feb 20 13:53:51 PST 2008 Christian <christian@whoop.org>
- Fix to __bro_openssl_read(), which handled some error cases
reported by BIO_read() incorrectly. (Robin Sommer)
- Clarifications to documentation of bro_conn_active() and
bro_conn_process_input().
- Version bump to 1.4.0.
------------------------------------------------------------------------
Thu Sep 13 13:56:58 PDT 2007 Christian <christian@whoop.org>
- autogen.sh now uses --force when running libtoolize, which at least
in some setups seems to be necessary to avoid bizarre build issues.
(In the particular case encountered, these looked like run-together
ar and runlib invocations). Thanks to Po-Ching Lin for helping nail
this down.
------------------------------------------------------------------------
Mon Sep 10 18:17:29 PDT 2007 Christian <christian@whoop.org>
- Broccoli now supports table and set container types. Have a look at
the bro_table_...() and bro_set_...() families of functions in
broccoli.h, the updated manual, and the updated broconn and brotable
examples in the test/ directory.
------------------------------------------------------------------------
Tue Sep 4 15:53:27 PDT 2007 Christian <christian@whoop.org>
- Major bugfix for capabilities exchange during handshake: Broccoli did
not convert into NBO, causing successful event exchange to fail. :(
Amazingly, this means disabling cache usage per Broccoli's request
never worked...
------------------------------------------------------------------------
Tue Sep 4 12:36:53 PDT 2007 Christian <christian@whoop.org>
- Changed the way compact argument passing to event callbacks works.
All event metadata is now represented by a single argument, a pointer
to a BroEvMeta structure. It contains the name of the event, the
number of arguments, and the arguments along with their types.
Updated documentation and broping demo accordingly.
NOTE: This introduces an API incompatibility. If you were previously
using the compact callback API, you will need to update your
code! I bumped up the library version info to 2:0:0 to signal
this.
- Fixed a bug in the implementation of BRO_CFLAG_YIELD and some SGML-
violating documentation of same.
------------------------------------------------------------------------
Thu Aug 16 15:24:51 CEST 2007 Christian <christian@whoop.org>
- Include autogen.sh in the distribution.
------------------------------------------------------------------------
Sat Aug 11 04:59:35 PDT 2007 Robin <robin@icir.org>
- New flag for Broccoli's connections: with BRO_CFLAG_YIELD,
bro_conn_process_input() processes at most one event at a time and then
returns (Robin Sommer).
- The new Broccoli function bro_conn_new_socket() creates a connection
from an existing socket, which can then be used with listen()/accept()
to have Broccoli listen for incoming connections (Robin Sommer).
------------------------------------------------------------------------
Fri Jul 6 18:18:05 PDT 2007 Christian <christian@whoop.org>
- Bumped up the version number to 1.3. Now that Broccoli is bundled
with Bro, it makes sense to keep Broccoli's release version number
in synch with Bro's.
- Added the automake-provided ylwrap wrapper script to the distribution.
This is for compatibility reasons: some automakes believe that
Broccoli requires ylwrap, others don't. The distcheck target however
needs ylwrap when it *is* required, so it's easiest to just provide
one. It can always be overwritten locally, should the need arise.
------------------------------------------------------------------------
Wed Mar 7 10:49:25 PST 2007 Christian <christian@whoop.org>
- Data format version number bumped up, in sync with Bro again.
------------------------------------------------------------------------
Mon Dec 4 12:07:12 PST 2006 Christian <christian@whoop.org>
- Updated broconn.c to new bro_record_get_named_val().
------------------------------------------------------------------------
Tue Nov 28 11:16:04 PST 2006 Christian <christian@whoop.org>
- Run-time type information is now also available for the values stored
in records (previously there was only type-checking, but no way to
obtain the type of the vals). See the manual and API documentation of
the functions bro_record_get_nth_val() and bro_record_get_named_val()
for details.
------------------------------------------------------------------------
Mon Nov 27 18:38:06 PST 2006 Christian <christian@whoop.org>
- Compact argument passing for event callbacks: as an alternative to the
argument passing style used so far for event callbacks (dubbed "expan-
ded"), one can now request "compressed" passing by using the
bro_event_registry_add_compact() variant. Instead of passing every
event argument as a separate pointer, compact passing provides only
the number of arguments, and a pointer to an array of BroEvArgs.
The elements of this array then provide pointers to the actual argu-
ments as well as pointers to the new BroValMeta metadata structure,
which currently contains type information about the argument.
This style is better suited for applications that don't know the type
of events they will have to handle at compile time, for example when
writing language bindings.
broping.c features example code, also see the manual for detailed
explanation.
------------------------------------------------------------------------
Mon Nov 27 16:32:52 PST 2006 Christian <christian@whoop.org>
- Bumped up version to 0.9
- I'm starting to use shared library version numbers to indicate API
changes. Their correspondence to the release version number will be
listed in VERSION.
- Fixed a warning in bro_packet.c
------------------------------------------------------------------------
Mon Nov 27 16:23:46 PST 2006 Christian <christian@whoop.org>
- Renamed cvs.pl to svn.pl
- Bumped up BRO_DATA_FORMAT_VERSION to 13, to match that of Bro trunk.
------------------------------------------------------------------------
Mon Nov 27 16:21:28 PST 2006 Christian <christian@whoop.org>
- Updating my commit script to SVN -- let's see if this works...
------------------------------------------------------------------------
Mon May 15 19:21:30 BST 2006 Christian <christian@whoop.org>
- Correction to the explanation of bro_event_registry_add(), pointed
out by Robin Sommer.
------------------------------------------------------------------------
Mon May 8 08:14:31 PDT 2006 Christian <christian@whoop.org>
- Added config.sub and config.guess versions that seem to work well with
MacOS X to the tree, to remove the dependency on the libtool/automake
versions installed on the machine where tarballs are built.
- Removed -f from libtoolize invocation in autogen.sh, so we don't
overwrite the above.
- Fixed COPYING, which wasn't actually referring to Broccoli. :)
------------------------------------------------------------------------
Sat May 6 20:17:32 BST 2006 Christian <christian@whoop.org>
- Last-minute tweaks bring last-minute brokenness, especially when
configuring without --enable-debug... :(
------------------------------------------------------------------------
Tue May 2 13:25:31 BST 2006 Christian <christian@whoop.org>
- Added generated HTML documentation to CVS, so it is guaranteed to be
included in tarballs generated via dist/distcheck, regardless of
whether GtkDoc support exists on the build system or not.
------------------------------------------------------------------------
Tue May 2 02:31:39 BST 2006 Christian <christian@whoop.org>
- Changed connection setup debugging output to state more clearly
whether an SSL or cleartext connection is attempted, as suggested
by Brian Tierney.
- New configuration item /broccoli/use_ssl to enable/disable SSL
connections, as suggested by Jason Lee. Documentation and sample
configuration in broccoli.conf updated accordingly, look at the latter
for a quick explanation.
- A bunch of small tweaks to get distcheck to work properly when invoked
from the Bro tree.
- Other doc/Makefile.am cleanups.
------------------------------------------------------------------------
Sat Apr 29 19:12:07 PDT 2006 Christian <christian@whoop.org>
- Fixed bogusness in docs/Makefile.am's dist-hook target. Should now
work much better in general, and in particular not bomb out with
non-GNU make.
------------------------------------------------------------------------
Fri Apr 7 23:52:20 BST 2006 Christian <christian@whoop.org>
- Bumped up BRO_DATA_FORMAT_VERSION to 12, to match the one in Bro's
CVS HEAD again.
------------------------------------------------------------------------
Mon Mar 27 22:59:04 BST 2006 Christian <christian@whoop.org>
- This should fix a memleak detected by Jim Mellander and reported with
a test case by Mark Dedlow.
------------------------------------------------------------------------
Fri Mar 3 16:40:56 GMT 2006 Christian <christian@whoop.org>
- Warning for invalid permissions on ~/.broccoli.conf has been upgraded
from debugging output to stderr, per request from Mark Dedlow.
- Only check validity of config file name assembled via getenv("HOME")
if it yields a filename different from the one assembled via the
passwd entry.
------------------------------------------------------------------------
Thu Mar 2 17:57:49 GMT 2006 Christian <christian@whoop.org>
- Reintroducing file needed for distcheck.
------------------------------------------------------------------------
Thu Mar 2 16:27:55 GMT 2006 Christian <christian@whoop.org>
- Debugging fixlet.
------------------------------------------------------------------------
Fri Feb 3 20:31:08 GMT 2006 Christian <christian@whoop.org>
- Embarrassing debugging output fixes.
------------------------------------------------------------------------
Fri Jan 27 23:40:23 GMT 2006 Christian <christian@whoop.org>
- Only do lock operations when there's any need for them.
------------------------------------------------------------------------
Fri Jan 27 18:30:06 GMT 2006 Christian <christian@whoop.org>
I am *so* fired. Overlooked a very clear warning that bro_io.c:lock()
wasn't returning a value.
------------------------------------------------------------------------
Wed Jan 18 10:45:33 GMT 2006 Christian <christian@whoop.org>
- Fixed call trace debugging inconsistencies, this will hopefully fix a
case of runaway call trace indentation depth that Robin + Stefan have
bumped into.
------------------------------------------------------------------------
Wed Jan 4 16:21:07 GMT 2006 Christian <christian@whoop.org>
- Documentation fixlet, pointed out by Stefan Kornexl.
------------------------------------------------------------------------
Thu Dec 22 00:48:20 GMT 2005 Christian <christian@whoop.org>
- Attempt at a more portable detecting of [g]libtoolize. Let me know if
this works any better.
------------------------------------------------------------------------
Mon Dec 19 17:48:19 PST 2005 Christian <christian@whoop.org>
- Moved brosendpkts.c and rcvpackets.bro from test/ to contrib/, i.e.,
out of the default build process. brosendpkts.c defines variables in
the middle of main(), which some compilers tolerate while others
don't. This should fix build issues reported by Brian Tierney.
------------------------------------------------------------------------
Thu Dec 15 18:38:18 GMT 2005 Christian <christian@whoop.org>
Configuration tweaks to run smoothly when invoked from a Bro build.
- Added AC_CONFIG_AUX_DIR(.) to make sure things are exclusively run
out of our tree.
- Added flags to autogen.sh and configure.in to indicate that we're
part of a Bro build.
------------------------------------------------------------------------
Fri Dec 2 14:04:05 GMT 2005 Christian <christian@whoop.org>
- Removed EXTRA_DIST for the test app policies, since they are included
in the tarball and installed anyway via pkgdata_DATA.
------------------------------------------------------------------------
Fri Dec 2 13:59:27 GMT 2005 Christian <christian@whoop.org>
- Added "brosendpkts", a test program for sending pcap packets to a Bro,
plus the accompanying Bro policy. Contributed by Stefan Kornexl and
Robin Sommer, with a tiny tweak to build only when pcap support is
available.
------------------------------------------------------------------------
Wed Nov 23 11:59:03 PST 2005 Christian <christian@whoop.org>
- Avoided the keyword "class" to prevent problems with using broccoli.h
in a C++ context. Pointed out by Stefan Kornexl.
------------------------------------------------------------------------
Tue Nov 8 14:10:23 PST 2005 Christian <christian@whoop.org>
- Added support for connection classes, updated documentation.
------------------------------------------------------------------------
Mon Oct 31 19:37:55 PST 2005 Christian <christian@whoop.org>
- Support for specifying type names along with values. This is done
through a new and optional argument to bro_event_add_val(), bro_
record_add_val(), and friends. See manual for details.
- Added a test program "broenum" for demonstrating this. When running
Bro with the provided broenum.bro policy, it sends a single event with
an enum val to the remote Bro, which will print both numerical and
string representations of the value. For example, broenum.bro defines
an enum type
type enumtype: enum { ENUM1, ENUM2, ENUM3, ENUM4 };
Given this,
$ broenum -n 0 yields Received enum val 0/ENUM1
$ broenum -n 1 yields Received enum val 1/ENUM2
$ broenum -n 4 yields Received enum val 4/<undefined>
You can also test predefined enums:
$ broenum -t transport_proto -n 1
yields
Received enum val 1/tcp
------------------------------------------------------------------------
Mon Oct 31 17:07:15 PST 2005 Christian <christian@whoop.org>
Changed commit script to pass the commit message through the generated
file via -F, instead of via -m and the command line. D'oh.
------------------------------------------------------------------------
Mon Oct 31 17:03:47 PST 2005 Christian <christian@whoop.org>
- Support for the new abbreviated serialization format for types. Need
to come up with a decent API for actually using this feature now.
------------------------------------------------------------------------
Mon Oct 31 11:25:22 PST 2005 Christian <christian@whoop.org>
Several changes to handshake implementation and API(!).
- Refactored the handshake code to make the multiple phases of the
connection's initialization phase more explicit. Our own and the peer's
handshake state are now tracked separately. conn_init_configure() takes
care of our state machine with a separate function per phase, and
__bro_io_process_input() handles the peer's state.
- Added support for capabilities. The only capability Broccoli currently
supports is a non-capability: it can ask the remote Bro not to use the
serialization cache. In order to do so, pass BRO_CONN_DONTCACHE as
a connection flag when obtaining the connection handle. Needs more
testing.
- Several API changes. Given the more complex handshake procedure that
is in place now, the old approach of only completing the handshake half-
way in bro_connect() so the user can requests before calling
bro_conn_await_handshake() (or alternatively, passing
BRO_CONN_COMPLETE_HANDSHAKE as a connection flag) is just too messy now.
The two steps of obtaining a connection handle and establishing a
connection have been split into separate functions, so the user can
register event handlers in between.
What was
BroConn *bc = bro_connect(..., BRO_CFLAGS_NONE);
bro_event_registry_add(bc,...);
bro_event_registry_add(bc,...);
bro_event_registry_request(bc);
bro_conn_await_handshake(bc);
/* ... */
bro_disconnect(bc);
is now
BroConn *bc = bro_conn_new(..., BRO_CFLAGS_NONE);
bro_event_registry_add(bc,...);
bro_event_registry_add(bc,...);
bro_conn_connect(bc);
/* ... */
bro_conn_delete(bc);
Note that the explicit call to bro_event_registry_request() is gone as
bro_conn_connect() will automatically request event types for which
handlers have been installed via bro_event_registry_add(). What was
BroConn *bc = bro_connect(..., BRO_CFLAGS_COMPLETE_HANDSHAKE);
bro_disconnect(bc);
is now
BroConn *bc = bro_conn_new(..., BRO_CFLAGS_NONE);
bro_conn_connect(bc);
/* ... */
bro_conn_delete(bc);
I might add bro_conn_disconnect() in the near future. It'd allow us
to keep a completely configured connection handle around and use it
repeatedly for establishing connections.
Sorry for the inconvenience but I really think this is a lot nicer than
the old API. The examples and documentation have been updated accor-
dingly.
------------------------------------------------------------------------
Sat Oct 29 15:43:18 PDT 2005 Christian <christian@whoop.org>
Added an optional age list to the hash table implementation. We'll
need this to duplicate Bro's object serialization caching strategy.
------------------------------------------------------------------------
Fri Oct 28 15:26:55 PDT 2005 Christian <christian@whoop.org>
Brothers and sisters, hallelujah! On the 27th day Christian looked at
record vals in the Broccoli, and he saw that it was leaking like a
sieve. So Christian ran the valgrind. On the 28th day Christian still
looked at Broccoli, with tired eyes, ground the vals[1] a bit more,
and he saw that it was plugged[2].
Amen. :)
[1] Really really bad pun. Sorry.
[2] I get zero memleaks on broping -r -c 100 now. :)
------------------------------------------------------------------------
Thu Oct 27 20:02:39 PDT 2005 Christian <christian@whoop.org>
First crack at reference-counted sobjects. I need reference counting
in order to get rid of objects in the serialization cache (since they
can contain nested objects etc -- it's nasty), which I had ignored so
far. There are still leaks in the event transmission code, dammit. :(
------------------------------------------------------------------------
Thu Oct 27 15:06:10 PDT 2005 Christian <christian@whoop.org>
Added my own list implementation due to suckiness of the TAILQ_xxx
macro stuff which I never liked anyway. The problem is that elements
of lists built using these macros can only have each member exactly
once as the prev/next pointers are part of the structs.
A few uses of TAILQ_xxx remain, these will go in the near future.
------------------------------------------------------------------------
Tue Oct 25 19:57:42 PDT 2005 Christian <christian@whoop.org>
Partial support for enum vals, per request from Weidong. Sending enum
vals should work, though the underlying enum types aren't fully handled
yet.
------------------------------------------------------------------------
Mon Oct 24 16:31:56 PDT 2005 Christian <christian@whoop.org>
TODO item: clean up generated parser/lexer files when we know we can
regenerate them. make clean currently does not erase them, which caused
Weidong some trouble.
------------------------------------------------------------------------
Fri Oct 21 17:48:51 PDT 2005 Christian <christian@whoop.org>
Clarification to the manual, after a question from Weidong.
------------------------------------------------------------------------
Fri Oct 14 18:05:39 PDT 2005 Christian <christian@whoop.org>
Transparent reconnects should work again (took all *day*, argh -- I
totally broke it with the connection sharing stuff). Try broping while
occasionally stopping and restarting the Bro side.
Fixed a number of memleaks -- broping is now leak-free according to
valgrind.
Clarifications in the debugging output.
------------------------------------------------------------------------
Fri Oct 14 12:07:10 PDT 2005 Christian <christian@whoop.org>
Added documentation for the new user data argument to
bro_event_registry_add().
------------------------------------------------------------------------
Fri Oct 14 11:48:00 PDT 2005 Christian <christian@whoop.org>
Added user data to event handler callbacks. This is necessary for
example when using class members in C++ as callbacks since the object
needs to be provided at the time of dereferencing. It's also easier to
use than the existing bro_conn_{set,get}_data() mechanism.
Updated documentation with more details on the broccoli-config script.
------------------------------------------------------------------------
Thu Oct 13 15:08:56 PDT 2005 Christian <christian@whoop.org>
When supporting packets (the default), check whether pcap.h actually
exists. This has thus far just been assumed. We don't actually use
the library, so there's no need to test for it.
------------------------------------------------------------------------
Mon Oct 10 20:37:15 PDT 2005 Christian <christian@whoop.org>
Changed bro_record_get_named_val() and bro_record_get_nth_val() to
return a pointer to the queried value directly, instead of through
a pointer argument. These arguments' type used to be void* though it
should really be void**, but switching to void** causes lots of warnings
with current GCCs ('dereferencing type-punned pointer will break
strict-aliasing rules'). NULL is perfectly usable as an error indicator
here, and thus used from now on. Updated manual, broping, and broconn
accordingly.
------------------------------------------------------------------------
Tue Sep 20 17:19:58 PDT 2005 Christian <christian@whoop.org>
Fixed a varargs buglet that is tolerated on Linux but not BSD. Pointed
out by Scott Campbell.
------------------------------------------------------------------------
Fri Sep 9 18:48:54 PDT 2005 Christian <christian@whoop.org>
Support for textual tags on packets, also an upgrade to more complex
handshake procedure that allows for synchronization of state (Robin
Sommer).
Note: as of this change, at least Bro 1.0a2 is required.
------------------------------------------------------------------------
Wed Aug 10 01:36:47 BST 2005 Christian <christian@whoop.org>
Fixed my insecure usage of snprintf.
------------------------------------------------------------------------
Tue Jul 19 10:11:49 PDT 2005 Christian <christian@whoop.org>
Forgot to include broconn's policy file in the distribution.
------------------------------------------------------------------------
Mon Jul 18 16:34:22 PDT 2005 Christian <christian@whoop.org>
Fixed a bug that caused the lookup of record fields by name to fail.
------------------------------------------------------------------------
Fri Jul 1 00:44:49 BST 2005 Christian <christian@whoop.org>
The sequence of tests determining which config file to read from
failed to fall back properly to the global config file in case of
incorrect user permissions. Fixed.
------------------------------------------------------------------------
Mon Jun 27 19:34:56 PDT 2005 Christian <christian@whoop.org>
Added bro_buf_reset() to the user-visible API.
------------------------------------------------------------------------
Mon Jun 27 17:58:53 PDT 2005 Christian <christian@whoop.org>
When a configuration item cannot be found in the current config file
section, a lookup is also attempted in the default section (the one
at the top of the file, before any sections are defined). This allows
the sections to override the default section, which is what one would
expect.
------------------------------------------------------------------------
Mon Jun 27 14:43:56 PDT 2005 Christian <christian@whoop.org>
Debugging output tweak. When providing the SSL cert passphrase via
the config file, do no longer report it in the debugging output.
------------------------------------------------------------------------
Mon Jun 27 12:33:52 PDT 2005 Christian <christian@whoop.org>
Cosmetics in the debugging output of __bro_openssl_write().
------------------------------------------------------------------------
Fri Jun 24 18:13:49 PDT 2005 Christian <christian@whoop.org>
Added --build flag to broccoli-config. It reports various details
about the build, for example whether debugging support was compiled in.
------------------------------------------------------------------------
Fri Jun 24 10:37:23 PDT 2005 Christian <christian@whoop.org>
I'm adding a little test app that subscribes to a few connection
events and prints out the fields of the received connection records,
both for testing and code demonstration purposes. So far it has
highlighted a bug in Bro that occurs when a remote app is a pure
requester of events and not sending anything. Fix pending.
------------------------------------------------------------------------
Mon Jun 20 18:21:24 PDT 2005 Christian <christian@whoop.org>
Show the names of requested events in the debugging output -- it
had to be deciphered from the hex string which isn't that much fun.
------------------------------------------------------------------------
Thu Jun 16 14:02:59 PDT 2005 Christian <christian@whoop.org>
Better documentation of how to extract record fields.
------------------------------------------------------------------------
Thu Jun 16 11:51:02 PDT 2005 Christian <christian@whoop.org>
- Added bro_string_get_data() and bro_string_get_length() to avoid
making people access BroString's internal fields directly.
- Moved BroString's internal storage format to uchar*.
------------------------------------------------------------------------
Sun Jun 12 19:17:31 PDT 2005 Christian <christian@whoop.org>
Debugging output now shows the correct function and line numbers again.
I had accidentially moved __FUNCTION__ and __LINE__ into bro_debug.c :(
------------------------------------------------------------------------
Fri Jun 3 15:00:48 PDT 2005 Christian <christian@whoop.org>
I broke the sanity checks for semaphore initialization when I moved
the semaphore structures to shared memory. Fixed.
------------------------------------------------------------------------
Mon May 16 22:25:41 PDT 2005 Christian <christian@whoop.org>
- Debugging output now goes to stderr instead of stdout. That keeps it
out of the way if an instrumented app dups() stdout to another file
descriptor.
- Debugging output is now disabled by default (even when compiled in),
so it needs to be enabled explicitly in the code or in the config file.
------------------------------------------------------------------------
Fri May 13 18:24:23 PDT 2005 Christian <christian@whoop.org>
Synchronization fixes and minor cleanups.
- Unsuccessful connection attempts to remote Bros in combination with
connection sharing caused the caller to hang indefinitely. This should
now be fixed, but required some fairly intricate tweaks to the locking
constructs. Still needs more testing.
- Bumped version to 0.8.
------------------------------------------------------------------------
Fri May 6 23:09:29 BST 2005 Christian <christian@whoop.org>
This is the 0.7.1 release.
------------------------------------------------------------------------
Fri May 6 14:44:53 PDT 2005 Christian <christian@whoop.org>
Documentation for shareable connection handles.
------------------------------------------------------------------------
Fri May 6 12:11:17 PDT 2005 Christian <christian@whoop.org>
Build fixlets.
- Don't only test for the first of the documentation extraction tools,
but also for those used later on.
- Few more signedness warnings fixed.
------------------------------------------------------------------------
Wed May 4 18:33:40 PDT 2005 Christian <christian@whoop.org>
Fixed a whole bunch of signedness warnings reported by gcc 4 on MacOS
10.4. Thanks to Roger for the quick reply.
------------------------------------------------------------------------
Wed May 4 17:41:40 PDT 2005 Christian <christian@whoop.org>
Fix for a little-endian bug that I managed to introduce when testing on
Solaris ... *sigh* :(
------------------------------------------------------------------------
Wed May 4 17:30:07 PDT 2005 Christian <christian@whoop.org>
A number of portability fixes after testing the build on Linux, FreeBSD
and Solaris.
------------------------------------------------------------------------
Mon May 2 20:17:04 PDT 2005 Christian <christian@whoop.org>
Fixed an obvious bug the config file parser. I'm baffled as to how it
could go unnoticed for so long.
------------------------------------------------------------------------
Mon May 2 20:11:25 PDT 2005 Christian <christian@whoop.org>
Portability fixes.
- Use -pthread (not -lpthread) in both the --cflags and --libs options
to broccoli-config, if required. -lpthread does not work on BSDs, where
-pthread has different effects on the linker.
- s/System V/SYSV/ in configure script output for consistency.
- Bumped version to 0.7.1.
It should build correctly on BSDs and Linux now. Still need to check
whether synchronization actually works on the BSDs.
------------------------------------------------------------------------
Fri Apr 29 23:12:01 BST 2005 Christian <christian@whoop.org>
If the configure script determines we need -lpthread, it's a good idea
to actually reflect that in broccoli-config.
------------------------------------------------------------------------
Fri Apr 29 22:36:26 BST 2005 Christian <christian@whoop.org>
Fix for SYSV semaphores pointed out by Craig Leres -- I completely
forgot to test the SYSV stuff before the release. *sigh*.
------------------------------------------------------------------------
Thu Apr 28 13:46:57 BST 2005 Christian <christian@whoop.org>
- This is the 0.7 release.
------------------------------------------------------------------------
Thu Apr 28 13:43:44 BST 2005 Christian <christian@whoop.org>
RPM spec file fixlet.
------------------------------------------------------------------------
Wed Apr 27 18:04:57 BST 2005 Christian <christian@whoop.org>
Preparations for the 0.7 release.
------------------------------------------------------------------------
Wed Mar 16 18:34:27 GMT 2005 Christian <christian@whoop.org>
I think shared connections w/ SSL work. :) They key aspects are
- We want to be able to use a single connection handle in arbitrary
process/thread scenarios: in sshd, a single handle created in the
listening process should work in all forked children (right now I'm
created separate ones in each child, yuck), in Apache it should work
in all servicing threads (creating a separate connection in each
servicing thread would be far too costly), etc.
- However, all SSL I/O on a single BIO must happen in the same *thread*
according to openssl-users -- same process seems intuitive because of
cipher streams etc; why it's per thread I don't know.
The approach is now as follows: when a connection handle is marked as
shareable, an I/O handler process is forked off during handle setup
that processes all I/O for a single connection handle exclusively.
Data are processed through separate tx/rx buffers that live in shared
memory and are protected by semaphores. Additionally, a number of
fields in the connection handle also live in shared memory so can be
used to send back and forth messages etc. By using global semaphores as
condition variables, rx/tx requests are dispatched to the I/O handler
process. Therefore this should work for all multi-process/thread
scenarios in which processes/threads are created after the connection
handle is set up.
This all is transparent when a connection is not marked shareable. The
main optimization left to do now is to make the locking more fine-
grained -- a throughput comparison is going to be interesting...
I haven't tried transparent reconnects again; I'd presume I managed
to break them in the process.
------------------------------------------------------------------------
Mon Mar 14 17:31:17 GMT 2005 Christian <christian@whoop.org>
- Lots of work on shared connection handles. This is going to take a
while to work robustly. For now steer clear of BRO_CFLAG_SHAREABLE.
- Fixed wrong ordering of semaphore locks in __bro_io_msg_queue_flush().
- The connection hack to work around OpenSSL's 'temporary unavailable'
beliefs is now only used when the problem occurs, namely during
reconnects.
- Fixed a bug in the Posix version of __bro_sem_new() that prevented
processes from creating more than one different semaphores. Doh.
- Bumped BRO_DATA_FORMAT_VERSION to 9, to sync up with Bro tree.
- Added __bro_sem_get(), returning the current value of a sempahore,
with implementations for Posix + SYSV.
- Lots of calltracing added.
------------------------------------------------------------------------
Mon Mar 14 10:24:54 GMT 2005 Christian <christian@whoop.org>
Code for shared connection handles with SSL enabled. Pretty much done,
but needs a lot of testing now.
------------------------------------------------------------------------
Sat Mar 12 18:13:58 GMT 2005 Christian <christian@whoop.org>
Beginning of support for sharing connection handles for SSL-enabled
connections. Since supporting this is complex, it will be optional,
and enabled by using the new BRO_CFLAG_SHAREABLE connection flag.
------------------------------------------------------------------------
Fri Mar 11 14:50:23 GMT 2005 Christian <christian@whoop.org>
Move to AC_PROG_LIBTOOL.
------------------------------------------------------------------------
Fri Mar 11 14:33:57 GMT 2005 Christian <christian@whoop.org>
Portability and robustness fixes.
- auto* calls in autgen.sh are now checked for success and cause the
script to abort on error.
- Instead of trying to figure out what libraries the various OSs need
in order to be able to use Posix semaphors, I'm now attempting to use
the -pthread flag directly. If that fails, we just fall back to SYSV
semaphores.
- All semaphore + shmem implementations are now included in the tarball,
the point is to include them selectively in the *build*.
- Stevens' ifdef magic for union semun doesn't work on at least OpenBSD
so I'm using the BSD_HOST macro from config.h now.
- Apparently AM_PROG_LIBTOOL causes some people trouble so we need to
check how to get that working realiably :(
------------------------------------------------------------------------
Mon Feb 21 14:45:51 GMT 2005 Christian <christian@whoop.org>
- Partial-write bugfix. When we succeed only partially in writing out
a message, report success, not failure. Failure is handled by queuing
the message for later transmission, but we have already sent it
partially and the rest is still stuck in the output buffer, so if we
queue it again, it'll get sent at least twice.
I had noticed that out of 100000 events sent by 100 processes in
parallel, typically around 100020 arrived :)
------------------------------------------------------------------------
Sat Feb 19 21:04:46 GMT 2005 Christian <christian@whoop.org>
- Lots of synchronization work. This generally seems to work now! :) It
required one major addition: support for shared memory. The problem is
that if multiple threads/processes attempt to write at the same time
and one write succeeds only partially, then *whichever* thread/process
gets to write next needs to write out the rest before writing any new
messages. The only alternative is to have write operations block until
entire messages are sent, which seems dangerous from an instrumentation
point of view. To share the remaining message data, shared memory is
required: both the tx and rx buffers now operate in shared memory and
are protected by semaphores. The current implementation uses SYSV shared
memory.
I think shared memory is a good idea in general; for example it could be
used during instrumentation to get information from one corner of an app
to another without changing the application's structure. I don't think
we'll need this right away, but it's nice to have a possible technique
for it.
- bro_disconnect() is now more tricky to use than before: if you use
it in a parallel setting, you *must* call it from the same process that
called bro_connect() and you must do so *after* all the other processes
have finished using the connection (typically this is not hard to do, so
I think we can live with that).
The reason is that semaphores + shared memory need to be uninstalled
specifically and I haven't yet figured out a way to automate reference
counting so that the last thread/process using a connection could do
this automatically. It would be very cool if the functions that are
used for deinstallation could be asked to fail while the IPC objects are
still in use, but that's not the case.
- You can still build the whole thing without semaphores or shared mem
and it'll work for single-threaded apps. The configure script now issues
a warning if not all tools required for stable parallel operation can be
found.
- Added bro_event_queue_length_max() to allow applications to find out
the maximum queue length before messages will get dropped. brohose uses
this to wait until the queue gets half full before insisting on a flush.
------------------------------------------------------------------------
Fri Feb 18 17:14:40 GMT 2005 Christian <christian@whoop.org>
- SYSV semaphore implementation. Configure checks are included
and work as follows: if both Posix + SYSV semaphores are found,
Posix are preferred, however the user can override this by passing
--disable-posix-semaphores. Semaphores are still not actually used.
------------------------------------------------------------------------
Thu Feb 17 22:24:12 GMT 2005 Christian <christian@whoop.org>
- First shot at semaphore support. Checking for Posix named semaphores
and making sure they actually work at configure time was the hardest
part; actual semaphore code untested and still unused. No ifdefs
anywhere :)
------------------------------------------------------------------------
Thu Feb 17 20:06:00 GMT 2005 Christian <christian@whoop.org>
- Incompletely sent chunks are now recognized and remaining parts are
shipped as soon as possible: repeated brohose -n 1 -e 1000 runs do not
take out Bro any more. :)
------------------------------------------------------------------------
Thu Feb 17 19:21:15 GMT 2005 Christian <christian@whoop.org>
- Added "brohose", which lets you hose a Bro with events by forking a
configurable number of processes, and having each process pump out an
event a configurable number of times as fast as possible. This is meant
as both a stress-testing tool for the protocol as well as obviously for
the synchronization stuff that'll go into Broccoli soon.
------------------------------------------------------------------------
Wed Feb 16 17:40:47 GMT 2005 Christian <christian@whoop.org>
- Documentation for the configuration options for debugging output.
------------------------------------------------------------------------
Thu Feb 10 11:39:57 GMT 2005 Christian <christian@whoop.org>
- Changed bro_event_queue_empty() to bro_event_queue_length(),
which is more useful in general and can be used to find out
whether the queue is empty, too.
------------------------------------------------------------------------
Tue Feb 8 14:45:58 GMT 2005 Christian <christian@whoop.org>
- This is release 0.6.
------------------------------------------------------------------------
Mon Feb 7 14:54:15 GMT 2005 Christian <christian@whoop.org>
- Additional byte swaps for IP addresses + subnets for compatibility
with Bro.
------------------------------------------------------------------------
Sun Feb 6 23:55:07 GMT 2005 Christian <christian@whoop.org>
- Debugging output can now be configured from the config file,
using the /broccoli/debug_messages and /broccoli/debug_calltrace
config items.
------------------------------------------------------------------------
Tue Feb 1 21:34:17 GMT 2005 Christian <christian@whoop.org>
- During handshake, data format compatibility is now confirmed as well
as matching protocol version.
------------------------------------------------------------------------
Tue Feb 1 21:04:43 GMT 2005 Christian <christian@whoop.org>
- Initial commit of support for sending/receiving libpcap packets.
Totally untested, and not documented yet. More on this once support
for packets is committed into the Bro tree.
------------------------------------------------------------------------
Tue Feb 1 18:39:02 GMT 2005 Christian <christian@whoop.org>
- Transparent reconnects now also work for non-SSL connections. I was
just lucky that the SSL handshake prevented the same problem from
occurring in the SSL-enabled case. Two fixes were necessary:
1) a separate attempt to connect to the peer that I have full control
over, and
2) a fixlet in queue management that caused the event that
triggers the reconnect to be sent before any handshake information
for the new connection, thus causing a connection teardown by the
Bro end because the version number was not seen at the right time.
------------------------------------------------------------------------
Mon Jan 31 19:38:36 GMT 2005 Christian <christian@whoop.org>
- Fixed a few spots where D_ENTER was not balanced with D_RETURN
- Added an int-to-string table for message types, for debugging
- Added a flag to the connection structure that prevents reconnect
attempts while one is already in progress
- Made io_msg_queue() private to bro_io.c because it was only called
from there.
------------------------------------------------------------------------
Fri Jan 28 12:35:03 GMT 2005 Christian <christian@whoop.org>
- Changed the error semantics of in __bro_io_msg_queue() so that queuing
a message after failure to send is not a failure. This fixes an issue
with handshake completion that I have observed with broping across
different machines, where events could still get lost despite explicit
request to complete the handshake.
------------------------------------------------------------------------
Sun Jan 16 20:45:42 GMT 2005 Christian <christian@whoop.org>
- Serialization/Unserialization for ports fixed, support for ICMP ports.
------------------------------------------------------------------------
Sat Jan 15 13:58:16 GMT 2005 Christian <christian@whoop.org>
- Sending and receiving IP addresses and subnets was broken, fixed now.
- Fixed a small memleak when first-time connection setup fails.
------------------------------------------------------------------------
Thu Jan 13 21:03:45 GMT 2005 Christian <christian@whoop.org>
- When using reconnects, Broccoli will now not attempt to reconnect
more than once every 5s.
------------------------------------------------------------------------
Thu Jan 13 20:43:13 GMT 2005 Christian <christian@whoop.org>
- Added connection flag BRO_CFLAG_ALWAYS_QUEUE that causes events
always to be queued in the connection's event queue regardless of
whether the peer is currently dead or not.
- Moved the test of whether the peer requested an event that is
about to be sent or not to the point where the event actually is
about to be sent, from the point where it is requested to be sent.
The difference is that now an event will get silently dropped on
the floor if after a connection outage and a reconnect, a change
in the events requested from the peer will prevent the old queued
events to be sent anyway, even if they are no longer requested.
------------------------------------------------------------------------
Wed Jan 12 20:46:10 GMT 2005 Christian <christian@whoop.org>
- Added support for transparent reconnects for broken connections.
When using BRO_CFLAG_RECONNECT, Broccoli now attempts to reconnect
whenever a peer died and the user tries to read from or write to
the peer. This can aways be triggered manually using
bro_reconnect().
- Added bro_conn_alive() to determine if a connection is currently
alive or not.
------------------------------------------------------------------------
Tue Jan 11 17:33:51 GMT 2005 Christian <christian@whoop.org>
- Added connection flags parameter to bro_connect() and
bro_connect_str(): BRO_CFLAG_COMPLETE_HANDSHAKE completes
the handshake right away before returning from bro_connect()/
bro_connect_str(), and BRO_CFLAG_RECONNECT still needs to be
implemented. Documentation updated accordingly.
------------------------------------------------------------------------
Sat Jan 8 21:07:30 CET 2005 Christian <christian@whoop.org>
- Allow empty (or comments-only) configuration files.
------------------------------------------------------------------------
Sat Jan 8 20:52:56 CET 2005 Christian <christian@whoop.org>
- Fixed the home directory lookup via getpwent() -- now correctly looks
up the entry of the current effective user. Doh.
- Beginning of code for connection flags to use when creating a
connection, for example for handshake behaviour, automatic reconnection
attempts, etc.
------------------------------------------------------------------------
Tue Jan 4 23:28:59 CET 2005 Christian <christian@whoop.org>
- constness fixes for functions that accept values for events and
record fields.
------------------------------------------------------------------------
Tue Jan 4 22:07:35 CET 2005 Christian <christian@whoop.org>
- Encrpyted connections now extract as much data as possible from
the underlying buffer by calling BIO_read() optimistically.
- For encrypted connections, the passphrase for the certificate's
private key can now be specified in the configuration file using key
"/broccoli/host_pass".
- Added support for the handshake message in the Bro protocol.
- If the ca_cert or host_cert keys are found in the config file, but
there is a problem loading the crypto files, don't attempt to connect.
- Completed documentation on encrypted communication, explaining the
use of ca-create and ca-issue.
- Fixed several bugs in the handling of sections in config files.
Matching of domain names is now case-insensitive.
- The ~/.broccoli.conf file is now only used when it is readable only
by the user owning it.
- More robustness for corner cases of buffer sizes.
- Fixed a bug in sending messages that consist of only a single chunk
(like the handshake message).
- The library now attempts to initialize the random number generator
in OpenSSL from /dev/random if possible.
------------------------------------------------------------------------
Fri Dec 24 11:58:08 CET 2004 Christian <christian@whoop.org>
- If the ca_cert or host_cert keys are found in the config file, but
there is a problem loading the crypto files, don't attempt to connect.
- Completed documentation on encrypted communication, explaining the
use of ca-create and ca-issue.
- Fixed several bugs in the handling of sections in config files.
------------------------------------------------------------------------
Thu Dec 23 14:33:56 GMT 2004 Christian <christian@whoop.org>
- Added sections support for configuration files. Sections can be
declared at arbitrary points in the config file, using the same syntax
as in OpenSSL config files. There can be a global section at the
beginning of the file, before the first declared sections. Sections are
selected using bro_conf_set_domain().
- Support for a per-user config file in ~/.broccoli.conf. This does
not override settings in the global config file but completely replaces
it, i.e., when the user-specific file is found, the global one is
ignored.
- Added bro_conn_await_handshake() that blocks for limitable amount of
time, waiting for the handshake of a new Bro connection to complete.
This still needs some fixing, but is definitely necessary to prevent
weird races from occurring when a client tries to use a new connection
that has not yet been established completely.
- Test applications are now linked to static libraries. This will
hopefully keep the build more portable.
- Use of LFLAGS and YFLAGS moved to AM_LFLAGS and AM_YFLAGS, given the
warnings issued when using automake 1.9.
- First shot at fixing the buffer flushing issues I see when using
encrypted connections.
------------------------------------------------------------------------
Fri Dec 10 16:31:26 GMT 2004 Christian <christian@whoop.org>
- Added + fixed OpenSSL code to support encrypted communication.
- Added OpenSSL as requirement to spec file.
- Changed broping policies to always use the same port
- Updated broccoli.conf: added keys for the CA's and the host's cert.
------------------------------------------------------------------------
Thu Dec 9 14:59:24 GMT 2004 Christian <christian@whoop.org>
- Build fixes in case documentation tools are not found
- Documentation polishing -- only SSL setup section todo still.
------------------------------------------------------------------------
Thu Dec 9 00:48:05 GMT 2004 Christian <christian@whoop.org>
- Final documentation passes for the 0.6 release.
------------------------------------------------------------------------
Mon Dec 6 17:18:55 GMT 2004 Christian <christian@whoop.org>
- More documentation, explaining the data types, records, Bro policy
configuration, started section on SSL setup (copied from Robin right
now), and minor fixes.
------------------------------------------------------------------------
Mon Dec 6 15:17:05 GMT 2004 Christian <christian@whoop.org>
- Added spec file for building RPMs -- seems to work
- Aest policies are now installed in $prefix/share/broccoli
------------------------------------------------------------------------
Mon Dec 6 00:22:02 GMT 2004 Christian <christian@whoop.org>
- Dropped the ..._raw() functions for records. These won't be used
internally ever. Their implementation moved to bro.c, and only the high-
level code remained in bro_record.c.
- Added bro_event_set_val() to replace a val in an existing event.
There's not much use in resending an existing event unless it is
identical, which is not that useful. High-level code is in
__bro_event_set_val().
- Made it more clear in the comments explaining the
bro_record_get_..._val() functions that the "result" argument must
actually be the address of a pointer. (void * as argument type means
that the compiler does not issue a warning when passing in, say, a
double * -- but it would do so if we would use void **.)
------------------------------------------------------------------------
Sun Dec 5 22:05:53 GMT 2004 Christian <christian@whoop.org>
- Updates to the cvs wrapper script: surround with date and name
only in the ChangeLog, not in the commit message itself.
------------------------------------------------------------------------
Sun Dec 5 02:15:29 GMT 2004 Christian <christian@whoop.org>
- Fixed a bug in __bro_val_clone(): forgot to handle BRO_INTTYPE_OTHER.
- Changed --enable-debugging flag to --enable-debug, for consistency
with the Bro tree.
- Fixed bugs in several cloning implementations that didn't call the
parent's implementation.
------------------------------------------------------------------------
Sun Dec 5 01:40:52 GMT 2004 Christian <christian@whoop.org>
- Added __bro_event_copy() to clone events internally.
- Events are now duplicated in __bro_io_event_queue() before they're
sent so the user's event remains unaffected (and thus could be sent
repeatedly etc).
- Extensive pass over the documentation; still a good deal to do.
------------------------------------------------------------------------
Sat Dec 4 03:09:05 GMT 2004 Christian <christian@whoop.org>
More work on documentation, much is outdated now.
------------------------------------------------------------------------
Sat Dec 4 02:05:30 GMT 2004 Christian <christian@whoop.org>
- Started a ChangeLog. No detailed ChangeLog information was kept
previous to this commit.
------------------------------------------------------------------------
Broccoli Python
0.62 | 2017-05-26 08:27:17 -0500
* Release 0.62.
0.61 | 2016-10-27 14:42:24 -0700
* Release 0.61.
0.60-3 | 2016-10-13 15:58:06 -0700
* Python 3 compatibility fixes. (Daniel Thayer)
0.60 | 2016-08-12 13:14:12 -0700
* Release 0.60.
* Add README.rst -> README symlink. (Johanna Amann)
0.59 | 2015-04-27 08:25:18 -0700
* Release 0.59
0.58-9 | 2015-02-13 18:01:05 -0600
* Install broccoli_intern.py (Jon Siwek)
0.58-8 | 2015-02-13 16:02:49 -0600
* In broccoli.py, import from broccoli_intern not _broccoli_intern.
(Jon Siwek)
0.58-1 | 2014-07-08 09:56:42 -0700
* Fix setup.py to work with path changes. Addresses BIT-1213.
(Nicholas Weaver)
0.58 | 2014-04-03 15:53:50 -0700
* Release 0.58
0.57-3 | 2014-01-23 16:59:19 -0800
* Supply connAlive() and connDelete() methods. (jpohlmann)
0.57 | 2013-11-06 00:21:37 -0800
* Installation section was missing necessary steps. (Johanna Amann)
0.56 | 2013-10-14 09:24:55 -0700
* Updating copyright notice. (Robin Sommer)
0.55-2 | 2013-10-02 10:34:29 -0700
* Fix mem leaks. (Jon Siwek)
0.55 | 2013-09-23 13:14:46 -0500
* Change submodules to fixed URL. (Jon Siwek)
* Switching to relative submodule paths. (Robin Sommer)
* s/bro-ids.org/bro.org/g. (Robin Sommer)
0.54 | 2012-08-01 13:56:08 -0500
* Fix configure script to exit with non-zero status on error (Jon Siwek)
0.53 | 2012-06-11 17:25:05 -0700
* Fix overflow problems in converting Python IP addresses to
Broccoli. (Jon Siwek)
* Update bindings to work with Broccoli's IPv4-mapped BroAddrs. (Jon
Siwek)
* Update IPv6 literal syntax in test scripts. (Jon Siwek)
* Update broccoli-python for IPv6 addr/subnet support. Addresses
#448. (Jon Siwek)
* Raise minimum required CMake version to 2.6.3. (Jon Siwek)
0.52 | 2012-01-10 16:56:13 -0800
* Submodule README conformity changes (Jon Siwek)
* Simplify finding of Python headers/libraries. Addresses #666 (Jon
Siwek)
0.51-3 | 2011-11-03 15:17:19 -0700
* Fixing compiler warnings. Addresses #388. (Jon Siwek)
0.51 | 2011-10-27 17:41:32 -0700
* Compile SWIG bindings with no-strict-aliasing. Addresses #644.
(Jon Siwek)
0.5 | 2011-10-25 20:18:20 -0700
* Make dist now cleans the copied source. (Jon Siwek)
* Add configure-time check that swig can generate python wrappers.
Addresses #642. (Jon Siwek)
* Updates for changes to communication API. (Seth Hall)
* Distribution cleanup. (Jon Siwek and Robin Sommer)
* Install binaries with an RPATH (Jon Siwek)
* Remove the 'net' type from Broccoli python bindings. Aaddresses
#535).
* Workaround for FreeBSD CMake port missing debug flags. (Jon Siwek)
* Adjust how python-broccoli test script prints floats.
* Allow record instances that don't initialize all fields. (Jon
Siwek)
* Update tests w/ example of sending a partial records. (Jon Siwek)
* Fix pybroccoli record instantiation. Declaring more than one
record_type could cause it to break because only the last-declared
record_type was used in the instantiation. (Jon Siwek)
* Change to fill in record field names. Before, it was sending
records with hardcoded "<unknown>" field names. (Jon Siwek)
* Adating CMake's include path based on output of python-config.
(Robin Sommer)
* Teaching CMake to use python-config for finding libraries. (Robin
Sommer)
* Making python-broccoli work with 64-bit integers. (Robin Sommer)
0.4 | 2011-07-19 17:54:35 -0700
* Improvements and fixes to record implementation. (Jon Siwek)
* Allow record instances that don't initialize all fields.
* Update tests w/ example of sending a partial records.
* Fix pybroccoli record instantiation.
* Change pybroccoli to fill in record field names.
* Update test bro script for new Bro policy script organization. (Jon Siwek)
* Cleanup (Seth Hall, Robin Sommer)
0.3 | 2011-05-05 20:42:47 -0700
* CMake build system. (Jon Siwek).
* Adapting to work with 64-bit integers, which Bro and Broccoli are
now using. (Robin Sommer)
0.2
- Repository switched to git, and README converted to reSt.
- License changed to BSD-style.
Broccoli Ruby
1.61 | 2017-05-26 08:27:27 -0500
* Release 1.61.
1.60 | 2016-10-27 14:42:49 -0700
* Release 1.60.
1.59 | 2016-06-30 16:55:31 -0700
* Add README.rst -> README symlink. (Johanna Amann)
1.58 | 2015-04-27 08:25:18 -0700
* Release 1.58
1.57 | 2014-04-03 15:53:50 -0700
* Release 1.57
1.56 | 2013-10-14 09:24:55 -0700
* Updating copyright notice. (Robin Sommer)
* Fix for setting REPO in Makefile. (Robin Sommer)
1.55 | 2013-09-23 14:42:03 -0500
* Update 'make dist' target. (Jon Siwek)
* Change submodules to fixed URL. (Jon Siwek)
* Switching to relative submodule paths. (Robin Sommer)
* s/bro-ids.org/bro.org/g. (Robin Sommer)
1.54 | 2012-08-01 13:56:22 -0500
* Fix configure script to exit with non-zero status on error (Jon Siwek)
1.53 | 2012-06-11 17:25:05 -0700
* Update bindings to work with Broccoli's IPv4-mapped BroAddrs. (Jon Siwek)
* Fix count/enum being treated same as addr. (Jon Siwek)
* Update broccoli-ruby for IPv6 addr/subnet support. Addresses #448.
(Jon Siwek)
* Raise minimum required CMake version to 2.6.3 (Jon Siwek)
1.52 | 2012-01-09 16:11:01 -0800
* Submodule README conformity changes (Jon Siwek)
1.51-10 | 2011-11-07 05:44:17 -0800
* Ignoring some SWIG warnings. Addresses #388. (Jon Siwek)
* Changes to broccoli-ruby installation scheme. Fixes #652.
- `--home` and `--prefix` configure options are now respected when
installing as the main CMake project. If not given, the Ruby
- When being installed as a CMake sub-project, then the
"home"-style installation is performed. (Jon Siwek)
1.51 | 2011-10-26 13:51:22 -0700
* Compile SWIG bindings with no-strict-aliasing (addresses #644).
(Jon Siwek)
1.5 | 2011-10-25 17:41:31 -0700
* Make dist now cleans the copied source. (Jon Siwek)
* Add configure-time check that swig can generate Ruby wrappers.
Addresses #642. (Jon Siwek)
* Distribution cleanup. (Robin Sommer)
* Updates to work with communication API changes. (Seth Hall)
* Reorganized the module names. From ruby, a user now loads the
"broccoli" module. This automatically pulls in the swig wrapper
named "broccoli_ext". (Seth Hall)
* Building with cmake completely works now. (Seth Hall)
* Updates for the change to 64-bit ints. (Seth Hall)
* Fixes for the example script. (Seth Hall)
* New example script that points out a bug in broccoli. (Seth Hall)
* Remove the 'net' type from Broccoli ruby bindings. Addresses #535.
(Jon Siwek)
* Install binaries with an RPATH (Jon Siwek)
1.4 | 2011-02-25 21:26:49 -0500
* Cleaning up and adding a configure script. (Seth Hall)
* Ruby 1.8 is now required. (Seth Hall)
* CMake fixes. (Seth Hall and Jon Siwek)
* Initial import. (Seth Hall)
Capstats
0.25 | 2017-05-26 08:27:56 -0500
* Release 0.25.
0.24-2 | 2016-12-24 11:06:37 +0100
* Fix the "-p" and "--write" command-line options. (Daniel Thayer)
0.24 | 2016-10-27 14:43:34 -0700
* Release 0.24.
0.23 | 2016-06-30 16:55:31 -0700
* Add README.rst -> README symlink. Addresses BIT-1413 (Vlad Grigorescu)
0.22 | 2015-04-27 08:25:19 -0700
* Release 0.22
0.21 | 2014-04-03 15:53:51 -0700
* Release 0.21
0.20 | 2013-10-14 09:24:55 -0700
* Release.
0.19-4 | 2013-10-07 17:07:40 -0700
* Fix getopt_long() usage. (Daniel Thayer)
* Updating copyright notice. (Robin Sommer)
* Fix for setting REPO in Makefile. (Robin Sommer)
0.19 | 2013-09-23 20:22:43 -0700
* Update 'make dist' target. (Jon Siwek)
* Correct a few errors in the README. (Daniel Thayer)
* s/bro-ids.org/bro.org/g. (Robin Sommer)
0.18 | 2012-08-01 13:57:19 -0500
* Fix configure script to exit with non-zero status on error (Jon Siwek)
0.17 | 2012-07-05 12:53:52 -0700
* Raise minimum required CMake version to 2.6.3 (Jon Siwek)
0.16 | 2012-01-09 16:11:02 -0800
* Submodule README conformity changes. (Jon Siwek)
* Fix parallel make portability. (Jon Siwek)
0.15 | 2011-10-25 17:41:31 -0700
* Make dist now cleans the copied source (Jon Siwek)
0.14-26 | 2011-10-14 15:09:34 -0700
* Distribution cleanup. (Robin Sommer)
0.14-25 | 2011-10-14 15:06:20 -0700
* Distribution cleanup. (Jon Siwek and Robin Sommer)
* config.h wasn't being configured by CMake correctly (Jon Siwek)
* Adding 'C' language to CMake project as some configure checks depend on it. (Jon Siwek)
0.14-14 | 2011-09-04 08:55:48 -0700
* FindPCAP now links against thread library when necessary (e.g.
PF_RING's libpcap). (Jon Siwek)
* Install binaries with an RPATH. (Jon Siwek)
* Workaround for FreeBSD CMake port missing debug flags. (Jon Siwek)
0.14-5 | 2011-07-24 08:28:06 -0700
* Fixing memory initialization error. (David Binderman)
0.14-3 | 2011-03-14 17:41:03 -0700
* CMake tuning. (Jon Siwek)
0.14-2 | 2011-02-01 12:38:34 -0800
* Linking directly to the found pcap library. (Jon Siwek)
0.14 | 2011-01-28 11:09:04 -0800
* Fix for compiling on OpenBSD and NetBSD. (Kevin Lo)
* Converting README from AsciiDoc to REST. (Robin Sommer)
* Ported to CMake. (Jon Siwek)
* New CHANGES format.
0.13
* Do not output anything to syslog if not explicitly enabled.
* New option -q <n> suppresses all normal output but exits with
status code 0 if at least n packets have been received, and 1
otherwise.
0.12
* New option --select/-N which for live pcap input uses select() to
check for new packets. This is primarily for testing purposes and
shouldn't produce any different results. The code mimics pretty
closely how the Bro NIDS uses select() on pcap file handles.
* Fix disabling option -d when compiled without DAG support. (Justin
Azoff)
* autotools compatibility fixes.
* Man page contributed by Justin Azoff.
0.11
* Fixed potential segfault.
0.1
* Initial release
Trace-Summary
0.87 | 2017-06-06 13:13:45 -0700
* Release 0.87.
0.86-15 | 2017-05-19 14:01:03 -0500
* Fix to correctly identify JSON logs when "--conn-version=2" is specified.
Also fixed identification of non-default field scope separator for
non-json logs. Minor cleanup of json and scope separator code, and
added test cases for json logs and logs with non-default scope
separator. (Daniel Thayer)
0.86-14 | 2017-05-18 14:19:25 -0500
* Add ability to read json format logs and logs with a non-default field
scope separator. (Pierre Gaulon)
0.86-11 | 2017-04-21 14:24:45 -0700
* Fix some failing tests on Python 3. (Daniel Thayer)
* Fix a minor bug when using Python 3, and improved the check to
detect if Python 3 is running. (Daniel Thayer)
0.86-8 | 2017-03-20 12:12:06 -0700
* Extended trace-summary to interpret Bro 2.x-style log header
lines. (Daniel Thayer)
* Improve error handling and performance when reading conn.log by
skipping redundant operations removing unused variables and
unnecessary exception handlers. (Daniel Thayer)
* Improved a few error messages by showing more info. (Daniel
Thayer)
0.86 | 2016-10-31 10:27:57 -0700
* Release 0.86.
* Update description of Bro in the README (Daniel Thayer)
* Let cmake put the python path in the hashbang line. (Daniel Thayer)
* Remove ipsumdump check. Addresses BIT-1736. (Johanna Amann)
0.85-5 | 2016-10-20 11:50:23 -0400
* Fix test canonifier to work with FreeBSD/OSx sed. (Daniel Thayer).
0.85-3 | 2016-10-10 08:15:55 -0700
* Python 3 compatibility fix. (Daniel Thayer)
0.85 | 2016-08-12 13:20:06 -0700
* Release 0.85.
0.84-16 | 2016-05-17 16:21:13 -0700
* Adjust IP address column widths as needed for IPv6 addrs to
improve readability of the output. Addresses BIT-1571. (Daniel
Thayer)
* Add README.rst -> README symlink. Addresses BIT-1413 (Johanna
Amann)
0.84-2 | 2015-08-18 07:54:36 -0700
* Fix typo in a TEST_DIFF_CANONIFIER script name. (Daniel Thayer)
0.84 | 2015-04-27 08:25:19 -0700
* Release 0.84
0.83-19 | 2015-03-06 14:52:27 -0800
* Update code to work with Python 3. Bump minimum required Python
version to 2.6. (Daniel Thayer)
* Fix timestamps to not loose precision unnecessarily. (Daniel
Thayer)
* Add more error checks so errors are reported more clearly. (Daniel
Thayer)
* Add regression tests. (Daniel Thayer)
0.83-9 | 2014-12-08 13:54:39 -0800
* Add man page for trace-summary. (Raúl Benencia)
0.83 | 2014-04-03 15:53:51 -0700
* Release 0.83
0.82 | 2013-10-14 09:24:55 -0700
* Updating copyright notice. (Robin Sommer)
0.81 | 2013-09-23 20:24:46 -0700
* Fixing sampling in pcap mode. (Robin Sommer)
* s/bro-ids.org/bro.org/g (Robin Sommer)
0.8 | 2012-07-05 12:54:50 -0700
* Fix typos. (Daniel Thayer)
* trace-summary now works with IPv6 traffic. It needs a current
pysubnettree for that. (Daniel Thayer)
* Raise minimum required CMake version to 2.6.3. (Jon Siwek)
0.73 | 2012-01-09 16:11:02 -0800
* Submodule README conformity changes. (Jon Siwek)
0.72 | 2011-10-25 17:57:00 -0700
* New make dist/distclean targets. (Jon Siwek)
* Adding executable permission back to script. (Robin Sommer)
* Cleaning up the distribution. (Robin Sommer)
* Updating README (Jon Siwek)
0.71-19 | 2011-09-08 12:52:20 -0700
* Now ignoring all lines starting with a pound Closes #602. (Robin
Sommer)
* Install binaries with an RPATH (Jon Siwek)
0.71-16 | 2011-08-03 16:18:15 -0700
* Switching to new update-changes script. (Robin Sommer)
0.71-15 | 2011-08-03 16:02:14 -0700
* trace-summary now parses both Bro 1.x and 2.x conn.log formats.
The default setting is to make an educated guess at the format,
but can be explicitly set via the new --conn-version switch. (Jon
Siwek)
0.71-6 | 2011-03-14 17:41:05 -0700
* CMake tweaks. (Jon Siwek)
* Prettyfing the message about sampling being in effect.
0.71-3 | 2011-01-15 14:14:07 -0800
* Updating update-changes. (Robin Sommer)
* Let CMake infer install prefix (Jon Siwek)
* Add warning when building and installing are done by different users (Jon Siwek)
0.71-1 | 2011-01-04 19:02:06 -0800
* Tweaking update-changes. (Robin Sommer)
0.71 | 2011-01-04 18:36:36 -0800
* Better error message when missing Python package. (Jon Siwek)
* Better error message if ipsumdump not installed. (Jon Siwek)
* Migrated from os.popen (deprecated since Python 2.6) to
subprocess.Popen (available since Python 2.4). (Jon Siwek)
* Switch to CMake-based installation (Jon Siwek)
0.7
- Repository switched to git, and README converted to reSt.
- Sample factor now included in output.
- Bugfix: Sample factor was not applied to local subnets
break-down.
0.6
License changed to BSD-style.
0.5
First release.
BinPAC
0.50 | 2018-08-28 10:25:02 -0500
* Fix array bounds checking
For arrays that are fields within a record, the bounds check was based
on a pointer to the start of the record rather than the start of the
array field. (Jon Siwek, Corelight)
0.49 | 2018-05-29 18:58:22 -0500
* BIT-1829: throw exceptions for negative array length expressions
(Corelight)
* BIT-1829: throw exceptions for excessive array sizes (Corelight)
* Improve parsing of known-length, static-size arrays (Corelight)
* GH-4: fix premature loop termination when parsing known-length arrays
(Tomas Bortoli: initial report/patch, Corelight: finalizations)
* BIT-1829: fix &length suppressing boundary checks for array elements
(Corelight)
0.48 | 2018-02-13 09:27:42 -0800
* Release 0.48
* Fix integer overflow in binpac generated code
(Philippe Antoince/Catena cyber).
0.47 | 2017-05-26 08:26:05 -0500
* Release 0.47.
0.46-1 | 2016-11-30 10:18:23 -0800
* Allow more than one &require attribute on a field. (François Pennaneach)
0.46 | 2016-10-27 14:41:38 -0700
* Release 0.46.
0.45 | 2016-08-02 11:09:42 -0700
* Release 0.45.
0.44-24 | 2016-08-02 11:08:13 -0700
* Fix memory leak in pac_parse.yy. (Bryon Gloden)
0.44-21 | 2016-06-14 17:41:28 -0700
* Bug fix for pac_swap function with int32 type of argument.
(Bartolo Otrit)
0.44-18 | 2016-05-23 08:25:49 -0700
* Fixing Coverity warning. (Robin Sommer)
0.44-17 | 2016-05-06 16:52:37 -0700
* Add a comment in the generated C++ code for fall through in
switch. Coverity raised an error about this. (Vlad Grigorescu)
0.44-11 | 2016-03-04 12:36:57 -0800
* Update for new CMake OpenSSL script. (Johanna Amann)
0.44-7 | 2016-01-19 10:05:37 -0800
* Fixed compiler complaining about recursive function. (Seth Hall)
0.44-3 | 2015-09-11 12:24:21 -0700
* Add README.rst symlink. Addresses BIT-1413 (Vlad Grigorescu)
0.44 | 2015-04-27 08:25:17 -0700
* Release 0.44.
0.43-8 | 2015-04-21 20:11:06 -0700
* Adding missing include. (Robin Sommer)
0.43-7 | 2015-04-21 13:45:20 -0700
* BIT-1343: Extend %include to work with relative paths. (Jon Siwek)
0.43-5 | 2015-04-09 12:09:04 -0700
* BIT-1361: Improve boundary checks of records that use &length.
(Jon Siwek)
0.43 | 2015-01-23 09:56:59 -0600
* Fix potential out-of-bounds memory reads in generated code.
CVE-2014-9586. (John Villamil and Chris Rohlf - Yahoo Paranoids,
Jon Siwek)
0.42-9 | 2014-11-03 10:05:17 -0600
* Separate declaration of binpac::init from definition. (Jon Siwek)
0.42-6 | 2014-10-31 17:42:21 -0700
* Adding a new binpac::init() function that must be called by the
host before anything else. Internally, this function compiles all
regular expressions, avoiding to do that inside the regexp
constructor. (Robin Sommer)
0.42 | 2014-04-08 15:24:11 -0700
* Release 0.42.
0.41-5 | 2014-04-08 15:23:48 -0700
* Request format macros from inttypes.h explicitly. This helps
ensure the availability of PRI* macros from .pac files, which
cannot create this definition themselves since the inclusion of
binpac.h is hardcoded to be placed very early in the generated
code and already includes inttypes.h itself. (Jon Siwek)
0.41 | 2013-10-14 09:24:54 -0700
* Updating copyright notice. (Robin Sommer)
0.4-5 | 2013-10-02 10:33:05 -0700
* Fix uninitialized (or unused) fields. (Jon Siwek)
* Generate initialization code for external types. Numeric/pointer
types can be initialized to 0. (Jon Siwek)
* Optimize negative string length check. (Jon Siwek)
* Fix for setting REPO in Makefile. (Robin Sommer)
0.4 | 2013-09-23 20:56:19 -0700
* Update 'make dist' target. (Jon Siwek)
* Change submodules to fixed URL. (Jon Siwek)
* Add virtual dtor to RefCount base class. (Jon Siwek)
0.34-24 | 2013-09-12 15:49:51 -0500
* Add missing break to switch statement case. (Jon Siwek)
* Remove unreachable code. (Jon Siwek)
* Add missing va_end()'s to match va_start()'s. (Jon Siwek)
* Fix two use-after-free bugs. (Jon Siwek)
* Fix double-free. (Jon Siwek)
* Remove some answers from the Q&A section of README (Daniel Thayer)
* Add BinPAC documentation from the old Bro wiki (Daniel Thayer)
0.34-11 | 2013-07-24 18:35:28 -0700
* Adding an interface to manually control the buffering for
generated parsers. (Robin Sommer)
This consists of two parts:
1. The generated Flow classes expose their flow buffers via a new
method flow_buffer().
2. Flow buffers get two new methods:
// Interface for delayed parsing. Sometimes BinPAC doesn't get the
// buffering right and then one can use these to feed parts
// individually and assemble them internally. After calling
// FinishBuffer(), one can send the uppper-layer flow an FlowEOF()
// to trigger parsing.
void BufferData(const_byteptr data, const_byteptr end);
void FinishBuffer(); (Robin Sommer)
0.34-8 | 2013-04-27 15:04:23 -0700
* Fix an exception slicing issue in binpac generated cleanup code.
(Jon Siwek)
* s/bro-ids.org/bro.org/g (Robin Sommer)
0.34-3 | 2012-11-13 17:24:24 -0800
* Add scoping to usages of binpac::Exception classes in generated
code. This allows analyzers to define their own types of the same
name without mistakingly overshadowing the usages of
binpac::Exception and its derived types in the generated parser
code. (Jon Siwek)
0.34 | 2012-08-01 13:54:39 -0500
* Fix configure script to exit with non-zero status on error (Jon
Siwek)
0.33 | 2012-07-24 09:05:37 -0700
* Silence warning for generated code when compiling with clang.
(Robin Sommer)
0.32 | 2012-06-11 17:25:04 -0700
* Change binpac.h integral typedefs and reimplement 64-bit
pac_swap(). Addresses #761. (Jon Siwek)
* Adding int64 and uint64 types to binpac. (Seth Hall)
* Raise minimum required CMake version to 2.6.3 (Jon Siwek)
0.31 | 2012-01-09 16:11:01 -0800
* Submodule README conformity changes. (Jon Siwek)
* Fix parallel make portability. (Jon Siwek)
0.3 | 2011-10-25 17:41:31 -0700
* Change distclean to only remove build dir. (Jon Siwek)
* Make dist now cleans the copied source. (Jon Siwek)
* Distribution cleanup. (Jon Siwek and Robin Sommer)
* Arrays now suport the &transient attribute.
If set, parsed elements won't actually be added to the array, and
read access to the array aren't permitted. This is helpful to save
memory in the case of large arrays for which elements don't need
(or can't) be buffered. (Robin Sommer)
* Install binaries with an RPATH. (Jon Siwek)
* Workaround for FreeBSD CMake port missing debug flags. (Jon Siwek)
0.2 | 2011-04-18 12:50:21 -0700
* Converting build process to CMake (Jon Siwek).
* Fixing crash with undefined case expressions. (Robin Sommer)
Found by Emmanuele Zambon.
* A command line -q flag to quiet the output, plus a fix for a small
compiler warning. (Seth Hall)
* Initial import of Bro's binpac subdirectory from SVN r7088. (Jon Siwek)
Bro-Aux
0.39 | 2017-05-26 08:26:51 -0500
* Release 0.39.
* Update plugin skeleton's CMakeLists.txt.
The project name is now derived from the actual plugin name instead of
just "Plugin" and it now sets up CPack packaging (e.g. RPM/DEB) to be
available via "make package" in the build dir. (Jon Siwek)
* Remove superfluous spaces from line endings. (Johanna Amann)
* Small update to plugin-support to use override instead of virtual (Johanna Amann)
0.38 | 2016-11-14 09:19:05 -0800
* Robustness fix for check-release script. (Robin Sommer)
0.37 | 2016-10-27 14:42:01 -0700
* Release 0.37.
* Reverting fix to build plugins on OS X; no longer necessary because plugins
do no longer have to pull in OpenSSL. (Johanna Amann)
0.36 | 2016-08-12 13:12:49 -0700
* Release 0.36.
* Update gen-mozilla-ca-list.rb to use a file instead of accessing the
Mozilla server. (Johanna Amann)
* Fix compiler warning. (Robin Sommer)
* Removing nftools. (Robin Sommer)
* Fix coverity warning (memory leak) in bro-cut (Daniel Thayer)
0.35-27 | 2016-06-21 18:31:33 -0700
* Fix bro-cut to allow unset or zero time values. (Daniel Thayer)
* Fix failure to build plugins on OS X 10.11 with init-plugin.
(Daniel Thayer)
0.35-18 | 2016-03-04 12:38:16 -0800
* Update for new CMake OpenSSL script. (Johanna Amann)
0.35-15 | 2016-02-01 12:37:46 -0800
* Fix the init-plugin script to be more portable. (Daniel Thayer)
0.35-8 | 2015-08-10 14:56:24 -0700
* Plugin skeletons now include a __preload__.bro that pulls in
types.bro for defining types. (Robin Sommer)
0.35-7 | 2015-08-10 12:58:35 -0700
* Fix a test for large time values that fails on some systems.
(Daniel Thayer)
0.35-6 | 2015-08-06 22:29:36 -0400
* Improved handling of malformed input, avoiding crashes. (Justin
Azoff and Daniel Thayer)
* Remove unused code and fix initialization of long_opts. (Daniel
Thayer)
0.35-4 | 2015-07-21 09:38:58 -0700
* Bringing back the ``--help`` option for bro-cut. (Justin Azoff)
0.35-2 | 2015-07-10 07:14:52 -0700
* Add more documentation for bro-cut. (Daniel Thayer)
0.35 | 2015-06-03 09:02:49 -0700
* Release 0.35.
0.34-5 | 2015-06-03 09:02:10 -0700
* Fix replace_version_in_rst function in update-changes script to
cope with "beta" in version string. (Daniel Thayer)
* Portability fix for plugin configure script. (Daniel Thayer)
* Fix minor typo in init-plugin error message. (Daniel Thayer)
0.34 | 2015-05-07 20:30:43 -0700
* Release 0.34.
* Change make-release to assume sign-file is in path (Johanna Amann)
0.33-76 | 2015-04-27 08:23:18 -0700
* Fix sed regex for replacing version in header file. (Jon Siwek)
0.33-74 | 2015-04-23 06:58:37 -0700
* Correct a few typos in update-changes script. (Daniel Thayer)
* Adding function to update-changes that updates version in a C
header file. (Robin Sommer)
* Fix plugin configure skeletons to work on more shells. (Jon Siwek)
0.33-68 | 2015-02-23 11:26:14 -0600
* Plugin skeleton updates. (Robin Sommer)
- Updating plugin skeleton license.
- Removing the plugin MAINTAINER skeleton file.
- Adding hooks to configure script so that plugins can add options
without modifying the scripts itself.
- BIT-1302: Extending plugin skeleton Makefile to reload cached
CMake variables when Bro has been reconfigured. (Robin Sommer)
- Removing bdist and sdist make targets. The former is superseded by
the new build process which always creates a binary distribution
tarball. The latter is easy enough to do manually now that all
dynamic stuff is in build/
- Added a VERSION file; content goes into name of the binary tarball
- Move README.edit-me to README.
- Allowing relative paths for --bro-dist
* Changing init-plugin to take an additional parameter specifying the
directory where to create the plugin skeleton. (Robin Sommer)
0.33-58 | 2015-02-12 12:15:39 -0600
* Fix bro-cut compile warning on FreeBSD (Johanna Amann)
0.33-56 | 2015-01-08 13:06:36 -0600
* Increase minimum required CMake version to 2.8. (Jon Siwek)
0.33-55 | 2014-12-08 13:49:37 -0800
* Add man page for bro-cut. (Raúl Benencia)
* Add --install-root to plugin skeleton's configure. (Robin
Sommer)
* Fix get-bro-env script to use sh equality operator. (Jon Siwek)
* Add an option to update-changes that prevents it from adding
author names to entries. (Robin Sommer)
0.33-45 | 2014-08-21 15:47:29 -0500
* Various tweaks to the plugin skeleton. (Robin Sommer)
0.33-38 | 2014-08-01 14:03:49 -0700
* bro-cut has been rewritten in C, and is hence much faster. (Daniel
Thayer, based on an initial version by Justin Azoff).
0.33-26 | 2014-07-30 15:51:42 -0500
* Remove a superfluous file from plugin skeleton. (Jon Siwek)
* init-plugin now creates a Plugin.h as well. (Robin Sommer)
* Adding a basic btest setup to the plugin skeleton. (Robin Sommer)
* Updating plugin skeleton to new API. (Robin Sommer)
* Updates to the init-plugin script/skeleton. (Robin Sommer)
* A script to setup a skeleton for a new dynamic plugin. (Robin Sommer)
0.33-11 | 2014-07-08 20:42:32 -0700
* Add more tests of bro-cut. (Daniel Thayer)
* Fix bug in bro-cut when duplicate fields are specified. (Daniel Thayer)
* Fix bug in bro-cut when log file has missing field. (Daniel Thayer)
* Fix bug in bro-cut output of "#types" header line. (Daniel Thayer)
* Fix bug in bro-cut when separator is not hexadecimal. (Daniel Thayer)
* Adding test target to top-level Makefile. (Robin Sommer)
0.33-4 | 2014-06-26 17:31:25 -0700
* Test-suite for bro-cut. (Daniel Thayer)
0.33-2 | 2014-06-26 17:27:09 -0700
* Change bro-cut UTC options to not always override local time.
(Daniel Thayer).
* Updated the bro-cut usage message to make it more clear that the
BRO_CUT_TIMEFMT environment variable affects only the -u and -d
options. (Daniel Thayer).
0.33 | 2014-05-08 16:27:10 -0700
* Release 0.33.
0.32-5 | 2014-05-08 16:25:55 -0700
* Adding git-move-submodules scriptm, which moves all submodules to
the head of a given branch and adapts parent modules
correspondingly. (Robin Sommer)
0.32-4 | 2014-04-22 21:34:23 -0700
* A git hook script to prevent pushs when the external test suite has
new commits pending. (Robin Sommer)
0.32 | 2013-11-01 05:24:56 -0700
* Extending Mozialla cert script to include source URL and copyright
in output. (Robin Sommer)
0.31 | 2013-10-14 09:24:54 -0700
* Release.
0.3-5 | 2013-10-07 17:19:14 -0700
* Fix for release script. (Robin Sommer)
* Updating copyright notice. (Robin Sommer)
0.3-3 | 2013-09-28 11:17:42 -0700
* Don't show error message in bro-cut when gawk not found, which
could appear on some systems. (Daniel Thayer)
0.3-1 | 2013-09-24 13:41:02 -0700
* Fix for setting REPO in Makefile, and some tweaks to release
scripts. (Robin Sommer)
0.3 | 2013-09-23 14:42:56 -0500
* Update 'make dist' target. (Jon Siwek)
* Change submodules to fixed URL. (Jon Siwek)
* make-release nows ignores modules that aren't tagged for release
or beta. (Robin Sommer)
* Prettyfing check-release output. (Robin Sommer)
* Update gen-mozilla-ca-list.rb to retrieve the Mozilla
root CA list from a current url. (Johanna Amann)
0.26-25 | 2013-09-18 14:44:35 -0700
* A set of README updates, including installation instructions and
description of bro-cut. (Daniel Thayer)
* Switching to relative submodule paths. (Robin Sommer)
0.26-21 | 2013-08-19 11:21:11 -0700
* Fixing git-show-fastpath handling of non-existing fastpath
branches. (Robin Sommer)
0.26-19 | 2013-07-31 20:09:52 -0700
* Making git-show-fastpath save against repositories that don't have
a fastpath. (Robin Sommer)
0.26-16 | 2013-05-17 07:45:24 -0700
* A negate option -n for bro-cut printing all fields *except* those
listed on the command-line. (Robin Sommer)
0.26-14 | 2013-03-22 12:17:54 -0700
* Fixing bro-cut to work with older gawk versions. (Chris Kanich)
* s/bro-ids.org/bro.org/g (Robin Sommer)
0.26-5 | 2012-11-01 14:24:25 -0700
* Portability fix: removing interface option on non-Linux. (Robin Sommer)
0.26-4 | 2012-10-31 14:39:03 -0700
* rst learns a new option "-i <if>" to set the interface to use.
(Vlad Grigorescu).
0.26 | 2012-08-24 15:10:04 -0700
* Fixing update-changes, which could pick the wrong control file. (Robin Sommer)
* Fixing GPG signing script. (Robin Sommer)
0.25 | 2012-08-01 13:55:46 -0500
* Fix configure script to exit with non-zero status on error (Jon Siwek)
0.24 | 2012-07-05 12:50:43 -0700
* Raise minimum required CMake version to 2.6.3 (Jon Siwek)
* Adding script to delete old fully-merged branches. (Robin Sommer)
0.23-2 | 2012-01-25 13:24:01 -0800
* Fix a bro-cut error message. (Daniel Thayer)
0.23 | 2012-01-11 12:16:11 -0800
* Tweaks to release scripts, plus a new one for signing files.
(Robin Sommer)
0.22 | 2012-01-10 16:45:19 -0800
* Tweaks for OpenBSD support. (Jon Siwek)
* bro-cut extensions and fixes. (Robin Sommer)
- If no field names are given on the command line, we now pass through
all fields. Adresses #657.
- Removing some GNUism from awk script. Addresses #653.
- Added option for time output in UTC. Addresses #668.
- Added output field separator option -F. Addresses #649.
- Fixing option -c: only some header lines were passed through
rather than all. (Robin Sommer)
* Fix parallel make portability. (Jon Siwek)
0.21-9 | 2011-11-07 05:44:14 -0800
* Fixing compiler warnings. Addresses #388. (Jon Siwek)
0.21-2 | 2011-11-02 18:12:13 -0700
* Fix for misnaming temp file in update-changes script. (Robin Sommer)
0.21-1 | 2011-11-02 18:10:39 -0700
* Little fix for make-release script, which could pick out the wrong
tag. (Robin Sommer)
0.21 | 2011-10-27 17:40:45 -0700
* Fixing bro-cut's usage message and argument error handling. (Robin Sommer)
* Bugfix in update-changes script. (Robin Sommer)
* update-changes now ignores commits it did itself. (Robin Sommer)
* Fix a bug in the update-changes script. (Robin Sommer)
* bro-cut now always installs to $prefix/bin by `make install`. (Jon Siwek)
* Options to adjust time format for bro-cut. (Robin Sommer)
The default with -d is now ISO format. The new option "-D <fmt>"
specifies a custom strftime()-style format string. Alternatively,
the environment variable BRO_CUT_TIMEFMT can set the format as
well.
* bro-cut now understands the field separator header. (Robin Sommer)
* Renaming options -h/-H -> -c/-C, and doing some general cleanup.
0.2 | 2011-10-25 19:53:57 -0700
* Adding support for replacing version string in a setup.py. (Robin
Sommer)
* Change generated root cert DN indices format for RFC2253
compliance. (Jon Siwek)
* New tool devel-tools/check-release to run before making releases.
(Robin Sommer)
* devel-tools/update-changes gets a new option -a to amend to
previous commit if possible. Default is now not to (used to be the
opposite). (Robin Sommer)
* Change Mozilla trust root generation to index certs by subject DN. (Jon Siwek)
* Change distclean to only remove build dir. (Jon Siwek)
* Make dist now cleans the copied source (Jon Siwek)
* Small tweak to make-release for forced git-clean. (Jon Siwek)
* Fix to not let updates scripts loose their executable permissions.
(Robin Sommer)
* devel-tools/update-changes now looks for a 'release' tag to
idenfify the stable version, and 'beta' for the beta versions.
(Robin Sommer).
* Distribution cleanup. (Robin Sommer)
* New script devel-tools/make-release to create source tar balls.
(Robin Sommer)
* Removing bdcat. With the new log format, this isn't very useful
anymore. (Robin Sommer)
* Adding script that shows all pending git fastpath commits. (Robin
Sommer)
* Script to measure CPU time by loading an increasing set of
scripts. (Robin Sommer)
* extract-conn script now deals wit *.gz files. (Robin Sommer)
* Tiny update to output a valid CA list file for SSL cert
validation. (Seth Hall)
* Adding "install-aux" target. Addresses #622. (Jon Siwek)
* Distribution cleanup. (Jon Siwek and Robin Sommer)
* FindPCAP now links against thread library when necessary (e.g.
PF_RING's libpcap) (Jon Siwek)
* Install binaries with an RPATH (Jon Siwek)
* Workaround for FreeBSD CMake port missing debug flags (Jon Siwek)
* Rewrite of the update-changes script. (Robin Sommer)
0.1-1 | 2011-06-14 21:12:41 -0700
* Add a script for generating Mozilla's CA list for the SSL analyzer.
(Seth Hall)
0.1 | 2011-04-01 16:28:22 -0700
* Converting build process to CMake. (Jon Siwek)
* Removing cf/hf/ca-* from distribution. The README has a note where
to find them now. (Robin Sommer)
* General cleanup. (Robin Sommer)
* Initial import of bro/aux from SVN r7088. (Jon Siwek)
BTest
0.57 | 2017-05-15 16:13:33 -0700
* Release 0.57.
0.56-22 | 2017-05-15 16:13:23 -0700
* Fixing broken version numbers. (Robin Sommer)
0.56-21 | 2017-05-15 16:05:18 -0700
* Catching CTRL-C and cleaning up. (Robin Sommer)
0.56-20 | 2017-03-21 17:56:10 -0700
* Catching exception that wasn't caught. (Robin Sommer)
0.56-19 | 2017-03-03 12:50:42 -0800
* Fix btest-progress output to stderr when run from btest. (Daniel
Thayer)
0.56-17 | 2017-03-02 16:24:31 -0800
* Cosmectics for progress output: Delete it before asking for
baseline updates. (Robin Sommer)
* Fixing missing output for back-to-back btest-progress calls.
Addresses BIT-1800. (Robin Sommer)
* Fix for augmented output to console. (Robin Sommer)
* Send btest-progress output to stderr as well. (Robin Sommer)
0.56-13 | 2017-02-23 10:14:56 -0800
* Prevent socket path length from exceeding system limits. Addresses
BIT-862. (Daniel Thayer)
0.56-11 | 2017-02-03 12:38:01 -0800
* Adding btest-progress to setup.py. (Robin Sommer)
0.56-10 | 2017-01-25 13:04:07 -0800
* Fix a failing test on FreeBSD. (Daniel Thayer)
* Fix a bug in btest-progress when using the "-q" option. (Daniel
Thayer)
* Fix some trivial errors in documentation and Makefile. (Daniel
Thayer)
* Add 'upload' Makefile target to upload to PyPi. (Jon Siwek)
0.56-5 | 2017-01-24 08:45:29 -0800
* Bugfix for recent btest-progress changes. (Robin Sommer)
0.56-4 | 2017-01-23 19:59:59 -0800
* New utility btest-progress to display progress messages while a
test is executing. These messages appear in real-time while the
rest is still running. When stdout is a tty, the progress messages
are incorporated into the colored one-line status message. By
default, btest-progress also prints the message to a test's
standard output. That can be suppressed by giving it an option -q.
(Robin Sommer)
* Experimental automatic generation of test reference documentation.
The new command-line option "-R <format>" prints out a list of all
tests in either Markdown (format 'md') or reStructuredText (format
'rst'). The list includes a documentation string with each test
that gets defined through a new "@TEST-DOC: <docstring>"
directive. This is experimental. (Robin Sommer)
* Fix pylint warnings. (Robin Sommer)
0.56 | 2016-10-31 14:23:57 -0700
* Release 0.56.
0.55-6 | 2016-10-31 14:23:24 -0700
* Python 3 compatibility fixes for btest-sphinx.py. (Daniel Thayer)
0.55-4 | 2016-10-25 09:31:25 -0700
* Fix diff-max-lines.test to work on openbsd. (Daniel Thayer)
0.55-2 | 2016-10-10 08:18:54 -0700
* Fix the btest-rst-cmd script to work with Python 3. (Daniel
Thayer)
0.55 | 2016-02-23 14:02:35 -0800
* Release 0.55.
0.54-65 | 2016-02-23 14:00:10 -0800
* Fine-tuning diagnostic output. It needlessly stripped leading
whitespace. (Robin Sommer)
0.54-63 | 2016-02-07 19:39:54 -0800
* Extending --groups to allow running everything *except* a set of
groups. (Robin Sommer)
* Fix portability issue with use of mktemp. (Daniel Thayer)
0.54-60 | 2015-11-16 07:30:38 -0800
* Updates for Python 3. (Fabian Affolter)
0.54-58 | 2015-10-01 16:04:51 -0700
* Improved test of TEST_DIFF_FILE_MAX_LINES. (Daniel Thayer)
* Added ability for a user to override the default number of lines
to show for diffs by setting the environment variable
TEST_DIFF_FILE_MAX_LINES. Reduced the default to 100. (Daniel
Thayer)
* When no baseline exists, changed btest-diff to always just show
the entire file. (Daniel Thayer)
0.54-55 | 2015-08-25 07:47:22 -0700
* Port to Python 3. (Daniel Thayer)
* Various cleanup, bug fix, simplifications, and smaller
improvements. (Daniel Thayer)
* Improve and extend test suite substantially. (Daniel Thayer)
0.54-9 | 2015-07-03 18:21:52 -0700
* Make sure IgnoreDirs works with toplevel globbing. (Robin Sommer)
0.54-8 | 2015-07-03 16:31:24 -0700
* Expanding globs in TestDirs, relative to TestBase. (Robin Sommer)
0.54-7 | 2015-06-22 13:07:42 -0700
* Allow BTEST_TEST_BASE overriding in alternative configuration.
(Vlad Grigorescu)
* Create README symlink for GitHub rendering. (Vlad Grigorescu)
0.54-1 | 2015-06-18 09:08:34 -0700
* Add support for BTEST_TEST_BASE environment variable for
overriding the test base directory. (Robin Sommer)
0.54 | 2015-03-02 17:22:22 -0800
* Release 0.54.
0.53-6 | 2015-03-02 17:21:26 -0800
* Improve documentation of timing functionality. (Daniel Thayer)
* Add a new section to documentation that lists the BTest
prerequisites. (Daniel Thayer)
* Add warning when btest cannot create timing baseline. (Daniel
Thayer)
0.53-3 | 2015-01-22 07:25:01 -0800
* Fix some typos in the README. (Daniel Thayer)
0.53-1 | 2014-11-11 13:21:10 -0800
* In diagnostics, do not show verbose output for tests known to
fail. (Robin Sommer)
0.53 | 2014-07-22 17:36:24 -0700
* Release 0.53.
0.52-2 | 2014-07-22 17:36:15 -0700
* Update MANIFEST.in and setup.py to fix packaging. (Jon Siwek)
0.52 | 2014-03-13 14:05:44 -0700
* Release 0.52.
0.51-14 | 2014-03-13 14:05:36 -0700
* Fix a link in the README. (Jon Siwek)
0.51-12 | 2014-02-11 16:12:44 -0800
* Work-around for systems reporting that a socket path is too long.
Addresses BIT-862. (Robin Sommer)
0.51-11 | 2014-02-11 15:37:40 -0800
* Fix for Linux systems that have the perf tool but don't support
measuring instructions. (Robin Sommer)
* No longer tracking tests that are expected to fail in state file.
(Robin Sommer)
* Refactoring the timing code to no longer execute at all when not
needed.(Robin Sommer)
0.51-7 | 2014-02-06 21:06:40 -0800
* Fix for platforms that don't support timing measurements yet.
(Robin Sommer)
0.51-6 | 2014-02-06 18:19:08 -0800
* Adding a timing mode that records test execution times per host.
This is for catching regressions (or improvements :) that lets
execution times divert significantly. Linux only for now. See the
README for more information. (Robin Sommer)
* Adding color to test status when writing to console. (Robin Sommer)
* A bit of refactoring to define the status messages ("ok", "failed")
only at a single location.
Also added a note when a test declared as expecting failure in fact
succeeds. (Robin Sommer)
0.51-2 | 2013-11-17 20:21:08 -0800
* New keyword ``TEST-KNOWN-FAILURE`` to mark tests that are
currently known to fail. (Robin Sommer)
0.51-1 | 2013-11-11 13:36:36 -0800
* Fixing bug with tests potentially being ignored when using
alternatives. (Robin Sommer)
0.51 | 2013-10-07 17:29:50 -0700
* Updating copyright notice. (Robin Sommer)
0.5-1 | 2013-10-07 17:26:30 -0700
* Polishing how included commands and files are shown. (Robin Sommer)
- Enabling CSS styling to command lines and shown file names
via the new "btest-include" and "btest-cmd" classes.
- Fix to enable showing line numbers in btest-sphinx generated
output.
- Fix to enable Pygments coloring in output.
0.5 | 2013-09-20 14:48:01 -0700
* Fix the btest-rst-pipe script. (Daniel Thayer)
* A set of of documentation fixes, clarifications, and extensions.
(Daniel Thayer)
* A set of changes to Sphinx commands and directives. (Robin Sommer)
btest-rst-*:
- Always show line numbers.
- Highlight the command executed.
- rst-cmd-include gets an option -n <i> to include only upto i lines.
- rst-cmd-include prefixes output with "<file>" to show what we're
including.
btest-include:
- Set Pygments language automatically if we show a file with an
extension we know (in particular ".bro").
- Prefix output with "<file>" to show what we're including.
0.4-63 | 2013-08-28 21:10:39 -0700
* btest-sphinx now provides a new directive btest-include. This
works like literalinclude (with all its options) but it also saves
a version of the included text as a test to detect changes. (Robin
Sommer)
0.4-60 | 2013-08-28 18:54:51 -0700
* Fix typos and reST formatting in README (Daniel Thayer)
* Fix a couple of error messages. (Daniel Thayer)
* Fixed a reference to a non-existent variable which was causing the
"-w" option to have no effect. (Daniel Thayer)
* Test portability fix. (Robin Sommer)
0.4-55 | 2013-08-22 16:09:21 -0700
* New "Sphinx-mode" for BTest, activated with -S. This allows to
capture a test's diagnostic output when running from inside
Sphinx; the output will now be inserted into the generated
document. (Robin Sommer)
* Adding an option -n to btest-rst-cmd that truncates output longer
than N lines. (Robin Sommer)
* Adding a PartFinalizer that runs a commmand at the completion of
each test part. (Robin Sommer)
0.4-51 | 2013-08-22 10:36:34 -0700
* Improve cleanup of processes that don't terminate with
btest-bg-wait. (Jon Siwek)
0.4-49 | 2013-08-13 18:43:03 -0700
* Fixing test portability problems. (Daniel Thayer)
* Adding TEST_BASE environment variable. The existing TESTBASE isn't
always behaving as expected and wasn't documented to begin with.
(Robin Sommer)
0.4-43 | 2013-08-12 16:04:53 -0700
* Bugfix for ignored tests. (Robin Sommer)
0.4-42 | 2013-07-31 20:46:30 -0700
* Adding support for "parts": One can split a single test across
multiple files by adding a numerical ``#<n>`` postfix to their
names, where each ``<n>`` represents a separate part of the test.
``btests`` will combine all of a test's parts in numerical order
and execute them subsequently within the same sandbox. Example in
the README. (Robin Sommer)
* When running a command, TEST_PART contains the current part
number. (Robin Sommer)
* Extending Sphinx support. (Robin Sommer)
* Adding tests for Sphinx functionality.
* Support for parts in Sphinx directives. If multiple btest
directives reference the same test name, each will turn into
a part of a single test.
* Internal change restructuring the btest Sphinx directive. We
now process it in two passes: one to save the test at parse
time, and one later to execute once everything has been
parsed.
* Adding Sphinx sandbox for testing.
* Fix for tests returning no output to render at all. (Robin Sommer)
0.4-28 | 2013-07-17 21:56:18 -0700
* btest-diff now passes the name of the file under consideration on to
canonifiers. (Robin Sommer)
0.4-27 | 2013-07-14 21:19:59 -0700
* When searching for tests, BTest now ignores a directories if it finds
a file ".btest-ignore" in there. (Robin Sommer)
0.4-26 | 2013-07-08 20:46:22 -0700
* Fixing bug with @TEST-START-NEXT naming. (Robin Sommer)
0.4-25 | 2013-07-08 13:25:50 -0700
* A test-suite for btest. Using, of course, btest. "make test" will
test most of btest's features. The main missing piece is testing
the Sphinx support, we will add that next. (Robin Sommer)
* When creating directories, we know also create intermediaries.
That in particular means that "@TEST-START-FILE a/b/c" now creates
a directory "a/b" automatically and puts the file in there. (Robin
Sommer)
* IgnoreDirs now also works for sub directories. (Robin Sommer)
* Documentation updates. (Robin Sommer)
* Adding "Initializer" option, which runs a command before each
test. (Robin Sommer)
* Adding "CommandPrefix" option that changes the naming of all btest
commands by replacing the "@TEST-" prefix with a custom string.
(Robin Sommer)
* Default configuration file can be overriden via BTEST_CFG
environment variable. (Robin Sommer)
* s/bro-ids.org/bro.org/g (Robin Sommer)
* Bugfix for -j without number. (Robin Sommer)
* New @TEST-ALTERNATIVE that activates tests only for the given
alternative. Renamed @TEST-NO-ALTERNATIVE to
@TEST-NOT-ALTERNATIVE, and allowing "default" for both
@TEST-ALTERNATIVE and @TEST-NOTALTERNATIVE to specify the case
that BTest runs without any alternative given. (Robin Sommer)
* Fix for alternative names containing white spaces. (Robin Sommer)
0.4-14 | 2013-01-23 18:11:22 -0800
* Fixing links in README and removing TODOs. (Robin Sommer)
0.4-13 | 2013-01-23 14:33:23 -0800
* Allowing use of -j without a value. BTest then uses the number of
CPU cores as reported by the OS. (Robin Sommer)
0.4-11 | 2013-01-21 17:50:40 -0800
* Adding a new "alternative" concept that combines filters and
substitutions, and adds per-alternative environment variables.
(Robin Sommer)
Instead of defining filters and substitutions separately, one now
specifies an alternative configuration to run with "-A <name>" and
that then checks for both "[substitutions-<name>]" and
"[filter-<name>]" section. In addition, "[environment-<name>]"
allows to define alternative-specific environment variables.
The old filter/substitutions options -F and -s are gone. The
sections for substitutions are renamed to "[substitutions-<name>]"
from "[subst-<name>]".
0.4-10 | 2013-01-07 09:45:35 -0800
* btest now sets a new environment variable TEST_VERBOSE, giving the
path of a file where a test can record further information about
its execution that will be included with btest's ``--verbose``
output. (Robin Sommer)
0.4-9 | 2012-12-20 12:20:44 -0800
* Documentation fixes/clarifications. (Daniel Thayer)
* Fix the btest "-c" option, which didn't work when the specified
config file was not in the current working directory. (Daniel
Thayer)
0.4-6 | 2012-11-08 16:33:51 -0800
* Putting a limit on how many input line btest-diff shows. (Robin
Sommer)
0.4-5 | 2012-11-01 16:14:29 -0700
* Making Sphinx module tolerant against docutils version change.
(Robin Sommer)
0.4-4 | 2012-09-25 06:24:59 -0700
* Fix a couple of reST formatting problems. (Daniel Thayer)
0.4-2 | 2012-09-24 11:41:06 -0700
* Add option -x to output test results in an XML (JUnit-like)
format. (Jon Siwek)
0.4 | 2012-06-15 15:15:13 -0700
* Remove code to expand environment variables on command line. (Not
needed because the command line is just passed to the shell.)
(Daniel Thayer)
* Clarify explanation about expansion of environment variables.
(Daniel Thayer)
* Fix errors in README and btest help output; added documentation
for the -q option. (Daniel Thayer)
* Fixed a bug in btest where it was looking for "filters-" (instead
of "filter-") in the btest config file. (Daniel Thayer)
0.31-45 | 2012-05-24 16:43:14 -0700
* Correct typos in documentation. (Daniel Thayer)
* Failed tests are now only recorded into the state file when we're
not updating. That allows to run "btest -r" repeatedly while
updating baselines in between. (Robin Sommer)
* Experimentation Sphinx directive to write a btest with a Sphinx
document. See README for more information.
* Fixing typos, plus an console output tweak. (Robin Sommer)
* Option -q now implies -b as well. (Robin Sommer)
0.31-33 | 2012-05-13 17:08:15 -0700
* New command to copy a file into a test's directory.
``@TEST-COPY-FILE: <file>``
Copy the given file into the test's directory before the test is
run. If ``<file>`` is a relative path, it's interpreted relative
to the BTest's base directory. Environment variables in ``<file>``
will be replaced if enclosed in ``${..}``. This command can be
given multiple times. (Robin Sommer)
* Suppressing error messages when btest-diff can't remove diag file.
(Robin Sommer)
* Adding option -q/--quiet to suppress informational non-error
output. (Robin Sommer)
* Option -F also takes a comma-separated list to specify multiple
filters , rather than having to give -F multiple times. (Robin
Sommer)
0.31-28 | 2012-05-06 21:27:15 -0700
* Separating semantics of groups and thread serialization into
separate options. -g still specifices @TEST-GROUPs that are to be
executed, but these groups don't any longer control which tests
get serialized in a parallel execution. For that, there's a new
"@TEST-SERIALIZE: <tag>" command that takes a tag and then makes
sure all tests with the same tag are run within the same thread.
(Robin Sommer)
* TEST-GROUPS can now be given multiple times now to assign a test
to a set of groups. (Robin Sommer)
* Extended -g to accept a comma-separated list of groups names to
run more than one test group. (Robin Sommer)
* New output handler for console output. This output is now the
default when stdout is a terminal. It prints out a compressed
output that updates as btest goes through; it also indicates the
progress so far. If btest's output is redirected to a
non-terminal, is switches back to the old style. (Robin Sommer)
* New test command @TEST-NO-FILTER: <filter>
This allows to ignore a test when running a specific filter. (Robin Sommer)
* Changing the way filters are activated.
-F <filter> now activates only the given filter, but doesn't run
the standard tests in addition. But one can now give -F a
command-separated list of filters to activate them all, and refer
to the standard tests without filter as ``-``. (Robin Sommer)
* Fix to allow numbered test to be given individually on the command
line. (E.g., integer.geq-3 for a file that contains three tests).
(Robin Sommer)
0.31-23 | 2012-04-16 18:10:02 -0700
* A number of smaller fixes for bugs, plus polishing, caused by the
recent restructuring. (Robin Sommer)
* Removing the error given when using -r with tests on the command
line. It's unnessary and confusing compared to when listing tests
in btest.cfg. (Robin Sommer)
* Adding a new "finalizer" option.
``Finalizer``
An executable that will be executed each time any test has
succesfully run. It runs in the same directory as the test itself
and receives the name of the test as its parameter. The return
value indicates whether the test should indeed be considered
succeeded. By default, there's no finalizer set. (Robin Sommer)
* btest is now again overwriting old diag files instead of appending
(i.e., back to as it used to be). (Robin Sommer)
* Diag output is now line-buffered. (Daniel Thayer)
0.31-13 | 2012-03-13 15:59:51 -0700
* Adding new option -r that reruns all tests that failed last time.
btest now always records all failed tests in a file called. (Robin
Sommer)
* Internal restructuring to factor output out into sublcasses.
(Robin Sommer)
* Adding parallel test execution to btest. (Robin Sommer)
- A new option "-j <n>" allows to run up to <n> tests in
parallel.
- A new @TEST-GROUP directive allows to group tests that can't
be parallelized. All tests of the same group will be
executed sequentially.
- A new option "-g <group>" allows to run only tests of a
certain group, or with "-g -" all tests that don't have a
group.
0.31-2 | 2012-01-25 16:58:29 -0800
* Don't add btest's path to PATH anymore. (Jon Siwek)
0.31 | 2011-11-29 12:11:49 -0600
* Submodule README conformity changes. (Jon Siwek)
0.3 | 2011-10-25 19:58:26 -0700
* More graceful error handling at startup if btest.cfg not found.
(Robin Sommer)
* Python 2.4 compat changes. (Jon Siwek)
* When in brief mode, btest-diff now shows full output if we don't
have a baseline yet. (Robin Sommer)
* Adding executable permission back to script. (Robin Sommer)
* Cleaning up distribution. (Robin Sommer)
0.22-28 | 2011-09-15 15:18:11 -0700
* New environment variable TEST_DIFF_BRIEF. If set btest-diff no
longer includes a mismatching file's full content it the
diagnostic output. This can be useful if the file being compared
is very large. (Robin Sommer)
0.22-27 | 2011-08-12 22:56:12 -0700
* Fix btest-bg-wait's kill trap and -k option. (Jon Siwek)
0.22-18 | 2011-07-23 11:54:07 -0700
* A new option -u for interactively updating baselines.
* Teach btest's TEST-START-FILE to make subdirectories (Jon Siwek)
* Output polishing. (Robin Sommer)
* Have distutils install 'btest-setsid' script. (Jon Siwek)
* A portable setsid. (Robin Sommer)
* Fixes for background execution of processes.
* Fixing exit codes. (Robin Sommer)
0.22-6 | 2011-07-19 17:38:03 -0700
* Teach btest's TEST-START-FILE to make subdirectories (Jon Siwek)
0.22-5 | 2011-05-02 08:41:34 -0700
* A number of bug fixes, and output polishing. (Robin Sommer)
* More robust background execution by btest-bg-*. (Robin Sommer)
0.22-4 | 2011-03-29 21:38:13 -0700
* A test command can now signal to btest that even if it fails
subsequent test commands should still run by returning exit code 100.
btest-diff uses this to continue in the case that no baseline has
yet been established.
* New test option @TEST-REQUIRES for running a test conditionally.
See the README for more information.
0.22-2 | 2011-03-03 21:44:18 -0800
* Two new helper scripts for spawning processes in the background.
See README for more information.
* btest-diff can now deal with files specificied with paths.
0.22 | 2011-02-08 14:06:13 -0800
* BTest is now hosted along with the other Bro repositories on
git.bro-ids.org.
0.21 | 2011-01-09 21:29:18 -0800
* In btest.cfg, option values can now include commands to execute in
backticks.
Example:
[environment]
CC=clang -emit-llvm -g `hilti-config --cflags`
* Limiting substitutions to replacing whole words.
* Adding "substitutions". Substitutions are similar to filters, yet
they do not adapt the input but the command line being exectued.
See README for more information.
* Instead of giving a test's file name on the command line, one can
now also use its "dotted" name as it's printed out when btest is
running (e.g., "foo.bar"). That allows for easier copy/paste.
* Starting CHANGES.
PySubnetTree
0.27 | 2017-05-26 08:28:10 -0500
* Release 0.27.
0.26-2 | 2016-12-06 12:33:17 -0800
* Fix compiler warnings on OpenBSD, add missing include. (Daniel Thayer)
0.26 | 2016-10-27 14:43:48 -0700
* Release 0.26.
0.25-4 | 2016-10-06 09:08:48 -0700
* Fix the prefixes() function to compile and work on Python 3.
(Daniel Thayer)
* Cleanup tests for the prefixes() function and improved comments
and error messages. (Daniel Thayer)
0.25 | 2016-08-12 13:19:45 -0700
* Release 0.25.
0.24-7 | 2016-01-25 14:22:14 -0800
* Added prefixes() method to return all prefixes in the tree as a
set of strings, with or without length. Also supports returning
IPv4 addresses in their "native" format. (James Royalty)
0.24 | 2015-05-07 20:24:57 -0700
* Release 0.24.
* Update dist Makefile target (Johanna Amann)
0.23-23 | 2015-03-23 10:37:20 -0500
* Update try..except syntax in one example in the README.
(Daniel Thayer)
* BIT-1303: Reorganize tests to use btest and add more test cases.
(Daniel Thayer)
0.23-19 | 2015-03-23 09:36:00 -0500
* Add IPv6 example in the docs (David Salisbury)
0.23-18 | 2015-03-17 09:27:14 -0700
* Python 3 compatibility fixes. (Jon Siwek)
0.23-12 | 2014-12-12 10:44:38 -0800
* Use IPv6 as canonical storage format (IPv4 -> IPv4-mapped IPv6).
Addresses GitHub issues #4 and maybe #5. (Jon Siwek)
* Update MANIFEST.in. Include SubnetTree.i in the distribution so
the swig file (SubnetTree.cc) can be regenerated with the tarball
if needed. . (Scott Kitterman)
0.23 | 2014-04-03 15:53:51 -0700
* Release 0.23.
0.23 | 2014-03-31 18:05:31 -0700
* Updated setup.py to work with setuptools. Uploaded to PyPI. (Henry
Stern/Robin Sommer)
0.22 | 2013-10-14 09:47:15 -0700
* Release.
0.21 | 2013-10-14 09:46:34 -0700
* Fixing version number in setup.cfg. Addresses BIT-1088. (Robin
Sommer)
* Remove dead code. (Jon Siwek)
* Fix allocator/deallocator mismatch. (Jon Siwek)
0.20 | 2013-09-23 11:53:17 -0700
* Fix an error in README and improve the examples (Daniel Thayer)
* Fix a broken link in the documentation. (Daniel Thayer)
0.19-9 | 2013-03-08 09:19:54 -0800
* Fix a compiler warning. (John Siwek)
* s/bro-ids.org/bro.org/g. (Robin Sommer)
0.19-3 | 2012-09-29 14:10:39 -0700
* Fix compile error with Python C API. Addresses #887. (Matthias
Vallentin)
0.19-1 | 2012-09-24 16:11:13 -0700
* Fixing memory leak. When deleting a PySubnetTree, the values
weren't unref'ed. (Simon Arlott)
0.19 | 2012-08-01 13:57:31 -0500
* Fix configure script to exit with non-zero status on error (Jon Siwek)
0.18 | 2012-07-05 12:33:40 -0700
* Improve check for SWIG/Python version incompatibility. Addresses
#843. (Jon Siwek)
0.17-16 | 2012-07-02 14:53:05 -0700
* Cleanup and update of SubnetTree_wrap.cc file. (Daniel Thayer)
* Fix compile warnings and dependencies of swig-generated files. (Jon Siwek)
* Fix typos. (Daniel Thayer)
0.17-8 | 2012-04-09 15:36:47 -0700
* pysubnettree now supports IPv6 addresses and prefixes. (Henry
Stern, updated by Daniel Thayer).
* SubnetTree now have a binary mode as well in in which single
addresses are passed in the form of packed binary strings as,
e.g., returned by `socket.inet_aton. (Henry Stern, updated by
Daniel Thayer).
* Raise minimum required CMake version to 2.6.3 (Jon Siwek)
0.17 | 2012-01-09 16:11:02 -0800
* Submodule README conformity changes. (Jon Siwek)
* Simplify finding of Python headers/libraries. Addresses #666. (Jon
Siwek).
0.16-3 | 2011-11-03 15:17:21 -0700
* Fixing compiler warnings. Addresses #388. (Jon Siwek)
0.16 | 2011-10-26 13:50:28 -0700
* Compile SWIG bindings with no-strict-aliasing (closes #644) (Jon Siwek)
0.15 | 2011-10-25 20:15:08 -0700
* New make dist/distclean targets. (Jon Siwek)
* Cleaning up the distribution. (Jon Siwek and Robin Sommer)
0.14-8 | 2011-09-04 09:19:08 -0700
* Install binaries with an RPATH. (Jon Siwek)
* Add check for incompatible swig+python versions. Addresses #562.
(Jon Siwek)
* Workaround for FreeBSD CMake port missing debug flags. (Jon Siwek)
0.14 | 2011-05-05 20:59:58 -0700
* CMake build setup. (Jon Siwek)
* Cleanup, and converting README to REST. (Robin Sommer)
* Initial import to git. (Robin Sommer)
Copyright 2016, The Bro Project.
Last updated on December 07, 2018.
Created using Sphinx 1.8.2.