Packet Acquisition and Control (PACF)

The PACF provides a transparent interface for dynamically changing both the input traffic that Zeek sees (packet acquisition) and the traffic that the network forwards (packet control). Scripts can use functions provided by the framework when they want to, e.g., block a host or start sampling a flow’s traffic. The framework than triggers that action in whatever way the user’s environment supports.

The user configures that support at startup time by registering instances of equipment-specific plugins that implement the corresponding functionality. That could be, e.g., an OpenFlow plugin that knows how to talk to the forwarding router; a load-balancer plugin that adapts the hardware frontend feeding traffic to a cluster; or even a BPF plugin that implements actions by adapting Zeek’s BPF filter. A user can instantiate multiple plugins simultaneously; for every requested action, they’ll be asked successively if they can carry out the desired operation; the first which does, gets to execute it. That gives us the flexibility to push individual actions out into the network as far as possible, while potentially falling back to less efficient schemes where no other support is available.

(There’s an older version of this proposal here.)

Zeek User API

The following types and functions options are exposed to other scripts to transparently manipulate packet acquisition and control:

module PACF;

# Type of Entity for defining an action.
type EntityType: enum {
    ADDRESS,                 # Activity involving a specific IP address.
    ORIGINATOR,              # Activity *from* a source IP address.
    RESPONDER,               # Activity *to* a destination IP address.
    CONNECTION,              # A (bi-directional) connection.
    FLOW,                    # A (uni-directional) flow.
    MAC,                     # A MAC address.
};

# An entity instance for defining an action.
type Entity: record {
    ty: EntityType:
    conn: conn_id &optional; # Used with CONNECTION.
    flow: flow_id &optional; # Used with FLOW.
    ip: subnet &optional;    # Used with ADDRESS; can specifiy a CIDR subnet.
    mac: string &options;    # Used with MAC.
};

# Types of targets to apply a rule to.
type TargetType: enum {
    MONITOR,    # Apply rule passively to traffic sent to Zeek for monitoring.
    FORWARD,    # Apply rule actively to traffic on forward path.
};

# Type of rules the framework supports.
#
# PLugins can add their own if they insist.
type RuleType: enum {
    DROP,       # Stop forwarding packets matching entity.
    SAMPLE,     # Begin sampling packets matching entity.
    REDIRED,    # Begin redirecting packets matching entity.
    LIMIT,      # Begin rate-limiting packets matching entity.
    MODIFY,     # Begin modifying packets matching entity.
    NOTIFY,     # Notify when entity matches condition.
};

# A rule that the framework will put in place.
type Rule: record {
    ty: RuleType;               # Type of rule.
    target: TargetType;         # Where to apply rule.
    entity: Entity;             # Entity to apply rule to.
    timeout: interval;          # Timeout after which to expire the rule.
    priority: int &default=0;   # Priority if multiple rules match an entity (larger value is higher priority).
    tag: string &optional;      # Optional custom tag for logging purposes.

    id: string &default="";     # Internal ID for this rule. Will  be set when added.

    # Rule-specific arguments, filled out as rule requires.
    # TODO: Needs to be fleshed out more in terms of what rule
    # types will need, and how to standarize the argumetns among
    # them.
    arg_int: int;
    arg_str: string;
    [...]
};

# Installs a rule. If succesful, returns an ID string unique to the
# rule that can later be used to refer to it. If unsuccessful,
# returns an empty string. The ID is also assigned to r$id.
#
# Note that "successful" means "a plugin knew how to handle the rule",
# it doesn't necessarily mean that it was indeed successfully put in place,
# because that might happen asynchronously and thus fail only later.
function add(r: Rule) : string;

# Removes a rule referenced by its ID, as returned by add(). Returns
# true if the relevant plugin indicated that ity knew how to
# handle the removal
#
# Note that again "success" means the plugin accepted the removal. They might still
# fail to put it into effect, as that  might happen asynchronously and thus go wrong
# at that point.
function remove(tag: string) : bool;

# Remove all rules for an entity.
global reset(e: Entity);

# Flush all state.
global clear();

### Asynchronous feedback.

# Confirms that a rule was put in place.
event confirmed(r: Rule, plugin: string, msg: string);

# Reports that a rule was removed due to a remove() call.
event removed(r: Rule, plugin: string, msg: string);

# Reports that a rule was removed internally due to a timeout.
event timeout(r: Rule, plugin: string);

# Reports that a rule was removed externally.
event removed_externally(r: Rule, plugin: string, msg: string);

# Reports that a rule was added externally.
event added_externally(r: Rule, plugin: string, msg: string);

# Reports an error when operating a on a rule.
event error(r: Rule, plugin: string, msg: string);

Plugin API

A plugin is defined by specifying a subset of functions implementing any of the actions that the public API offers:

module PACF:

# Definitioan of a plugin. A plugin implements just the functions it supports.
# Each function returns true if it could execute the corresponding action (and
# hence no other plugin will be asked to do so); and false if it couldn't
# support it (and hence we'll check further plugins down the chain). Not
# defining a function is equivalent to always returning false.
type Plugin: record {
    # Returns a descriptive name of the plugin instance, suitable for use in
    # logging messages. Note that this function is not optional.
    name: function() : string;

    # One-time initialization functionl called  when plugin gets registered, and
    # befire any ther methods are called.
    init: function() &optional;

    # One-time finalization function called when a plugin is shutdown; no
    # further functions will be called afterwords.
    done: function() &optional;

    # Implements the add() operation. If the plugin accepts the
    # operation, it returns an ID unique to the command that can
    # later be used to refer to it. In that case, it must
    # flag success/failure through the corresponding reporting
    # events. If the plugin doesn't support the operation, it must
    # return an empty string.
    function add(r: Rule) : string;

    # Implements the remove() operation. Success/failure must be
    # reported through the corresponding reporting events().
    function remove(tag: string) : bool;

    # A transaction groups a number of operations. The plugin can
    # add them internally and postpone putting them into effect
    # until committed. This allows to build a configuration of multiple
    # rules at once, including replaying  a previous state.
    transaction_begin: function() &optional;
    transaction_end: function() &optional;

    # Returns a list of all current rules. This can include rules
    # put in place externally (or in previous run) and then mapped to Zeek's command
    # structure. It's basically a dump of the plugin is currently
    # configured to do.
    #
    # Ideally this wouldn't be optional, but it might be difficult
    # to implement for some plugins, so we'll probably need to
    # leave it informational-only.
    rules: function(): vector[Rule] &potional;

    # Table for the plugin to store instance-specific configuration information.
    config: table[string] of string;
}

# Register a plugin. The higher the priority, the earlier it will be checked
# whether it supports an operation, relative to other plugins.
global register(p: Plugin, priority: int);

Example

Say we have plugins for OpenFlow and BPF. Both can control the input Zeek sees; the former by talking to a frontend switch, the latter by adding BPF filters on the fly. They are defined like this:

module PACF;

global OpenFlow: Plugin = [
    name = [....]
    add = ...
    remove = ...
    ...
    };

global BPF: Plugin = [
    name = [....]
    add = ...
    remove = ...
    ...
    };

Both plugins also provide instantiation functions:

PACF::openflow_create(controller: addr);
PACF::bpf_create(iface: string);

These create an instance of the corresponding plugin class and use the configuration table to store their arguments and whatever else they need.

Now say in our network we want to use OpenFlow whenever it support an operation, and fall-back to BPF otherwise:

event bro_init()
     {
     local openflow = PACF::openflow_create(192.168.1.1);
     local bpf = PACF::bpf_create("eth1);

     PACF::register(openflow, 2); # High priority.
     PACF::register(bpf, 1);      # Low priority.
     }

Now other scripts can adjust what Zeek sees:

# Stop analyzing 10.0.0.1 for the next 5 minutes.
local e = Entity($ty=PACF::ADDRESS, $ip=10.0.0.1);
PACF::add([ty=PACF:DROP, target=PACF:MONITOR, entity=e, $timeout=5min]);

Notes

• It still remains unclear to me what the best strategy is for managing the rules across the restarts. I see two options:

  • Zeek keeps the authorative version of all rules, and saves them to disk persistently. At startup, it replays it back to the plugins (which would work even if plugins have been reconfigured in the meantime).
  • The plugins keep the authoritative version and decide themselves on startup/shutdown what to do. Zeek doesn’t really care.

  While the 1st option would be nice conceptually (as it unifies this across all plugins), it the 2nd is probably more realisitc. The 2nd also plays more nicely with rules put in place externally.

  However, assuming we keep the transaction support in there, Bro can generally replay configurations if it wants to.

• Not quite sure what should generally happen at Zeek termination with the current control rules. Leave it place? Remove? Provide an option?

• We’ll probably need a more sophisticated structure eventally for the plugin instances than a sorted list. We could do a tree where an operation is always passed on to all childs. For example, we could have multiple BPF plugin instances, one for each load-balanced interface; they’d all get a drop operation so that all can suppress the corresponding traffic. We’d then create "meta IDs" that internally map to list of individual plugin IDs. But this is all something we can leave to version 2.

• Can we get control for load-balancing into the API? Or is that something better handled seperately? My hunch right now is the latter, however a plugin here might end up talking to the same subsystem as is controlling the load-balancing.