base/utils/urls.bro

Functions for URL handling.

Source File:/scripts/base/utils/urls.bro

Summary

Options

url_regex: pattern &redef

A regular expression for matching and extracting URLs.

Types

URI: record

A URI, as parsed by decompose_uri.

Functions

decompose_uri: function
find_all_urls: function	Extracts URLs discovered in arbitrary text.
find_all_urls_without_scheme: function	Extracts URLs discovered in arbitrary text without the URL scheme included.

Detailed Interface

Options

url_regex

Type:pattern Attributes:&redef Default:

/^?(^([a-zA-Z\-]{3,5})(:\/\/[^\/?#"'\r\n><]*)([^?#"'\r\n><]*)([^[:blank:]\r\n"'><]*|\??[^"'\r\n><]*))$?/

A regular expression for matching and extracting URLs.

Types

URI

Type:

record

scheme: string &optional

The URL’s scheme..

netlocation: string

The location, which could be a domain name or an IP address. Left empty if not specified.

portnum: count &optional

Port number, if included in URI.

path: string

Full including the file name. Will be ‘/’ if there’s not path given.

file_name: string &optional

Full file name, including extension, if there is a file name.

file_base: string &optional

The base filename, without extension, if there is a file name.

file_ext: string &optional

The filename’s extension, if there is a file name.

params: table [string] of string &optional

A table of all query parameters, mapping their keys to values, if there’s a query.

A URI, as parsed by decompose_uri.

Functions

decompose_uri

Type:function (uri: string) : URI

find_all_urls

Type:function (s: string) : string_set

Extracts URLs discovered in arbitrary text.

find_all_urls_without_scheme

Type:function (s: string) : string_set

Extracts URLs discovered in arbitrary text without the URL scheme included.