Searching¶

Syntax¶

The search syntax is very close to the Lucene syntax. By default all message fields are included in the search if you don’t specify a message field to search in.

Messages that include the term ssh:

ssh

Messages that include the term ssh or login:

ssh login

Messages that include the exact phrase ssh login:

"ssh login"

Messages where the field type includes ssh:

type:ssh

Messages where the field type includes ssh or login:

type:(ssh login)

Messages where the field type includes the exact phrase ssh login:

type:"ssh login"

Messages that have the field type:

_exists_:type

Messages that do not have the field type:

NOT _exists_:type

Messages that match regular expression ethernet[0-9]+:

/ethernet[0-9]+/

By default all terms or phrases are OR connected so all messages that have at least one hit are returned. You can use Boolean operators and groups for control over this:

"ssh login" AND source:example.org
("ssh login" AND (source:example.org OR source:another.example.org)) OR _exists_:always_find_me

You can also use the NOT operator:

"ssh login" AND NOT source:example.org
NOT example.org

Note that AND, OR, and NOT are case sensitive and must be typed in all upper-case.

Wildcards: Use ? to replace a single character or * to replace zero or more characters:

source:*.org
source:exam?le.org
source:exam?le.*

Note that leading wildcards are disabled to avoid excessive memory consumption! You can enable them in your Graylog configuration file:

allow_leading_wildcard_searches = true

Also note that message, full_message, and source are the only fields that are being analyzed by default. While wildcard searches (using * and ?) work on all indexed fields, analyzed fields will behave a little bit different. See wildcard and regexp queries for details.

Fuzziness: You can search for similar terms:

ssh logni~
source:exmaple.org~

This example is using the Damerau–Levenshtein distance with a default distance of 2 and will match “ssh login” and “example.org” (intentionally misspelled in the query).

You can change the distance like this:

source:exmaple.org~1

You can also use the fuzzyness operator to do a proximity search where the terms in a phrase can have different/fuzzy distances from each other and don’t have to be in the defined order:

"foo bar"~5

Numeric fields support range queries. Ranges in square brackets are inclusive, curly brackets are exclusive and can even be combined:

http_response_code:[500 TO 504]
http_response_code:{400 TO 404}
bytes:{0 TO 64]
http_response_code:[0 TO 64}

You can also do searches with one side unbounded:

http_response_code:>400
http_response_code:<400
http_response_code:>=400
http_response_code:<=400

It is also possible to combine unbounded range operators:

http_response_code:(>=400 AND <500)

Escaping¶

The following characters must be escaped with a backslash:

&& || : \ / + - ! ( ) { } [ ] ^ " ~ * ?

Example:

resource:\/posts\/45326

Relative time frame selector¶

The relative time frame selector lets you look for messages from the selected option to the time you hit the search button. The selector offers a wide set of relative time frames that fit most of your search needs.

Absolute time frame selector¶

When you know exactly the boundaries of your search, you want to use the absolute time frame selector. Simply introduce the dates and times for the search manually or click in the input field to open up a calendar where you can choose the day with your mouse.

Keyword time frame selector¶

Graylog offers a keyword time frame selector that allows you to specify the time frame for the search in natural language like last hour or last 90 days. The web interface shows a preview of the two actual timestamps that will be used for the search.

Here are a few examples for possible values.

• “last month” searches between one month ago and now
• “4 hours ago” searches between four hours ago and now
• “1st of april to 2 days ago” searches between 1st of April and 2 days ago
• “yesterday midnight +0200 to today midnight +0200” searches between yesterday midnight and today midnight in timezone +0200 - will be 22:00 in UTC

The time frame is parsed using the natty natural language parser. Please consult its documentation for details.

Field statistics¶

Compute different statistics on your fields, to help you better summarize and understand the data in them.

The statistical information consist of: total, mean, minimum, maximum, standard deviation, variance, sum, and cardinality. On non-numeric fields, you can only see the total amount of messages containing that field, and the cardinality of the field, i.e. the number of unique values it has.

Quick values¶

Quick values helps you to find out the distribution of values for a field. Alongside a graphic representation of the common values contained in a field, Graylog will display a table with all different values, allowing you to see the number of times they appear. You can include any value in your search query by clicking on the magnifying glass icon located in the value row.

Field graphs¶

You can create field graphs for any numeric field, by clicking on the Generate chart button in the search sidebar. Using the options in the Customize menu on top of the field graph, you can change the statistical function used in the graph, the kind of graph to use to represent the values, the graph interpolation, as well as the time resolution.

Once you have customized some field graphs, you can also combine them by dragging them from the hamburger icon on the top corner of the graph, and dropping them into another field graph. You can see the location of the hamburger icon and the end result in the following screenshots:

Field graphs appear every time you perform a search, allowing you to compare data, or combine graphs coming from different streams.

Syslog severity mapper¶

The syslog severity mapper decorator lets you convert the numeric syslog level of syslog messages, to a human readable string. For example, applying the decorator to the level field in your logs would convert the syslog level 4 to Warning (4).

To apply a syslog severity mapper decorator, you need to provide the following data:

• Source field: Field containing the numeric syslog level
• Target field: Field to store the human readable string. It can be the same one as the source field, if you wish to replace the numeric value on your search results

Format string¶

The format string decorator provides a simple way of combining several fields into one. It can also be used to modify the content of a field in, without altering the stored result in Elasticsearch.

To apply a format string decorator you need to provide the following data:

• Format string: Pattern used to format the resulting string. You can provide fields in the message by enclosing them in ${}. E.g. ${source} will add the contents of the source message field into the resulting string
• Target field: Field to store the resulting value
• Require all fields (optional): Check this box to only format the string when all other fields are present

For example, using the format string Request to ${controller}#${action} finished in ${took_ms}ms with code ${http_response_code}, could produce the text Request to PostsController#show finished in 57ms with code 200, and make it visible in one of the message fields in your search results.

Pipeline Decorator¶

The pipeline decorator provides a way to decorate messages by processing them with an existing processing pipeline. In contrast to using a processing pipeline, changes done to the message by the pipeline are not persisted. Instead, the pipeline is used at search time to modify the presentation of the message.

The prerequisite of using the pipeline decorator is that an existing pipeline is required.

When you are done creating a pipeline, you can now add a decorator using it on any number of streams. In order to create one, you proceed just like for any other decorator type, by clicking on the Decorator sidebar, selecting the type (“Pipeline Processor Decorator” in this case) and clicking the Apply button next to one.

Upon clicking Apply, the pipeline to be used for decorating can be selected.

After selecting a pipeline and clicking Save, you are already set creating a new pipeline decorator.

Debugging decorators¶

When a message is not decorated as expected, or you need to know how it looked like originally, you can see all changes that were done during decoration by clicking “Show changes” in the message details.

In this view, deleted content is shown in red, while added content is shown in green. This means that added fields will have a single green entry, removed fields a single red entry and modified fields will have two entries, a red and a green one.

Further functionality¶

If the existing decorators are not sufficient for your needs, you can either search the Graylog marketplace, or write your own decorator.

Enabling/Disabling search result highlighting¶

Using search result highlighting will result in slightly higher resource consumption of searches. You can enable and disable it using a configuration parameter in the graylog.conf of your Graylog nodes:

allow_highlighting = true

Query time range limit¶

Sometimes the amount of data stored in Graylog is quite big and spans a wide time range (e. g. multiple years). In order to prevent normal users from accidentally running search queries which could use up lots of resources, it is possible to limit the time range that users are allowed to search in.

Using this feature, the time range of a search query exceeding the configured query time range limit will automatically be adapted to the given limit.

The query time range limit is a duration formatted according to ISO 8601 following the basic format P<date>T<time> with the following rules:

Examples:

More details about the format of ISO 8601 durations can be found on Wikipedia.

Relative time ranges¶

The list of time ranges displayed in the Relative time frame selector can be configured, too. It consists of a list of ISO 8601 durations which the users can select on the search page.