Raw Log Search query syntax

Raw Log Search is a feature of Data Explorer that lets you search through raw, unparsed logs forwarded from your on-premises systems and cloud applications. How far back you can search depends on your data retention policy. You can use Raw Log Search for operational and security-related tasks like validating a configuration change or investigating a security alert.

Tip: You can also search the Arctic Wolf observation pipeline for parsed and analyzed event logs. See Data Explorer for more information.

Search expression syntax

A raw log search can be as simple as entering a single word or as complex as a Boolean expression with multiple operators.

Tip:

Not entering a search term returns all logs in a specified time range.

A complex search expression uses conditional logic to limit the number of matches. This logic is expressed as a combination of search terms and one or more operators using this general syntax:

BASH 
<search_term><space><operator><space><search_term>

Where:

  • search_term is a string value. For example, a word or number sequence. It can include whitespaces, quotation marks, or special characters.
  • space is the 0x20 character.
  • operator is the AND, OR, or NOT Boolean operator.

Parentheses, (), are also operators. Parentheses allow you to group Boolean expressions and specify the order of operations.

For more information about writing search expressions, see:

Tip:
  • To write a complex search expression, you can use a query template that is similar to the search you want to run, and then modify the search expression. See Run a search for more information.
  • If you are unsure about how to structure a complex search expression, you can start with a broad search, and then review the results to decide how best to refine your expression.

Exact string matching

Raw Log Search uses exact string matching. This means that if a search term includes whitespaces, quotation marks, or special characters, Raw Log Search considers those characters to be part of the same string.

Note:
  • Wildcard characters are not supported.
  • Raw logs are not parsed. Therefore, Raw Log Search is unable to determine if a string match in log data represents a field name or field value and will return all matches.

Search expressions use exact matching, so you can modify a search term to make your search broad or narrow. For example, if you want to find Cisco ASA logs with message ID 201009, the search expression 201009 results in a search that is too broad. Logs that match this search expression would include all logs that contain the substring 201009, even if 201009 is not part of an message ID. For example:

  • MSWinEventLog ... "EventID":4625 ... "RecordNumber":201009 ...
  • MSWinEventLog 1 Microsoft-Windows-GroupPolicy/Operational 1201009 ...
  • "@type":"awn-agent" ... "systemTime\":\"2024-02-05T23:03:50.2010099Z\" ...

A more refined search expression is %ASA-201009.

Operators

These are the available search operators and their uses:

Note:
  • You must include a single space character on both sides of the AND, OR, and NOT operators, and you must write these operators in all capital letters.
  • A search expression can have up to 10 operators.

Operator

Description

AND

Separates individual search terms, requiring that both are present in any order in the log line.

OR

Separates individual search terms, and returns results where either term is present anywhere in the log line.

NOT

Excludes searches from the results that contain the term following NOT in the search expression.

( )

Controls the order of operations when you use multiple operators in your search expression. Without parentheses, the default precedence is NOT, AND, then OR. To search for parentheses contained within log data, you must use the backslash \ character to escape the parentheses operation in your search expression.

For example, NOT aa OR bb AND cc is a valid search expression to find records that do not contain aa, or contain both bb and cc in any order. NOT aa OR bb AND cc is equivalent to (NOT aa) OR (bb AND cc) and would match these log lines:

  • logID 0001: bb cc ii
  • logID 0027: aa bb cc
  • logID 0378: zz yy hh
  • logID 0690: bb hh vv
  • logID 1822: gg ii cc

Case sensitivity

Search expressions are case-sensitive by default. You can deselect the Case sensitive option to ignore capitalization.

Character limits

  • A search expression must contain at least 2 non-whitespace characters. Search expressions with only one non-whitespace character, for example a or b, are invalid.
  • A search expression must not contain more than 1000 characters including whitespace characters. Search expressions that exceed this length are invalid. Note that UTF-8 characters, for example, a mathematical symbol, might count as multiple characters.

Wildcards

Wildcard characters, for example * or ?, are not supported. All search expressions are substring matched, so the * wildcard character is implied on both sides of the search word. For example, entering error is equivalent to entering *error*.

Reserved characters

Reserved characters are characters that you cannot use in search expressions. Search expressions that contain any of these characters are invalid:

  • The line feed or new line character, 0x0A, often represented as \n.
  • The carriage return character, 0x0D, often represented as \r.
  • The vertical tab character, 0x0B, often represented as \v.
  • The form feed character, 0x0C, often represented as \f.
  • The shift out and shift in characters, 0x0E and 0x0F.
Note:

This list of reserved characters is subject to change based on system requirements or technical restrictions. Changes are usually applied for non-printable characters in the low-ASCII range, 0x00 to 0x1F, or the high UTF-8 range.

Tab characters

For logs that include tab-separated values, you can add horizontal tab characters to your search term to limit the number of matches.

In Windows event logs, event IDs are tab-separated. To search for a Windows event ID, in your search term, insert one horizontal tab character, 0x09, before and after the event ID. This prevents the search expression from matching other log data that might contain the same series of numbers, like timestamps or UUIDs.

To insert a horizontal tab character, on the search bar, click Indent.

For example, to search for all failed and successful login attempts for user jsmith, use this search expression:

BASH 
MSWinEventLog AND (   4624   OR   4625   ) AND jsmith

Where:

  • MSWinEventLog limits the search to Windows event logs.
  • jsmith is the username.
  • 4625 is the event ID for a failed login attempt.
  • 4624 is the event ID for a successful login attempt.
Note:

In this example, there is a horizontal tab character before and after each event code and a single space before and after the OR operator.

The order of search terms and whitespace characters is:

BASH 
MSWinEventLog<space>AND<space>jsmith<space>AND<space>(<tab>4624<tab><space>OR<space><tab>4625<tab>)

Where:

  • space is the 0x20 character.
  • tab is the 0x09 character.

Examples of log search expressions

These examples illustrate how you can modify your search expression syntax to refine your search:

Search expression

Anticipated search result

Match

Description

error code: 400

Authorization failed with error code: 400

Yes

There is an exact match for the search string in the log line.

error code: 400

Aborted operation returned error code: 4009217

Yes

Although extra characters follow the search term, there is an exact substring match.

error code: 400

HTTP request failure error code: 400

No

The character following the colon is a tab character ( 0x09) and not a space ( 0x20) in the raw log line, and therefore does not match the search expression.

error code: 400

HTTP request failure error code: 400

No

There are two spaces, instead of a single space, following the colon in the raw log line, so there is not an exact match to the search expression.

error code: 400

Device code error: 400

No

All words are present in the log line, but the order of the words do not match the search expression.

jsmith AND (failed login)

ERROR:: failed login for user: jsmith

Yes

The search expression matches logs that contain jsmith and the exact phrase failed login, including whitespace.

jsmith AND (fail OR denied)

ERROR:: failed login for user: jsmith

Yes

The first search term and only one of the OR terms appear in the search results.

Note:

The word fail is a substring of failed since wildcards on both sides of a search string are implied.

jsmith AND (fail OR denied)

Permission denied to jsmith attempting deletion of object 0x943150897

Yes

The first search term and one of the OR terms appear in the search results.

jsmith AND (fail OR denied)

Login denied: user 'ajsmith' failed verification question

Yes

All the search terms appear in the log line.

HTTP (404)

HTTP (404) POST /widgets/missing

No

The syntax is invalid because the parentheses function as operators in this search expression. This syntax returns an error similar to Query contains syntax error(s): 'HTTP (404)' at position: 6.

Tip:

Add backslash characters to render the parentheses in this search expression as normal characters in the search string.

HTTP \(404\)

HTTP (404) POST /widgets/missing

Yes

The search string appears in the log line.

"failed login"

ERROR:: "failed login"

Yes

The search expression only matches logs that contain the exact terms, including whitespace and quotes.

Get help

If you have questions about Raw Log Search, contact your Concierge Security® Team (CST) at security@arcticwolf.com.