enrich

Allows using CSV lookup to enrich data

Example: Add known network ports to events

file: lookup.csv

port,service
22,ssh
80,http
443,https

input:

{"port":22}
{"port":80}
{"port":100}
{"port":443}

action:

enrich:
  - lookup-file: lookup.csv
    match:
      - type: num
        event-field: port
        lookup-field: port
    add:
      event-field: service
      lookup-field: service

output:

{"port":22,"service":"ssh"}
{"port":80,"service":"http"}
{"port":100}
{"port":443,"service":"https"}

Field NameDescriptionTypeDefault
lookup-fileA file containing lookupsstring-
matchDetail on what to match on, associating event fields and lookup fields--
addDetail on what to add to the event, based on the match.
If there is no default value, then the output field will not be added to the event.
--

lookup-file

A file containing lookups

Type: string

match

Detail on what to match on, associating event fields and lookup fields

Field NameDescriptionTypeDefault
typeType of match, one of the following: str, num, cidr, ip, num-range, num-list, str-liststring-
event-fieldEvent field matchedfield-
lookup-fieldLookup field matchedfield-

type

Type of match, one of the following: str, num, cidr, ip, num-range, num-list, str-list

Type: string

Example: matching strings

file: lookup1.csv

port,service
22,ssh
80,http
443,https

input:

{"service":"ssh"}
{"service":"http"}
{"service":"unknown"}
{"service":"https"}

action:

enrich:
  - lookup-file: lookup1.csv
    match:
      - type: str
        event-field: service
        lookup-field: service
    add:
      event-field: port
      lookup-field: port

output:

{"service":"ssh","port":"22"}
{"service":"http","port":"80"}
{"service":"unknown"}
{"service":"https","port":"443"}

Example: matching CIDR

file: lookup2.csv

office,network
Splitpoint,192.168.26.0/28
Panoptix,192.168.85.0/24

input:

{"address":"192.168.26.10"}
{"address":"192.168.26.100"}
{"address":"192.168.85.100"}

action:

enrich:
  - lookup-file: lookup2.csv
    match:
      - type: cidr
        event-field: address
        lookup-field: network
    add:
      event-field: office
      lookup-field: office

output:

{"address":"192.168.26.10","office":"Splitpoint"}
{"address":"192.168.26.100"}
{"address":"192.168.85.100","office":"Panoptix"}

Example: IP address (if event match.event-field does not match ip address format, the event will be filtered out)

file: lookup3.csv

fqdn,address
panoptix.io,206.189.28.194

input:

{"address":"192.168.26.10"}
{"address":"206.189.28.194"}

action:

enrich:
  - lookup-file: lookup3.csv
    match:
      - type: ip
        event-field: address
        lookup-field: address
    add:
      event-field: fqdn
      lookup-field: fqdn

output:

{"address":"192.168.26.10"}
{"address":"206.189.28.194","fqdn":"panoptix.io"}

Example: Type is a range of numbers

file: lookup4.csv

range,grouping
0-3,small numbers
3-10,larger numbers

input:

{"number":1}
{"number":10}
{"number":100}

action:

enrich:
  - lookup-file: lookup4.csv
    match:
      - type: num-range
        event-field: number
        lookup-field: range
    add:
      event-field: grouping
      lookup-field: grouping

output:

{"number":1,"grouping":"small numbers"}
{"number":10,"grouping":"larger numbers"}
{"number":100}

Example: Type is a list of strings

file: lookup5.csv

lists,grouping
"zero,two,four",even
"one,three,five",odd

input:

{"number":"one"}
{"number":"two"}
{"number":"three"}

action:

enrich:
  - lookup-file: lookup5.csv
    match:
      - type: str-list
        event-field: number
        lookup-field: lists
    add:
      event-field: grouping
      lookup-field: grouping

output:

{"number":"one","grouping":"odd"}
{"number":"two","grouping":"even"}
{"number":"three","grouping":"odd"}

Example: CDIR with multiple matches

file: lookup6.csv

source,destination,label
192.168.26.0/24,192.168.26.0/24,sameSide
192.168.85.0/24,192.168.85.0/24,sameSide
192.168.26.0/24,192.168.85.0/24,sameSide
192.168.26.0/24,0.0.0.0/0,outbound
192.168.85.0/24,0.0.0.0/0,outbound
0.0.0.0/0,192.168.26.0/24,inbound
0.0.0.0/0,192.168.85.0/24,inbound
0.0.0.0/0,0.0.0.0/0,unknown

input:

{"src":"192.168.26.10","dst":"192.168.26.11"}
{"src":"192.168.26.10","dst":"192.168.85.10"}
{"src":"192.168.85.10","dst":"192.168.85.11"}
{"src":"192.168.26.10","dst":"192.168.86.10"}
{"src":"192.168.86.10","dst":"192.168.26.10"}
{"src":"192.168.86.10","dst":"192.168.86.11"}

action:

enrich:
  - lookup-file: lookup6.csv
    match:
     - type: cidr
       event-field: src
       lookup-field: source
     - type: cidr
       event-field: dst
       lookup-field: destination
    add:
      event-field: traffic-direction
      lookup-field: label

output:

{"src":"192.168.26.10","dst":"192.168.26.11","traffic-direction":"sameSide"}
{"src":"192.168.26.10","dst":"192.168.85.10","traffic-direction":"sameSide"}
{"src":"192.168.85.10","dst":"192.168.85.11","traffic-direction":"sameSide"}
{"src":"192.168.26.10","dst":"192.168.86.10","traffic-direction":"outbound"}
{"src":"192.168.86.10","dst":"192.168.26.10","traffic-direction":"inbound"}
{"src":"192.168.86.10","dst":"192.168.86.11","traffic-direction":"unknown"}

event-field

Event field matched

Type: field

lookup-field

Lookup field matched

Type: field

add

Detail on what to add to the event, based on the match. If there is no default value, then the output field will not be added to the event.

Field NameDescriptionTypeDefault
event-fieldField name to be added to the eventfield-
lookup-fieldField (CSV header) to lookup data to be place in event-fieldfield-
default-valuefield-
event-fieldsAdd multiple fields to a single event based on a single match, providing a default--

event-field

Field name to be added to the event

Type: field

lookup-field

Field (CSV header) to lookup data to be place in event-field

Type: field

default-value

???

Type: field

Example: type=cidr

file: lookup7.csv

office,network
Splitpoint,192.168.26.0/28
Panoptix,192.168.85.0/24

input:

{"address":"192.168.26.10"}
{"address":"192.168.26.100"}
{"address":"192.168.85.100"}

action:

enrich:
  - lookup-file: lookup7.csv
    match:
      - type: cidr
        event-field: address
        lookup-field: network
    add:
      event-field: office
      lookup-field: office
      default-value: unknown

output:

{"address":"192.168.26.10","office":"Splitpoint"}
{"address":"192.168.26.100","office":"unknown"}
{"address":"192.168.85.100","office":"Panoptix"}

event-fields

Add multiple fields to a single event based on a single match, providing a default

The limitation with this shortcut is that the lookup field name should be the same as the event field

Example: type=cidr

file: lookup8.csv

office,network
Splitpoint,192.168.26.0/28
Panoptix,192.168.85.0/24

input:

{"address":"192.168.26.10"}
{"address":"192.168.26.100"}
{"address":"192.168.85.100"}

action:

enrich:
  - lookup-file: lookup8.csv
    match:
      - type: cidr
        event-field: address
        lookup-field: network
    add:
      event-fields:
        - office: unknown

output:

{"address":"192.168.26.10","office":"Splitpoint"}
{"address":"192.168.26.100","office":"unknown"}
{"address":"192.168.85.100","office":"Panoptix"}