expand

Converts simple separated data into JSON

This is called expand because it pulls JSON data (values,key-value pairs or JSON) from a field.

Field NameDescriptionTypeDefault
input-fieldField containing datafield-
delimThe delimiter (separator) to be usedstring,
removeWhether to remove field after its value is usedboolfalse

expand csv

Converts CSV rows into JSON records

Field NameDescriptionTypeDefault
headerWhether to treat first line as a CSV headerboolfalse
gen-headersWhether to generate automatic CSV headersboolfalse
fieldsSpecify fields and their types (str, num, bool, num)array of key-value pairs-
field-fileSpecify fields and their types (str, num, bool, num)File containing "name:type" pairs-
header-fieldField containing header (CSV column names)--
header-field-typesField containing header has types specified (with name:type format)-false
relaxed-schemaAllow picking just the first given number fields, and giving those names-false

Whether to treat first line as a CSV header

Type: bool

Example

input:

text,number
one,1
two,2

action:

expand:
  input-field: _raw
  remove: true
  csv:
    header: true

output:

{"text":"one","number":1}
{"text":"two","number":2}

gen-headers

Whether to generate automatic CSV headers

Type: bool

Example

input:

one,1

action:

expand:
  input-field: _raw
  remove: true
  csv:
    gen-headers: true

output:

{"_0":"one","_1":1}

fields

Specify fields and their types (str, num, bool, num)

Type: array of key-value pairs

Example

input:

{"data":"one,1"}

action:

expand:
  input-field: data
  csv:
    fields:
      - text: str
      - number: num

output:

{"data":"one,1","text":"one","number":1}

field-file

Specify fields and their types (str, num, bool, num)

Type: File containing "name:type" pairs

Example

file: some-file

text:str
number:num

input:

{"data":"one,1"}

action:

expand:
  input-field: data
  csv:
    field-file: some-file

output:

{"data":"one,1","text":"one","number":1}

header-field

Field containing header (CSV column names)

Example

input:

{"header":"text,number","row":"one,1"}

action:

expand:
  input-field: row
  remove: true
  csv:
    header-field: header

output:

{"text":"one","number":1}

header-field-types

Field containing header has types specified (with name:type format)

Example

input:

{"header":"text:str,number:str","row":"one,1"}

action:

expand:
  input-field: row
  remove: true
  csv:
    header-field: header
    header-field-types: true

output:

{"text":"one","number":"1"}

relaxed-schema

Allow picking just the first given number fields, and giving those names

Example

input:

{"row":"1,2,3,4,5"}

action:

expand:
  input-field: row
  remove: true
  csv:
    relaxed-schema: true
    fields:
      - first: num
      - second: str

output:

{"first":1,"second":"2"}

expand key-value

Converts data in key=value format

Example: Something basic

input:

one=1 two=2

action:

expand:
  input-field: _raw
  remove: true
  delim: ' '
  key-value:
    key-value-delim: '='

output:

{"one":"1","two":"2"}

Example: Quoted strings

input:

male="john doe" female="jane doe"

action:

expand:
  input-field: _raw
  remove: true
  delim: ' '
  key-value:
    key-value-delim: '='

output:

{"male":"john doe","female":"jane doe"}
Field NameDescriptionTypeDefault
autoconvertAutomatically convert to numbers if possibleboolfalse
key-value-delimSet the delimiter between the key and the valuebool=
multipleIn the case of repeated, choose whether to choose first, last, or to place all in an arraystring-

autoconvert

Automatically convert to numbers if possible

Type: bool

Example

input:

one=1 two=2.0

action:

expand:
  input-field: _raw
  remove: true
  delim: ' '
  key-value:
    autoconvert: true

output:

{"one":1,"two":2}

key-value-delim

Set the delimiter between the key and the value

Type: bool

Example

input:

one:1 two:2

action:

expand:
  input-field: _raw
  remove: true
  delim: ' '
  key-value:
    key-value-delim: ':'

output:

{"one":"1","two":"2"}

multiple

In the case of repeated, choose whether to choose first, last, or to place all in an array

Type: string

Example: first

input:

name=john name=jane

action:

expand:
  input-field: _raw
  remove: true
  delim: ' '
  key-value:
    multiple: first

output:

{"name":"john"}

Example: last

input:

name=john name=jane

action:

expand:
  input-field: _raw
  remove: true
  delim: ' '
  key-value:
    multiple: last

output:

{"name":"jane"}

Example: array

input:

name=john name=jane

action:

expand:
  input-field: _raw
  remove: true
  delim: ' '
  key-value:
    multiple: array

output:

{"name":["john","jane"]}

expand json

Treat incoming data as already in JSON format

Type: bool

Example

input:

{"row":"{\"one\":1,\"two\":2}"}

action:

expand:
  input-field: row
  remove: true
  json: true

output:

{"one":1,"two":2}

expand events

Converts a single event into multiple events

Field NameDescriptionTypeDefault
output-split-fieldA field where resulting split data is to be placedfield-
skip-list--

output-split-field

A field where resulting split data is to be placed

Type: field

Example

input:

{"names":"john jane","family":"doe"}

action:

expand:
  input-field: names
  remove: true
  delim: ' '
  events:
    output-split-field: name

output:

{"family":"doe","name":"john"}
{"family":"doe","name":"jane"}

skip-list

???

input-field

Field containing data

Type: field

Example

input:

{"good":"fast,safe","bad":"slow,dangerous"}

action:

expand:
  input-field: good
  remove: true
  csv:
    fields:
    - speed: str
    - trust: str

output:

{"bad":"slow,dangerous","speed":"fast","trust":"safe"}

delim

The delimiter (separator) to be used

Type: string

Example

input:

{"row":"1 2"}

action:

expand:
  delim: " "
  input-field: row
  csv:
    fields:
    - one: num
    - two: str

output:

{"row":"1 2","one":1,"two":"2"}

remove

Whether to remove field after its value is used

Type: bool

Example

input:

{"row":"1 2"}

action:

expand:
  delim: " "
  remove: true
  input-field: row
  csv:
    fields:
    - one: num
    - two: str

output:

{"one":1,"two":"2"}