stream

Create a new field calculated on historical data

Currently the only operation supported is delta, which gives the difference between the current value and the last value, together with the elapsed time in milliseconds.

If marker is specified, a new event will be created, otherwise the existing event will be enriched

Example

input:

{"time":"2018-07-09T15:09:06.479Z","data":1}
{"time":"2018-07-09T15:09:07.479Z","data":2}
{"time":"2018-07-09T15:09:08.179Z","data":4}

action:

stream:
  input-time: time
  watch: data
  marker:
    - delta
  operation: delta

output:

{"_marker":"delta","delta":1,"elapsed":0}
{"time":"2018-07-09T15:09:06.479Z","data":1}
{"_marker":"delta","delta":1,"elapsed":1000}
{"time":"2018-07-09T15:09:07.479Z","data":2}
{"_marker":"delta","delta":2,"elapsed":700}
{"time":"2018-07-09T15:09:08.179Z","data":4}

Field NameDescriptionTypeDefault
input-timeUse incoming time, instead of current system timedefault_iso-
watchA field to watch for changesfield-
operationA stream operation (delta is the only supported value)--
only-changesLet elaspsed field contain the time in milliseconds since the actual change occuredbool-
output-fieldUse this field to store the differencefielddelta
elapsed-fieldField name to store elapsed timefieldelasped
group-byTrack differences between field values belonging to different groupsfield-
markerThe marker to use for the new generated event(string,string) pair-

input-time

Use incoming time, instead of current system time

Type: default_iso

watch

A field to watch for changes

Type: field

operation

A stream operation (delta is the only supported value)

only-changes

Let elaspsed field contain the time in milliseconds since the actual change occured

Type: bool

output-field

Use this field to store the difference

Type: field

elapsed-field

Field name to store elapsed time

Type: field

group-by

Track differences between field values belonging to different groups

Type: field

Example: Here the change in state is tracked for each value of name

input:

{"time":"2020-02-03T15:34:45.149Z","name":"john","state":0}
{"time":"2020-02-03T15:34:46.149Z","name":"jane","state":0}
{"time":"2020-02-03T15:34:47.149Z","name":"john","state":1}
{"time":"2020-02-03T15:34:48.149Z","name":"jane","state":1}
{"time":"2020-02-03T15:34:49.149Z","name":"john","state":1}
{"time":"2020-02-03T15:34:50.149Z","name":"jane","state":1}

action:

stream:
  input-time: time
  watch: state
  group-by: name
  operation: delta

output:

{"time":"2020-02-03T15:34:45.149Z","name":"john","state":0,"delta":0,"elapsed":0}
{"time":"2020-02-03T15:34:46.149Z","name":"jane","state":0,"delta":0,"elapsed":0}
{"time":"2020-02-03T15:34:47.149Z","name":"john","state":1,"delta":1,"elapsed":2000}
{"time":"2020-02-03T15:34:48.149Z","name":"jane","state":1,"delta":1,"elapsed":2000}
{"time":"2020-02-03T15:34:49.149Z","name":"john","state":1,"delta":0,"elapsed":2000}
{"time":"2020-02-03T15:34:50.149Z","name":"jane","state":1,"delta":0,"elapsed":2000}

marker

The marker to use for the new generated event

Type: (string,string) pair

Example: With a marker a new event is created

input:

{"time":"2018-07-09T15:09:06.479Z","data":1}
{"time":"2018-07-09T15:09:07.479Z","data":2}
{"time":"2018-07-09T15:09:08.179Z","data":4}

action:

stream:
  input-time: time
  watch: data
  operation: delta
  marker: [DELTA]

output:

{"_marker":"DELTA","delta":1,"elapsed":0}
{"time":"2018-07-09T15:09:06.479Z","data":1}
{"_marker":"DELTA","delta":1,"elapsed":1000}
{"time":"2018-07-09T15:09:07.479Z","data":2}
{"_marker":"DELTA","delta":2,"elapsed":700}
{"time":"2018-07-09T15:09:08.179Z","data":4}

Example: Without marker the existing event is enriched

input:

{"time":"2018-07-09T15:09:06.479Z","data":1}
{"time":"2018-07-09T15:09:07.479Z","data":2}
{"time":"2018-07-09T15:09:08.179Z","data":4}

action:

stream:
  input-time: time
  watch: data
  operation: delta

output:

{"time":"2018-07-09T15:09:06.479Z","data":1,"delta":1,"elapsed":0}
{"time":"2018-07-09T15:09:07.479Z","data":2,"delta":1,"elapsed":1000}
{"time":"2018-07-09T15:09:08.179Z","data":4,"delta":2,"elapsed":700}