Event Ingestion

Warning! Uncharted territory!

This feature is currently in early beta stage and by no means production ready yet.

The API, the configuration as well as underlying data structures are subject to change and for the latter there is no guarantee of a migration path to the official release.

Event ingestion is the process of sending event data from third-party systems to Camunda Optimize related to business processes that are not fully automated with Camunda BPM yet. Based on this data it is possible to create process models inside Optimize - called event based processes - that can be used in reports.

To enable that beta feature please refer to Event Based Process Configuration

Event Ingestion API

Events can be ingested via a REST API.

Authentication

To ingest events a shared secret needs to be provided for requests to be accepted. Please refer to Ingestion Configuration for the particular configuration key to set this secret.

An example configuration looks like the following:

eventBasedProcess:
  eventIngestion:
    apiSecret: secret

Once configured on every HTTP Request a header X-Optimize-API-Secret with the secret value needs to be provided. Given a configured example value of mySecret, the header would need to be set like this:

X-Optimize-API-Secret: mySecret

Batch Ingestion

Method

PUT /api/ingestion/event/batch

Request Body

A JSON array of event objects:

Name	Value/Type	Required	Description
id	String	yes	A global unique id (GUID) that uniquely identifies a particular event.
group	String	no	A group identifier that may allow to easier identify a group of related events for a user at the stage of mapping events to a process model. An example could be a domain of events that are most likely related to each other, e.g. `billing`. Note: The triplet of `group`, `source` and `eventName` will be used as a unique identifier for classes of events.
source	String	no	A source identifier that identifies the originating system, e.g. `order-service`. It’s purpose is to make it easier for a user to uniquely identify event classes at the stage of mapping events to a process model. A functional use-case might be if you have conflicting eventNames across different sources, e.g. an `eventName:OrderProcessed` originating from both `order-service` and `shipping-service`. In this case the `source` field provides means to clearly separate these different types of events even though their name is equal. Note: The triplet of `group`, `source` and `eventName` will be used as a unique identifier for classes of events.
eventName	String	yes	An event name or type that identifies the nature of the event, e.g. `OrderCreated` in combination with it’s `group` and `source` it’s the most important property to distinguish between events when mapping events to a process model. Note: The triplet of `group`, `source` and `eventName` will be used as a unique identifier for classes of events.
traceId	String	yes	A traceId is a correlation key that relates multiple events to a single business transaction or process instance speaking in BPMN terms. Events with the same traceId will be correlated into one event based process instance.
timestamp	Long	yes	Milliseconds since Epoch Time of when the event occurred.
data	Object	no	A generic JSON object that includes data related to the event. You may provide any JSON object here but only simple properties of that object will be converted to process instance variables of an event based process instance. The following is example of a valid simple properties: `{ “reviewSuccessful”: true, “amount”: 10.5, “customerId”: “lovelyCustomer1” }` Nested objects like `customer` like in this example however would get ignored: `{ “customer”: { “firstName”:“John”, “lasTName”:“Doe” } }`

Result

This method returns no content.

Response Codes

Code	Description
204	Request successful.
400	Returned if some of the properties in the request body are invalid or missing.
401	No or wrong secret provided in HTTP Header `X-Optimize-API-Secret`. See [Authentication](#authentication) on how to authenticate.
500	Some error occured while processing the ingested event, best check the Optimize log.

Example

Request

PUT /api/ingestion/event/batch

Request Body:

 [
   {
      "id": "1edc4160-74e5-4ffc-af59-2d281cf5aca341",
      "group": "shop",
      "source": "order-service",
      "eventName": "orderCreated",
      "traceId": "id1",
      "timestamp": 1576589400831,
      "data": {
          "numberField": 1,
          "stringField": "example"
      }
  },
  {
      "id": "1edc4160-74e5-4ffc-af59-2d281cf5aca342",
      "group": "shop",
      "source": "order-service",
      "eventName": "orderValidated",
      "traceId": "id1",
      "timestamp": 1576589400831,
      "data": {
          "numberField": 1,
          "stringField": "example"
      }
  },
  {
      "id": "1edc4160-74e5-4ffc-af59-2d281cf5aca343",
      "group": "shop",
      "source": "shipping-service",
      "eventName": "packageShipped",
      "traceId": "id1",
      "timestamp": 1576589800831
  }
]

Response

Status 204.

Event Based Process Configuration

In order to make use of ingested events and create event based process mappings based on them, the event based process beta feature needs to be enabled in the Optimize configuration. This also includes authorizing particular users by their userId to access this feature in the user interface.

A full configuration example enabling the event based process beta feature, authorizing the user demo to access it as well as configuring an Event Ingestion API secret with the value secret looks like the following:

eventBasedProcess:
  enabled: true
  authorizedUserIds: ['demo']
  eventIngestion:
    apiSecret: secret

Options