Event Ingestion

Purpose

The Event Ingestion REST API serves the purpose to ingest business process related event data from any third-party system to Camunda Optimize. These events can then be correlated into an Event Based Process in Optimize to get business insights into business processes that are not yet fully modeled nor automated using the Camunda Platform.

Functionality

The Event Ingestion REST API has the following functionality

Ingest new event data in batches, see Example - Ingest three cloud events
Reingest/Override previously ingested events, see Example - Reingest cloud events

CloudEvents Compliance

To provide the best interoperability possible, the Optimize Event Ingestion REST API implements the CloudEvents - Version 1.0 specification, which is hosted by the Cloud Native Computing Foundation (CNCF).

In particular the Optimize Event Ingestion REST API is a CloudEvents consumer implemented as a HTTP Web Hook, as defined by the CloudEvents HTTP 1.1 Web Hooks for Event Delivery - Version 1.0 specification. Following the Structured Content Mode of the HTTP Protocol Binding for CloudEvents - Version 1.0, event context attributes and Event Data is encoded in the JSON Batch Format of the CloudEvents JSON Event Format v1.0.

Authorization

As required by the CloudEvents HTTP 1.1 Web Hooks for Event Delivery - Version 1.0 specification, every Event Ingestion REST API Request needs to include and authorization token either as an Authorization request header or as a URI Query Parameter named access_token.

Given a valid token, mySecret, the header would need to be set in one of the following ways:

Authorization: mySecret

Authorization: Bearer mySecret

For sending the token as a query parameter the HTTP Query would look like the following:

POST /api/ingestion/event/batch?access_token=mySecret

The token to be used to access the Optimize Event Ingestion REST API is a configurable shared secret. Please refer to Event Ingestion REST API Configuration for the particular configuration key to set this token.

The following is an example configuration with a token value of secret:

eventBasedProcess:
  eventIngestion:
    accessToken: secret

Method & HTTP Target Resource

POST /api/ingestion/event/batch

Request Headers

The following request headers have to be provided with every ingest request:

Header	Constraints	Value
Authorization	REQUIRED	See Authorization
Content-Length	REQUIRED	Size in bytes of the entity-body, also see Content-Length.
Content-Type	REQUIRED	Must be one of: `application/cloudevents-batch+json` `application/json`

Request Body

JSON Batch Format compliant JSON Array of CloudEvent JSON Objects:

Name	Type	Constraints	Description
specversion	String	REQUIRED	The version of the CloudEvents specification which the event uses, must be `1.0`. See CloudEvents - Version 1.0 - specversion.
id	String	REQUIRED	Uniquely identifies an event, see CloudEvents - Version 1.0 - id.
source	String	REQUIRED	Identifies the context in which an event happened, see CloudEvents - Version 1.0 - source. A use-case could be if you have conflicting types across different sources, e.g. a `type:OrderProcessed` originating from both `order-service` and `shipping-service`. In this case the `source` field provides means to clearly separate between the origins of a particular event. Note: The triplet of `type`, `source` and `group` will be used as a unique identifier for classes of events.
type	String	REQUIRED	This attribute contains a value describing the type of event related to the originating occurrence, see CloudEvents - Version 1.0 - type. Note: The triplet of `type`, `source` and `group` will be used as a unique identifier for classes of events.
time	Timestamp	OPTIONAL	Timestamp of when the occurrence happened, see CloudEvents - Version 1.0 - time. String encoding: RFC 3339. If not present a default value of the time the event was received will be created.
data	Object	OPTIONAL	Event payload data that is part of the event, see CloudEvents - Version 1.0 - Event Data. This CloudEvents Consumer API only accepts data encoded as `application/json`, the optional attribute CloudEvents - Version 1.0 - datacontenttype is thus not required to be provided by the producer. Furthermore there are no schema restrictions on the `data` attribute and thus the attribute CloudEvents - Version 1.0 - dataschema is also not required to be provided. Producer may provide any valid JSON object but only simple properties of that object will get converted to variables of a process instances of an Event Based Process instance later on. The following is an example of a valid simple properties `data` value. Each of those properties would be available as a variable in any Event Based Process where an event containing this as `data` was mapped: `{ “reviewSuccessful”: true, “amount”: 10.5, “customerId”: “lovelyCustomer1” }` Nested objects, such as `customer` in this example, would not be available as a variable in Event Based Processes where an event containing this as `data` value was mapped: `{ “customer”: { “firstName”:“John”, “lasTName”:“Doe” } }`
group	String	OPTIONAL	This is an OPTIONAL CloudEvents Extension Context Attribute that is specific to this API. A group identifier that may allow to easier identify a group of related events for a user at the stage of mapping events to a process model. An example could be a domain of events that are most likely related to each other, e.g. `billing`. Note: The triplet of `type`, `source` and `group` will be used as a unique identifier for classes of events.
traceid	String	REQUIRED	This is a REQUIRED CloudEvents Extension Context Attribute that is specific to this API. A traceid is a correlation key that relates multiple events to a single business transaction or process instance in BPMN terms. Events with the same traceId will get correlated into one process instance of an Event Based Process.

Result

This method returns no content.

Response Codes

Possible HTTP Response Status codes:

Code	Description
204	Request successful.
400	Returned if some of the properties in the request body are invalid or missing.
401	Secret incorrect or missing in HTTP Header `X-Optimize-API-Secret`. See Authentication on how to authenticate.
403	The Event Based Process feature is not enabled.
429	The maximum number of requests that can be serviced at any time has been reached. The response will include a `Retry-After` HTTP header specifying the recommended number of seconds before the request should be retried. See Configuration for information on how to configure this limit.
500	Some error occurred while processing the ingested event, best check the Optimize log.

Example

Ingest cloud events

Request

POST /api/ingestion/event/batch

Request Body:

 [
   {
      "specversion": "1.0",
      "id": "1edc4160-74e5-4ffc-af59-2d281cf5aca341",
      "source": "order-service",
      "type": "orderCreated",
      "time": "2020-01-01T10:00:00.000Z",
      "traceid": "id1",
      "group": "shop",
      "data": {
          "numberField": 1,
          "stringField": "example"
      }
  },
  {
      "specversion": "1.0",
      "id": "1edc4160-74e5-4ffc-af59-2d281cf5aca342",
      "source": "order-service",
      "type": "orderValidated",
      "time": "2020-01-01T10:00:10.000Z",
      "traceid": "id1",
      "group": "shop",
      "data": {
          "numberField": 1,
          "stringField": "example"
      }
  },
  {
      "specversion": "1.0",
      "id": "1edc4160-74e5-4ffc-af59-2d281cf5aca343",
      "source": "shipping-service",
      "type": "packageShipped",
      "traceid": "id1",
      "group": "shop",
      "time": "2020-01-01T10:00:20.000Z"
  }
]

Response

Status 204.

Reingest cloud events

The API allows you to update any previously ingested cloud event by just ingesting an event using the same event id. The following request would update the first cloud event that got ingested in the Ingest three cloud events sample. Please note that on an update the cloud event needs to be provided as a whole, it’s not possible to perform partial updates through this API.

In this example an additional field newField is added to the data block of the cloud event with the id 1edc4160-74e5-4ffc-af59-2d281cf5aca341.

Request

POST /api/ingestion/event/batch

Request Body:

 [
   {
      "specversion": "1.0",
      "id": "1edc4160-74e5-4ffc-af59-2d281cf5aca341",
      "source": "order-service",
      "type": "orderCreated",
      "time": "2020-01-01T10:00:00.000Z",
      "traceid": "id1",
      "group": "shop",
      "data": {
          "numberField": 1,
          "stringField": "example",
          "newField": "allNew"
      }
  }
]

Response

Status 204.

Options

Event Ingestion

Purpose

Functionality

CloudEvents Compliance

Authorization

Method & HTTP Target Resource

Request Headers

Request Body

Result

Response Codes

Example

Ingest cloud events

Request

Response

Reingest cloud events

Request

Response