Upgrade notes (3.3 to 3.4)

Heads Up!

To upgrade Optimize to version 3.4.0 please perform the following steps: Migration & Upgrade Instructions.

Here you will find information about:

  • limitations,
  • known issues,
  • changes in the supported environments,
  • any unexpected behavior of Optimize (e.g due to a new feature)

Breaking Changes

Elasticsearch

Optimize no longer supports Elasticsearch versions 7.3.x and 7.4.x as these reached their EOL. Furthermore 7.5.0 is not supported anymore as we encountered problems related to snapshotting that got fixed with 7.5.1, the minimum Elasticsearch version required for Optimize 3.4.0 is thus 7.5.1. See the Supported Environments sections for the full range of supported versions.

Camunda Platform

Optimize now requires at least Camunda Platform 7.13.5 and 7.12.x is not supported anymore. See the Supported Environments sections for the full range of supported versions.

Cannot use camunda as a source value for ingested events

When ingesting events using the Ingestion REST API, Optimize will now reject any events that use camunda (or are case variations of) as a value of the source field. Optimize will not change this for any previously ingested events. If you do have any previously ingested events with this value that are used in reports and notice any unexpected behaviour, please contact us for assistance in migrating these events to comply with this new restriction.

Unexpected behaviour

Flow Node Status is now a filter

With this version, the Flow Node status configuration option is changed into a Flow Node data filter. Therefore, If you previously changed the Flow Node status configuration in your report, your change will appear as a Flow Node data filter instead. There are also changes to the filter behaviour, which may affect your report result. For more information please see the Flow Node status filter documentation.

Changes in requirements

Number of shards per instance index

With Optimize 3.4 come some significant changes in the Elasticsearch schema: instead of storing the instance data for all definitions in one index, Optimize now creates one instance index per process and decision definition which stores all instances of this definition. Consequently, we have updated our default shard size per instance index to 1. If you have previously changed this configuration you may want to update it accordingly.

Hardware Recommendations

Due to the above described schema change for dedicated instance indices, Optimize hardware recommendations are now more directly linked to the amount of definitions and instances in your data sets. We recommend reading the updated hardware requirements section and adjusting your setup accordingly if required.

Known issues

Potential NullPointerException during import

As described above, Optimize 3.4 stores instance data in definition key dependent indices. This means in order to route instance data to the correct instance index during import, Optimize accesses the definition key information stored in the instance data provided by the Camunda Platform Rest API. Unfortunately, in rare cases where users have very old running instances, the instance data Optimize fetches from the Camunda Platform may not include a definition key, which then leads to a NullPointerException during import. This exception can look as follows in your logs:

13:23:39.414 [ImportJobExecutor-pool-11] ERROR o.c.o.s.e.j.i.CompletedIncidentElasticsearchImportJob - Error while executing import to elasticsearch
java.lang.NullPointerException: null

Please note that this only occurs for instances which have been started by very old Camunda Platform versions (<7.3) and which are still running.
The issue has been fixed with Optimize 3.5, so the easiest way to resolve it is to upgrade your Optimize version. If you prefer not to upgrade, you can also adjust your Camunda Platform data to enrich it with the missing definition key information. To do so, you will need to update both the data in your process instance table ACT_HI_PROCINST as well as the activity data for your instances in the ACT_HI_ACTINST table.

Update the process instance data:

UPDATE
    ACT_HI_PROCINST 
SET
    ACT_HI_PROCINST.PROC_DEF_KEY_ = ACT_RE_PROCDEF.KEY_
FROM
    ACT_HI_PROCINST H
INNER JOIN
    ACT_RE_PROCDEF R
ON 
    H.PROC_DEF_ID_ = R.ID_
WHERE H.PROC_DEF_KEY_ = '' or H.PROC_DEF_KEY_ is NULL;

Update the activity instance data:

UPDATE
    ACT_HI_ACTINST 
SET
    ACT_HI_ACTINST.PROC_DEF_KEY_ = ACT_RE_PROCDEF.KEY_
FROM
    ACT_HI_ACTINST H
INNER JOIN
    ACT_RE_PROCDEF R
ON 
    H.PROC_DEF_ID_ = R.ID_
WHERE H.PROC_DEF_KEY_ = '' or H.PROC_DEF_KEY_ is NULL;