Roughly every quarter of a year a new minor version of Optimize is released. These document guides you through the process of migrating your Optimize from one Optimize minor version to the other.

If you want to upgrade Optimize by several versions, you cannot do that at once but you need to perform the upgrades in a sequential order. For instance, if you want to upgrade from 2.5 to 3.0, you need to upgrade first from 2.5 to 2.6, then from 2.6 to 2.7 and finally from 2.7 to 3.0. The following table shows the recommended upgrade paths to the latest version:

Migrations Instructions

You can migrate from one version of Optimize to the next one without losing the data. To migrate to the latest version, please perform the following steps:

First steps

• Make sure that Elasticsearch has enough memory. To do that shutdown Elasticsearch and go the config folder of your Elasticsearch distribution. There you should find a file called jvm.options. Change the values of the two properties Xms and Xmx to at least 1g so that Elasticsearch has enough memory configured. For instance, the two lines could look like the following:

-Xms1g
-Xmx1g

• Restart the Elasticsearch and make sure that the instance is up and running throughout the whole migration process.
• The current Optimize is shutdown, so there is a downtime during the whole migration process.
• Back-up your Elasticsearch instance in case something goes wrong during the migration process. (Highly recommended, but optional)
• Make sure that you have enough storage available to perform the migration. During the migration process it can be the case that up to twice the amount of the storage of your ElasticSearch data is needed. (Highly recommended)
• Back-up your environment-config.yaml and environment-logback.xml located in the environemnt folder of the root directory of your current Optimize. (Optional)

Perform a rolling upgrade to the new ElasticSearch version

You only need to execute this step if you want to upgrade the ElasticSearch (ES) version during the upgrade. In case the ES version stays the same you can skip this step.

The ElasticSearch (ES) upgrade is usually being done in a rolling fashion. Read all about how to do the ES upgrade in the general ElasticSearch Upgrade Guide and consult the rolling ugprade guide of the ES documentation on how to conduct the rolling upgrade. If you have a very simple setup, for instance, a cluster with only one ES node without plugins installed nor machine learning jobs nor special configuration, the upgrade would essentially boil down to the following steps:

• Install the new ES version, e.g. using Docker, your favorite package manager or just by downloading and extracting the new tar/zip archive to a new directory.
• Copy the data from the old ES to the new ES. If you don’t expect any new data coming to your old ES you can just copy the data folder from the old ES distribution and overwrite the data folder in the new ES distribution.
• Copy your old configuration (config/elasticsearch.yml) over to the new ES installation.
• Stop the old ES instance.
• Start the new ES instance and check that everything looks fine.

Although the steps mentioned above are the ones that are at least required, it’s still highly recommended to read through the Elastic documentation do be sure nothing will break.

Perform the migration

• Go the enterprise download page and download the new version of Optimize you want to upgrade to. For instance, if your current version is Optimize 2.2 you should download the version 2.3. Extract the downloaded archive in your preferred directory. The archive contains the Optimize application itself and the executable to upgrade Optimize from your old version to the new version.
• In the environment folder of your current Optimize version, you have defined all configuration defined in the environment-config.yaml file , e.g. for Optimize to be able to connect to the engine and Elasticsearch. Copy the old configuration file and place it in the environment folder of your new Optimize distribution. Bear in mind that the configuration settings might have changed and thus the new Optimize won’t recognize your adjusted settings or complain about settings that are outdated and therefore refuses to startup. Checkout the configuration documentation (you might need to select the correct Optimize version) to get an overview of the allowed/possible settings and their current values.
• Now open up a terminal and change to the root directory of your new Optimize version and run the following command (the name of the jar file depends on the version upgrade): ./upgrade/upgrade.sh on Linux or upgrade/upgrade.bat on Windows
• During the execution the executable will output a warning to ask you to back-up your Elasticsearch data. Type ‘yes’ to confirm that you have backed-up the data.
• In case there are any errors during the migration process, feel free to file a support case.
• To get more verbose information about the upgrade, you can adjust the logging level as it is described in the configuration documentation.

After the migration

• It is very likely that you configured the logging of Optimize to your needs and therefore you adjusted the environment-logback.xml in the environment folder of the root directory of your old Optimize. You can now use the backed up logging configuration and put it in the environment folder of the new Optimize to keep your logging adjustments. (Optional)
• If you are using Optimize plugins it might be required to adjust those plugins to the new version. Go the project, where you developed your plugins. Increase the project version in maven to new Optimize version and build the plugin again (checkout the plugin guide for the details on that). Afterwards add the plugin jar to the plugin folder of your new Optimize distribution. (Optional)
• Start the new Optimize version, as described in the installation guide.

Typical errors

• Wrong execution path:

The upgrade has to be executed from the "upgrade" folder in the Optimize root directory!

 This error is stating that you should execute the upgrade jar from the `upgrade` folder of your new Optimize directory.

• Using an upgrade script that does not match your version:

Schema version saved in Metadata does not match required [2.X.0]

Let’s assume have Optimize 2.1 and want to upgrade to 2.3 and use the jar to upgrade from 2.2 to 2.3. Then this error occurs, because the jar expects Elasticsearch to have the schema version 2.1. Therefore, you downloaded the wrong Optimize artefact, which contained the wrong upgrade jar version.

Force reimport of engine data in Optimize

It can be the case that new features do work on data that was imported with the old version of Optimize. However, if you still want to use the new features on the old data, you can force a reimport of the engine data to Optimize.

To do so, please perform the following steps (this will keep all you defined reports, dashboards and alerts):

• Stop Optimize, but keep Elasticsearch running.
• From the Optimize Installation root run ./reimport/reimport.sh on Linux or reimport/reimport.bat on Windows and wait for it to finish
• Start Optimize again. Optimize will now import all the engine data from scratch.