There are two possible setups where multiple engines can be used:
Please check which scenario corresonds to your setup, because the configuration of multiple engines to Optimize is not always suited for the best import performance.
There is one restriction for the multiple engines feature: The engines are assumed to have distinct process definitions, which means that the one process definition (same key and version) is not deployed on two or more engines at the same time.
Multiple engines with a shared Database
In this scenario you have multiple engines distributed in a cluster, where each engine instance is connected to a shared database. See the following diagram for an illustration:
Now it could be possible to connect each engine to Optimize. However, the would conclude that Optimize imports the data from each engine. Since every engine accesses the same data through the shared database, Optimize would import the engine data multiple times. What is more desired in this scenario is to balance the work load on each engine during the import. Therefore, it is adviced to place a load balancer between the engines and Optimize. Thus, Optimize would only communicate to the load balancer. The following diagram depicts the described setup:
Multiple engines with a distributed Database
In this scenario you have multiple engines distributed in a cluster, but this time each engine has its own database. See the following diagram for an illustration:
Now you are able to connect each engine to Optimize. The data will then automatically be imported into Optimize. The following diagramm depicts the setup:
In order to set up the connections to the engines, you need add the information to the configuration file. For the sake of simplicity, let’s assume we have two departments, sales and marketing, each having their own engine with its own database and processes. Both are accessible in the local network. The sales engine has the port
8080 and the marketing engine the port
1234. Now an excerpt of the configuration could look as follows:
marketing are custom names that we chose in order to later on distinguish from where the data was originally imported.