One of the central components of the Pacemaker is the
ProcessPipelines module. The concept behind
this module is the
Pipeline Design Pattern.
To describe this pattern in short: You need to split up each process into multiple stages. Each stage has at least one step. While the stages have a clear sequence, the steps inside these stages can run in parallel or in random order, since they are not depending on each other, but on the stage.
In the following image, you can see the responsibilities of both units.
The heartbeat is responsible for the initialization of new pipelines and publishing of messages for the runner if a step can be executed. And since the status of a pipeline is an aggregation of all status of its steps the heartbeat is also responsible for this aggregation.
The runner executes each step, which is in the queue and does not take care of its status or anything else. This gives us great scalability since these runners can run on different or multiple machines.
The pipelines are usually configured in XML files, which are merged between the modules. This happens in the
same way as you already know from Magento’s configuration XMLs (e.g.
Pipelines can also be defined by code (without XML files) if you need to create pipelines in a dynamic way. Read Pipeline Initializer documentation for details.
Please refer to the How to Configure a ProcessPipeline page
A pipeline configuration has at least one condition, which will be checked periodically by the heartbeat. If a pipeline is ready to be spawned the heartbeat creates a new pipeline in the database.
Please refer to the How to create a pipeline condition page
Every step within a pipeline has at least one condition, which will be checked periodically by the heartbeat. If a step is ready to be executed the heartbeat pushes an execution message to the RabbitMQ.
Please refer to the How to create a step condition page
Every message in the RabbitMQ will be executed by the next free runner. This means that the amount of runners is at the same time the limiting factor for parallel execution. The runner instantiates and executes executor class.
Please refer to the How to create an executor page