Process Pipelines
One of the central components of the Pacemaker is the ProcessPipelines
module. The concept behind
this module is the
Pipeline Design Pattern.
The Pattern
To describe this pattern in short: You need to split up each process into multiple stages. Each stage has at least one step. While the stages have a clear sequence, the steps inside these stages can run in parallel or in random order, since they are not depending on each other, but on the stage.
Realization
In the Get Started section, you already touched the both major
parts of ProcessPipelines
, while configuring the heartbeat cron job
(which is the organizational unit) and
configuring the pipeline runners (which are the execution unit).
Heartbeat and Runner
In the following image, you can see the responsibilities of both units.
![Pipeline Architecture](../_images/Pipeline-Architecture.png)
The heartbeat is responsible for the initialization of new pipelines and publishing of messages for the runner if a step can be executed. And since the status of a pipeline is an aggregation of all status of its steps the heartbeat is also responsible for this aggregation.
The runner executes each step, which is in the queue and does not take care of its status or anything else. This gives us great scalability since these runners can run on different or multiple machines.
Configuration and Process Execution
The pipelines are usually configured in XML files, which are merged between the modules. This happens in the
same way as you already know from Magento’s configuration XMLs (e.g. config.xml
, di.xml
).
Pipeline Initializer
Pipelines can also be defined by code (without XML files) if you need to create pipelines in a dynamic way. Read Pipeline Initializer documentation for details. |
-
Please refer to the How to Configure a ProcessPipeline page
A pipeline configuration has at least one condition, which will be checked periodically by the heartbeat. If a pipeline is ready to be spawned the heartbeat creates a new pipeline in the database.
-
Please refer to the How to create a pipeline condition page
Every step within a pipeline has at least one condition, which will be checked periodically by the heartbeat. If a step is ready to be executed the heartbeat pushes an execution message to the RabbitMQ.
-
Please refer to the How to create a step condition page
Every message in the RabbitMQ will be executed by the next free runner. This means that the amount of runners is at the same time the limiting factor for parallel execution. The runner instantiates and executes executor class.
-
Please refer to the How to create an executor page