How to configure the runners

Runners are doing the job. They execute the major logic of each step if the step is ready to run. You need to have at least one runner up and running. You should have multiple runners to use the multiprocessing capabilities of Pacemaker.

How to execute a runner

A Pacemaker runner is a message queue consumer. Therefore you need to run the default Magento queue:consumers:start command to start a Pacemaker pipeline runner.

bin/magento queue:consumers:start pipelineRunner

Using supervisor for Pacemaker runners

We recommend to use supervisor in order to run multiple runners and ensure, that all runners are up and running all the time. See following supervisor configuration example:

[program:pipelineRunner]
stderr_logfile = <MAGENTO_ROOT>/var/pipelineRunner.log
autorestart = 1
autostart = 1
startsecs = 0
startretries = 0
directory = <MAGENTO_ROOT>
numprocs = 4
process_name = %(program_name)s_%(process_num)02d
user = www-data
command = /usr/bin/env php <MAGENTO_ROOT>/bin/magento queue:consumers:start --max-messages 1 pipelineRunner

Limit runner messages (--max-messages 1)

We recommend using the --max-messages option with value 1to avoid issues within stateful PHP code. Of course - it depends on how you implement your job executors, but there is still vendor code, which could have a state and this would raise issues, which are easy to fix but hard to detect.

startsecs config option

This option is required, since we have executors, which are running faster than one second. In this case supervisor would stop restarting the runners. Please refer to the supervisor configuration documentation for more details

If you’re using MacOS X please refer to how to setup RabbitMQ on MacOS X