How to configure the runners
Runners are doing the job. They execute the major logic of each step if the step is ready to run. You need to have at least one runner up and running. You should have multiple runners to use the multiprocessing capabilities of Pacemaker.
A Pacemaker runner is a message queue consumer. Therefore you need to run the default Magento
queue:consumers:start command to start a Pacemaker pipeline runner.
bin/magento queue:consumers:start pipelineRunner
We recommend to use supervisor in order to run multiple runners and ensure, that all runners are up and running all the time. See following supervisor configuration example:
[program:pipelineRunner] stderr_logfile = <MAGENTO_ROOT>/var/pipelineRunner.log autorestart = 1 autostart = 1 startsecs = 0 startretries = 0 directory = <MAGENTO_ROOT> numprocs = 4 process_name = %(program_name)s_%(process_num)02d user = www-data command = /usr/bin/env php <MAGENTO_ROOT>/bin/magento queue:consumers:start --max-messages 1 pipelineRunner
We recommend using the
--max-messages option with value
1to avoid issues within
stateful PHP code. Of course - it depends on how you implement your job executors, but there is
still vendor code, which could have a state and this would raise issues, which are easy to fix
but hard to detect.
This option is required, since we have executors, which are running faster than one second. In this case supervisor would stop restarting the runners. Please refer to the supervisor configuration documentation for more details
If you’re using MacOS X please refer to how to setup RabbitMQ on MacOS X