Add Steps to an existing Pipeline

Sometimes it is required to add additional steps to an existing import pipeline. Use cases could be

  • Add re-indexations and cache invalidation steps

  • Add cache warming processes

  • Add image cropping executions

  • and others

Manipulate the pipeline

By creating a own pipeline.xml in the etc folder of our module, we are able to create own pipelines and extend existing.

In this example, we want to add additional steps to the existing pacemaker_import_catalog pipeline.

Content of app/code/TechDivision/CustomCatalogImport/etc/pipeline.xml
<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:TechDivision_ProcessPipelines:etc/pipeline.xsd">
    <pipeline name="pacemaker_import_catalog">
        <step name="reindex_attributes" executorType="TechDivision\ProcessPipelines\Model\Executor\Reindex" sortOrder="100" description="Run full reindex">
            <conditions>
                <step_condition type="TechDivision\ProcessPipelines\Helper\Condition\Step\AttemptsLimit\Limit1" description="Try once."/>
                <step_condition type="TechDivision\AddNewStepsToExistingPipeline\Virtual\Condition\Step\IsReindexingStage" description="Ensure it is time for reindexing."/>
            </conditions>
        </step>
        <step name="drop_cache" executorType="TechDivision\ProcessPipelines\Model\Executor\DropCache" sortOrder="110" description="Drop relevant caches">
            <conditions>
                <step_condition type="TechDivision\ProcessPipelines\Helper\Condition\Step\AttemptsLimit\Limit1" description="Try once."/>
                <step_condition type="TechDivision\ProcessPipelines\Helper\Condition\Step\PreviousStepsCompleted" description="Previous step needs to be finished."/>
            </conditions>
        </step>
    </pipeline>
</config>

We create a pipeline node with the name "pacemaker_import_catalog" and add here two step nodes.

With the sortOrder attribute we define the position within our pipeline.

In this example, both steps will run as last.