Transform foreign import sources

Often the import files are not ready to be imported when we receive them from the foreign systems.

Commonly, there is data missing, which needs to be filled with Magento insides like category paths or website codes. There is a transformation step inside the given import pipelines.

See the example, where we will replace the default executor with custom logic.

Manipulate the pipeline

By creating a own pipeline.xml in the etc folder of our module, we can create own pipelines and overwrite existing.

In this example, we want to change the executor of the step product_transformation step within the pacemaker_import_catalog pipeline.

Content of app/code/MyModule/CustomCatalogImport/etc/pipeline.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:TechDivision_ProcessPipelines:etc/pipeline.xsd">
    <pipeline name="pacemaker_import_catalog">
        <step
                name="product_transformation"
                executorType="MyModule\CustomCatalogImport\Model\Executor\TransformationExecutor"
                sortOrder="40"
        />
    </pipeline>
</config>

We creating a pipeline node with the name "pacemaker_import_catalog" and add here a single step node with the name "product_transformation".

Both nodes are merged between the pipelines. By defining an own executorType we will change the previous defined executor since the sequence in the module.xml of our module gives our module higher priority.

Transformation logic

Once we created our module and changed the executor for the transformation step, we need to make the executor class. In the previous part, we already chose the class name MyModule\CustomCatalogImport\Model\Executor\TransformationExecutor, which means we need to create the file app/code/MyModule/CustomCatalog/Import/Model/Executor/TransformationExecutor.php according to the PSR-4 autoloader.

We recommend the directory structure Model/Executor, Model/Condition/Step and Model/Condition/Pipeline for the corresponding classes.

Place the following code; we already have a valid executor.

<?php
declare(strict_types=1);

namespace MyModule\CustomCatalogImport\Model\Executor;

use TechDivision\ProcessPipelines\Api\StepInterface;
use TechDivision\ProcessPipelines\Model\Executor\AbstractExecutor;

class TransformationExecutor extends AbstractExecutor
{
    /**
     * @inheritdoc
     */
    public function process(StepInterface $step): void
    {

    }
}

In order to test your current implementation you can add following code into the body of your process method and execute the pipelines .

$this->getLogger()->info("It works ;)");

Use the CLI to observe the progress of your pipeline (bin/magento pipeline:status -w 2).

See the details for your pipeline with bin/magento pipeline:detail <PIPELINE_ID> and retrieve the log for your transformation step with bin/magento pipeline:step:show <STEP_ID>. The result should be something like

--- ATTEMPT 1 ---
[2020-02-11 11:10:40] executor.INFO: It works ;)

Now you’re able to add your custom logic to this executor to transform the given files.

The file paths for this import pipeline are present in the $step as arguments.

Execute $files = (array)$step->getArgumentValueByKey('files'); to fetch a list of file paths.

Examples

Checkout following resource for an example module.

This example shows how to implement a mapping layer, which transforms data from CSV to CSV before running the import.

Transform foreign import sources

Manipulate the pipeline

Transformation logic

Examples

WIR HÄTTEN IHNEN DIESE MELDUNG AUCH GERNE ERSPART ...