Clean-up

As deleting data can reduce performance significantly, the clean-up functionality should be used carefully!

Pacemaker Community Edition (CE) provides a clean-up functionality that offers the possibility to remove values for empty columns or relations that are no longer part of the CSV file.

It is beneficial when using the add-update operation because it will only be added or updated in general if already available.

Product import

Clean-Up functionality for the product import is activated by default and available for following listing below and as well as for all product attributes (clean-up-empty-columns) which needs additional configuration.

To activate the clean-up functionality for product attributes, as well as additional attributes, the columns that has to be cleaned need to be specified in the clean-up-empty-columns array like "clean-up-empty-columns": [ "activity", "erin_recommends" ].

Remember that columns specified under 'additional_attributes' must be cleaned and provided with empty values, e.g. if the following columns 'activity' and 'in_recommends' must be cleaned, the column 'additional_attributes' requires the value '"activity=,erin_recommends="'.

To deactivate the clean-up functionality, add a snippet, e.g. <custom-configuration-dir>/operations.json and override the corresponding clean-up-* flags like

{
  "operations": {
    "general": {
      "catalog_product": {
        "add-update.url-rewrites": {
          "plugins": {
            "subject": {
              "id": "import.plugin.subject",
              "subjects": [
                {
                  "id": "import_product_url_rewrite.subject.url.rewrite",
                  "file-resolver": {
                    "prefix": "url-rewrite"
                  },
                  "params": {
                    "clean-up-url-rewrites": false
                  },
                  "observers": [
                    {
                      "import": [
                        "import_product_url_rewrite.observer.url.rewrite.update"
                      ]
                    }
                  ]
                }
              ]
            }
          }
        }
      },
      "catalog_product_tier_price": {
        "add-update": {
          "plugins": {
            "subject": {
              "id": "import.plugin.subject",
              "listeners": [
                {
                  "plugin.process.success": [
                    "import_product_tier_price.listener.delete.obsolete.tier_prices"
                  ]
                }
              ],
              "params": {
                "clean-up-tier-prices": false
              },
              "subjects": [
                {
                  "id": "import_product_tier_price.subject.tier_price",
                  "listeners": [
                    {
                      "subject.import.success": [
                        "import_product.listener.register.sku.to.pk.mapping"
                      ]
                    }
                  ],
                  "file-resolver": {
                    "prefix": "product-import-tier-price"
                  },
                  "observers": [
                    {
                      "import": [
                        "import_product_tier_price.observer.tier_price.update"
                      ]
                    }
                  ]
                }
              ]
            }
          }
        }
      }
    },
    "ce": {
      "catalog_product": {
        "add-update": {
          "plugins": {
            "subject": {
              "id": "import.plugin.subject",
              "subjects": [
                {
                  "id": "import_product.subject.bunch",
                  "file-resolver": {
                    "prefix": "product-import"
                  },
                  "params": {
                    "copy-images": false,
                    "clean-up-media-gallery": false,
                    "clean-up-empty-image-columns": false,
                    "clean-up-website-product-relations": false,
                    "clean-up-category-product-relations": false,
                    "clean-up-empty-columns": []
                  },
                  "observers": [
                    {
                      "import": [
                        "import_product.observer.composite.base.add_update"
                      ]
                    }
                  ]
                }
              ]
            }
          }
        }
      }
    }
  }
}

In most cases, it is beneficial to delete images that are no longer in the CSV files from the database.

The image type, e.g. thumbnail as an attribute, links to the product, and the name and the position are stored in separate tables.

To clean-up both of them, it is necessary to set the flags clean-up-empty-image-columns and clean-up-media-gallery to true, to the default values.

If you’re not confident what you’re doing, both flags should have the same value.

Category relations

In many cases, product category relations don’t change at all or change, not really often.

In case the product category relations change frequently, and it’ll be necessary to update them with the add-update operation, this can be done by setting the flag clean-up-category-product-relations to true.

As the product category relation is not only persisted in a column, this relation can not be cleaned by adding a column name to the array clean-up-empty-columns.

Website relations

In many cases, product website relations don’t change at all or change, not often. In case the product website relations change frequently, and it’ll be necessary to update them with the add-update operation, this can be done by setting the flag clean-up-website-product-relations to true.

As the product website relation is not only persisted in a column, this relation can not be cleaned by adding a column name to the array clean-up-empty-columns.

Tier prices

Prices nearly always need to be up-to-date, so if tier prices are imported, the clean-up functionality should be activated by setting the flag clean-up-tier-prices to true.

URL rewrites

Cleaning up URL rewrites will be necessary in most cases and should be activated by setting the flag clean-up-url-rewrites to true,

Categories

As the category functionality in Magento is less complicated, the clean-up only provides one flag for the

As for the product import, the flags have to be configured on a subject level like

{
  "operations": {
    "ce": {
      "catalog_category": {
        "add-update.url-rewrite": {
          "plugins": {
            "subject": {
              "id": "import.plugin.subject",
              "subjects": [
                {
                  "id": "import_category.subject.bunch",
                  "file-resolver": {
                    "prefix": "category-url-rewrite"
                  },
                  "params": {
                    "clean-up-url-rewrites": false
                  },
                  "observers": [
                    {
                      "import": [
                        "import_category.observer.url.rewrite.update"
                      ]
                    }
                  ]
                }
              ]
            }
          }
        }
      }
    }
  }
}

Category url rewrites

Cleaning-Up URL rewrites will be necessary in most cases. To deactivate it, set the flag clean-up-url-rewrites to false.