Validation
Up with version 3.8.0, validation for all entity types will activate by default.
Especially in case of large CSV files, let us suppose > 100 MB, validation can slow down the import process massively. Therefore, the validation is highly customizable and can, if necessary, completely be switched off.
Switch-Off validaton
If you don’t want your CSV files validated, you are able to override the appropriate shortcuts with a snippet, e.g. etc/configuration/shortcuts.json.
For example, if you want to remove the validation from the add-update process of your product import, delete
the operation general/catalog_product/validate from the appropriate shortcut.
Finally, the snippet has to look like the following code:
{
"shortcuts": {
"ce": {
"add-update": [
"general/general/global-data",
"general/general/move-files",
"general/catalog_product/collect-data",
"general/eav_attribute/convert",
"general/eav_attribute/add-update.options",
"general/eav_attribute/add-update.option-values",
"general/eav_attribute/add-update.swatch-values",
"general/catalog_category/convert",
"ce/catalog_category/sort",
"ce/catalog_category/add-update",
"ce/catalog_category/add-update.path",
"ce/catalog_category/add-update.url-rewrite",
"general/catalog_category/children-count",
"ce/catalog_product/add-update",
"ce/catalog_product/add-update.variants",
"ce/catalog_product/add-update.bundles",
"ce/catalog_product/add-update.links",
"ce/catalog_product/add-update.grouped",
"ce/catalog_product/add-update.media",
"general/catalog_product/add-update.msi",
"general/catalog_product/add-update.url-rewrites"
]
}
}
}
}Custom validations
Use for each entity type a snippet, that declares the available operations in the corresponding repository
configuration folder etc/configuration/operations.json.
Each of these files contains the configuration for the validation operation.
By overriding this definition, e.g. with a custom snippet like custom-configuration-dir>/operations.json,
you’ve full control what will get validated and how the validation works.
In general, the validation operation is based on a validator subject, an observer, some listeners, and various callbacks.
Finally, the validations are implemented as callbacks that allow you to register one or more validation callbacks for each column.
Pacemaker Community Edition (CE) comes with some specialized callbacks that only can be used for the corresponding columns and a custom regex validator that can be used to integrate custom, regex-based validations.
To go into more details, let’s take a look at the validation operation of the product import, which is declared in the operations.json of the techdivision/import-product repository.
To make it easier to understand, we’ve removed the unnecessary parts for you in this example:
{
"operations": {
"general": {
"catalog_product": {
"validate": {
"plugins": {
"subject": {
"id": "import.plugin.subject",
"subjects": [
{
...
"params" : {
"custom-validations" : {
"sku" : [ "/.+/" ],
"product_type": [ "simple", "virtual", "configurable", "bundle", "grouped", "giftcard" ],
"visibility": [ "Not Visible Individually", "Catalog", "Search", "Catalog, Search" ]
}
},
"callbacks": [
{
"sku": [ "import.callback.custom.regex.validator" ],
"store_view_code": [ "import.callback.store.view.code.validator" ],
"attribute_set_code": [ "import.callback.attribute.set.name.validator" ],
"product_type": [ "import.callback.custom.array.validator" ],
"tax_class_id": [ "import_product.callback.validator.tax.class" ],
"product_websites": [ "import.callback.store.website.validator" ],
"visibility": [ "import.callback.visibility.validator" ],
"related_skus": [ "import_product.callback.validator.link" ],
"upsell_skus": [ "import_product.callback.validator.link" ],
"crosssell_skus": [ "import_product.callback.validator.link" ],
"created_at" : [ "import.callback.validator.datetime" ],
"updated_at" : [ "import.callback.validator.datetime" ],
"special_price_to_date" : [ "import.callback.validator.datetime" ],
"special_price_from_date" : [ "import.callback.validator.datetime" ],
"custom_design_to" : [ "import.callback.validator.datetime" ],
"custom_design_from" : [ "import.callback.validator.datetime" ],
"new_to_date" : [ "import.callback.validator.datetime" ],
"new_from_date" : [ "import.callback.validator.datetime" ],
"price" : [ "import.callback.validator.number" ],
"special_price" : [ "import.callback.validator.number" ],
"map_price" : [ "import.callback.validator.number" ],
"msrp_price" : [ "import.callback.validator.number" ],
"qty" : [ "import.callback.validator.number" ]
}
]
}
]
}
}
}
}
}
}
}|
you face a problem with a validator or don’t need it, because your file doesn’t contain the column, feel free to remove the corresponding callback from your overriding snippet. |
Regex validator
The regex validator, used to validate the SKU, uses a regular expression to check the corresponding column’s value.
For the SKU, the regular expression results in /.+/ what allows any character with at least one char.
If you want to enable digitals only, you can change the regular expression in the params/custom-validations array from /.+/ to /\+d/.
|
Our recommendation for a regex builder is regex 101, which will help you a lot to find the right expression for your needs. |
Custom array validator
If you don’t want to use a regular expression, but you have a list of allowed values, the custom array validator will be a good decision. By default, we use it to validate the product type.
To add your own product type, simply extend the list in the params/custom-validations array, e.g. to:
-
"simple", "virtual", "configurable", "bundle", "grouped", "giftcard", "my_product_type".