FAQ

Composer runs into auth issues on my Mac OS X machine.

Question

As Solution Partner or Customer, you received a ext12345 username together with a token. This gives you access via composer to the necessary libraries. These credentials have to be added in your auth.json, which can, for example, be in the source directory of your project like src/auth.json.

Example:

{
   "http-basic": {
      "gitlab.met.tdintern.de": {
         "username": "ext00000",
         "password": "asaZIjkUIollKSADnrlm"
      }
   }
}

Whenever composer will be invoked, these credentials will be used for the HTTP download, composer will do. In some cases, you’ll receive a message from composer that you’ll not have the necessary access rights to install pacemaker or one of its packages.

Answer

macOS saves the credentials in the keychain on the host level (https://gitlab.met.tdintern.de). When invoking composer the first time, and it doesn’t matter from which project, the first host entry from the keychain will be used and not the one from the auth.json anymore. This may lead to the problem that the Pacemaker libraries can not be loaded anymore because of missing access.

As generally, you’ll access the GIT repositories over SSH, in case of pacemaker it will be necessary to disable the caching of the credentials of the system for HTTPS calls. Therefore execute the following commands to remove the credential helper from the GIT configuration

git config --local --unset-all credential.helper \
  && git config --global --unset-all credential.helper \
  && git config --system --unset-all credential.helper

After that, the credential helper has to re-initialized empty (because of any other tool like xCode for example may also use it) with the following command

git config --global --add credential.helper "" && composer clear-cache

Finally, search in the keychain tool for met and delete the entry gitlab.met.tdintern.de. If the keychain has been deactivated, in the future GIT should ALWAYS use the credentials from the auth.json from your project or the global one.

The performance on the production/stating system is worse than on my local machine!

Question

The performance between your local and any other system differs significantly. What can be the reason?

Answer

Probably the MySQL transaction log has different settings. The option innodb_flush_log_at_trx_commit by default has the value 1. This means, that the transaction log will be written by MySQL after each commit. This option controls the balance between strict ACID compliance for commit operations and higher performance that is possible when commit-related I/O operations are rearranged and done in batches. Setting this value to 2, you can achieve better performance, but then you can lose transactions in a crash.

Possible values are

0

write and flush once per second

1

write and flush at each commit

2

write at commit, flush once per second

For example, switching this value from 1 to 2 the import performance improves from 02:06:10 to 00:03:29 h which is for sure significant

ID Pipeline Created Finished Duration

219

xxx_import_catalog

Oct 24, 2019 2:47:04 PM

Oct 24, 2019 4:53:14 PM

02:06:10 h

221

xxx_import_catalog

Oct 24, 2019 5:02:02 PM

Oct 24, 2019 5:05:31 PM

00:03:29 h

Supervisor does not restarts the runners any more

Problem

After working for a while as a charm the supervisor stops restarting the runner consumers. The supervisor log displays an error like FATAL Exited too quickly (process log may have details).

Answer

Even if a process exits with an “expected” exit code, supervisor will consider the start as a failure if the process exits quicker than startsecs. Which is default 1 second. You need to set the option startsecs=0 to the supervisor configuration in order to avoid this behaviour.

Please refer to How to configure the runners.

Timestamp Detection doesn’t work

Problem

I’m using the Professional Edition > 3.8, when I try to activate the timestamp detection with --use-timestamp=true, everything seems to work fine but the performance is at the same level as when I activate the change-set detection with --use-change-set=true. The logfile doesn’t contain any error messages, as well as the console.

Answer

This may result out of the problem, that the date format in your CSV file is different from the default format and the necessary date format has not been configured in the configuration. Assuming you are using Magento Commerce, to change the date format to Y-m-d H:i:s, create a snippet named <magent-install-dir>/app/etc/configuration/operations.json that contains the following content

{
  "operations": {
    "ee": {
      "catalog_product": {
        "validate": {
          "plugins": {
            "subject": {
              "id": "import.plugin.subject",
              "listeners": [
                {
                  "plugin.process.start": [
                    "import.listener.reset.loader.eav.attribute.option.value"
                  ],
                  "plugin.process.success": [
                    "import.listener.stop.validation"
                  ]
                }
              ],
              "subjects": [
                {
                  "id": "import.subject.validator",
                  "create-imported-file": false,
                  "date-converter": {
                    "source-date-format": "Y-m-d H:i:s"
                  },
                  "file-resolver": {
                    "prefix": "product-import"
                  },
                  "listeners": [
                    {
                      "subject.artefact.header.row.process.start": [
                        "import.listener.validate.header.row"
                      ]
                    }
                  ],
                  "params": {
                    "custom-validations": {
                      "sku": [
                        "/.+/"
                      ],
                      "product_type": [
                        "simple",
                        "virtual",
                        "configurable",
                        "bundle",
                        "grouped",
                        "giftcard",
                        "designyourown"
                      ],
                      "visibility": [
                        "Not Visible Individually",
                        "Catalog",
                        "Search",
                        "Catalog, Search"
                      ]
                    }
                  },
                  "observers": [
                    {
                      "import": [
                        "import_product.observer.composite.base.validate"
                      ]
                    }
                  ],
                  "callbacks": [
                    {
                      "sku": [
                        "import.callback.custom.regex.validator"
                      ],
                      "store_view_code": [
                        "import.callback.store.view.code.validator"
                      ],
                      "attribute_set_code": [
                        "import.callback.attribute.set.name.validator"
                      ],
                      "product_type": [
                        "import.callback.custom.array.validator"
                      ],
                      "tax_class_id": [
                        "import_product.callback.validator.tax.class"
                      ],
                      "product_websites": [
                        "import.callback.store.website.validator"
                      ],
                      "visibility": [
                        "import.callback.visibility.validator"
                      ],
                      "related_skus": [
                        "import_product.callback.validator.link"
                      ],
                      "upsell_skus": [
                        "import_product.callback.validator.link"
                      ],
                      "crosssell_skus": [
                        "import_product.callback.validator.link"
                      ],
                      "created_at": [
                        "import.callback.validator.datetime"
                      ],
                      "updated_at": [
                        "import.callback.validator.datetime"
                      ],
                      "special_price_to_date": [
                        "import.callback.validator.datetime"
                      ],
                      "special_price_from_date": [
                        "import.callback.validator.datetime"
                      ],
                      "custom_design_to": [
                        "import.callback.validator.datetime"
                      ],
                      "custom_design_from": [
                        "import.callback.validator.datetime"
                      ],
                      "new_to_date": [
                        "import.callback.validator.datetime"
                      ],
                      "new_from_date": [
                        "import.callback.validator.datetime"
                      ],
                      "price": [
                        "import.callback.validator.number"
                      ],
                      "special_price": [
                        "import.callback.validator.number"
                      ],
                      "map_price": [
                        "import.callback.validator.number"
                      ],
                      "msrp_price": [
                        "import.callback.validator.number"
                      ],
                      "qty": [
                        "import.callback.validator.number"
                      ],
                      "is_returnable": [
                        "import_product_ee.callback.rma.validator"
                      ]
                    }
                  ]
                }
              ]
            }
          }
        },
        "replace": {
          "plugins": {
            "subject": {
              "id": "import.plugin.subject",
              "subjects": [
                {
                  "id": "import_product_ee.subject.bunch",
                  "date-converter": {
                    "source-date-format": "Y-m-d H:i:s"
                  },
                  "file-resolver": {
                    "prefix": "product-import"
                  },
                  "params": {
                    "copy-images": false
                  },
                  "observers": [
                    {
                      "import": [
                        "import_product_ee.observer.composite.base.replace"
                      ]
                    }
                  ]
                }
              ]
            }
          }
        },
        "add-update": {
          "plugins": {
            "subject": {
              "id": "import.plugin.subject",
              "subjects": [
                {
                  "id": "import_product_ee.subject.bunch",
                  "date-converter": {
                    "source-date-format": "Y-m-d H:i:s"
                  },
                  "file-resolver": {
                    "prefix": "product-import"
                  },
                  "params": {
                    "copy-images": false,
                    "clean-up-media-gallery": true,
                    "clean-up-empty-image-columns": true,
                    "clean-up-website-product-relations": true,
                    "clean-up-category-product-relations": true,
                    "clean-up-empty-columns": [
                      "special_price",
                      "special_price_from_date",
                      "special_price_to_date"
                    ]
                  },
                  "observers": [
                    {
                      "import": [
                        "import_product_ee.observer.composite.base.add_update"
                      ]
                    }
                  ]
                }
              ]
            }
          }
        }
      }
    }
  }
}