FAQ

Composer runs into auth issues on my Mac OS X machine.

Question

As Solution Partner or Customer, you received a ext12345 username together with a token. It gives you access via Composer to the necessary libraries.

These credentials have to be added in your auth.json, which can, for example, be in the source directory of your project like src/auth.json.

Example:

{
   "http-basic": {
      "gitlab.met.tdintern.de": {
         "username": "ext00000",
         "password": "asaZIjkUIollKSADnrlm"
      }
   }
}

Whenever Composer is invoked, these credentials will be used for the HTTP download, Composer will do.

In some cases, you will receive a message from Composer that you will not have the necessary access rights to install Pacemaker or one of its packages.

Anwser

MacOS saves the credentials in the keychain on the host level (https://gitlab.met.tdintern.de).

When invoking Composer the first time, and it doesn’t matter from which project, the first host entry from the keychain will be used and not the auth.json anymore.

It may lead to the problem that the Pacemaker libraries can not be loaded anymore because of missing access.

In the case of Pacemaker, it will be necessary to disable the caching of the system credentials for HTTPS calls.

Therefore execute the following commands to remove the credential helper from the GIT configuration

git config --local --unset-all credential.helper \
  && git config --global --unset-all credential.helper \
  && git config --system --unset-all credential.helper

After that, the credential helper has to re-initialized empty (because of any other tool like xCode for example may also use it) with the following command

git config --global --add credential.helper "" && composer clear-cache

Finally, search in the keychain tool for met and delete the entry gitlab.met.tdintern.de.

If the keychain has been deactivated, in the future GIT should always use the credentials from the auth.json from your project or the global one.

The performance on the production/stating system is worse than on my local machine.

Question

The performance between your local and any other system differs significantly. What can be the reason?

Anwser

Probably the MySQL transaction log has different settings.

The option innodb_flush_log_at_trx_commit by default has the value 1.

It means, that the transaction log will be written by MySQL after each commit.

This option controls the balance between strict ACID compliance for commit operations and higher performance that is possible when commit-related I/O operations are rearranged and done in batches.

Setting this value to 2, you can achieve better performance, but then you could lose transactions in a crash.

Possible values are

0

write and flush once per second

1

write and flush at each commit

2

write at commit, flush once per second

For example, switching this value from 1 to 2 the import performance improves from 02:06:10 to 00:03:29 h which is for sure significant

ID Pipeline Created Finished Duration

219

xxx_import_catalog

Oct 24, 2019 2:47:04 PM

Oct 24, 2019 4:53:14 PM

02:06:10 h

221

xxx_import_catalog

Oct 24, 2019 5:02:02 PM

Oct 24, 2019 5:05:31 PM

00:03:29 h

Timestamp Detection does not work.

Question

I’m using the Pacemaker Professional Edition (PE) > 3.8.0, when I try to activate the timestamp detection with --use-timestamp=true, everything seems to work fine but the performance is at the same level as when i activate the change-set detection with --use-change-set=true.

The log file does not contain any error messages, as well as the console.

Anwser

It may result out of the problem that the date format in your CSV file is different from the default format and the necessary date format has not been configured in the configuration.

Pretending you are using Magento Commerce, to change the date format to Y-m-d H:i:s, create a snippet named <magent-install-dir>/app/etc/configuration/operations.json that contains the following content

{
  "operations": {
    "ee": {
      "catalog_product": {
        "validate": {
          "plugins": {
            "subject": {
              "id": "import.plugin.subject",
              "listeners": [
                {
                  "plugin.process.start": [
                    "import.listener.reset.loader.eav.attribute.option.value"
                  ],
                  "plugin.process.success": [
                    "import.listener.stop.validation"
                  ]
                }
              ],
              "subjects": [
                {
                  "id": "import.subject.validator",
                  "create-imported-file": false,
                  "date-converter": {
                    "source-date-format": "Y-m-d H:i:s"
                  },
                  "file-resolver": {
                    "prefix": "product-import"
                  },
                  "listeners": [
                    {
                      "subject.artefact.header.row.process.start": [
                        "import.listener.validate.header.row"
                      ]
                    }
                  ],
                  "params": {
                    "custom-validations": {
                      "sku": [
                        "/.+/"
                      ],
                      "product_type": [
                        "simple",
                        "virtual",
                        "configurable",
                        "bundle",
                        "grouped",
                        "giftcard",
                        "designyourown"
                      ],
                      "visibility": [
                        "Not Visible Individually",
                        "Catalog",
                        "Search",
                        "Catalog, Search"
                      ]
                    }
                  },
                  "observers": [
                    {
                      "import": [
                        "import_product.observer.composite.base.validate"
                      ]
                    }
                  ],
                  "callbacks": [
                    {
                      "sku": [
                        "import.callback.custom.regex.validator"
                      ],
                      "store_view_code": [
                        "import.callback.store.view.code.validator"
                      ],
                      "attribute_set_code": [
                        "import.callback.attribute.set.name.validator"
                      ],
                      "product_type": [
                        "import.callback.custom.array.validator"
                      ],
                      "tax_class_id": [
                        "import_product.callback.validator.tax.class"
                      ],
                      "product_websites": [
                        "import.callback.store.website.validator"
                      ],
                      "visibility": [
                        "import.callback.visibility.validator"
                      ],
                      "related_skus": [
                        "import_product.callback.validator.link"
                      ],
                      "upsell_skus": [
                        "import_product.callback.validator.link"
                      ],
                      "crosssell_skus": [
                        "import_product.callback.validator.link"
                      ],
                      "created_at": [
                        "import.callback.validator.datetime"
                      ],
                      "updated_at": [
                        "import.callback.validator.datetime"
                      ],
                      "special_price_to_date": [
                        "import.callback.validator.datetime"
                      ],
                      "special_price_from_date": [
                        "import.callback.validator.datetime"
                      ],
                      "custom_design_to": [
                        "import.callback.validator.datetime"
                      ],
                      "custom_design_from": [
                        "import.callback.validator.datetime"
                      ],
                      "new_to_date": [
                        "import.callback.validator.datetime"
                      ],
                      "new_from_date": [
                        "import.callback.validator.datetime"
                      ],
                      "price": [
                        "import.callback.validator.number"
                      ],
                      "special_price": [
                        "import.callback.validator.number"
                      ],
                      "map_price": [
                        "import.callback.validator.number"
                      ],
                      "msrp_price": [
                        "import.callback.validator.number"
                      ],
                      "qty": [
                        "import.callback.validator.number"
                      ],
                      "is_returnable": [
                        "import_product_ee.callback.rma.validator"
                      ]
                    }
                  ]
                }
              ]
            }
          }
        },
        "replace": {
          "plugins": {
            "subject": {
              "id": "import.plugin.subject",
              "subjects": [
                {
                  "id": "import_product_ee.subject.bunch",
                  "date-converter": {
                    "source-date-format": "Y-m-d H:i:s"
                  },
                  "file-resolver": {
                    "prefix": "product-import"
                  },
                  "params": {
                    "copy-images": false
                  },
                  "observers": [
                    {
                      "import": [
                        "import_product_ee.observer.composite.base.replace"
                      ]
                    }
                  ]
                }
              ]
            }
          }
        },
        "add-update": {
          "plugins": {
            "subject": {
              "id": "import.plugin.subject",
              "subjects": [
                {
                  "id": "import_product_ee.subject.bunch",
                  "date-converter": {
                    "source-date-format": "Y-m-d H:i:s"
                  },
                  "file-resolver": {
                    "prefix": "product-import"
                  },
                  "params": {
                    "copy-images": false,
                    "clean-up-media-gallery": true,
                    "clean-up-empty-image-columns": true,
                    "clean-up-website-product-relations": true,
                    "clean-up-category-product-relations": true,
                    "clean-up-empty-columns": [
                      "special_price",
                      "special_price_from_date",
                      "special_price_to_date"
                    ]
                  },
                  "observers": [
                    {
                      "import": [
                        "import_product_ee.observer.composite.base.add_update"
                      ]
                    }
                  ]
                }
              ]
            }
          }
        }
      }
    }
  }
}

General

Question

General

Anwser

Field contents should be enclosed in double-quotes ( ").