Configuration
|
As of version To avoid complex adjustments of the configuration, version |
If no configuration file is specified, it will be loaded and merged from individual parts delivered with the respective repositories.
In the case of the import:products, only the operations, specified by the given shortcut, e.g. add-update, will be used and executed.
By default, the configuration contains whether database configuration nor an image directory.
The command-line options can specify the database configuration, it is necessary to provide a custom configuration snipped
that activates the flags "copy-images":true, "clean-up-media-gallery": true" and "clean-up-empty-image-columns": true and
contains the paths to the image files.
This snippet requires the JSON format, and can be named with any random name, e.g. images.json and has to be placed in the
usage
By default, the value for the custom-configuration-dir is <magento-install-directory>/app/etc/configuration.
Configuration snippets
All configuration snippets must be in JSON format and have to be located in the custom configuration dir.
In general, nearly all available configuration options/arguments can and should be defined in those snippets, instead of passing them as command-line options.
The possibility to override configuration values on the command-line should only be used during development or in individual cases, e.g. when the values will be loaded from a third-party system.
The structure is separated into a general configuration section, the database configuration, the logger configuration, and the configuration for the available operations.
General configuration
The general configuration contains necessary metadata like the Magento edition and version, which will be needed when
Pacemaker Community Edition (CE) will be invoked from a directory other than the Magento installation directory and
without the --installation-dir option.
Also, it can be used to specify default values, if you want to change the default operation from add-update to replace for example.
The change that general metadata we recommend to place a snipped, e.g., with the name <custom-configuration-dir>/configuration.json in
the custom configuration directory.
{
"magento-edition": "CE",
"magento-version": "2.3.1",
"operation-name" : "replace",
"table-prefix": "test_",
"installation-dir" : "/var/www/magento"
}Global parameter
Global parameter in the configuration file enable developers to pass specific configuration values from the configuration file itself.
To serve the same purpose, the command-line (using the --params option) or an additional file (using the --params-file option)
through their import logic, e.g., project-specific observers can be used.
In a configuration snippet
It’s possible to create a snippet with params, e.g. <custom-configuration-dir>/params.json which contains values like
{
"params": {
"my-website-country-mapping": {
"DE": [ "de_DE", "de_AT", "de_CH" ],
"EN": [ "en_US", "en_UK" ]
}
}
}These parameter can then be used wherever access to the configuration object is available, e.g. in an observer like
namespace My\Project;
use TechDivision\Import\Observers\AbstractObserver;
/**
* A custom observer implementation.
*/
class MyObserver extends AbstractObserver
{
/**
* Return's the global param with the passed name.
*
* @param string $name The name of the param to return
* @param mixed $defaultValue The default value if the param doesn't exist
*
* @return string The requested param
* @throws \Exception Is thrown, if the requested param is not available
*/
public function getGlobalParam($name, $defaultValue = null)
{
return $this->getSubject()->getConfiguration()->getConfiguration()->getParam($name, $defaultValue);
}
/**
* Will be invoked by the action on the events the listener has been registered.
*
* @param \TechDivision\Import\Subjects\SubjectInterface $subject The subject instance
*
* @return array The modified row
*/
public function handle(SubjectInterface $subject)
{
// load the params from the configuration
$myWebsiteMapping = $this->getGlobalParam('my-website-country-mapping');
// do something with the configuration value
}
}As command-line option
Besides the params that can be defined in the configuration file itself, it is possible to specify additional params using the command-line as an option
bin/import-cli-simple.phar import:products \
--params='{ "params": { "my-website-country-mapping": { "DE": [ "de_LI" ] } } }'which will append the value de_LI to the DE param of the my-website-country-mapping.
As file, defined as command-line option
Besides the possibility of specifying the params directly as a command-line option, it is also possible to specify a path to a file containing the JSON encoded params. The file requires the following format.
{
"params": {
"my-website-country-mapping": {
"DE": [ "de_LI" ]
}
}
}|
Values from the configuration file will be overwritten with values delivered by command-line use, which again will be overwritten with
the values from an additional file that has been specified with the |
Extend Pacemaker Community Edition (CE) with additional libraries
In more complex projects, it’ll we possible, that additional libraries are necessary. As the Pacemaker Community Edition (CE) Console Tool uses a Symfony DI container. It is required to register the other library by adding it to the configuration file. Depending on how the Pacemaker Community Edition (CE) Console Tool has been installed, there are two options.
|
If you write an extension library, do not forget to provide the Symfony DI configuration. |
Extension libraries
Assuming that the Pacemaker Community Edition (CE) Console Tool has been installed as Composer library, together with a Magento 2 installation,
the simplest way to register an additional extension is to add a snippet, e.g. <custom-configuration-dir>/extension-libraries.json that
contains the name of the library, like
{
"extension-libraries" : [
"techdivision/import-product-magic360"
]
}|
This is only possible if the extension library uses the same Composer autoloader as Pacemaker Community Edition (CE) Console Tool does. As a natural outcome, if you’re using the PHAR version of Pacemaker Community Edition (CE), it is required use the |
Additional vendor directories
Suppose that the Pacemaker Community Edition (CE) Console Tool PHAR archive will be used.
The Composer class loader of the additional library vendor directory must be added by a
snippet, e.g. <custom-configuration-dir>/additional-vendor-dirs.json, which contains the following content
{
"additional-vendor-dirs" : [
{
"vendor-dir": "vendor",
"relative": true,
"libraries": [
"techdivision/import-adapter-custom"
]
}
]
}Events
Beside the configured worklfow with Plugins, Subjects, Observers and Callbacks, addtional events are available to add custom functionality.
| Event Name | Description | Since Version |
|---|---|---|
app.set.up |
Is triggered before the import will be processed. |
|
app.tear.down |
Is triggered after the import has been processed. |
|
app.process.transaction.start |
Is triggered before the application start’s the transaction. |
|
app.process.transaction.success |
Is triggered after the application has the transaction committed successfully. |
|
app.process.transaction.failure |
Is triggered after the application rollbacked the transaction. |
|
app.process.transaction.finished |
Is triggered after the application transaction has been finished (either it has been successful or not). |
3.8.0 |
subject.artefact.process.start |
Is triggered before an import artifact will be processed. |
|
subject.artefact.process.success |
Is triggered when an import artifact has successfully been processed. |
|
subject.artefact.process.failure |
Is triggered when an import artifact can not be processed. |
|
subject.artefact.row.process.start |
Is triggered when an import artifact has successfully been processed. |
|
subject.artefact.row.process.success |
Is triggered when an import artifact has successfully been processed. |
|
subject.artefact.header.row.process.start |
Is triggered before an import artifact’s header row will be processed. |
3.8.0 |
subject.artefact.header.row.process.start |
Is triggered when an import artifact’s header row has successfully been processed. |
3.8.0 |
plugin.process.start |
Is triggered before the plugin’s |
3.4.0 |
plugin.process.success |
Is triggered after a plugin’s |
3.4.0 |
plugin.process.failure |
Is triggered when an exception has been thrown during the plugin’s |
3.4.0 |
plugin.export.start |
Is triggered before a plugin’s |
3.8.0 |
plugin.export.success |
Is triggered after a plugin’s |
3.8.0 |
plugin.export.failure |
Is triggered when an exception has been thrown during the plugin’s |
3.8.0 |
subject.import.start |
Is triggered before a subject’s |
3.4.0 |
subject.import.success |
Is triggered after a subject’s |
3.4.0 |
subject.import.failure |
Is triggered when an exception has been thrown during the subject’s |
3.4.0 |
subject.export.start |
Is triggered before a subject’s |
3.4.0 |
subject.export.success |
Is triggered after a subject’s |
3.4.0 |
subject.export.failure |
Is triggered when an exception has been thrown during the subject’s |
3.4.0 |
Besides the possibility of registering global events, up with version 3.4.0, it is possible to register events on a plugin and subject level. It avoids execution of events that only provide functionality for a dedicated plugin or subject, so keep in mind that they will not be fired for every plugin or every subject.
Plugin level
The listeners will only be executed before, after, or on the plugin’s failure, for the configured event.
Configuration in a snippet, e.g. <customer-installation-dir>/operations.json, can look like
{
"operations": {
"general": {
"catalog_product_tier_price": {
"add-update": {
"plugins": {
"subject": {
"id": "import.plugin.subject",
"listeners": [
{
"plugin.process.success": [
"import_product_tier_price.listener.delete.obsolete.tier_prices"
]
}
],
"subjects": [ ... ]
}
}
}
}
}
}
}Subject level
As subjects are responsible for importing and exporting artifacts, events for both steps have been added.
The listeners will only be executed before, after, or on the subject’s failure, for the configured event.
Configuration in a relevant snippet, e.g. <customer-installation-dir>/operations.json, can look like
{
"operations": {
"general": {
"catalog_product_tier_price": {
"add-update": {
"plugins": {
"subject": {
"id": "import.plugin.subject",
"subjects": [
{
"id": "import_product_tier_price.subject.tier_price",
"listeners": [
{
"subject.import.success": [
"import_product.listener.register.sku.to.pk.mapping"
]
}
],
"observers": [ ... ]
}
]
}
}
}
}
}
}
}Default listeners
By default, Pacemaker Community Edition (CE) comes with several listeners registered.
{
"listeners": [
{
"app.set.up": [
"import.listener.render.ansi.art",
"import.listener.render.operation.info",
"import.listener.render.mysql.info",
"import.listener.render.debug.info",
"import.listener.initialize.registry"
],
"app.process.transaction.success": [
"import.listener.finalize.registry",
"import.listener.archive",
"import.listener.clear.artefacts",
"import.listener.clear.directories",
"import.listener.operation.report"
],
"app.process.transaction.failure": [
"import.listener.finalize.registry",
"import.listener.render.validations"
],
"app.tear.down": [
"import.listener.import.history",
"import.listener.clear.registry"
]
}
]
}The first one, named import.listener.render.ansi.art renders the nice ASCII art when invoking the CLI.
The second and third parties are mandatory as they are responsible for initializing the registry of the actual import.
|
As a solution partner, feel free to replace the ASCII art renderer with one of your choices, e.g. rendering your company logo. |
Database
The configuration allows the registration of multiple databases, for example in a snippet <custom-configuration-dir>/databases.json,
which can look like
{
"databases": [
{
"id": "local",
"default": false,
"pdo-dsn": "mysql:host=127.0.0.1;dbname=appserver_magento2_ee212",
"username": "your-username",
"password": "your-password"
},
{
"id": "remote",
"default": true,
"pdo-dsn": "mysql:host=127.0.0.130;dbname=appserver_magento2_ee212",
"username": "your-username",
"password": "your-password"
}
]
}Depending on the command-line option --use-db-id and the specified value, the database with the given ID will be used.
If the command-line options are not specified, the one with the flag "default": true will be used.
If not found, the first configured database will be used.
If a value for the command-line option --db-pdo-dsn has been specified, the --use-db-id option will be ignored, and
the given DSN value will be used for database connection instead. Additionally, the credentials,
by using the --db-username and --db-password options also need to be specified.
Loggers
Pacemaker Community Edition (CE) uses Monolog to provide the basic logging functionality.
Therefore, at least one logger instance is necessary.
By default, if no logger has been configured, a system logger will be installed, which writes log messages to the error
log that has been set in the php.ini file of the used PHP installation.
To add additional loggers, e.g. in case you want to send mails if an exception has been thrown, the configuration can be
extended, e.g. with a snippet <custom-configuration-dir>/loggers.json which looks like
{
"loggers": {
"mail": {
"id": "import.logger.factory.monolog",
"channel-name": "logger/mail",
"handlers": [
{
"id": "import.logger.factory.handler.swift",
"formatter": {
"id": "import.logger.factory.formatter.line",
"params": {
"format": "[%datetime%] %channel%.%level_name%: %message% %context% %extra%",
"date-format": "Y-m-d H:i:s",
"allow-inline-line-breaks": true,
"ignore-empty-context-and-extra": true
}
},
"params": {
"log-level": "error",
"bubble": false
},
"swift-mailer" : {
"id" : "import.logger.factory.transport.swift.smtp",
"params" : {
"to" : "tw@techdivision.com",
"from" : "pacemaker@techdivision.com",
"subject": "Something Went Wrong",
"content-type" : "text/plain"
},
"transport" : {
"params" : {
"smtp-host" : "mail.techdivision.com",
"smtp-port" : 25
}
}
}
}
]
}
}
}You can also use a handler for Native Mail Sender, which takes the settings of the PHP configuration.
{
...,
"handlers": [
{
"id": "import.logger.factory.handler.native.mail.log",
"formatter": {
"id": "import.logger.factory.formatter.line",
"params": {
"format": "[%datetime%] %channel%.%level_name%: %message% %context% %extra%",
"date-format": "Y-m-d H:i:s",
"allow-inline-line-breaks": true,
"ignore-empty-context-and-extra": true
}
},
"params": {
"to": [
"tw@techdivision.com"
],
"subject": "Something Went Wrong",
"from": "pacemaker@techdivision.com",
"level": "error",
"bubble": true,
"maxColumnWidth": 500
}
}
],
...
}This will add a mail logger and set the default log level to error.
Aliases
Aliases can be used to override classes with custom functionality, e.g. provided by a project-specific library.
For example, if a custom adapter for caching is used, the default class behind the Symfony DI configuration of
the cache.adapter can be replaced by the customer DI identifier ` import_parallel .cache .adapter.redis` in a
snippet, e.g. <custom-installation-dir>/aliases.json.
{
"aliases": [
{
"id": "cache.adapter",
"target": "import_parallel.cache.adapter.redis"
}
]
}|
Overriding classes can be dangerous and should only be done when you aware of what you’re doing. |
Images
When you want to copy images from a source directory to the Magento pub/media directory, you can add a
snippet, e.g. <custom-configuration-dir>/operations.json, that overrides the default operation.
In case images for the products has to be copied from directory var/importexport/media/wysiwyg to the appropriate
target directory pub/media/catalog/product the copy-images flag has to be set to true as well as
the values for the media-directory (which is the target directory), and the images-file-directory (which is the source directory)
has to be also specified.
-
With the option
"override-images": true|falseyou are able to configure how to deal with images during an import that already exists in the target directory. -
Is the option set to
override-images": false, the existing file is left as it is, but the filename of the new file increments with a counter, and the latest version is assigned to the product. -
If the option is
override-images": true, the existing image will be overwritten.
Example Configuration:
{
"operations": {
"ce": {
"catalog_product": {
"add-update": {
"plugins": {
"subject": {
"id": "import.plugin.subject",
"subjects": [
{
"id": "import_product.subject.bunch",
"file-resolver": {
"prefix": "product-import"
},
"params": {
"copy-images": true,
"override-images": false,
"media-directory" : "pub/media/catalog/product",
"images-file-directory" : "var/importexport/media/wysiwyg",
"clean-up-media-gallery": true,
"clean-up-empty-image-columns": true,
"clean-up-website-product-relations": true,
"clean-up-category-product-relations": true,
"clean-up-empty-columns": [
"special_price",
"special_price_from_date",
"special_price_to_date"
]
},
"observers": [
{
"import": [
"import_product.observer.composite.base.add_update"
]
}
]
}
]
}
}
}
}
}
}
}In the CSV file, the path to the images has to start with a / like /m/b/mb01-blue-0.jpg.
Pay attension to the example files in the repository
techdivision/import-sample-data;
|
When you copy images, you must create resized versions of them. Therefore, you can invoke the
command Please also keep in mind that it won’t usually make sense to copy the images during the import process, as this will slow down the performance significantly. It is strongly recommended to copy the pictures to the appropriate folder in your Magento installation and only importing the path to the images. |
Header mappings
In some cases, it can be convenient to map column names to the appropriate attributes.
It’s a built-in feature and can simply be configured by adding a snippet, e.g. <custom-configuration-dir>/header-mappings.json that
contains an array with header mappings like
{
"header-mappings": {
"catalog_product": {
"my_sku_column": "sku",
"my_qty_column": "qty",
...,
}
}
}In the example above, the column name my_sku_column will automatically be mapped to the mandatory column SKU.
It happens before any columns will be processed and allows vendors to configure any column to the appropriate
attribute without custom development.
|
Keep in mind that the header mappings are valid for all operations within the configuration file |