Introduction

This page describes the Perfion Connector for Shopware.

Applicable versions

The following Shopware version have currently been tested and are approved for this version of the Perfion Connector:

• Shopware 6.4 (all 2.x.x versions)

• Shopware 6.5 (from version 3.0.1 and forward)

The basics of what this connector does

The connector (depicted in a very simplified way in Figure 1) works as follows:

Open

Shopware asks Perfion for data based on the configured channels in Perfion. It is possible to extend this functionality using the built in fluent PHP API. The Perfion Connector will process the received data and create the custom fields, properties, categories, products, and images.

Installation

Prerequisites

Before installing the connector make sure, you have the following prerequisites set up:

1. You have a running Shopware (6.4 or 6.5) installation with PHP 7.3, PHP 7.4 or PHP 8.1 running.

2. You have a Perfion Database v. 4.6 (or newer).

3. You have a Perfion API set up on IIS Perfion running at least 4.6 (or newer). The Perfion API endpoint must be accessible from the Shopware server.

   1. Please note: If a firewall protects your Perfion instance in your infrastructure setup, please put all Shopware server IPs in the allow list of your firewall.

4. Your Perfion license must include licenses for the Perfion Ecommerce API with at least one Ecommerce Channel and the Shopware connector.

Setting up the needed features in Perfion

Before the connector can be used, you need to ensure that you have active the Ecommerce API in Perfion, this gives you access to the API mapping functionality.

Installing the connector in Shopware

The Shopware connector can be installed from the Shopware store and using composer. Please be aware that all Connector versions 2.x.x are compatible with Shopware 6.4 and all above 3.x.x are compatible with Shopware 6.5.

Configuring the connector

This chapter takes you through all configuration needed for setting up the connector for the first time. First section will take you through the necessary steps in the Shopware backend interface. Next section will take you through the steps needed in Perfion.

When you have completed this chapter, you have a very basic setup of Shopware with the Perfion Connector.

Configuration in Shopware

After the connector has been installed though the Shopware store, it needs to be configured to reach the Perfion API. Please note, that it is possible to have a unique configuration for each of your sales channels if you want to.

It is often a good idea not to activate the integration on “All sales channels”, but instead type in a general configuration on the “all sales channels” level (1), and override the configuration inheritance, and activate the unique sales channels one by one (2). If you activate all your sales channels, the integration will try executing once per channel, even if you don’t need data on a particular sales channel.

Please refer to the Configuration section for configuring Perfion section, for an overview of the mandatory features you need to setup in Perfion.

Open

Plugin configuration

When you enter the plugin configuration section in Shopware, you have the following options.

Perfion channel name - The name of the channel you have setup in the Ecommerce Mapping in Perfion.

Activate channel - Only active channels are processed when running the integration.

Perfion API Url – The URL to your Perfion API

Perfion user name – If you are using a secure service, type in your Perfion username

Perfion password - If you are using a secure service, type in your Perfion password

When this is enabled. Custom fields are imported as complex fields - You can enable / disable how custom fields are imported. If complex custom fields are enabled, fields will be imported as selectables / lists etc in Shopware, at the cost of import speed. If disabled, all custom fields are imported as simple values. Multi selects are imported as string arrays. This improved import performance.

Default active status – When enabling this feature, products are only activated in Shopware, if they have the ‘active’ field mapped in Perfion, and this field is set to true for your products. If this features is disabled, products are always considered active.

Deactivate Perfion categories in Shopware that are removed from Perfion – When this feature is active, categories that are removed from the Perfion catalog, will be deactivated in Shopware.

Deactivate Perfion products in Shopware that are removed from Perfion– When this feature is active, products that are removed from the Perfion catalog, will be deactivated in Shopware.

Import prices – If enabled, also imports prices (via the price mapping in Perfion) into Shopware.

Import stock – If enabled, also imports stock  (via the stock mapping in Perfion) into Shopware.

Perfion batch count – The amount of products downloaded from Perfion in a single batch.

How many seconds before an attempted connection is dropped – Timeout for the Perfion API requests connection initiation.

How many seconds before the whole request is dropped – Timeout for the duration of an entire HTTP request to the Perfion API.

Enable scheduled tasks – If enabled, the Shopware scheduled tasks will download products. 1 Full import every night, and 1 delta import every hour. It is highly suggested that you use CRON jobs, and not this feature, since it is mainly implemented for testing purposes.

Index rebuild on import – Check this to immediately rebuild your index, when you import data.

Server time offset – DEPRECATED – no longer in use.

Root category – The category that will be used as the root, when importing categories into Shopware.

Use "Filter" view in Perfion – When this feature is enabled, Shopware will use the chosen view below, to figure out which features should be created as properties. Mapped features not selected in this view, will still be created as custom fields.

The View in Perfion you use to control features – If the above setting is enabled, type in the view name, that you wish to use to select which features should be properties in Shopware. (Note: as of Perfion 5.3.2, only WebDetail, WebList or WebVariant can be used)

Test API Button – The Test API button, will call Perfion, and validate that a connection can be established. Furthermore, the Test API button will also validate common mapppings and settings in Perfion and Shopware, and report back any issues found. This should help you figure out, if you are missing anything in your configuration.

Configuration in Perfion

Please note that a complete configuration example is shown at the end of this section.

Creating a channel

Since Perfion supports multiple output channels, and multiple channel mappings, you need to predefine your Shopware channel first.

Go to your “EcommerceSystem” feature data section, and ensure that the feature “Shopware” exists. If it does not, then create it.

Next, go to your “EcommerceChannel” feature data section, and add your new Shopware channel. In this example, the channel I simply called Shopware, using the EcommerceSystem also called Shopware.

Language Deviations

Go to your EcommerceMapping feature data section.

In your EcommerceMapping you need to set up language deviations for your channel, such that the Ecommerce API can properly fit you localizable data into Shopware.

The input (left hand side) of your Ecommerce Mapping should point to the Perfion language you would like to send to Shopware. Language Deviation “Output Kind” is empty. On the right hand side, in the “Ecommerce To” column, you have to type in the Shopware language code, that matches your Perfion language. In the example shown below, English(EN) is mapped to en-GB, and Danish(DAN) is mapped to da-DK. The language code must exactly match the Shopware language code.

Category mapping

Set up your category structure and map required fields in EcommerceMapping

Create a selectable string feature in Perfion and configure it as a multiselectable localizable string with allowed hierarchyas seen in screenshot below. This catalog is no different than any other catalog you create in Perfion.

It would proable also be helpful, to define a Perfion section for your Shopware catalog, to ease your work when categorizing your website products; However this is not a requirement.

Category Ecommerce mapping

For categories, you can map the following fields. Please note, that required field, are required for the integration to work. Required fields must always have the exact value in the TO column as shown below. Certain system field like “.id” and “.childCount” must always be spelled exactly as in the “Mapping” column. Read “Note” column carefully.

Additional custom fields:

If you map any other Perfion feature in your category mapping, they will be created as custom fields on the category.

Eg. this mapping:

Will result in a “headline” custom field being created in the category custom field set.

Full example of a standard category mapping without any additional custom fields

How to setup a category search

Create a new feature under “EcommerceSearches”. This search must be the same name as you defined in your EcommerceMapping’s Context mapping seen in the example above (Category/WebDetail/ShopwareCategories); in this example it is “ShopwareCategories”. Right click the search, and provide a query, if you need to filter the categories for Shopware futher. A common thing to do, is to filter away categories, that have been deactivated via a Boolean feature, which you can see an example of below.

Product mapping

Ecommerce Mapping

Since you have already created your language deviations which will also work for products, we can jump directly to the EcommerceMapping of products.

You can find the currently supported fields in the table below. Please bear in mind, that any mapped field that is not in the table below, will be created as a custom field on your products in Shopware. Any fields to your “filterable_properties” mapping, your fields will be created as properties on your products in Shopware. Filterable properties is also described in this document.

Product / Variant mapping

Ecommerce API Specific mappings

Remember that, as with any other use of the Perfion Ecommerce API, you also need to map the special Ecommerce API mappings, to enable variant setup.

For products and variants, you need to map the following (see complete mapping screenshot for more help):

EcommerceVariantDimensions

EcommerceType

EcommerceVariantOfProduct

How does the “filterable_products” mapping work?

When you are mapping the filterable_products mapping, you have the opportunity to select which features you would like to create as Properties in Shopware.

Properties in Shopware are features that can be used as filtering, or as variant configurations.

The filterable_products mapping, is a text string enclosed by apostrophes ‘’. The values inside the apostrophes are the output mapping names (in the Ecommerce TO column) of the features you want to configure as properties. These must be semi-colon seperated.

Remember, if you do not add your mappings to the filterable_properties mapping, the fields will be created as custom fields on the products, and this cannot be used for filtering or variant dimensions.

YOU ALWAYS HAVE TO INSERT THE NAMES OF FEATURES THAT ARE USED AS VARIANT DIMENSIONS, OTHERWISE VARIANTS CANNOT BE CREATED CORRECTLY.

Example 1:

Lets say you have a product that has 2 variant dimensions, color and size. Before you can use these dimensions, you will need to map them in your Ecommerce Mapping; note that the TO / output name is an arbitrary name. I chose color_dimension and size, eg:

Before variants can be created, the two mapping names (color_dimension, size), needs to be added to the filterable_properties like so:

Example 2:

Lets say that you added another feature, that you would like to be able to filter on in Shopware, you could do the following.

Add the feature mapping:

Add the Ecommerce TO name, to the filterable_properties like so:

How to setup a product search

To setup a product search, go to the EcommerceSearches feature data section, and create a new search to filter your products. As with categories, the search should be names the same, as the search part of your Context mapping.

In this case, the search is called ShopwareProductSearch.

Right click the search and input your product search filter query. It is common to limit your search, to products that are contained in the active categories that you are using for Shopware, eg:

Ecommerce Settings - Activating variants

If you are working with variants, you need to enable variants in the EcommerceSettings feature data section, by setting VariantSupportEnabled to True.

Additional required EcommerceSettings

• To enable incremental / delta imports to Shopware, you need to create a settings feature called “TimeZone”, and set it to “utc”

• You also need to type in your FileServiceUrl and ImageServiceUrl (see screenshot above).

Complete category, product and variant mapping

How to setup your products and variants in the Perfion grid

When you are done mapping your Ecommerce API setup, the next step is to configure your products. In this example, we have configured a product to have two variant dimensions, Color and Material.

In the screenshot below, you will see the needed column for such a setup.

EcommerceVariantOfProduct: In this column you need to type in the Value/Name of your root product column. In the example, all the variants below “CHAMBORD”, have CHAMBORD in the EcommerceVariantOfProduct column, to “connect” them to their parent product.

EcommerceType: Here you need to chose which product is the parent product, and which ones are the variants.

EcommerceVariantDimensions: In this field you can type in a comma separated list of Perfion feature names(unique name), that you want to use as the variant dimensions. The names have to match the Perfion unique name exactly. If you chose eg. Color, Material, you also need to ensure, that the variants actually have values in their “Variant dimension” features. This is why the Material and Color columns are filled in, as shown below.

Setting up Related Products

To set up related products, you need to add one or more related products features as children to your products in Perfion. This should look like this in the feature section in Perfion:

Remember to add the new related product types, to your product configuration in Perfion:

When you have set up your related product types. You need to add them to your product mapping in the Feature Data section, under the Ecommerce Mappping:

When these are imported to Shopware, the related product features captions are used as the cross selling groups in Shopware, and products are added to these groups.

Notes on integration

Product activation for full imports

For full imports, products that have been removed from your catalog will be deactivated in Shopware, if you have applied the configuration setting: “Deactivate products removed in Perfion”. This is not the case in delta import, since the complete catalog isn’t known at that point.

Requirements for transferrable data

It is a requirement for features to be transferred to Shopware, that they have a translation on the primary language that you have mapped to Shopware. The connector will warn you if you are missing translations, and these can be seen in the connector logs.

Using a Filter view in Perfion to select properties for Shopware

When enabling the configuration feature “Use Filter view” in the Shopware Configuration, Shopware will use the view selections you have chosen in the Perfion product configuration, to know which features are allowed as properties. The view used, is the view you have configured in the Shopware plugin configuration page. Additionally, this also means, that the “filterable_properties” mapping will be unused for the chosen channel.

Figuring out what is missing in the configuration.

When executing the connector via the CLI, the connector will return information about missing, or misconfigurations in Shopware and Perfion.

Furthermore, if you use the “Test API” button in the plugin configuration in Shopware, it will also return a report of possible issues or missing fields in your configuration.

Automating the integration and using the CLI commands

The integration is initiated using either CLI commands, or automatic the CLI commands via CRON Jobs.

Setting up cronjobs

By default, the connector does not automatically fetch data from your Perfion instance. You can always get the data manually in the CLI. If you want to automate the import of the Perfion data, you need to set up cronjobs to fetch and process the data automatically.

Setting up the cronjobs depends on how often you change product data in Perfion and how quickly you need the product data changes imported in your Shopware installation.

To use the commands in your CLI, you have the following possibilities:

php /path/to/bin/console perfion:import CODE –FLAGS

CODE can be the following:

all = do a full import of products, categories, images,category images.

product – Get products (including images)

category – Get categories (including images)

FLAGS can be configured with these settings:

--delta=true – Do an incremental / delta import in stead of a full import

--skipimages=true – Don’t download images in import.

--debug=true – Provide a much larger amount of logging and output data.

--retention=0 – Delete batches from before X amount of days

--vv – Add all log levels to output

--skipextensions – Skips executing the final extensions step of the connector execution

--extensions – Only executes extensions, skips import (use: per:import extensions)

 --import-latest – Imports the latest successful batches again bypassing the API endpoints

You can set up cronjobs for the following commands:

Imports

php /path/to/bin/console perfion:import CODE --FLAGS > /dev/null 2>&1

The placeholder CODE refers to one of the valid entities that can be downloaded from Perfion:

all = do a full import of products, categories, images,category images.

product – Get products

category – Get categories

Example – high frequency incremental/delta import command:

php /path/to/bin/console perfion:import all –delta=true > /dev/null 2>&1

Example – Full import for only products:

php /path/to/bin/console perfion:import product –no-debug –env=prod

For even more verbosity run the command with –v

Important import flags which are highly suggested to set

When running imports via CLI, it is very important to use the flags –no-debug –env=prod. If you do not use these flags, the integration will consume a very large amount of memory, due to excessive database logging/logging in the standard Symfony / Doctrine API’s.

Furthermore, if you have a significant amount of data that needs to be transferred, execute commands via the PHP Cli using the memory_limit configuration, eg:

php –d memory_limit=-1 bin/console perfion:import all –delta=true –no-debug –env=prod

Batches

In Shopware you can configure the size of the batches that are retrieved from Perfion. When the connector communicates with Perfion, it downloads the batches into a local folder on the Shopware installation, such that you always have the opportunity to return and check the XML payloads of the batches /path/to/shopware/pim-batches.

A reference to the batches is also stored in the database table perfion_batch.

If the retrieval of a batch fails due to server errors, timeouts or other network issues, the connector will retry up to 3 times, in order to try and complete the integration job. Retries are done every 1,4,9 minutes. If no retries are successful, the connector will stop the integration. Errors from Ecommerce API from Perfion will also be shown as exceptions and logged.

To ensure that the disk doesn’t get filled up quickly, it is very important that you set up CRON jobs, to purge old batches from the system.

You can purge old batches older than 7 days, by issuing the CLI command manually or via CRON jobs like this:

php /path/to/bin/console perfion:import retention 7

Appendix

Easier Shopware Sales Channel assignment in Perfion

Since the Perfion Shopware Connector needs one or more sales channel ids from Shopware, the “normal” way of inserting these, is not very user friendly when navigating the Perfion User Interface. A suggestion for a better setup, would be to create the following:

• Create a multi selectable string called “ShopwareSalesChannels” in the Perfion feature section

• Create a simple string called ShopwareSalesChannelId in the Perfion feature section.

• Assign the new string ShopwareSalesChannelId, to the ShopwareSalesChannels selectable configuration.

• Assign the ShopwareSalesChannels selectable to your product configuration.

• Create a new string feature in the feature section, call it ShopwareSalesChannelIdsRemote.

• Assign the feature to your product configuration.

• For this feature, create a new remote query for it, and assign it; Use the query at the bottom of this section for the query. You need to swap out the highlighted IDs, with the PerfionIDs for your new ShopwareSalesChannels selectable, and your new ShopwareSalesChannelId string.

• Select “SalesChannelsSemicolonSeperated” as your datafield:

• You should now have these features:

• Create selectables for your ShopwareSalesChannels feature data, and assign the associated ShopwareSalesChannelId.

• You should be able to see a semicolon separated string containing the chosen sales channels:

• Map the ShopwareSalesChannelIdsRemote, to the ‘sales_channels’ mapping, in the EcommerceMapping feature section.

SQL Query

WITH SalesChannels AS(

SELECT i.ID AS KEYVALUE,SpecsView.ItemId,SpecsView.Item FROM Items i

LEFT JOIN SpecsAllView as SpecsView ON SpecsView.ItemId = i.Id 

WHERE i.BaseType = 100 AND

i.ID in (#@ID:24363#) AND SpecsView.SpecType= 502 ← ShopwareSalesChannels-PerfionID

), SalesChannelIds AS(

SELECT s.KEYVALUE,saw.String,s.Item FROM SalesChannels s

LEFT JOIN SpecsAllView saw ON saw.ItemId = s.Item

WHERE saw.SpecType=503 ← ShopwareSalesChannelId-PerfionId

)SELECT KEYVALUE,string_agg(String,';') as SalesChannelsSemicolonSeperated FROM SalesChannelIds GROUP BY KEYVALUE

Integration timeouts

If you encounter integration timeouts, often identified with the message “error fetching http headers”, you can do the following to fine tune how long a timeout is allowed.

When running the perfion integration commands either manually or using CRON jobs, you can specify a PHP timeout, by prepending the command:

php -d default_socket_timeout=<SECONDS> <COMMAND>, where <SECONDS> is a number representing the amount of seconds before a request is dropped (the number -1 represents no timeout), and <COMMAND> is the perfion command you wish to execute.

Example for fetching and processing items for all channels: php -d default_socket_timeout=-1 bin/console perfion:import all

PHP Insufficient memory issues

Depending on the server setup regarding the maximum PHP memory consumption, you might need to specify a higher memory limit, when executing commands either manually or in CRON jobs.

To execute a command with a specific memory limit, you can prepend your commands with:

php -d memory_limit=<MAX_LIMIT_IN_BYTES> <COMMAND>, where <MAX_LIMIT_IN_BYTES> is the maximum allowed memory consumption in bytes (-1 is unlimited), and <COMMAND> is the perfion command you wish to execute.

Example for fetching and processing items for all channels: php -d memory_limit=-1 bin/console perfion:import all

You can specify multiple settings when executing commands. Eg. if you need to set both timeout limit, and memory limit, it looks as follows:

php -d default_socket_timeout=-1 -d memory_limit=-1 bin/console perfion:import all

Known issues

Importing images

When importing images to Shopware, a thumbnail generation message is create for each updated image. When importing a large amount of images, it is necessary to configure the Shopware queue systems heartbeat and timeout.

You can configure these values in the .env file of your shopware installation, by looking for the following entry:

'AMQP_URL' => 'amqp://HOSTNAME?connect_timeout=30&heartbeat=0',

Updating feature datatypes

Due to the internals of Shopware, you can experience issues when changing Perfion feature datatypes, after your custom fields has been created in Shopware. The reason for this is, that Shopware indexes custom fields and types in the ElasticSearch indexmapping, which cannot be updated when they have been created.

You will encounter a serializer exception when running your import, that indicates this.

The solution to this exception is to reset your elasticsearch index, and reindex it again afterwards, by using the following commands:

bin/console es:reset

bin/console es:index

(Extensible) Post processing overview

When the Shopware Connector imports product data, some processing is done as the final step of an import. The reason for this is, that all data needs to be available (across all downloaded batches), before this can be done correctly.

An example of this, is Related Products, where all Perfion products must be known before they can be assigned correctly, since related products might be split across multiple batches; and all initial processing is done “Per-Batch”.

This post processing concept can be extended, if you wish to do additional work, with the product data that was received from Perfion. The Shopware Connector automatically executes all post processing types, that has been injected into the code base, and thus, you can extend the system, by following the procedure in the next section.

Example:

Create a new type that implements the interface “ShopwareProductPostProcessor” or “ShopwareCategoryPostProcessor”. These interfaces are located in the “Wexo\ShopwarePerfionConnector\WexoShopwarePerfionIntegration\Interfaces” namespace.

This interface requires you to implement one method called “process(array $input) : void”

When you implement this method, the $input parameter will contain all the products or categories that were retrieved in the current integration execution. From this function, you can use the data to update Shopware in ways that is not part of the standard connector package. Remember to inject your new type; You can find an example of how this should be done, by looking at the DI XML file located in the connector code base (/src/Resources/config/services.xml), under the service definition for “TaskIntegrationService”

(Wexo\ShopwarePerfionConnector\WexoShopwarePerfionIntegration\Tasks\TaskIntegrationService)

(Extensible) Pre processing overview

When the Shopware Connector imports product data, it is now possible to inject a class, that does pre processing.

An example of this is, that the connector itself uses this concept, when the “use Filter View in Perfion” functionality is enabled. It gathers information on which Perfion features should be used as properties in Shopware.

The pre processing concept can be used to inject your own functionality into the connector, if there is anything you wish to be done, before the “regular” integration executes. This could for example be using the Perfion Fluent API, to gather additional information needed, before the integration executes.

If you wish to inject a pre processor, you can create a class that implements the ShopwarePreProcessor interface.

You can find an example of how such a class could be implemented in the code found here “src/WexoShopwarePerfionIntegration/PreProcessors/ShopwarePerfionFiltersPreProcessor.php”

When you have created your class, you need to define it as a service. In the example found in the code above, it is done as such:

Remember to add your new service, as a sub-argument of the “ShopwarePreProcessor” collection, of the TaskIntegrationService

Custom extensions / SDK  overview

An extension system has been implemented along with a simple SDK in the Shopware Connector, that allows you to call the standard (Non E-Commerce) Perfion API, and build your own custom functionality, based on data retrieved from Perfion.

Since the Perfion Ecommerce API is optimized for Products and Categories, it cannot be used to retrieve additional objects from Perfion, however by utilizing the standard Perfion API, everything can be requested, and used as you wish, if you need custom features. To make this process easier, we have developed an extension system, that gives you services to communicate with Perfion, and build your own custom implementations without initial overhead of implementing communication services.

To retrieve data from Perfion, you need to implement the appropriate classes, and utilize the Perfion Fluent API. The fluent API has a simple syntax that corresponds with the Perfion Query Language, and does all the query building, serialization, deserialization, request / response and so forth, needed to retrieve an XML response from Perfion.

Using this service, you are no longer limited to retrieving “Product” and “Category” features, but can retrieve any feature from Perfion, by defining the feature name (or id) in the ->from(“”) mehod, eg:

Example:

1 Create a new type, that implements the interface ShopwarePerfionExtension found in the namespace “Wexo\ShopwarePerfionConnector\WexoShopwarePerfionIntegration\Extensions”.

This interface requires you to implement one method, called extend(). The extend() method is called after all regular importing of data has been done, and can also be executed as a standalone job, via the CLI, by using the command:

bin/console perfion:import product –extensions-only

You can check out the very simple example extension, found here: “/src/WexoShopwarePerfionIntegration/Extensions/HelloWorldShopwarePerfionExtension.php”.

Remember to inject your extension into the TaskIntegrationService of the Shopware Connector and your newly created Extension service.

Please note, that the service arguments for the HelloWorld service, are simply examples, you can inject what you find useful here.

You can check “/src/Resources/config/services.xml” and find the HelloWorldShopwareExtension service definition:

And find the TaskIntegrationService dependencies, to see how your new extension is added to the connector. You should add your new extensions, as sub arguments, for the ShopwarePerfionExtension argument, of the TaskIntegrationService.

The Perfion Fluent API:

The Fluent API is a relatively simple abstraction layer, that converts a fluent syntax to Perfion XML requests, does the appropriate conversions, and issues these to the normal Perfion API, in a way that the API understands. used in a Shopware context, the fluent API

To get an idea of how the Fluent API works, you can check out the connector code base file “src/WexoPerfionSDK/Service/Perfion/PerfionProductHelloWorldService.php”.

The getData() method has an example of calling Perfion using batches, for large requests.

The getOne() methods shows you how to call Perfion for non batched requests.

If you need to, the get rawQuery() method shows you how to issue a Perfion request, using a raw XML query.

The Fluent API is setup created with a familiar SQL like syntax in mind, eg. Select->From->Where->Skip->Take etc.