Shopify Connector uses a synchronization manager, which is responsible for reading configuration parameters and based on these parameters determines when actual synchronization should start.
Synchronization Manager
Synchronization manager continuously runs and once in a while checks the ECommerce configuration settings from Perfion. The frequency of how often this check is performed is based on parameters set in Perfion ECommerce Settings. If parameters related to configuration update intervals or synchronization frequency are not set in Perfion, e.g. default check periods are used, then it defaults to its internal check interval, which is 10 minutes. There are three parameters used to determine how often manager has to check for new configurations and also how often data synchronization should run: ConfigurationUpdateFrequencyInMin, SyncFrequencyInMin, DataMaxAgeInMin. Refer to “ECommerce Data Mapper – Settings” for more information about how these parameters work.
Synchronization manager writes what it is doing and when to the log file. Please note, that if there is some configuration issue in Perfion, e.g. ECommerce mappings are set incorrectly for ECommerce API, or mappings are missing some parameters needed for Shopify Connector (e.g. Sync type parameters are not defined correctly or mappings do not have all mandatory parameters), then synchronization manager will report about these failures to log file and wait some time before it will try to read configuration data again. Therefore, if the ECommerce mappings were set up correctly and works, it is not recommended to modify any settings parameters in Perfion without having access to Shopify Connector log file. Any failures in synchronization manager will not be reported back to Perfion.
Logging
There are two types of logging implemented in Shopify Connector: log file and log in Perfion.
Shopify Connector Log File
This log file will have information about everything what is happening in Shopify Connector during synchronization manager check of configuration and also when performing actual product and category synchronizations. The location of the log file and how detailed logging will be, one can define in Shopify Connector configuration file. Each Shopify Connector instance will have its own log file.
There is one log file created each day and there are no log file cleanup procedures, so if disk storage is limited, then users have to cleanup old log files manually.
Log in Perfion
Shopify Connector can report some of log related data back to Perfion. The information reported to Perfion is, however, limited and only information related to synchronization of individual product or category items is logged. If during synchronization something wrong will happen with one product, then Shopify Connector will be able to send this information back to Perfion and save it to the Perfion feature configured to be used with SyncMsg mapping parameter. The information will be saved as part of the original product data, which failed in synchronization. Therefore, if there is something wrong with one product data, then after synchronization, all information of what went wrong with that particular product, will be available for users as part of that product’s data.
The log information per synchronization entity item will be updated each time synchronization runs and selects that item for synchronization. For example, if a product is chosen to be synchronized based on its sync parameters. If the product failed to synchronize for the first time, but was synchronized later without any errors, then log related information will be handled accordingly. On error it will be update with new error and warning messages, and on success, any old log information will be removed.
SyncMsg parameter can be defined for product and for category. Each log information written to SyncMsg related feature will be related to the actual product or category. Any information sent to Perfion will be also written to Shopify Connector log file, so for full information what is happening one may need to refer to this file. Even though the information provided for each synchronization item is limited, it allows to detect most of data related issues without leaving Perfion.
If ECommerce data is used with more than one channel, then logging will work in the same way, but we recommend to define independent SyncMsg parameter features for each channel to avoid interference. If two or more channels are synchronizing the same entity item and there is only one SyncMsg feature defined for all channels, then each Shopify Connector instance will overwrite data set by other Shopify Connector instances, which synchronize other channels.
Shopify Connector Basic Concepts and Error Handling
The Shopify Connector was designed to run forever. Any errors happening inside Shopify Connector should not stop it completely. In most cases it will report the errors to the log file and will continue its designed cycle of life. For example, restarting PC where Shopify Connectors run should be no problem.
Error Handling
Any errors while retrieving configuration data from Perfion and also any ECommerce configuration parsing validation errors are considered critical errors in Shopify Connector and synchronization manager will not start any synchronizations until all required data is in order.
The synchronization will stop if there are any errors while retrieving initial information about data in Perfion and in Shopify before the actual synchronization of product and category items. For example, when product synchronization starts, Shopify Connector tries to get all necessary data from Perfion to know which products it has. The similar procedure is repeated for Shopify products, so it can compare both and find differences. Any failures in this process will be treated as critical errors and will stop synchronization.
If synchronization initialization is successful and Shopify Connector starts, let’s say, product synchronization, then product items will be processed independently. This means that if there is any failure with one product, it will not directly affect synchronization of other products. Any errors happening within a single product item synchronization process will be logged independently to Shopify Connector log file and also back to Perfion. Note, the Connector’s log file will have more information about any errors or warning detected while performing synchronization.
Shopify Connector Recovery from Failure
The Shopify Connector was designed to recover from any crash and it is also safe to restart at any time. It will download the newest configuration data and will continue synchronization. In order for it to do it automatically, the Shopify Connector must be set up as Windows service.
In case the last synchronization failed, let’s say at 50%, then after it was restarted, it will continue further from where it left, because any item synchronization in Shopify Connector is final. This means that after each entity item is synchronized with Shopify, its status will be updated back to Perfion. The next time Shopify Connector will read data, it will no longer pick the same items, which were synchronized previously.
NOTE: regardless of what are predefined synchronization intervals, the first synchronization will run immediately after Connector first start, unless synchronization is disabled using parameters in Perfion.
Shopify Connector was designed so, that synchronization can run at any time regardless of what are the synchronization intervals. The Perfion is considered as master of data and therefore (in theory) the more often the synchronization runs, the closer data in Shopify is to the data in Perfion. If data synchronization is not desired at some point in time, then active measures must be taken to set up configurations in Perfion not to perform any updates at that time interval. For example, if some existing data is currently being updated (modified) in Perfion and should not be sent to Shopify yet until modification/update is complete, then users have to set those particular items not to be synchronized. If this is not done, then users may experience situations, when data is synchronized with Shopify in the middle of modification process. There are many ways to make sure that some data is not sent to Shopify.
Independent Item Error Handling
All requests sent to ECommerce API and Shopify API may fail for various reasons, e.g. because of timeout or error can be received because of invalid data sent to API, etc. Shopify Connector is designed to handle these errors by using configurable timeout intervals for both APIs and also by using number of retries configurations for both APIs. These settings can be set in Shopify Connector configuration file (refer to “Shopify Connector Configuration File”).
By default Shopify Connector is configured to retry three times after any failure while sending or receiving data to/from ECommerce API or Shopify API. There are various types of errors which may happen when sending and receiving data from both APIs and timeout and retry parameters allow to mitigate most of them.
Shopify Connector will report about all problems which happen during sending or receiving of data to the log file, but it will report errors about individual failed items only in a case, when after predefined number of failed request, it could not finish the job. That means that only the last known error will be reported back to Perfion, while log file will have all error messages, which were received each time it tried. Moreover, each sent request has its unique auto-generated ID, which allows to find information about all related requests and responses. Note: to access this information, logging level must be set to “Info” or “Debug” level.
Individual Item Synchronization
Individual item synchronization starts when synchronization manager initializes it based on configurations it has.
Individual item synchronization consist of two parts: category synchronization and product synchronization.
In order to handle product relations to categories correctly, it is required that all categories are up to date and also that all categories known to Perfion exist in Shopify. Therefore, before starting product synchronization Shopify Connector will always perform category synchronization. Category synchronization and product synchronization are independently controlled using ECommerce configurations, so in cases when categories are not used, this step will be skipped.
Regardless if it is category or product synchronization, the typical procedure of synchronization is more or less standard:
Task # | Task | Description |
1 | Get reduced entity data from Perfion | Retrieves information from Perfion about all available entity items. Only limited data of each entity item data set is downloaded. For example it reads entity item key, sync parameters and some other important information, which will be used to understand the current status of data in Perfion. |
2 | Get reduced entity data from Shopify | Retrieves information from Shopify about all available entity items. Only limited data of each entity item data set is downloaded, for example, item ID and if the item is active (published). |
3 | Compare data | Compares the data received in task #1 and task #2 and finds out which data has to be synchronized. The result after processing is three lists of entity items: items for creating new, items for updating existing and items for deletion. During this process the data will be selected based on various rules. Refer to “Synchronization Algorithm” for more information how items are selected. |
4 | Create new items | The task will create all selected items as new data in Shopify. |
5 | Update existing items | The task will update all selected items in Shopify. |
6 | Delete not existing items | The task will delete all selected items from Shopify. |
Synchronization Algorithm
All data in Perfion related to synchronization must have a unique key (defined by KeyField in ECommerce mappings for product and category entities). Any items retrieved via ECommerce API from Perfion which do not have a key, will not be used in synchronization. The items with duplicate key will also be rejected.
The similar check is also performed for SyncID field, which is used to store entity item’s ID value from Shopify. There can be no duplicate entity items in Perfion, which use the same SyncID value. If any of these items are found, they will be removed.
Before synchronization Shopify Connector retrieves information about all available entity items from Perfion and also from Shopify. It retrieves only basic information needed to determine if item has to be synchronized. For entity item data from Perfion it includes all sync type parameters and also the parameter, which shows if item is published or not. Items from Shopify will be retrieved with item IDs and parameter, which shows if item is published or not.
In the first step Shopify Connector will use a common algorithm for all entity types to select preliminary items for further processing. This selection does not take into account any other item data but instead validates item keys and sync IDs and creates three list of items: items for creation, for update and for deletion. The table below shows how entity items will be selected to these three lists.
Condition | Action |
SyncID value in Perfion is not set or SyncID value is set, but the Shopify ID value was not found in any of items retrieved from Shopify | The item is added to the list of items for creation in Shopify |
SyncID value in Perfion is set and it matches one of Shopify item IDs | The item is added to the list of items for update in Shopify |
Shopify item ID is not found in any of Perfion item SyncID values | The item is added to the list of items for deletion from Shopify |
The second step in determination which item has to be synchronized uses entity specific rules and apply them to each item in the lists selected in the first step.
Order | Condition | Action | Reason |
Context: New category creation rules, Data source: The list of categories for creation | |||
1 | If Shopify Connector is not set to allow creation of new items, (parameter SyncType does not have “Add” option, refer to “ECommerce Data Mapper – Settings”). | No Sync | Shopify Connector is not allowed to add new items to Shopify. |
2 | If the item’s SyncCollection value is set to “false”. | No Sync | The category item is selected not to be used in Shopify. |
3 | Always. | Sync | In any other case a new category will be created in Shopify. |
Context: Existing category update rules, Data source: The list of categories for update | |||
1 | If Shopify Connector is not set to allow update of existing items, (parameter SyncType does not have “Update” option, refer to “ECommerce Data Mapper – Settings”). | No Sync | Shopify Connector is not allowed to update any existing data in Shopify. |
2 | If the item’s SyncCollection value is set to “false”. | No Sync And Add item to deletion list | The category is selected not to be used in Shopify. Since the category already exist in Shopify, it must be removed and therefore, it will be added to the list of categories for deletion. Note: category items for deletion list is processed the last. |
3 | If the item is marked as not published in Perfion, item is marked as not published in Shopify and EC Settings parameter UpdateUnpublishedItems is set to “false”. | No Sync | There is no need to update deactivated items in Shopify, which cannot be used anyway. However, in some cases Shopify allows to see how deactivated items may look if they were activated and in this case one can overwrite this rule by setting EC Settings parameter UpdateUnpublishedItems to “true”. |
4 | If the item’s SyncDate value is not set. | Sync | Missing SyncDate value is a designed way to force item synchronization. |
5 | If the item’s SyncDate value is set, but DataMaxAgeInMin parameter is not used (refer to “ECommerce Data Mapper – Settings”). | No Sync | DataMaxAgeInMin parameter controls how often data has to be refreshed in Shopify. If it is not set, then the category will not be periodically synchronized with Shopify and users will need to manually select categories for synchronization, e.g. by removing the item’s SyncDate value. |
6 | If the item’s SyncDate value is set, and DataMaxAgeInMin parameter is set (refer to “ECommerce Data Mapper – Settings”), then Shopify Connector will check SyncDate value. If date is not older than maximum allowed data age. | No Sync | Data refresh is enabled, but the time did not come yet for the current item to be re-synchronized. |
7 | Always. | Sync | In any other case the category item will be updated in Shopify. In this case it will be a category item, which has expired date. |
Context: Existing category deletion rules, Data source: The list of categories for deletion | |||
1 | If Shopify Connector is not set to allow deletion or deactivation of Shopify items, (parameter SyncType does not have “Delete” or “Deactivate” option, refer to “ECommerce Data Mapper – Settings”). | No Sync | Shopify Connector is not allowed to delete or deactivate existing items in Shopify. |
2 | If Shopify Connector is set to deactivate Shopify items (parameter SyncType has “Deactivate” option, refer to “ECommerce Data Mapper – Settings”) and the item in Shopify is not published. | No Sync | The Shopify Connector is allowed to deactivate Shopify items, but the item in Shopify is already deactivated, so there is no reason to deactivate again. |
3 | Always. | Sync | In any other case the category item will be deleted or deactivated in Shopify. |
Context: New product creation rules, Data source: The list of products for creation | |||
1 | If Shopify Connector is not set to allow creation of new items, (parameter SyncType does not have “Add” option, refer to “ECommerce Data Mapper – Settings”). | No Sync | Shopify Connector is not allowed to add new items to Shopify. |
2 | If the item’s SyncType parameter is set to “NoUpdate” value (refer to “Product SyncType Parameter”). | No Sync | The product item is used in Shopify, but marked not to be synchronized. |
3 | If the item’s SyncProduct value is set to “false”. | No Sync | The product item is selected not to be used in Shopify. |
4 | Always. | Sync | In any other case a new product will be created in Shopify. |
Context: Existing product update rules, Data source: The list of products for update | |||
1 | If Shopify Connector is not set to allow update of existing items, (parameter SyncType does not have “Update” option, refer to “ECommerce Data Mapper – Settings”). | No Sync | Shopify Connector is not allowed to update any existing data in Shopify. |
2 | If the item’s SyncType parameter is set to “NoUpdate” value (refer to “Product SyncType Parameter”). | No Sync | The product item is used in Shopify, but marked not to be synchronized. |
3 | If the item’s SyncProduct value is set to “false”. | No Sync And Add item to deletion list | The product is selected not to be used in Shopify. Since the product already exist in Shopify, it must be removed and therefore, it will be added to the list of products for deletion. Note: product items for deletion list is processed the last. |
4 | If the item is marked as not published in Perfion, item is marked as not published in Shopify and EC Settings parameter UpdateUnpublishedItems is set to “false”. | No Sync | There is no need to update deactivated items in Shopify, which cannot be used anyway. However, in some cases Shopify allows to see how deactivated items may look if they were activated and in this case one can overwrite this rule by setting EC Settings parameter UpdateUnpublishedItems to “true”. |
5 | If the item’s SyncDate value is not set. | Sync | Missing SyncDate value is a designed way to force item synchronization. |
6 | If the item’s SyncDate value is set, but DataMaxAgeInMin parameter is not used (refer to “ECommerce Data Mapper – Settings”). | No Sync | DataMaxAgeInMin parameter controls how often data has to be refreshed in Shopify. If it is not set, then the product will not be periodically synchronized with Shopify and users will need to manually select products for synchronization, e.g. by removing the item’s SyncDate value. |
7 | If the item’s SyncDate value is set, and DataMaxAgeInMin parameter is set (refer to “ECommerce Data Mapper – Settings”), then Shopify Connector will check SyncDate value. If data is not older than maximum allowed data age. | No Sync | Data refresh is enabled, but the time did not come yet for current item to be re-synchronized. |
8 | Always. | Sync | In any other case the product item will be updated in Shopify. In this case it will be a product item, which has expired date. |
Context: Existing product deletion rules, Data source: The list of products for deletion | |||
1 | If Shopify Connector is not set to allow deletion or deactivation of Shopify items, (parameter SyncType does not have “Delete” or “Deactivate” option, refer to “ECommerce Data Mapper – Settings”). | No Sync | Shopify Connector is not allowed to delete or deactivate existing items in Shopify. |
2 | If Shopify Connector is set to deactivate Shopify items (parameter SyncType has “Deactivate” option, refer to “ECommerce Data Mapper – Settings”) and the item in Shopify is not published. | No Sync | The Shopify Connector is allowed to deactivate Shopify items, but the item in Shopify is already deactivated, so there is no reason to deactivate again. |
3 | Always. | Sync | In any other case the product item will be deleted or deactivated in Shopify. |
NOTE: for each context in the table the rules are checked in defined order until the condition is matched exactly. If rule is not matched, then the rules will be checked until one does. In each context, the last rule is the one which will be always matched, e.g. it does not have any condition.
Product Update
The table below shows how product update is performed and how many requests each task requires. The tasks described in this table are also used for product creation, custom collection update and custom collection creation. The difference in between different entity updates or creations will be only in which tasks are used for each. The product update is the most complex procedure of them all, so it uses all tasks, while other entities will use only some of these task. Refer to following sections to see which of tasks shown in the table will not be used with other entities.
Task Nr. | Task | Requests (source) | Conditions | Description |
#-1 | Get category keys and IDs | 1 (ECommerce API) 1 or more (Shopify API) | If categories are set up for synchronization | This is a preparation task for all other following tasks and it is performed only once. If categories are set up for synchronization, then before product synchronization the category data is retrieved from Perfion and Shopify to know which categories are available in Shopify and Perfion. Product synchronization will use only those categories, which are known to Perfion and to Shopify, so if category synchronization failed for some categories (e.g. some new categories were not created in Shopify during category synchronization), then product synchronization will know nothing about them and will not create any relations in Shopify for those categories (e.g. in task #9). NOTE: Shopify does not allow to read all available data at once, so how many times data will be read from Shopify depends on how many items there is to read. The default Shopify data read page size is 250 items per request. |
#0 | Get a chunk of data from Perfion | 1-2 (ECommerce API) | Always performed | This is a preparation task for all other following tasks. A chunk of data is retrieved from Perfion with a full data set. Chunk size is user configurable (default is 25 items). All downloaded items will be independently processed in parallel, while tasks from #1 to #11 for each item will be executed sequentially. At the same time Shopify Connector will start retrieval of next chunk of data from Perfion in parallel to item processing, so that when all items have finished processing, the next chunk of data is available without any extra delays. When this tasks is started for the first time, it sends two request, while any extra chunk retrievals use only one request. |
#1 | Get all metafields | 1 (Shopify API) | If metafields are used | Received metafield data is used to prepare item data with new and modified metafields, which then is sent to Shopify in task #4. All metafields for deletion are also determined at this task, which then are handled later in task #10. |
#2 | Variant data retrieval from Shopify | Number of variants * 2 + 1 (Shopify API) | If variant data is chosen to be kept as it is in Shopify | Variant data update from Shopify. During product update all variants are recreated using data only set up in Perfion. In case user chose to keep existing variant data by setting EC Settings KeepExistingVariantData parameter value to ‘true’, the variant data will be retrieved from Shopify in this task. Connector will get all product variant data which takes 1 request, then Inventory Item data for each variant and finally, the Inventory Level data for each Inventory Item. |
#3 | Product data fix | 1-2 (Shopify API) | If existing product data is not compatible with new product data | If product variant option position was changed or if updated product will have less options than the current product in Shopify, then product options and variant data must be reset, which may take up to two extra requests. This is a rare case. |
#4 | Update data without images | 1 (Shopify API) | Always performed | If product has variants and images, then product cannot be created using only one request. To create a product with variants, which have images, one must know image IDs and variant IDs in Shopify. This means that first images has to be created, which then can be linked to variants, or variants has to be created first and then images added at the same time adding references to variants. In this step the basic item data is added/updated in Shopify (including variants and variant options) and the next task (#5) will handle any image related updates. |
#5 | Update images | 1 (Shopify API) | Performed if product or product variants has images and images are set to be synchronized | Even though image presence require to send two requests in order to set data correctly in Shopify, both requests do not take more time than it would take by sending all data in one request. The overhead time needed to send an extra request is insignificant for a product update which has images. |
#6 | Get all collects | 1 (Shopify API) | Performed if product to category relations are used | Collect is Shopify object, which links Shopify product to Shopify custom collection. This task will get all known relations (collects) from Shopify and will determine what corrections has to be done so that product has all relations as it is set up in Perfion. |
#7 | Update Inventory Item data | Number of variants (Shopify API) | Always performed | Inventory Item entity is variant data extension entity and is always updated. |
#8 | Add/Update/Delete Inventory Level data | Number of Variants * (Number of locations + 1) (Shopify API) | Performed if Variant (Inventory Item) is tracked | Inventory level data has information about variant quantities and there is 1 entity for each location. For each variant first all available inventory levels are retrieved from Shopify and then the difference is handled by synchronizing (adding missing, removing not used and updating existing). |
#9 | Add/Delete collects | Number of changes needed (Shopify API) | Performed if product to category relations are used and there are any changes | Adds and deletes collects if there are any changes as compared to current product in Shopify. Each operation will use independent request. |
#10 | Delete metafields | Number of metafields to delete (Shopify API) | If metafields are used, and if there are any for deletion, and if keeping of deleted metafields option is not enabled | Deletes metafields one by one. Each metafield deletion will use independent request. |
#11 | Update Perfion | 1 (ECommerce API) | Always performed | Updates synchronization related parameters in Perfion. In case of error, the information about error will be updated (SyncMsg) and in case of success, other parameters will be updated (SyncID, SyncDate, SyncType, SyncMsg). It also depends in which task the process fails. If a new item was successfully created in Shopify, but, let’s say, deletion of metafields failed, then item synchronization will be marked as failed, but item’s Shopify ID will still be saved in Perfion. In this case, the next time the synchronization is performed, it will start with update of existing item instead of starting from scratch. |
NOTE: tasks #-1 and #0 are preparation for synchronization tasks and tasks from #1 to #11 are independent item processing tasks.
New Product Creation
Creation of a new product is a simplified operation as compared to product update procedure (refer to “Product Update”). Creation of new product will perform only the following tasks from product update table: #-1, #0, #4, #5, #7, #8, #9 and #11.
Product creation does not require to cleanup old data, or update variant data, so tasks #1, #2, #3, #6 and #10 are not required.
The task #9 will not delete any collects when new product is created, because collects does not exist in Shopify until the new product is created and collects are manually added. At this task, only new collects will be added.
Custom Collection Update
Update of custom collection is very similar to product update (refer to “Product Update”), but since custom collections are less complex objects, it does not have to perform some of tasks shown in product update table. It will perform tasks: #0, #1, #4, #10 and #11.
Task #-1 is a preparation task, which is used only with product synchronization.
Custom collections does not have variants and may have only one image, so tasks #2, #3, #5, #7 and #8 are not required.
Custom collection references to products (collects) are handled only from product data synchronization, so tasks #6 and #9 are not performed. Deletion of collects are automatically cleaned up by Shopify after custom collection is removed. If the new custom collection is added, then it does not have any product relations yet and product update will handle any relation updates if necessary. If custom collection data is updated, then it does not affect anyhow its relation to products.
New Custom Collection Creation
Creation of a new custom collection is a simplified operation as compared to product update procedure (refer to “Product Update”). Creation of a new custom collection will perform only the following tasks from product update table: #0, #4 and #11.
Task #-1 is a preparation task, which is used only with product synchronization.
Creation of a new custom collection does not require to cleanup old data, so tasks #1, #6 and #10 are not required.
Custom collections does not have variants and may have only one image, so task #2, #3, #5, #7 and #8 are not required.
Task #9 is not required, because product to category relations are handled only via product synchronization.
Product Deletion or Deactivation
The deletion or deactivation of product uses only one request. After deleting the main entity item Shopify will cleanup all related data and will remove any custom collection to product relations (collects) and will also automatically remove any related metafields.
In case deactivation is used instead of deletion, then instead of delete request Shopify Connector will update product “publish” parameter value to “false”. In this case, all the extra data belonging or related to this product (collects and metafields) will remain as they were before deactivation.
Custom Collection Deletion or Deactivation
The deletion or deactivation of custom collection uses only one request. After deleting the main entity item Shopify will cleanup all related data and will remove any custom collection to product relations (collects) and will also automatically remove any related metafields.
In case deactivation is used instead of deletion, then instead of delete request Shopify Connector will update custom collection “published” parameter value to “false”. In this case, all the extra data belonging or related to this custom collection (collects and metafields) will remain as they were before deactivation.
Synchronization Performance
Synchronization performance is directly related to how fast the data can be handled via ECommerce API and Shopify API.
ECommerce API has no predefined limits, but it is not designed to return quick responses for record by record requests. Its main task is to combine many types of data in some special format acceptable for ECommerce applications and also to return data in larger quantities. ECommerce API uses Perfion API for raw data request to get data from Perfion, then it combines that data and finally sends response back to user.
Shopify Admin API is designed to handle many requests and responds promptly, but it has usage limitations, which limits how much data can be requested or sent to Shopify. Shopify uses a so called “leaky bucket” algorithm to control the amount of calls users can use in a period of time. Simplifying how their algorithm works, one can say in conclusion, that Shopify allows two API calls per second (or four, if one uses Shopify Plus account). Refer to https://help.shopify.com/api/getting-started/api-call-limit for more information of how it works.
The Shopify limits mentioned above are more theoretic limits and does not take into account how much time each request-response takes. For example, if product has multiple images, then a single request to Shopify to update this product can take more than three seconds. If data is sent to Shopify sequentially, then it is difficult to exceed the limits specified by Shopify. This, however is not good enough in order to transfer large amounts of data to Shopify, because it does not allow data handling in bulk mode, e.g. to create several products in one request. To handle data in Shopify one needs to work with independent entities and sometimes some entities need more than one request to be properly created/updated. This leads to situation, where update of single entity item, e.g. product, takes three or more request to Shopify.
Shopify Connector handles all request to Shopify and also to ECommerce API in parallel to increase performance. Instead of sending one product update request to Shopify, which may take more than three seconds each, the Shopify Connector will send up to 20 request to Shopify to update multiple products at the same time. Moreover, Shopify Connector will control amount of requests it sends to Shopify in order not to violate its data transfer rules. This allows to send more data to Shopify and to maximize the utilization of allowed number of requests.
Shopify Connector handles data from Perfion in chunks, e.g. it takes 25 products from ECommerce API and then processes all those products one by one in parallel mode. Then it takes another 25 products, etc. This allows to optimize performance, because while data is sent to Shopify, the data from ECommerce API can be retrieved in parallel without wasting any additional time.
The overall data synchronization performance in Shopify Connector, in most cases, depends on Shopify performance. There is no universal figure to show how much time data synchronization will take, because it all depends on how much data is sent to Shopify. The main factors influencing synchronization time:
How large images are used and how many of them are used in each entity. A simple product update takes under one second in Shopify, but any additional image will slow down update significantly. If large images are used and there are many of them, one should not expect fast synchronization performance.
How images are transferred to Shopify.
If images are defined using URL, then Shopify will need to make additional requests to external server(s) to download those images. That means that if the product has 10 images, then Shopify will have to make another 10 requests before it can complete product creation/update.
If images are embedded in request using Base64 encoded inline data (binary data in an ASCII string format), then Shopify will not need to take additional several requests to get data from external server(s), but then the request with such data may take more time to send to Shopify. Moreover, Base64 encoded inline data takes more space than raw binary data, which can be downloaded directly from external image server. According to Wikipedia, Base64 encoded binary data uses about 137% of the original data size.
If user have chosen to keep the existing variant data in Shopify when updating product (by setting ECommerce Settings KeepExistingVariantData parameter to ‘true’), then each product update will take many extra requests to Shopify to get existing variant data. In this case Connector will need to retrieve all product variants, all variant inventory items and all variant inventory levels from Shopify in order to have all necessary data for update.
If metafields are used. If product or custom collection metafields are used, then Shopify Connector will need minimum one additional request to Shopify for each entity item to read existing metafields before they could be updated. Moreover, if there are any metafields, which has to be deleted for product or custom collection and Shopify Connector is not set up to keep deleted metafields, then deletion of each metafield will take one request to Shopify. If 10 metafields has to be deleted, then metafield deletion time may take longer than total time of typical product data synchronization.
Inventory level (variant quantity data for single location) data update. If variant quantity data is used, then each product update will require several requests to be sent to Shopify in order to update all quantities for all variants and for all locations. The number of required requests will be: number_of_variants * (number_of_locations + 1). If there are 5 variants and 5 locations, then Connector will need to send 30 requests to Shopify.
If product to custom collection relations are used and how many relations for each product there are. If product to custom collection relations are enabled, then product synchronization will use minimum one additional request to Shopify to get information about existing relations. Moreover, any addition of new relation or removal of old relation will take an extra request to Shopify. So if product was related to one category and then it was assigned to two new categories, then updating this information will take three extra requests to Shopify. One request to get information about existing relations, and then two more to add two new relations.
Number of channels used at the same time. If two or more instances of Shopify Connector are used at the same time to synchronize several Shopify stores, and all are actively synchronizing data, then there could be some delays from ECommerce API. ECommerce API retrieves the same data from Perfion for all channels and handles all requests synchronously. This does not add any significant performance issues, but it depends on data quantity in Perfion and in some cases it could affect overall performance. Multiple channels does not affect so much Shopify performance, because each channel belongs to independent Shopify store and all data transfer limits are based per Shopify store.
The most time is used when the entity item has to be updated in Shopify, because then Shopify Connector must check what data exist in Shopify and handle all changes. Refer to “Product Update” to see how product update works and how many requests it takes to update its data. In order to update products which have a lot of changed data, the procedure could be speedup by replacing products (removing old one and then creating it again as a new one) instead of updating. When product is deleted from Shopify, then Shopify will handle all related data deletion, which is much faster than trying to do this from Shopify Connector. This, however, have implications, that product gets a new Shopify entity ID and one cannot keep the old product data (e.g. deleted metafields or existing variant data cannot be used for update).
Image Handling
Products, variants and custom collections may have images in Shopify. Custom collection and variant may have only one image, while products may have many.
The images can be in .png, .gif or .jpg format.
Product and variant images in Shopify are product images. The only difference is that variant may have only one image, which is defined as one of product’s images. Practically product and variant can share the same image in Shopify. Shopify allows the total amount of 250 product and variant images.
The product and variant images are handled differently in Perfion. Products and variants in Perfion are separate entities and each have their own images. This have some limitations of how images can be organized, but it also offers some benefits.
For example, the same image cannot be shared by product and variant if data is synchronized from Perfion, but one can achieve the same effect by not defining product image at all and instead defining only variant image. Then, the image of the first variant will become a product image, which will be also a variant image.
Since product and variant images are controlled separately in Perfion, then it is possible to have more than one image for each product variant. Shopify Connector allows to utilize this concept. Read more about it in “Using Multiple Images for Variants”.
Product and variant images support metafields in Shopify, but Shopify Connector cannot use metafields, because of how image data is handled by ECommerce API.
Image Data Configuration in ECommerce Mapping
Images for Shopify Connector can be defined by using different ECommerce Output Kinds and multiple parameters can be added to ECommerce Mapping feature to combine images of different types (URL vs embedded) at the same time. Each image related parameter can represent single or multi-value image or file type features, and it is also allowed to use a simple string feature with single or multi-value URL values as a source for images. The example of how one can configure multiple types of images for product is shown in the Figure 3.
Figure 3 shows all ECommerce Output Kinds which can be used to define images, but there can be multiple features used for each Output Kind. All feature names, defined in ECommerce Mapping column must be unique. One can also use a literal value with “Field” Output Kind, but then, only one image mapping can be defined in such way.
Another important fact is, that all images, which are defined using URLs must be publically accessible from Shopify. Refer to “Supported Image Types” for more info.
If images are defined using several Output Kinds, then images will be handled in predefined order based on Output Kind:
Order | Output Kind | Description |
1 | Image(URL) or Image(Embedded) | Images defined by URL or embedded images have the same priority and will be handled in the order they are defined. Images can also be defined using other Image Output Kind types, e.g. by using patterns. Please refer to “ECommerce API” manual for more information for all available options and how to set them up. |
2 | Field | Field Output Kind can use only image URL to define an image. It also allows using literal value definition. Any images defined by Field Output Kind will be handled in the order they are defined. |
3 | Attachment(URL) or Attachment(Embedded) | Attachment Output Kind is directly related to Perfion feature, which has File base type. Not all files are images, therefore one should make sure that data specified using Attachment Output Kind is an image supported by Shopify. Shopify Connector do not check the content of the binary data and will send data as specified further to Shopify, which may issue an error if the data is not in correct format. Images defined by file URL or embedded files have the same priority and will be handled in the order they are defined |
Supported Image Types
There are two types of images one can import to Shopify:
By using URL of an image. Only absolute URLs can be used and images must be publically accessible.
By using Base64 encoded string of an image.
When images are imported to Shopify, it will save all images locally. If image data in Shopify was added by using URL, then Shopify will retrieve data from remote image server where that image is hosted and will save internally in its own image server. Therefore, it is important to make sure that all images sent to Shopify by using URL are publicly accessible. If they are not, then Shopify will not be able to retrieve those images and will issue an error. If image URLs are not publicly accessible, then one can use embedded images instead.
If images are defined using Base64 encoded inline data (binary data in an ASCII string format), then Shopify will not need to take additional request to get data from external server, but then the request with such data may take more time to send to Shopify. Base64 encoded inline data takes more space than raw binary data which can be downloaded directly from external image server. According to Wikipedia, Base64 encoded binary data size is about 137% of the original data size or to be more precise: Base64 encoded image size = original binary image size * 1.37 + 814 bytes.
Category Image Handling
There can be only one image assigned to custom collection in Shopify, but Perfion allows using multiple images for categories. The category images are selected in the same way as for products and variants using the concept described in “Image Data Configuration in ECommerce Mapping”.
Even though one can use multiple images in Perfion and also set up ECommerce mappings to use multiple images, the Shopify Connector will use only the first found image and will also issue a warning if there are more than one image specified for the category. Therefore, it is recommended to use only one image for categories in Perfion.
Using Multiple Images for Variants
Shopify supports only one image for variant, but Perfion controls variants independently and allows to have multiple images for each variant. Since the ECommerce API does not limit how many images can be transferred for each variant, the Shopify Connector has an extra feature, which allows to send more than one variant image to Shopify.
Images for product and variant are configured the same by using different types of mappings as it is described in “Image Data Configuration in ECommerce Mapping”. In order to allow multiple images for each variant, one has to set variant’s SyncAllImages parameter value to “true”. This will allow Shopify Connector to add all found variant images as product images and they will all be added together, e.g. before images for other variant are added. The Shopify will not “officially” use all variant’s extra images as variant images and only the first image will be assigned to the variant. However, Shopify uses templates for showing products and one can modify the standard template to manage product images differently. Figure 4 shows how Shopify product images with multiple images for single variant will look like in Shopify store.
Product Image Order Determination
Shopify allows specifying product image positions how they will appear in Shopify store, e.g. which product image will be the first, the second, etc. Perfion does not support this feature, because ECommerce API does not support adding extra data to images.
Moreover, the product images in Shopify consist of product and variant images. Perfion controls product and variant images independently, and therefore, there is no simple way to control all product image positions explicitly. The table below shows in which order images will be handled in Shopify Connector and then sent to Shopify.
Order | Image Source | Description |
1 | Product | All product images are added first. Product images are handled in the order which is specified in “Image Data Configuration in ECommerce Mapping”. |
2 | Variant #1 | One or multiple images from the first variant are added. If multiple variant images are used, then they are handled in the order which is specified in “Image Data Configuration in ECommerce Mapping”. |
3 | Variant #2 | The same as for Variant #1 |
… | … | … |
N | Variant #(N-1) | The same as for Variant #1 |
NOTE: the order in which variants are selected is not fixed in Perfion. It depends on various parameters specified for features, which store product and variant data. By default, the variants will be handled in Shopify Connector in the same order they are delivered by ECommerce API. To know more about how ECommerce API handles variant data, refer to “ECommerce API” manual. There is, however, a possibility to define positions for each variant by using variant Position parameter. The Shopify Connector will use this parameter to reorder variants in ascending order before they are sent to Shopify.
NOTE: even though images are sent to Shopify in some predefined order, Shopify store may show them in different order. For example, if product has three images and it has two variants, then by opening general product page Shopify will select data for the first found variant and will show its image, even though there are several other images defined for that product which have higher position in Shopify. The images in Shopify’s product image gallery will be presented in correct order (e.g. in the same order they are added to the product by Shopify Connector), but Shopify will define which image of those available will be shown as large image. This depends on Shopify configuration and is a subject for change.
Image Caching Setup
Ecommerce API uses Perfion API to deliver image content. Before using images with Shopify Connector, one has to setup ECommerce parameters in Perfion and also make sure that image/file caching works correctly.
In ECommerce Settings add correct image and file services related parameters.
ImageServiceUrl and FileServiceUrl parameters. These parameters define URLs to Perfion image and file services. These URLs are used when Perfion API handles images or files (one can use images as attachments with Shopify Connector and attachments are handled via file services, so file service setup is as necessary as image service setup). If URLs are not accessible, then ECommerce API may fail. It is also important to know that if Perfion image or file services are slow, then it will directly affect ECommerce API response times. The ECommerce API responses may also fail if image or file services are not accessible.
An example of ImageServiceUrl parameter value: http://localhost:8080/perfion/image.aspx.
For more details about these parameters refer to “ECommerce API” manual.
ImageServiceUrlParameters, ImageHeight, ImageWidth and OutputFolder parameters. These parameter are not so important for Shopify Connector, but they are related to image and file services and may also be useful to know about. Refer to “ECommerce API” manual for more information.
Setup image and file services to use caching. This is part of Perfion API setup and has to be done in the server where Perfion API is installed. The caching is enabled by default, but one must make sure that Perfion API service user has necessary write access rights to the Perfion API cache folder. The procedure below shows how to add Perfion API service user with write access rights to cache folder’s properties:
Go to the folder, where Perfion API is installed and open Perfion\Cache folder properties (right click with a mouse on the “Cache” folder and choose “Properties”).
Example of Perfion API cache folder location: C:\inetpub\wwwroot\PerfionAPI\Perfion\Cache.
Open “Security” tab and add a new user.
In “Select Users, Computers, Service Accounts, or Groups” window click “Locations…” button.
Select the top location and click OK.
In “Enter the object names to select” text box enter the user name of Perfion API site application pool, which is set up in IIS (Microsoft Internet Information Services).
The format: "IIS Apppool\<Perfion API app pool name>" (without quotes)
Example: "IIS Apppool\PerfionAPI_Shopify"
Click "Check Names" button. If the name was specified correctly, it should be resolved and if not, then a new "Name Not Found" window will open.
Once the user name is specified correctly and found, click OK button.
Select added new user from the list and allow full control permissions. Click OK.
More information about application pool identities: https://docs.microsoft.com/en-us/iis/manage/configuring-security/application-pool-identities .
Test if image service cache works.
Find any image type feature in Perfion which has an image or create one for test purposes (it can be deleted after the test).
Select the image and open its preview. This can be done by right clicking with mouse on image icon (in grid or in item editor) and choosing “View Image…” from context menu.
A new “Preview” windows will open, which will show the image and also some extra image parameters. In the bottom left corner there is image GUID. This is image ID and it will be used when testing how cache works.
Open a web browser and use the following URL to see if the same image from Perfion is shown in web browser.
The format of URL: http://<PerfionApiSiteAddress>/Perfion/image.aspx?id=<image_GUID>
Or: <ImageServiceUrl>?id=<image_GUID>
Example: http://localhost:8080/Perfion/image.aspx?id=da00e137-9932-4ebf-9a7a-ba6e81a51555
Check if a copy of image was saved in the image cache folder. Go to Perfion API image cache folder, e.g. “\Perfion\Cache\Images”. In that folder there should be a copy of the same image, which was shown in the browser. The name of the cached image has the same image GUID in its file name.
Example of cached image file name: da00e137-9932-4ebf-9a7a-ba6e81a51555_#_#_#_True_False_#_image2.jpg
Metafield Handling
Metafields are special parameters in Shopify, which can be added to the main Shopify entities to allow users to save extra data which cannot be saved otherwise using standard Shopify parameters for each entity.
Metafields in Shopify are assigned to the main entities such as product, custom collection (category), variant and image, but they are a separate entity and are stored separately from main entities. This affects how they can be added, modified and removed in Shopify and they do not work like normal parameters of each entity. The differences of how metafields are handled in Shopify Connector based on to which entity they belong is shown in the table below.
Entity | Parent Entity | Keep Deleted Metafields Support | Description |
Product | - | Yes | Products in Shopify can use metafields and Perfion supports it with some limitations (refer to “Metafield Limitations”). |
Category | - | Yes | Custom collections in Shopify can use metafields and Perfion supports it with some limitations (refer to “Metafield Limitations”). |
Variant | Product | Yes | Variants in Shopify can use metafields and Perfion supports it with some limitations (refer to “Metafield Limitations”). |
Product Image | Product | - | Product images in Shopify can use metafields, but Shopify Connector does not support image metafields. ECommerce API at the moment cannot return extra data for images. |
Variant Image | Product | - | Variant images in Shopify can use metafields, but Shopify Connector does not support image metafields. ECommerce API at the moment cannot return extra data for images. Variant images in Shopify are product images with an extra reference ID saved in variant data and therefore they are handled (including metafields) the same as product images. The only difference is that variant images are stored with variant data in Perfion. |
Category Image | Category | - | Custom collection may have only one image and Shopify does not allow using metafields with that image. |
Refer to “Product Update” to see how metafields are handled for product update. Product creation, custom collection update and custom collection creation uses the same procedures as product update.
Variant metafields are handled differently in synchronization as compared to, for example, product update, because variant data is always recreated in Shopify during new product creation or existing product update.
Metafield Format
Metafields are complex parameters as compared to typical product or custom collection parameters. The typical parameter has a name and a single value or multiple values. Metafields, however, are defined by using 5 parameters, where 4 are mandatory:
Namespace – Parameter must have min 2 characters, max 20 characters and together with Key parameter defines the unique combined metafield “Key”. Allowed characters are A-Za-z0-9_.~-@#*$. Namespace is case sensitive.
Key – Parameter must have min 3 characters, max 30 characters and together with Namespace parameter defines the unique combined metafield “Key”. Allowed characters are A-Za-z0-9_.~-@#*$. Key is case sensitive.
Type – defines metafield value type. The values are shown in the table below.
Shopify API version 2021-07 introduced breaking changes and the previously supported 2 types were deprecated by replacing them with new types. The table below shows the currently supported types and how they relate to the previously used types. The type values are case insensitive.
Value – The actual value of metafield. The requirements for a value will depend based on metafield type.
Description (optional) – This parameter is optional and defines metafield value description.
The unique combined metafield “Key” is required to be unique within given Shopify entity. For example, the product may have several metafields with Key parameter value “Instructions”, but then Namespace parameter value for each must differ. For example, metafields “Installation”-“Instructions” (Namespace-Key) and “Washing”-“Instructions” will be unique.
Type | Old Type | Description |
String <no value> | string | A string (default). In case the type is not defined or defined using the old ‘string’ type, then Shopify ‘string’ type will be used by default. NOTE: the Shopify API after changing metafield type specifications starting from version 2021-07 introduced some breaking changes. The Shopify API by default is documented that it no longer uses String type and metafields of this type will be treated as Date type metafields. Date type metafield values have very strict requirements and in most cases could lead to invalid metafields. However, the metafields when retrieved via Shopify API still support String type even officially based on Shopify documentation String type is not available. Therefore, Perfion will continue using String type metafields just like before version 2021-07. Example: “Some text” |
SingleLineText
| string | A single-line text field. Example: “Some text” |
MultiLineText
| string | A multi-line text field. Example: “Item list: Table Chair” |
PageRef | string | A reference to a page on the online store. This is a special Shopify URL. Example: “gid://shopify/OnlineStorePage/1” |
ProductRef | string | A reference to a product on the online store. This is a special Shopify URL. Example: “gid://shopify/Product/1” |
VariantRef | string | A reference to a product variant on the online store. This is a special Shopify URL. Example: “gid://shopify/ProductVariant/1” |
FileRef | string | A reference to a file on the online store. This is a special Shopify URL. Example: “gid://shopify/MediaImage/123” |
Integer Int | integer, int | A whole number in the range of +/-9,007,199,254,740,991. Example: “10” |
Decimal | string | A number with decimal places in the range of +/-9999999999999.999999999. Example: “10.4” |
Date | string | A date in ISO 8601 format without a presumed time zone. Example: “2021-02-02” |
DateTime | string | A date and time in ISO 8601 format without a presumed time zone. Example: “2021-01-01T12:30:00” |
Url | string | A URL with one of the allowed schemes: https, http, mailto, sms, tel. Example: “https://www.shopify.com ” |
Json | string | A JSON-formatted string. Example: “[{ "k": "v1" }, { "k": "v2" }]” |
Bool | string | A true or false value. One can also use numbers 0 and 1. Example: “true” |
Color | string | The hexadecimal code for a color including leading # char. Example: “#fff123” |
Weight | string | A value and a unit of weight. Valid unit values: oz, lb, g, kg. This is a JSON-formatted string. Example: “{ "unit": "kg", "value": 2.5 }” |
Volume | string | A value and a unit of volume. Valid unit values: ml, cl, l, m3 (cubic meters), us_fl_oz, us_pt, us_qt, us_gal, imp_fl_oz, imp_pt, imp_qt, imp_gal. This is a JSON-formatted string. Example: { "unit": "ml", "value": 20.0 } |
Dimension | string | A value and a unit of length. Valid unit values: in, ft, yd, mm, cm, m. This is a JSON-formatted string. Example: { "unit": "cm", "value": 25.0 } |
Rating | string | A rating measured on a specified scale. This is a JSON-formatted string. Example: { "value": "3.5", "scale_min": "1.0", "scale_max": "5.0" } |
Metafield Limitations
Perfion ECommerce setup allows defining only simple (name vs. value) parameters and one cannot define complex metafield parameter without specifying it in some special way. In order to support metafields, the special metafield definition was chosen. The metafields are defined in ECommerce setup as normal parameters, but metafield parameter name and value must be formatted following some rules.
Metafield name format |
Metafield|<Namespace>|<Key>|<Type> |
Where:
Metafield – is a fixed keyword which tells to Shopify Connector that it is a metafield. Case insensitive.
Namespace – is the value of metafield Namespace parameter. Case sensitive.
Key – is the value of metafield Key parameter. Case sensitive.
Type – defines the type of the metafield. This should be one of the values form the table with supported metafield types. Case insensitive.
Some examples of metafield names are shown in the table below:
Metafield | Metafield Parameter Name in “ECommerce To” Mapping | Description | ||
Namespace | Key | Type | ||
Global | Size | Metafield|Global|Size | Normal SingleLineText type metafield | |
Global | Size | string | Metafield|Global|Size|string | Normal SingleLineText type metafield |
Global | Size | SingleLineText | Metafield|Global|Size|SingleLineText | Normal SingleLineText type metafield |
Global | Size | integer | Metafield|Global|Size|integer | Normal Integer type metafield |
Global | size | string | Metafield|Global|Size|string | Invalid metafield definition, the case sensitivity rule is violated. Note that such error will not be reported by Shopify Connector. |
global | title_tag | string | Metafield|global|title_tag|string | Special product metafield |
global | description_tag | string | Metafield|global|description_tag|string | Special product metafield |
Metafield value may consist of 2 parameters and therefore, it must be specified using 2 EC Mapping values where one mapping is created for the metafield value and the other mapping is created for metafield value description.
To define the metafield value one has to create a normal Field type EC mapping where:
ECommerce Mapping column should have the literal metafield value which will be used with all entity (category/product/variant) items or the feature name which will specify the metafield value for each entity item independently. Refer to the Figure 5 to see an example.
ECommerce Output Kind column value should be set to “Field”.
ECommerce To column value should define the metafield name.
ECommerce Entity column should define the entity name to which the metafield belongs.
To define the metafield value description one has to add another mapping. The metafield value description is an optional field, so this has to be done only if metafield value description has to be used. The definition is very similar to how the metafield value was defined. There are 2 differences:
ECommerce Output Kind column value should be set to “Attribute”.
ECommerce To column value should be set to the same name as for the metafield value mapping, but with added suffix “@description”. Refer to the Figure 5 to see an example.
Metafield Synchronization Process
Metafield synchronization with Shopify depends on the entity they belong to.
Variant is not tracked in Perfion independently and therefore variant data (including metafields) is always replaced with a new variant data even if the data have not changed. Even though the variant data is always recreated, the Perfion Connector has a feature which allows keeping the existing variant data in Shopify even if it does not exist in Perfion anymore. This allows any 3rd party tools to update variant data which is not available in Perfion. To enable this feature one must enable EC Settings parameter KeepExistingVariantData. In addition, one must set the product parameter SyncKeepDeletedMetafields to “true” in order to keep the metafields in Shopify which are not defined in Perfion. If the feature which allows to keep the current variant data is activated, then the variant data will still be recreated in Shopify, but Shopify Connector will first read the data from Shopify, merge it with the data available in Perfion and create that data again in Shopify.
Product and custom collection metafields are synchronized in the same way using this process:
Before product or custom collection synchronization all metafields for each entity are retrieved from Shopify.
Metafield data is updated in Shopify together with the data of their parent entity, therefore, the data received from Shopify is used to determine which metafields has to be updated, which has to be added and which has to be deleted.
New and existing metafields data are added to their parent entity data and are synchronized all at once together with their parent entity.
Deleted metafields must be handled independently in Shopify. That means that all metafields, which has to be deleted, will be deleted one by one. This process, however, is optional and one can chose to skip it by setting SyncKeepDeletedMetafields parameter to “true” value for product and/or category.
Product special metafields can be handled in two different ways in Shopify. It allows to define these metafields directly as fixed product’s data, but these metafields technically are no different than any other product metafield and can also be updated independently. Shopify Connector will synchronize these special metafields together with all other metafields and does not support the other method where data can be defined directly.
Shopify Inventory System
Shopify have recently changed how product variants are handled in their stores and the newest Shopify Connector version supports it. The new system is significantly more complicated than the one before and users should be aware how to properly use it.
The old product variants had a single quantity parameter and that value was valid for any locations where the variant was sold. The old system was like a physical store would have, e.g. in that physical store the same variant will have only one quantity definition. For example, there are 10 “cups” for sale in store “A”.
The new inventory system, however, introduced a more complex configuration, where product variant got some dimensions and it is not any longer treated as a variant in one physical store, but rather as a global product variant which can be sold in multiple stores. So now the single product variant is defined as: 5 “cups” for sale in store “A”, 3 “cups” for sale in store “B” and 2 “cups” for sale in store “C”. The definition of the variant is still the same, but now it may have multiple quantities which belong to different Shopify locations. Even though the new system is more complicated, the Shopify users can still continue to use only a single location like before. For example, the product variant in Perfion is still defined almost in the same way as before, but in the new system one may define multiple quantities instead of 1.
Even though the difference from user perspective is not so big, the structural changes in Shopify how the data is handled become significantly more complicated. The new inventory system in Shopify looks like shown in Figure 6.
As shown in Figure 6, the “Product Variant” block shows how the old product variant looked like in the old system and product quantity was a single parameter defined directly in “Product Variant” block. In the new system each product variant have got a new entity called “InventoryItem” and each inventory item could be related to multiple locations via so called “InventoryLevel” entity. The quantities in the new system are now saved in “InventoryLevel” block.
From Perfion perspective, when one has to properly configure a product variant for synchronization, the following has to be done:
Any locations used to control product variant quantities must be defined in ECommerce Settings mappings as reference locations (“RefLoc_*” parameters).
Any parameters related to product variant, which in new inventory system were expanded into multiple entities, will still be configured as before in ECommerce Variant mappings.
NOTE: in the new system several new parameters were added, such as “Cost”, “Tracked” and various country related code parameters. There were also some old parameters moved from the main product variant entity to Inventory Item entity, e.g. “SKU” parameter.
ECommerce Variant mappings configuration have slightly changed for product variant quantity definition. Before there was a single quantity field and now there can be more than 1. In order to be able to map all data correctly, Perfion uses a special quantity assignment format. The parameter name for quantity definition has the following format - “Quantity_<reference_location>”. Here “reference_location” is the name of reference location as it is specified in ECommerce Settings mappings, e.g. using “RefLoc_*” parameters.
Even though the configuration from Perfion side is rather simple, one still has to understand how Shopify inventory system works in order to define data correctly. For example, Shopify uses fulfillment services and if one enables such service, then product variant will be able to handle only a single quantity value. If in such case there will be more than a single quantity defined in variant mappings, then Shopify will not allow to update variants. It is strongly recommended to learn how the new inventory system works in order to understand the following:
What is fulfillment service and how inventory system works when such service is enabled
How normal locations differs from fulfillment locations
How switching between normal locations and fulfillment locations work
Most of ECommerce Variant mapping parameters have names which refer to parameters used by Shopify in Product Variant, Inventory Item and Inventory Level entities. In case there are some questions how to set those parameters or what they mean, please refer to Shopify documentation, e.g. “https://help.shopify.com/en/api/reference/products/product-variant ” and “https://help.shopify.com/en/api/reference/inventory ”.
To read more about Shopify inventory system please refer to “https://help.shopify.com/en/api/reference/inventory/inventorylevel ”.