Overview

Perfion Actions is a powerful tool that allows you to:

• Extract, transform and load data from many data sources into Perfion

• Export data from Perfion and deliver it in many formats to external systems/channels

• Import files and images from hot folders

• Export files and (transformed) images to specific locations

• Manipulate data inside Perfion

• Send email notifications

• Execute scripts and applications outside of Perfion

Formats

The supported formats are:

• Excel

• Csv

• Xml

• Fixed Width text

• SQL

• Perfion API Query

Via custom Add-In's it is possible to extend the tool to support any proprietary format.

Structure

Almost every supplier has their own unique way to structure the data and name the columns or elements in their data documents. One supplier might call a value Quantity and another Qty, and you need this data to end up in the same feature in Perfion. Furthermore, the values may be in different units from different suppliers, and some suppliers may pollute the values with metadata, like e.g. including the unit together with the value, or even worse include multiple different types of values merged together.

In addition to this, the actual structure of the supplied data may vary. In Xml-format it is obvious that elements and values can be nested in myriads of ways, but even in a flatter Excel-format, data can be divided on multiple sheets, and is not necessarily positioned to start in the top-left corner of a sheet, and the direction of data could be both horizontal and vertical. So even with Excel, there are many options to deliver data.
With Perfion Actions it is easy to take data from any structure, and transform both the structure, data and metadata to a form that can be understood by Perfion.

Other tools

There are several tools in the market for Data Mapping, such as Microsoft Biztalk Server, Altova, Flowheater among others. We do not want to obsolete these tools, since some of these tools are almost "programing languages" and as such able to handle everything. The purpose of Perfion Actions is to make 95% of all mapping cases very simple and fast to define. Very difficult and obscure structures will need other tools.

Typical approach of other tools

Most other tools use an approach similar to what is displayed in the following figure.

At first glance it looks simple enough. Via simple lines you just have to map which elements from the input goes where in the output.
However, it is rarely ever this simple. Due to the aforementioned structural differences it usually ends up like something as displayed in the following figure.

It is clear that the above is not simple anymore.
Since the purpose of Perfion Actions is to be simple and fast to work with, we did not want to use the same approach, but rather rethink from scratch.

Perfion Actions approach

After analyzing a lot of different input structures and typical approaches to dealing with them, we found that one of the big flaws of the existing tools is the very thing that seems so simple at first sight.
The flaw is the ability to point to one or several elements in the input and tie these values directly to somewhere in the output. It would be the equivalent of a large city having no roads and traffic regulation, so cars could go directly across the fields (with bumps and holes along the way). It would look like a disaster and would be unsustainable.
The solution is to enforce highways and traffic controls into the approach.
This means that the Perfion Data Mapping approach will enforce a process, which includes restrictions on what is possible to do when and where in the various steps of the process.
Some of the important goals will be to:

• Eliminate crossing lines

• Collect small roads "lines" into highways

• Restrict value manipulation and validation to relevant stages in the process

Approach Overview

The following figure shows a high-level presentation of the Mapping process.

Selection

The first step in the process is called selection. In this step the only focus is to open the input-format and extract the desired information from the structure in the input-format. The output of this process is a flattened table of data in a uniform format that is much easier to deal with in the following steps.
Since especially Xml-structures can be nested in many ways, it will be difficult to select everything you want in one go. This is solved by allowing the selection to be divided into as many selections as you need, each of which will be merged in the same way to the final uniform result table.

Transform

For some selections it will be needed to transpose the data (e.g. because columns are represented in rows). Following each selection, it is possible to transform the selected data prior to joining them onto the final uniform result table.

Mapping

Now that everything we need is held in a uniform flat table structure, it will be very easy to map the data. For many columns it will be simple renaming.
For the most advanced columns, it will be to define the output column name, and link this to a single function that has access to all the input columns, and can do whatever it needs to do, to create the output value. This may sound advanced, however, it is 1 function, and it is only possible to execute at this particular place, so it is easy to identify which values depend on a function, and what function it is.
All formatting and validation is done against the mapped output columns, and this is the only allowed place to do it, so it is easy to identify what is manipulated and how.
A library of typical formatting and validation functions is available, and it is possible to extend the library with custom functions via the Perfion AddIn capabilities.

Mapping Example

Below figure shows an example of how simple a mapping from the BMEcat Xml-format is with Perfion Actions.

The lines between the boxes in the figure are wider to illustrate that they are actually carrying more than one field or element, which with a typical mapping approach would have been many confusing crossing lines, but here they are assembled into a "highway".
Which fields or elements are carried through the "highway" is illustrated with the next figure, where the top-left Select box has been opened, to show a mapping of how it selects data from the input-data source.

Perfion Actions Overview

Perfion Actions is a framework designed to perform any kind of processing on data. This is accomplished by Actions, Commands, Data Sources and Action Maps, which will be described below.

Actions

An Action is nothing more than a named container of some functionality that can be executed. The Action can be enhanced with extra metadata to describe it.

Commands

A Command is a pre-defined function that delivers a specific piece of functionality. What kind of functionality that can be accomplished by Perfion Actions, ultimately depends on the available Commands.
However, this framework allows us to continually enhance the capabilities by adding new relevant Commands.

Data Sources

A Data Source can be anything that a Command can understand as input or deliver as output. There is nothing in the framework that limits the types of Data Sources. However, it will of course only make sense to use sources that are understood by a Command.
The most often used types of Data Sources are:

• In-Memory Data-Table object

• In-memory Xml Document

• A File-Path (to data content)

• A Stream (of data content) this can only be used programmatically via API.

The In-Memory Data-Table and Xml document data types are the 2 most important types for holding data during processing. Which data type to use will mostly depend on the structure of each of the data sources you need to process. You can convert between the sources and perform lookups into the other sources.
The File-path and Stream data sources are generally only used by specific Input / Output Commands, to point to the source or destination of data in specific file formats, such as e.g. Excel, Delimited or Fixed Width Text data formats, Xml-files etc.

Data Sources Stack

The Actions framework creates a stack of named data sources, to keep track of all the sources that are created or used by any of the Commands during the execution of an Action.

Data Source Names

You freely decide the names of Data Sources used during processing. However, if no names are defined the Framework will automatically assign names.
Any unnamed 'From' source will automatically be named 'FROM'
Any unnamed 'To' source will automatically be named 'TO'
Most likely you will define your own names, but these automatic names are especially relevant to know about for a) the initial data source supplied when working in the Perfion Application, and b) for passing sources from the EXECUTE.ACTION Command to the Action it references.

Data Source Scope

Any Data Source will only exist while an Action is executing.
It is possible to define an Action to call other Actions. This can be done with the Command EXECUTE.ACTION, and will be described in further detail for this Command.
However, when Actions call other Actions, all Data Sources will not be visible to sub-Actions. One input Data Source can be passed to a sub-Action. If desired, it is also possible to define global sources by prefixing the name with a $-sign, such as e.g. $MyGlobalData, in which case they will be visible to any Action that is involved. If possible, to keep it simple, it is recommended to divide the functionality so sub-Actions only depend on one input Data Source.

Special Data Sources

The following Data Sources are used for specific purposes by the framework, so these names should be avoided in your maps.
$LOG – This source appears, and contains log entries if any are needed to report on the execution, such as e.g. warnings, failed items or complete fails and reasons.
$VARIABLES – This source contains a stack of all current variable values, if any are used within script code.

Action Map

In order to make an Action do anything, its Action Map must be defined. The Action Map is a simple generic way to describe a process of functional steps that will be executed.
Below figure shows a graphical example of an Action Map.

The Action Map consists of references to Commands and the Data Sources they need as input or generate as output. In the above graphical example, the Commands are shown with blue background and the Data Sources with green background.
Each Command may need extra information to guide its processing in the form of additional mapping, which can be added below the Command (as seen above under the Select Command).
The above graphic illustration is used as an example to explain the Action Map. When defined within Perfion a working example of the above would look as shown below:

In the example above, we see that the first line contains a Command named SELECT.XMLEXCEL. The purpose of this Command is to find and select data out of an Xml-Excel document. Furthermore, the output from the Command will be delivered to a Data Source named 'My1stRESULT', and since it does not specify an input Data Source it will try to get the Xml-input from a Data Source named 'FROM'. If this Data Source does not exist, the above example will fail. Blank values ImportData'. This is subsequently used in the next action command step IMPORT as the input data source which is typically how actions are constructed - the output from one command is fed in as the input of another.

Blank values for Data Sources will always be automatically named as 'FROM' and 'TO' for the From and To columns respectively. These names are also the entry and exit source-names within a sub-Action, so it is recommended to use blank values when defining sub-Actions to pass sources to and from a sub-Action.

Comments / Ignore

It is possible to tell the Action Map to ignore certain lines by having two '-' characters, as the first two characters in the From part of a Map-line. Anything on that line will then be ignored, whether it is a Command or any additional mapping for a Command. Please note commenting out is not inherited - commenting out a parent step will not apply to children of that step.

Installing Actions

Perfion Actions rely on pre-defined Perfion Features that have been designed to be the building blocks – the Lego® pieces, if you will – that you will use to build your map of actions. All features are auto-generated by the installation action have their names prefixed "ActionMap".
In order to perform the installation, you must be logged in as an Administrator.

Navigate to Tools > Settings > Integration and click Install for Actions Setup.

Note it is recommended you repeat this process after a major upgrade as this will ensure your system has any new functions or Action Commands.