Binary Data Migration
- Morten Lindequist Køhler
Binary data migration from one location to another is performed by Migration Manager which can be started in different ways:
When binary data storage location is changed the Migration Manager will be started automatically.
User can control the Migration Manager from Settings -> Binary Storage -> Migration Manager area where all controls for Migration Manager are located. Refer to Figure 8.
The Migration Manager will start automatically with Perfion Windows client for the administrator user.
Binary data migration is a safe process which can be started and stopped at any time.
Binary data migration is done in the background, so if it was started manually, then the Settings window can be closed and Perfion will continue to work as usual. The data migration will be done by moving binaries to the new location and once the binaries are moved, Perfion will seamlessly switch to using binary data from the new location. This will work globally, e.g. not just for the Perfion Windows client where Migration Manager runs. Perfion web client and media services will work seamlessly as well while binary data is under migration.
Migrating Binary Data to DB vs. Perfion Asset Cloud Storage
There are slight differences based on where from the binary data is moved to where. The binary migration (content copying) operation will be more or less the same, but the Cloud Storage is treated differently compared to DB, e.g. how the old data is cleaned up.
DB to Perfion Asset Cloud storage migration. Once the binary data is migrated to the Perfion Asset Cloud storage, the binary data content will be deleted from the database.
Perfion Asset Cloud storage to DB migration. When the binary data is migrated to the database, the binary data will never be deleted from the Perfion Asset Cloud storage and it will be always accessible. The binary data can only be deleted as part of binary data cleanup procedure. Refer to Cleanup of Binary Data in the Perfion Asset Cloud.
Running Multiple Instances of Migration Manager
The Migrator Manager is designed to coexist with other Migration Mangers if there are more than 1 running, e.g. in another Perfion Windows client. However, the Migration Manager is optimized for performance and running multiple instances of it could lead to various errors. For example, 2 independent Migration Managers could try to migrate the same binary data at the same time or the number of connections to Storage service could increase to a point where timeouts could occur for various data update requests which are made by Migration Manager. Such errors would lead to situation where a lot of resources will be wasted to handle conflicts by retrying the migration of particular binary data again and again which could lead to performance slowdowns. Therefore, it is recommended to run only a single instance of Migration Manager for flawless binary data migration process.
Migration Manager Status
Based on what the Migration Manager does there are several status types used.
Status type | Description |
- | The status is unknown. One must press the “Update Status” button to show the actual status. |
Stopped | Migration Manager is stopped, e.g. there are no binary data migrations running in the Perfion Windows client. |
Running | The Migration Manager is running. |
Stopping | The Migration Manager is stopping. Usually this happens when the “Stop” button is pressed or binary data location is changed. The stopping procedure could take some time and it usually involves finishing migrating binary data which was already under migration when the stopping command was received. Migration Manager processes multiple binaries at the same time in parallel and the time it takes Migration Manager to stop depends on how large the binaries are and how many are currently under migration. |
Cleanup | The cleanup process usually happens only after migration of binary data from the database to the Perfion Asset Cloud storage. At the time the cleanup process begins, the binary data is already fully moved from the database to the Perfion Asset Cloud storage and cleanup process only deletes the old binary data from Perfion database. The cleanup of old binary data can take a lot of time, e.g. even longer than migration process itself. However, this process is not so important for actual data migration and can be postponed if necessary. The cleanup process is important if the Perfion database size should be reduced after binary data is migrated to the Perfion Asset Cloud storage. Even though Perfion will not use the binary data which is still stored in the database, until the cleanup process is done the database cannot be compressed, e.g. to save the space or to simplify database backup. |
Note that Migration Manager status is not updated automatically. One must press the “Update Status” button to refresh the status.
The Migration Manager status is only reported for the current Perfion Windows client. If there are any other Perfion Windows client running and using Migration Manager, then the status of those other migrators will not be shown.
Migration Manager Progress
The Migration Manager shows the process of how many binaries are in different types of storage locations and how many are in failed state. The migration process is considered finished when all binaries are located in the preselected binary storage location.
Number of binaries | Description |
- | The status is unknown. One must press the “Update Status” button to show the actual status. |
Local | The number of binaries which are located in Perfion database. |
Cloud | The number of binaries which are located in the Perfion Asset Cloud storage. |
Failed | The number of binaries which are currently under migration and has some issue. Note that this status just indicates that some binaries have issues while being migrated, but the binary data itself is not affected in any way. The binary will still be accounted as in Local or Cloud storage and this status is only meant to show possible errors. Note also that the Failed binaries status will be updated while the Migration Manager is running, because it is setup to use multiple retries and in most cases all failed binaries will be migrated correctly. If multiple Migration Managers are running at the same time, then there could be many errors reported because of collisions, dropped requests, etc. Therefore, if multiple Migration Managers are running at the same time and they report a lot of errors, it is recommended to leave only 1 Migration Manager running. |
Cleanup of Binary Data in the Perfion Asset Cloud
Binary data in the Perfion Asset Cloud cleanup procedure is meant to allow deleting some old binaries from the Perfion Asset Cloud storage, because there is no other way to delete this data. The binaries are never deleted from the Perfion Asset Cloud storage even though the binary data location for binaries in some chosen Perfion database were selected to be stored in database. Perfion Migration Manager will migrate all the binaries from the Perfion Asset Cloud storage to the database, but original data stored in the Perfion Asset Cloud storage will never be deleted.
The binary data cleanup could be a dangerous procedure and there is a risk losing binary data so it must always be performed with caution.
The cleanup procedure deletes all binaries from the Perfion Asset Cloud storage which are not known to the current Perfion client which is running the cleanup process. This means that if there are multiple Perfion databases each with different setup and all of them uses the same Perfion Asset Cloud storage, then this cleanup procedure will keep all the binaries which are known to the current Perfion DB, but delete all binaries which belong to those other Perfion databases. The procedure will also delete all the binaries which are not known to any Perfion database, e.g. some old binaries which are no longer used by Perfion.
Before running the cleanup procedure check how many binaries there are for deletion. Press the “Check Status” button in the “Cloud Storage Cleanup” area and Perfion will show how many binaries it found which qualify to be deleted. Refer to Figure 8. Then, after double checking that everything is as expected one can press the button “Delete Binaries From Cloud” to perform the cleanup procedure.
Database Cleanup
If the binary data in Perfion was moved from the storage in the database to the Cloud storage, then the database will be cleaned up by removing the old binary data from the database by the Migration Manager. However, this may not affect the actual size of the Perfion database, e.g. how much space it physically occupies in SQL server. Therefore, after binary data migration process to the Cloud storage it is recommended to perform database “Shrink” procedure which will allow clearing unused space in database and will make it physically smaller.
To Perform database “Shrink” operation open SQL Management Studio, select Perfion database which has to be cleaned up and then from the context menu select Task -> Shrink -> Database and/or Task -> Shrink -> Files as shown in the Figure 9. Follow on screen instructions or Microsoft SQL Management Studio documentation for any further details about database “Shrink” operation.
