Best practices when moving your operations data to a new home

One of the most important and, very often, labour intensive tasks when moving operational systems is your data migration.  This is usually due to the sheer volume of data being moved between systems and also the cleanup of the data prior to importing it.  Most business do not realise the amount of data they already have and where to find it so if you know where all of your data is kept and can access it easily then that is a great head start on the process.

ServiceTracker has managed many successful data migrations over the years with varying obstacles and difficulties overcome with each migration.  No two migrations are ever the same and if the process is not structured properly then there can be things that come back to bite you later on down the line.

Here are a few helpful tips to get you started on your data migration journey…

1. Identify the data you want to migrate

Choose which information you wish to migrate and identify the sources of this information. Information could be held in various systems, filing cabinets, email, various computers throughout the office or, as we have experienced, even in the boot of your car!  The first step in migrating your data is to take stock and decide which information you are going to use and the information you are going to archive if you should ever need to access it in the future

For example, current customer’s Contact information, Contractual information, scheduling, last known visit information, documents etc should all make it on to the moving list however customer’s that have not traded with you in perhaps 10 years may want to be archived?

However you undertake this task, at the end of it you should have a clear understanding of your data and the volume you will be transferring over to your new system.

2. Scrub the data

One of the most common mistakes when importing data into a new system is to simply dump it all in and try to clean it up later.  This does not go down well with users of the system and often the cleanup never takes place.  Moving to a new system means a fresh start and involves some form of cleanup operation to ensure there are no duplications, addresses of your customers are correct, telephone numbers are entered etc.  You have the ability to see where key information has been missed and go back over it to import clean and concise data..

3. Decide what is important to you

At this point you have identified missing information and it is time to rectify mistakes.  You can see any information that has not been entered correctly into your previous system and decide early on what is very important to you. For example, you may be looking at a long list of customers with no email address.  You can decided that in future no records can be created without an email address.  Write these requirements down ready for the migration and also for the first step in customising your system

4. Choose a method for importing your data

This task is generally managed by a consultant or your IT department however it is worth noting the available tools on the market for data import as it will help you understand the process, how long it should take and the costs incurred.  Some systems  have built in tools to import data whilst others rely on third party software to manage the data transfer.  There are many ways to move the data back and forth and it is important to note any limitations along the way such as the number of records that can be imported, the length of data etc.  

5. Create templates for your data

Once you have decided on the method of importing your data you will, in many cases, have to create a data import  template.  MS Excel is a good tool to start with as you can export records as CSVs, a widely used format for most data imports. The best way to create a template is to perform a data export from your new system using reporting tools or any data migration tools.  This will provide you with the field names, structure and example data to use when mapping your data into the new system

Identify the required fields for each table (mark these in red on your spreadsheet for easy viewing) and also identify relationships between the data (for example a contact may need to relate to a business before it is imported). These relationships dictate the order of data migration. For example, you should load the Business record first, then the contacts, then the Contractual data.

6. Test your templates

Review the data in your templates to make sure you have captured everything you need before populating the system with a small number of records. For example: Load one record, check the results, then load all records if you are successful.  This will save you time having to re-do the template if any data fails to import.

7. Migrate the data

Ensure that the data mapping is correct and you have the correct tables and fields in the system to match your data set. Your consultant or IT specialist will usually be able to create missing fields in the system to store legacy ID information to help maintain relationships and help you build custom reports for validation later on..

8. Validate the data

Use these techniques to validate your migration:

  • Create reports that validate record counts and provide an overall snapshot of migration.
  • Spot check the data.
  • Review exception reports to see what data was not migrated.​

 

With all of these steps mastered you are on your way to a very successful data migration but remember the most important rule of all; Always keep a backup just in case there are any problems post migration that need to be rectified or you need to refer back to your data in future.

If you would like to know more about how ServiceTracker have completed many successful data migrations or how our products streamline the migration process then please get in touch on 033 022 31022 or email us at info@servicetracker.uk.com