Data Migration

The EasyMetaHub Harvester provides a fast, easy, and reliable process to move data from relational databases to NoSQL databases. The process of moving data can be accomplished in days as opposed to weeks or even months.

The most common process is currently:

1. Run manual SQL queries
2. Dump data into CSV files on disk
3. Manually load data, one table at a time, into one document per row structure
4. Write code to merge dependent entities into a single document
5. Analyze source data
6. Write code to insert the reference data
7. Write libraries to cleanup source data
8. Write data enrichers
9. Write code to format and store canonical documents

These processes are labor, time and resource heavy. Another potential problem is that you will not know if the data is clean or dirty until well into the process. 

EasyMetaHub Harvester can perform the process in a tenth the time or less. It also determines the quality of the data before the data is ingested into the NoSQL database. You want to make sure that your data lake does not become a data swamp!

1. Harvest model from RDBMS with statistical data
2. Analyze and validate data (before loading any of it)
   1.  Add any missing referential integrity using the graphical display 
   2.  Analyze the data through the statistics and random samples 
3. Score the data
4. Create industry standard schemas of the target data (XML, JSON or RDF schemas)
5. Harvest the data as canonical documents