Copying data from one MSSQL database to another

I have database that has data imported from another server using import and export wizard of SQL Server Management Studio. There is also empty database with same tables but it also has primary keys, foreign keys and indexes. How to get data from first database to another? Here is the description of my crusade. And believe me – it is not nice one.

Bugs in import and export wizard

There is some awful bugs in import and export wizard that makes data imports and exports possible only on very limited manner:

• wizard is not able to analyze foreign keys,
• wizard wants to create tables always, whatever you say in settings.

The result is faulty and useless package. Now let’s go step by step and make things work in our scenario.

Database

There are two databases. Let’s name them like this:

• PLAIN – contains data imported from remote server (no indexes, no keys, no nothing, just plain dumb data)
• CORRECT – empty database with same structure as remote database (indexes, keys and everything else but no data)

Our goal is to get data from PLAIN to CORRECT.

1. Create import and export package

In this point we will create faulty SSIS package using SQL Server Management Studio.

1. Run import and export wizard and let it create SSIS package that reads data from CORRECT and writes it to, let’s say, CORRECT-2.
2. Make sure you enable identity insert.
3. Make sure there are no views selected.
4. Make sure you don’t let package to create tables (you can miss this step because it wants to create tables anyway).
5. Save package to SSIS.

2. Modify import and export package

Now let’s clean up the package and remove all faulty crap.

1. Connect SQL Server Management Studio to SSIS instance.
2. Select the package you just saved and export it to your hard disc.
3. Run Business Intelligence Studio.
4. Create new SSIS project (DON’T MISS THIS STEP).
5. Add package from disc as existing item to project and open it.
6. Move to Control Flow page do one of following:
   1. Remove all preparation SQL-tasks and connect Data Flow tasks.
   2. Modify all preparation SQL-tasks so the existence of tables is checked before table is created (yes, you have to do it manually).
7. Add new Execute-SQL task as first task in control flow:
   1. Open task properties.
   2. Assign destination connection as connection to use.
   3. Insert the following SQL as command:
      EXEC sp_MSForEachTable ‘ALTER TABLE ? NOCHECK CONSTRAINT ALL’
      GO
      EXEC sp_MSForEachTable ‘DELETE FROM ?’
      GO
   4. Save task.
8. Add new Execute-SQL task as last task in control flow:
   1. Open task properties.
   2. Assign destination connection as connection to use.
   3. Insert the following SQL as command:
      EXEC sp_MSForEachTable ‘ALTER TABLE ? CHECK CONSTRAINT ALL’
      GO
   4. Save task
9. Now connect first Execute-SQL task with first Data Flow task and last Data Flow task with second Execute-SQL task.
10. Now move to Package Explorer tab and change connections under Connection Managers folder.
    1. Make source connection to use database PLAIN.
    2. Make destination connection to use database CORRECT.
11. Save package and rebuilt the project.
12. Update package using SQL Server Management Studio. Some hints:
    1. Make sure you take the package from solution folder because it is saved there now.
    2. Don’t overwrite existing package. Use numeric suffix and let Management Studio to create a new version of package.

Now you are done with your package. Run it to test it and clean out all the errors you find.

TRUNCATE vs DELETE

You can see that I used DELETE FROM instead of TRUNCATE. Why? Because TRUNCATE has some nasty limits (taken from MSDN):

“You cannot use TRUNCATE TABLE on a table referenced by a FOREIGN KEY constraint; instead, use DELETE statement without a WHERE clause. Because TRUNCATE TABLE is not logged, it cannot activate a trigger.

TRUNCATE TABLE may not be used on tables participating in an indexed view.”

As I am not sure what tables you have and how they are used I provided here the solution that should work for all scenarios. If you need better performance then in some cases you can use TRUNCATE table instead of DELETE.

Conclusion

My conclusion is bitter this time although I am very positive guy. It is A.D. 2010 and still we have to write stupid hacks for simple things. Simple tools that existed before are long gone and we have to live mysterious bloatware that is our only choice when using default tools. If you take a look at the length of this posting and the count of steps I had to do for one easy thing you should treat it as a signal that something has went wrong in last years.

Although I got my job done I would be still more happy if out of box tools are more intelligent one day.

References

• T-SQL Trick for Deleting All Data in Your Database (Mauro Cardarelli)
• TRUNCATE TABLE (MSDN Library)
• Error Handling in SQL 2000 – a Background (Erland Sommarskog)
• Disable/Enable Foreign Key and Check constraints in SQL Server (Decipher)