How To Replicate Data Between Two Databases
-
Hi Folks, I have developed an inventory warehouse application (VB.Net UI) that uses a SQL 2000/2005 database. A Microsoft gold partner has asked me to seamlessly integrate my application with a Microsoft ERP application that also uses a SQL database. What I thought would be rather easy issue has really got me bogged down. I was wondering how to handle this bridge between the two applications. An example of the bridge would be: A sales order/Order Lines record(s) are inserted into MS ERP database table(s), this record(s) needs to be replicated into my database (Sales Order/Order Lines table(s)). The Schema’s of the table(s) are not the same. If a record(s) are delete/updated then these actions would also have to be reflected in my database. This would only be a one way replication, meaning my database can not perform inserts/Updates/deletes on the MS ERP database table(s). I have look at doing this at the database level using SQL replication of a view, but I would like to write a windows service that the user can define the particular table(s)/Field(s) to bridge. If you’re wondering why not just lookup the MS ERP database table(s) when needed, well part of the spec supplied to me was that the MS ERP database can be taken offline, but my application must still able to pick the outstanding ordered items that were in the ERP Database. I would be grateful for you comments.
Regards Ian
-
Hi Folks, I have developed an inventory warehouse application (VB.Net UI) that uses a SQL 2000/2005 database. A Microsoft gold partner has asked me to seamlessly integrate my application with a Microsoft ERP application that also uses a SQL database. What I thought would be rather easy issue has really got me bogged down. I was wondering how to handle this bridge between the two applications. An example of the bridge would be: A sales order/Order Lines record(s) are inserted into MS ERP database table(s), this record(s) needs to be replicated into my database (Sales Order/Order Lines table(s)). The Schema’s of the table(s) are not the same. If a record(s) are delete/updated then these actions would also have to be reflected in my database. This would only be a one way replication, meaning my database can not perform inserts/Updates/deletes on the MS ERP database table(s). I have look at doing this at the database level using SQL replication of a view, but I would like to write a windows service that the user can define the particular table(s)/Field(s) to bridge. If you’re wondering why not just lookup the MS ERP database table(s) when needed, well part of the spec supplied to me was that the MS ERP database can be taken offline, but my application must still able to pick the outstanding ordered items that were in the ERP Database. I would be grateful for you comments.
Regards Ian
If I am understanding you correctly, then you should just use a triggers on the ERP server and perform inserts, updates, and deletes accross databases. You can do something like the following: INSERT INTO DatabaseName.[owner].TableName (CompanyName, Phone) VALUES ('Snowflake Shipping', '(503)555-7233') - Where database name is, use your database name - Where owner name is, use the owner name of your database. The default is dbo - Where TableName is, use the table name on your database where you want to insert. If I am correct in my assumption, then this solves your problem of processing data across databases easily. However, The biggest problem you are going to run into is associating the records that are in the ERP database with your database. For that purpose, you should have ids that are in your database that corresponds to the ERP database. That way you can perform updates and deletes. If associating records through ids becomes a hassle, then you should probably use replication. Tyquaun -- modified at 23:55 Thursday 27th July, 2006
-
If I am understanding you correctly, then you should just use a triggers on the ERP server and perform inserts, updates, and deletes accross databases. You can do something like the following: INSERT INTO DatabaseName.[owner].TableName (CompanyName, Phone) VALUES ('Snowflake Shipping', '(503)555-7233') - Where database name is, use your database name - Where owner name is, use the owner name of your database. The default is dbo - Where TableName is, use the table name on your database where you want to insert. If I am correct in my assumption, then this solves your problem of processing data across databases easily. However, The biggest problem you are going to run into is associating the records that are in the ERP database with your database. For that purpose, you should have ids that are in your database that corresponds to the ERP database. That way you can perform updates and deletes. If associating records through ids becomes a hassle, then you should probably use replication. Tyquaun -- modified at 23:55 Thursday 27th July, 2006
Thanks for the quick reply Tyquaun, Your assumptions are correct. Your answer was on my list of possible ways to handle this task. I wanted to put this task out to the programming community to see what idea’s came back. I thought of using COM+ distributed transaction, but this would not work because its not my UI that initiates the transaction,MS ERP system does. The record id’s are not a problem because they do match the ERP database id's. What if my database is not running on the same server as the ERP database? Microsoft has three ERP applications and one of them does not use a MS SQL database, I just know they are going to ask me work with all three products after a period of time. It may be helpful to reads of my post if I throw my idea’s in here as well. 1. Use SQL Replication I am against this a bit because it almost needs the purchasing customer of my application to have a DBA on site to set this up or maintain the replication. 2. Place triggers on the appropriate tables of the ERP Database to replicate the required data to my database. 3. Use a COM+ distributed transaction (as I said above thinking this would not work) 4. Write a utility tool that would load the data from both databases into separate datasets then compare the ERP dataset (Master) to my dataset and replicate any ERP dataset (Master) changes. This utility could run as a windows service and load up the datasets every few minutes or so, I could also put UI into it that reads both the databases schema’s and lets an on site administrator select the table(s)/Column(s) to replicate. I sort of like this idea because the ERP database could be anything as long as there is an ODBC driver for it, but speed could be a problem. 5. Use MSSQL DTS tool. -- modified at 4:17 Friday 28th July, 2006
Regards Ian
-
Thanks for the quick reply Tyquaun, Your assumptions are correct. Your answer was on my list of possible ways to handle this task. I wanted to put this task out to the programming community to see what idea’s came back. I thought of using COM+ distributed transaction, but this would not work because its not my UI that initiates the transaction,MS ERP system does. The record id’s are not a problem because they do match the ERP database id's. What if my database is not running on the same server as the ERP database? Microsoft has three ERP applications and one of them does not use a MS SQL database, I just know they are going to ask me work with all three products after a period of time. It may be helpful to reads of my post if I throw my idea’s in here as well. 1. Use SQL Replication I am against this a bit because it almost needs the purchasing customer of my application to have a DBA on site to set this up or maintain the replication. 2. Place triggers on the appropriate tables of the ERP Database to replicate the required data to my database. 3. Use a COM+ distributed transaction (as I said above thinking this would not work) 4. Write a utility tool that would load the data from both databases into separate datasets then compare the ERP dataset (Master) to my dataset and replicate any ERP dataset (Master) changes. This utility could run as a windows service and load up the datasets every few minutes or so, I could also put UI into it that reads both the databases schema’s and lets an on site administrator select the table(s)/Column(s) to replicate. I sort of like this idea because the ERP database could be anything as long as there is an ODBC driver for it, but speed could be a problem. 5. Use MSSQL DTS tool. -- modified at 4:17 Friday 28th July, 2006
Regards Ian
What other database is the ERP using?
-
What other database is the ERP using?
-
It may be helpful to reads of my post if I throw my idea’s in here as well. 1. Use SQL Replication I am against this a bit because it almost needs the purchasing customer of my application to have a DBA on site to set this up or maintain the replication. 2. Place triggers on the appropriate tables of the ERP Database to replicate the required data to my database. 3. Use a COM+ distributed transaction (as I said above thinking this would not work) 4. Write a utility tool that would load the data from both databases into separate datasets then compare the ERP dataset (Master) to my dataset and replicate any ERP dataset (Master) changes. This utility could run as a windows service and load up the datasets every few minutes or so, I could also put UI into it that reads both the databases schema’s and lets an on site administrator select the table(s)/Column(s) to replicate. I sort of like this idea because the ERP database could be anything as long as there is an ODBC driver for it, but speed could be a problem. 5. Use MSSQL DTS tool
Regards Ian
-
It may be helpful to reads of my post if I throw my idea’s in here as well. 1. Use SQL Replication I am against this a bit because it almost needs the purchasing customer of my application to have a DBA on site to set this up or maintain the replication. 2. Place triggers on the appropriate tables of the ERP Database to replicate the required data to my database. 3. Use a COM+ distributed transaction (as I said above thinking this would not work) 4. Write a utility tool that would load the data from both databases into separate datasets then compare the ERP dataset (Master) to my dataset and replicate any ERP dataset (Master) changes. This utility could run as a windows service and load up the datasets every few minutes or so, I could also put UI into it that reads both the databases schema’s and lets an on site administrator select the table(s)/Column(s) to replicate. I sort of like this idea because the ERP database could be anything as long as there is an ODBC driver for it, but speed could be a problem. 5. Use MSSQL DTS tool
Regards Ian
Ian, I hope I am not too late. However, this is what I think. Maybe you should use the triggers as I suggested before. Also, for the databases that not SQL server. Create a windows service that will call a webservice and from there you can sync the data. Tyquaun
-
Ian, I hope I am not too late. However, this is what I think. Maybe you should use the triggers as I suggested before. Also, for the databases that not SQL server. Create a windows service that will call a webservice and from there you can sync the data. Tyquaun
Not too late Tyquaun, still working on the solution. Triggers in the past have always come back to bit me on the ... , but in this case I may have no choice. As for creating a windows service that will call a webservice, I'am currently working on some beta code now that may also bypass the need for using triggers. I will keep you posted, if I get something solid working. Reagards Ian
Regards Ian
-
Not too late Tyquaun, still working on the solution. Triggers in the past have always come back to bit me on the ... , but in this case I may have no choice. As for creating a windows service that will call a webservice, I'am currently working on some beta code now that may also bypass the need for using triggers. I will keep you posted, if I get something solid working. Reagards Ian
Regards Ian
Cool. Keep me updated.