Sync between a local and remote Dataset
-
I am looking for an help in the best way to accomplish a sync between two dataset of the same type. Meaning I have a LOCAL and REMOTE version of the database and I'm needing to sync them nightly. I have a web service that will accept a dataset and update the REMOTE version but because of a large amount of data I have a LOCAL copy that represents the REMOTE version. I first parse in data from flat files into a RAW virsion and need to sync data between the RAW and LOCAL versions, ( RAW will trump the LOCAL). I then use a web service to pass ONLY the changed data to the REMOTE version of the database. Overall I'm looking of and easy, processor and memory friendly way to sync this data. Any and all ideas on the topic would be helpful.
I'm listening but I only speak GEEK.
-
I am looking for an help in the best way to accomplish a sync between two dataset of the same type. Meaning I have a LOCAL and REMOTE version of the database and I'm needing to sync them nightly. I have a web service that will accept a dataset and update the REMOTE version but because of a large amount of data I have a LOCAL copy that represents the REMOTE version. I first parse in data from flat files into a RAW virsion and need to sync data between the RAW and LOCAL versions, ( RAW will trump the LOCAL). I then use a web service to pass ONLY the changed data to the REMOTE version of the database. Overall I'm looking of and easy, processor and memory friendly way to sync this data. Any and all ideas on the topic would be helpful.
I'm listening but I only speak GEEK.
foreach ( dsLogPro.CompaniesRow row_Raw in ds_Raw.Companies.Rows ) { dsLogPro.CompaniesRow found_Local = ds_Local.Companies.FindByCompanyCode( row_Raw.CompanyCode ); if ( found_Local != null ) { if ( !found_Local.Equals( row_Raw ) ) { // raw will trump local foreach ( DataColumn col in ds_Local.Companies.Columns ) { found_Local[ col ] = row_Raw[ col ]; // update local with raw } } } else { // this must be a new row found_Local = ds_Local.Companies.NewCompaniesRow(); foreach ( DataColumn col in ds_Local.Companies.Columns ) { found_Local[ col ] = row_Raw[ col ]; // update local with raw } ds_Local.Companies.Rows.Add( found_Local ); } } This is the direction I'm currently moving towards. I don't believe the tables will change but the columns might change in the future so I'm really trying to keep this flexable. I'm listening but I only speak GEEK.
-
I am looking for an help in the best way to accomplish a sync between two dataset of the same type. Meaning I have a LOCAL and REMOTE version of the database and I'm needing to sync them nightly. I have a web service that will accept a dataset and update the REMOTE version but because of a large amount of data I have a LOCAL copy that represents the REMOTE version. I first parse in data from flat files into a RAW virsion and need to sync data between the RAW and LOCAL versions, ( RAW will trump the LOCAL). I then use a web service to pass ONLY the changed data to the REMOTE version of the database. Overall I'm looking of and easy, processor and memory friendly way to sync this data. Any and all ideas on the topic would be helpful.
I'm listening but I only speak GEEK.
private void CompaniesCompare() { rowsAdded = 0; rowsDeleted = 0; rowsUpdated = 0; start = DateTime.Now; // This will update and add the rows m_parentProcess.OnPostMessageEventHandler( this, new MigrationEventArgs( "\tcomparing company data." ) ); foreach ( dsLogPro.CompaniesRow row_Raw in ds_Raw.Companies.Rows ) { dsLogPro.CompaniesRow found_Local = ds_Local.Companies.FindByCompanyCode( row_Raw.CompanyCode ); if ( found_Local != null ) { if ( !found_Local.Equals( row_Raw ) ) { // raw will trump local foreach ( DataColumn col in ds_Local.Companies.Columns ) { found_Local[ col ] = row_Raw[ col ]; // update local with raw } rowsUpdated++; } } else { // this must be a new row found_Local = ds_Local.Companies.NewCompaniesRow(); foreach ( DataColumn col in ds_Local.Companies.Columns ) { found_Local[ col ] = row_Raw[ col ]; // update local with raw } ds_Local.Companies.Rows.Add( found_Local ); rowsAdded++; } UpdateProgressBar(); } // This will remove un-needed rows foreach ( dsLogPro.CompaniesRow row_Local in ds_Local.Companies.Rows ) { dsLogPro_Live.CompaniesRow found_Raw = ds_Raw.Companies.FindByCompanyCode( row_Local.CompanyCode ); if ( found_Raw == null ) { // was not present in the raw so remove it. ds_Local.Companies.Rows.Remove( row_Local ); rowsDeleted++; } UpdateProgressBar(); } } This is what I have ended up with and would like someone elses opinion on the code. I'm listening but I only speak GEEK.