I have a problem in a SSIS solution !!!
-
I have a problem in a SSIS solution that is i have many flat file data sources "Text Files" want to load them in a SQL SERVER Database as a
destination, but flat files overridden every day with new data and i want only to take new data , so how can i take the new data ??? -
I have a problem in a SSIS solution that is i have many flat file data sources "Text Files" want to load them in a SQL SERVER Database as a
destination, but flat files overridden every day with new data and i want only to take new data , so how can i take the new data ???How do you determine which data is new? I would likely import the data to a table that is designed for that purpose and then execute a separate SQL statement to move only the new data to the actual destination. The import area would get cleared out before each import.
-
How do you determine which data is new? I would likely import the data to a table that is designed for that purpose and then execute a separate SQL statement to move only the new data to the actual destination. The import area would get cleared out before each import.
-
I have a problem in a SSIS solution that is i have many flat file data sources "Text Files" want to load them in a SQL SERVER Database as a
destination, but flat files overridden every day with new data and i want only to take new data , so how can i take the new data ???The way we do this type of operation is to move the data into and archive folder and load from there. If a new file arrives in the hot folder it has not been loaded so move it and load from the archive folder. We also do NOT transform during the load, the target table exactly reflects the source table and all fields are varchar. We then use a stored procedure to do the transforms (launched by the SSIS package).
Never underestimate the power of human stupidity RAH
-
The source data is a text files that contain the attendance data empID, datetime and etc and every day each text file come with the new and old attendance data, i am using SSIS solution
Then it's easy.
-
The way we do this type of operation is to move the data into and archive folder and load from there. If a new file arrives in the hot folder it has not been loaded so move it and load from the archive folder. We also do NOT transform during the load, the target table exactly reflects the source table and all fields are varchar. We then use a stored procedure to do the transforms (launched by the SSIS package).
Never underestimate the power of human stupidity RAH
Another point on which we agree. :thumbsup:
-
Then it's easy.
-
See Mycroft's post. Clear out a temporary area, import all the data there, then copy only the new data to the final location.