Distributed Parallel ETL Load processing
-
I'm not sure if this is the correct forum to post this in but here it goes. I'm working on designing a more robust ETL process for our Enterprise Data Warehouse. The idea is, there would be two servers set up as primary and secondary dispatchers along with (n) number of client listeners. I would like to be able to have each listener perform some SQL Query and/or a stored procedure using the resources on that specific machine in an effort to maximize both ETL process speed and resource utilization. My question is: Is this a situation where I could/should use a Parallel Data Warehouse architecture? I wouldn't really want, let's say, 20+ computers all performing rather large queries against the same SQL Server instance... that would negate any performance gains I may get from a Distributed architecture. Thank you for reading this and I appreciate any information you could provide.
I hate users. Not all of them, just the ones who talk.CP member: Al Einstien