The best architecture solution for working with multiple data sources
-
You almost convinced me, but one last concern: With a single gateway you risk to get a monolythic solution (everything is stuffed into one place). What do you plan to prevent this? How to you keep good extensibility and maintainability?
-^-^-^-^-^- no risk no funk
For this reasons (performance, security,maintainability) we can use several instances of services. They can share common repository or have it's own. Besides data sources these services will be able to communicate with each other or any external web or wcf-services. In general we are going to achieve extensibility and maintainability by using some common model of business logic that will accommodate our needs.
-
rokford wrote:
To avoid this we decided to develop some "common gateway" - middle tier that will be used by all applications that need to access any data sources. Also we plan to implement WCF for this task.
I don't think I understand. You are going to run all queries and result sets through the gateway machine?
-
I like WCF for the communication, but I am concerned that you are looking to roll your own gateway mechanism. The problem with this approach is that you have to cater for all situations because you can guarantee that, sooner or later, somebody will want to do something that your gateway just isn't designed to do. More importantly, they will have a valid reason for doing this, and you will either have to extend your gateway or you will constrain what they can do. Another issue, is that there really does come a time when your application needs to know what the data source is. For instance, you may want to insert a record into a database that has an automatically incremented primary key - but not all databases support this, so you need to have some mechanism to generate the key for this database.
Please visit http://www.readytogiveup.com/ and do something special today. Deja View - the feeling that you've seen this post before.
From my point of view accessing a data source and retrieving data is the task that can be formalized quite well. ADO.NET is a good example of this. Unlike ADO.NET we plan to implement some middle tier that will common for all applications (developers). So the main task is to access different data sources using some abstract model. You should not think that application "knows nothing" about database - it knows enough to get required information from database. But there will not be connection strings, stored procedures ,etc. "Another issue, is that there really does come a time when your application needs to know what the data source is. For instance, you may want to insert a record into a database that has an automatically incremented primary key - but not all databases support this, so you need to have some mechanism to generate the key for this database." In our company we always use stored procedure instead of direct sql statements. So in this case there must be a procedure like "insert_something" that hides away how primary key is generated - with auto increment or using sequences.Of course,it's implementation will depend on what database is being using, but at the same time it will transparent to client applications. So we do not want to create some universal language for all databases- but organize all these databases (connection strings,user credentials,stored procedures,ets) is some abstract model and make it accessible by all applications using different protocols and communication technologies (due to WCF)
-
For this reasons (performance, security,maintainability) we can use several instances of services. They can share common repository or have it's own. Besides data sources these services will be able to communicate with each other or any external web or wcf-services. In general we are going to achieve extensibility and maintainability by using some common model of business logic that will accommodate our needs.
After reading the other posts, too, I think that you have a nice concept for your solution. Reminds me somewhat of CAB (Composite UI Application Block) where the CAB is the central part that coordinates all others, just like your Gateway coordinates DataSources and client applications. Best luck with your approach. And maybe you can provide us with some information how it worked out in a CodeProject article - would be very interesting.
-^-^-^-^-^- no risk no funk
-
From my point of view accessing a data source and retrieving data is the task that can be formalized quite well. ADO.NET is a good example of this. Unlike ADO.NET we plan to implement some middle tier that will common for all applications (developers). So the main task is to access different data sources using some abstract model. You should not think that application "knows nothing" about database - it knows enough to get required information from database. But there will not be connection strings, stored procedures ,etc. "Another issue, is that there really does come a time when your application needs to know what the data source is. For instance, you may want to insert a record into a database that has an automatically incremented primary key - but not all databases support this, so you need to have some mechanism to generate the key for this database." In our company we always use stored procedure instead of direct sql statements. So in this case there must be a procedure like "insert_something" that hides away how primary key is generated - with auto increment or using sequences.Of course,it's implementation will depend on what database is being using, but at the same time it will transparent to client applications. So we do not want to create some universal language for all databases- but organize all these databases (connection strings,user credentials,stored procedures,ets) is some abstract model and make it accessible by all applications using different protocols and communication technologies (due to WCF)
But you still haven't answered the question about what happens when somebody legitimately needs to do something that your gateway just isn't designed to do? I haven't got a problem with the idea of formalizing the way you communicate with databases, this is just good design, but there will come a time when the gateway just won't cope 100%. The question I'm asking is, what is your strategy going to be at this point in time? Are you going to have a team to manage the gateway separate from the team who write the applications? It's not a good idea for the two groups to be one and the same.
Please visit http://www.readytogiveup.com/ and do something special today. Deja View - the feeling that you've seen this post before.
-
From my point of view accessing a data source and retrieving data is the task that can be formalized quite well. ADO.NET is a good example of this. Unlike ADO.NET we plan to implement some middle tier that will common for all applications (developers). So the main task is to access different data sources using some abstract model. You should not think that application "knows nothing" about database - it knows enough to get required information from database. But there will not be connection strings, stored procedures ,etc. "Another issue, is that there really does come a time when your application needs to know what the data source is. For instance, you may want to insert a record into a database that has an automatically incremented primary key - but not all databases support this, so you need to have some mechanism to generate the key for this database." In our company we always use stored procedure instead of direct sql statements. So in this case there must be a procedure like "insert_something" that hides away how primary key is generated - with auto increment or using sequences.Of course,it's implementation will depend on what database is being using, but at the same time it will transparent to client applications. So we do not want to create some universal language for all databases- but organize all these databases (connection strings,user credentials,stored procedures,ets) is some abstract model and make it accessible by all applications using different protocols and communication technologies (due to WCF)
rokford wrote:
Unlike ADO.NET we plan to implement some middle tier that will common for all applications (developers). So the main task is to access different data sources using some abstract model.
That is what a DAL is for. There is a very good reason that a DAL is a library and not a tier. By making it a tier you are adding significant application performance penalties for a perceived benefit that is still completely unclear to me.
-
rokford wrote:
Unlike ADO.NET we plan to implement some middle tier that will common for all applications (developers). So the main task is to access different data sources using some abstract model.
That is what a DAL is for. There is a very good reason that a DAL is a library and not a tier. By making it a tier you are adding significant application performance penalties for a perceived benefit that is still completely unclear to me.
We found some interesting solution but not sure it will accommodate our needs. http://wcf.netfx3.com/files/folders/development\_tools/entry11198.aspx
-
But you still haven't answered the question about what happens when somebody legitimately needs to do something that your gateway just isn't designed to do? I haven't got a problem with the idea of formalizing the way you communicate with databases, this is just good design, but there will come a time when the gateway just won't cope 100%. The question I'm asking is, what is your strategy going to be at this point in time? Are you going to have a team to manage the gateway separate from the team who write the applications? It's not a good idea for the two groups to be one and the same.
Please visit http://www.readytogiveup.com/ and do something special today. Deja View - the feeling that you've seen this post before.
-
After reading the other posts, too, I think that you have a nice concept for your solution. Reminds me somewhat of CAB (Composite UI Application Block) where the CAB is the central part that coordinates all others, just like your Gateway coordinates DataSources and client applications. Best luck with your approach. And maybe you can provide us with some information how it worked out in a CodeProject article - would be very interesting.
-^-^-^-^-^- no risk no funk
I think you should see this: www.datumnode.com
-
I think you should see this: www.datumnode.com
Thanks for the interesting link.
-^-^-^-^-^- no risk no funk