No more stored procedures
-
Of course it depends, everything *depends*, it depends on how the query was written and the knowledge and experience of the developer. Knowledgeable and experienced programmers are always going to wrap queries into the fewest possible trips to the server anyway which has nothing to do with stored procedures or dynamic sql so that's a moot point. The only difference between a stored procedure and dynamic sql is compilation time of the query which is cached anyway in most rdbm's which makes the whole argument moot, but for the sake of it even assuming it isn't cached, how much difference do you think the end user of the application is going to see between one query that uses a precompiled execution plan (which may in fact be recalculated anyway) to one that doesn't? User's care about features and functionality, sacrificing either of those to save that user a total of maybe 10 seconds waiting time over the entire course of a day isn't really very smart now is it?
John Cardinal wrote:
Of course it depends, everything *depends*, it depends on how the query was written and the knowledge and experience of the developer.
Exactly - which is why you cannot make a blanket statement about how raw/dynamic SQL is better than an SP - if the wrong developer reads that things can go very, very wrong.
John Cardinal wrote:
Knowledgeable and experienced programmers are always going to wrap queries into the fewest possible trips to the server anyway which has nothing to do with stored procedures or dynamic sql so that's a moot point.
Depends on the DB access layers and/or wrappers available to the developer's involved. Believe it or not, not all of them are
Open-Batch-Batch-Batch-Closecapable.John Cardinal wrote:
The only difference between a stored procedure and dynamic sql is compilation time of the query which is cached anyway in most rdbm's which makes the whole argument moot, but for the sake of it even assuming it isn't cached, how much difference do you think the end user of the application is going to see between one query that uses a precompiled execution plan (which may in fact be recalculated anyway) to one that doesn't?
And the query plan? Too many unique queries can get your previous query knocked out of the cache, requiring recompiling, redoing the query plan, etc. And not all software development is user-centric. To a single user, taking 1/10th of a second longer for a sequence of operations may not be a problem. When you are working with 100's or 1000's of users, or with connected high-throughput systems, things are a bit different; delay always adds up add up - desktop development is very different than truly distributed systems development.
John Cardinal wrote:
User's care about features and functionality, sacrificing either of those to save that user a total of maybe 10 seconds waiting time over the entire course of a day isn't really very smart now is it?
I have never had to reduce functionality or features because of using stored procedures. I have also been able to do both - use SPs for the main functionality of the system as well as offer a "query builder" interface for creating custom queries. SPs and raw/dynam
-
John Cardinal wrote:
Stored procedures should never be used in a commercial software application.
How are you defining commercial software? I sell business solutions and have used SPs for donkeys years and never had a single issue.
John Cardinal wrote:
- All data access should be done through a data access layer in a properly stratified design, i.e. presentation layer, business object layer, data access layer (and usually more in between). This gurantees portability, forcing your end user into a particular database brand when it's easy to write an independant DAL for different DB's is just stupid.
That makes sense, but it doesn't preclude using SPs on the database.
Michael CP Blog [^] Development Blog [^]
No, to be fair I'm just so fed up with "faith based" programmers spouting so called common knowledge, taking things on faith without really trying it out for themselves to see it's not true that I was a little too harsh there, commercial software developers can and will do whatever they want to. I think it's wrong from a practical standpoint of selling and supporting software.
-
Getting rid of the stored procedures is only one small step in preparing to write an application that runs on mulitple database systems, and isn't even an essential one. Writing a data access layer that is sufficiently abstract is what you have to do, and it's a lot of extra work if you are never going to make use of it. For your particular example recordset.open("insert data into table; select scope_identity() as userid") is the way that I'd work around that particular request. Most developers don't make enough use of batching SQL requests using stored procedures or not.
I can imagine the sinking feeling one would have after ordering my book, only to find a laughably ridiculous theory with demented logic once the book arrives - Mark McCutcheon
Andy Brummer wrote:
Writing a data access layer that is sufficiently abstract is what you have to do
5!
Andy Brummer wrote:
Most developers don't make enough use of batching SQL requests
5! Well said! Marc
People are just notoriously impossible. --DavidCrow
There's NO excuse for not commenting your code. -- John Simmons / outlaw programmer
People who say that they will refactor their code later to make it "good" don't understand refactoring, nor the art and craft of programming. -- Josh Smith -
I've just recieved an email from my supervisor, asking me not to use any server-side functions, stored procedures, views or queries and to keep all database coding within the code itself - just in case we need to change databases or sell to a client that doesn't use the same database that we do. We write in-house web apps (classic ASP and C#) using SQL Server 2000 and have so far sold a total of zero applications to third parties (We are not a software house - just a small IT department serving the rest of the company). Pseudo-code for the offending stored procedure that prompted the new policy is shown below:
begin insert data into table select scope_identity() as userid endI was instructed to change it to two separate calls from within the code:recordset.open( "insert data into table" ) ... recordset.open( "select @@identity" )Any thoughts? I have mine, but I'd be interested in hearing from others...
The StartPage Randomizer | The Timelapse Project | A Random Web Page
Miszou wrote:
I was instructed to change it to two separate calls from within the code: recordset.open( "insert data into table" ) ... recordset.open( "select @@identity" )
That is likely to be broken from the start in any multiuser environment. There is a very high likelihood that the value returned by the second select will not even be related to the identity value created in the first, since there is ample opportunity for another user to insert a record into the same table.
-
Stored procedures should never be used in a commercial software application. 1) Performance is not noticeably better to the end user. That used to be true a very long time ago supposedly, I don't know from personal experience because I wasn't databasing in the 70s and 80's. I can testify without a shadow of a doubt that it is no longer true for FireBird, MS SQL server or Oracle. I had a major app that was targetted exclusively at MS SQL server, about halfway through development we came to our senses and decided to target different databases, I personally have benchmarks and a lot of experience converting the app to DAL and there is no question that the end user will never see a difference in peformance either way. 2) All data access should be done through a data access layer in a properly stratified design, i.e. presentation layer, business object layer, data access layer (and usually more in between). This gurantees portability, forcing your end user into a particular database brand when it's easy to write an independant DAL for different DB's is just stupid. 3) There are many *many* things you can not do with stored procedures that you can easily do in code. 4) There is little or nothing you can do to protect your intellectual property when it's in plain sight in the database in stored procedures. 5) There is no security issue with using dynamic sql as long as you know what you are doing. I agree completely with your boss on the first part, on the second part about inserting a record adn retreiving it's unique id that's completely wrong for dynamic sql, instead generate the ID's at the application and insert the record in one trip. We use Guid's for that exactly to avoid the whole server generated identity trap.
John Cardinal wrote:
Stored procedures should never be used in a commercial software application.
I think that is a way too general statement. It's like saying that no othe language should be used except C++.
John Cardinal wrote:
there is no question that the end user will never see a difference in peformance either way.
I beg to differ. I did my own benchmarks and in many complex queries SPs performed a lot better than plain SQL through code. Talk about a difference of 6 hrs and 15 minutes in one case.
John Cardinal wrote:
forcing your end user into a particular database brand when it's easy to write an independant DAL for different DB's is just stupid.
Agreed!
John Cardinal wrote:
- There are many *many* things you can not do with stored procedures that you can easily do in code.
There are many things you can do in SPs which may be difficult in code.
Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. -Brian Kernighan
-
The attitude that Stored Procedures run quicker is a holdover from older RDBMS systems. Most decent RDBMS have very sophisticated caching mechanisms that render the differences moot. I've written responses about this in the past, and it is something that I can get quite hot under the collar about. What I will argue is that you should never render your dynamic SQL via something like:
string sql = string.Format("insert into mytable values ('{0}')", name);This opens up all sorts of security holes - but by using proper parameters then you can get past this quite nicely.
the last thing I want to see is some pasty-faced geek with skin so pale that it's almost translucent trying to bump parts with a partner - John Simmons / outlaw programmer
Deja View - the feeling that you've seen this post before. -
John Cardinal wrote:
Of course it depends, everything *depends*, it depends on how the query was written and the knowledge and experience of the developer.
Exactly - which is why you cannot make a blanket statement about how raw/dynamic SQL is better than an SP - if the wrong developer reads that things can go very, very wrong.
John Cardinal wrote:
Knowledgeable and experienced programmers are always going to wrap queries into the fewest possible trips to the server anyway which has nothing to do with stored procedures or dynamic sql so that's a moot point.
Depends on the DB access layers and/or wrappers available to the developer's involved. Believe it or not, not all of them are
Open-Batch-Batch-Batch-Closecapable.John Cardinal wrote:
The only difference between a stored procedure and dynamic sql is compilation time of the query which is cached anyway in most rdbm's which makes the whole argument moot, but for the sake of it even assuming it isn't cached, how much difference do you think the end user of the application is going to see between one query that uses a precompiled execution plan (which may in fact be recalculated anyway) to one that doesn't?
And the query plan? Too many unique queries can get your previous query knocked out of the cache, requiring recompiling, redoing the query plan, etc. And not all software development is user-centric. To a single user, taking 1/10th of a second longer for a sequence of operations may not be a problem. When you are working with 100's or 1000's of users, or with connected high-throughput systems, things are a bit different; delay always adds up add up - desktop development is very different than truly distributed systems development.
John Cardinal wrote:
User's care about features and functionality, sacrificing either of those to save that user a total of maybe 10 seconds waiting time over the entire course of a day isn't really very smart now is it?
I have never had to reduce functionality or features because of using stored procedures. I have also been able to do both - use SPs for the main functionality of the system as well as offer a "query builder" interface for creating custom queries. SPs and raw/dynam
-
Miszou wrote:
I was instructed to change it to two separate calls from within the code: recordset.open( "insert data into table" ) ... recordset.open( "select @@identity" )
That is likely to be broken from the start in any multiuser environment. There is a very high likelihood that the value returned by the second select will not even be related to the identity value created in the first, since there is ample opportunity for another user to insert a record into the same table.
Rob Graham wrote:
That is likely to be broken from the start in any multiuser environment.
Not true: From docs: @@IDENTITY and SCOPE_IDENTITY will return the last identity value generated in any table in the current session.
Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. -Brian Kernighan
-
Christian Graus wrote:
I guess that depends. How can you claim that it never is when 1 - stored procs are precompiled 2- stored procs lower the amount of network traffic required to make a request
I said *noticeably* different. If we forget entirely and conveinently the fact that execution plans are cached on most database servers now regardless of whether they came from dynamic sql or sp, we're talking maybe 10 seconds total over the entire course of day. Stored procedures do *not* reduce network traffice, well designed queries do.
Christian Graus wrote:
How is this possible, when a stored proc is just a bunch of SQL
Because you can write the sql dynamically in your code as your application is running, there are many areas you can take advantage of this when you are not fettered by set in stone stored procedures, I use it for filtering and sorting in a 3rd party datagrid that has complex filtering built into it that isn't sql friendly, I use it for very performant knowledgebase type searching, complex reporting.
Christian Graus wrote:
John Cardinal wrote: There is little or nothing you can do to protect your intellectual property when it's in plain sight in the database in stored procedures. If you're writing a C++ app, this is true. Assuming someone has access to your SQL Server, they can see your SQL.
Well my primary concern here is that I don't want my well meaning end users to go messing about with the stored procedures which they will do on their own given an easy opportunity to do so, but secondarily since my logic is primarily in my source code and it's obfuscated it's harder to get at that in the end which is all we can really do.
John Cardinal wrote:
Stored procedures do *not* reduce network traffice, well designed queries do.
So, you're saying that 20 lines of SQL generate as much network traffic as a proc name ?
John Cardinal wrote:
Because you can write the sql dynamically in your code as your application is running, there are many areas you can take advantage of this when you are not fettered by set in stone stored procedures, I use it for filtering and sorting in a 3rd party datagrid that has complex filtering built into it that isn't sql friendly, I use it for very performant knowledgebase type searching, complex reporting.
In other words, you're saying that you run some SQL, then do some stuff in code, then run some SQL ? That's not an apples for apples comparison at all. You can turn those SQL snippets into procs and get teh same effect.
John Cardinal wrote:
which they will do on their own given an easy opportunity to do so
I've never had a problem with this, but every app I've deployed, the SQL Server instance was secure from the rank and file, and the people who had access understood that their warranty expired if they changed code themselves.
Christian Graus - Microsoft MVP - C++ Metal Musings - Rex and my new metal blog
-
Pete O`Hanlon wrote:
What I will argue is that you should never render your dynamic SQL via something
Yeah, I'm assuming for the purposes of this discussion that goes without saying. :)
John Cardinal wrote:
Yeah, I'm assuming for the purposes of this discussion that goes without saying
What scares me is the number of people who aren't aware of this. If they were, then Colin would be out of a job:-D
the last thing I want to see is some pasty-faced geek with skin so pale that it's almost translucent trying to bump parts with a partner - John Simmons / outlaw programmer
Deja View - the feeling that you've seen this post before. -
When it costs you real hard dollars out of your own pocket to support software (and we never charge for support because we think it's extortion but that's another discussion) you make damn sure you cover every eventuality that might prevent the user from being able to use it productively. I can't begin to count the number of times end users have messed with databases for all manner of bizarre reasons. The single biggest security threat to most commercial software is well meaning people using it in the first place, not malicious hackers.
Your business model is flawed. I never charge for support, either. But, support ends when people mess with the program, simple as that. Can I pull my car to pieces, then call for warranty support because it's broken ? No.
Christian Graus - Microsoft MVP - C++ Metal Musings - Rex and my new metal blog
-
John Cardinal wrote:
Stored procedures should never be used in a commercial software application.
I think that is a way too general statement. It's like saying that no othe language should be used except C++.
John Cardinal wrote:
there is no question that the end user will never see a difference in peformance either way.
I beg to differ. I did my own benchmarks and in many complex queries SPs performed a lot better than plain SQL through code. Talk about a difference of 6 hrs and 15 minutes in one case.
John Cardinal wrote:
forcing your end user into a particular database brand when it's easy to write an independant DAL for different DB's is just stupid.
Agreed!
John Cardinal wrote:
- There are many *many* things you can not do with stored procedures that you can easily do in code.
There are many things you can do in SPs which may be difficult in code.
Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. -Brian Kernighan
Rama Krishna Vavilala wrote:
I think that is a way too general statement. It's like saying that no othe language should be used except C++.
Yeah it's hyperbole I agree, I just wanted to get the juices flowing in this discussion! ;)
Rama Krishna Vavilala wrote:
I beg to differ. I did my own benchmarks and in many complex queries SPs performed a lot better than plain SQL through code. Talk about a difference of 6 hrs and 15 minutes in one case.
Woops! That's not something I would be willing to admit, there is no difference other than compilation time and the quality of the query written in the first place.
-
Stored procedures should never be used in a commercial software application. 1) Performance is not noticeably better to the end user. That used to be true a very long time ago supposedly, I don't know from personal experience because I wasn't databasing in the 70s and 80's. I can testify without a shadow of a doubt that it is no longer true for FireBird, MS SQL server or Oracle. I had a major app that was targetted exclusively at MS SQL server, about halfway through development we came to our senses and decided to target different databases, I personally have benchmarks and a lot of experience converting the app to DAL and there is no question that the end user will never see a difference in peformance either way. 2) All data access should be done through a data access layer in a properly stratified design, i.e. presentation layer, business object layer, data access layer (and usually more in between). This gurantees portability, forcing your end user into a particular database brand when it's easy to write an independant DAL for different DB's is just stupid. 3) There are many *many* things you can not do with stored procedures that you can easily do in code. 4) There is little or nothing you can do to protect your intellectual property when it's in plain sight in the database in stored procedures. 5) There is no security issue with using dynamic sql as long as you know what you are doing. I agree completely with your boss on the first part, on the second part about inserting a record adn retreiving it's unique id that's completely wrong for dynamic sql, instead generate the ID's at the application and insert the record in one trip. We use Guid's for that exactly to avoid the whole server generated identity trap.
John Cardinal wrote:
There is little or nothing you can do to protect your intellectual property when it's in plain sight in the database in stored procedures.
BTW I don't think transferring everything to code solves the problem either. I have used tools such as SQL Profiler/ODBC Tracing to reverse engineer many applications to find out why theye were not working or why the behavior was not a expected.
Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. -Brian Kernighan
-
John Cardinal wrote:
Stored procedures do *not* reduce network traffice, well designed queries do.
So, you're saying that 20 lines of SQL generate as much network traffic as a proc name ?
John Cardinal wrote:
Because you can write the sql dynamically in your code as your application is running, there are many areas you can take advantage of this when you are not fettered by set in stone stored procedures, I use it for filtering and sorting in a 3rd party datagrid that has complex filtering built into it that isn't sql friendly, I use it for very performant knowledgebase type searching, complex reporting.
In other words, you're saying that you run some SQL, then do some stuff in code, then run some SQL ? That's not an apples for apples comparison at all. You can turn those SQL snippets into procs and get teh same effect.
John Cardinal wrote:
which they will do on their own given an easy opportunity to do so
I've never had a problem with this, but every app I've deployed, the SQL Server instance was secure from the rank and file, and the people who had access understood that their warranty expired if they changed code themselves.
Christian Graus - Microsoft MVP - C++ Metal Musings - Rex and my new metal blog
Christian Graus wrote:
So, you're saying that 20 lines of SQL generate as much network traffic as a proc name ?
:rolleyes: So what? If it means more features for end users, easier to maintain code and easier to support application?
Christian Graus wrote:
and the people who had access understood that their warranty expired if they changed code themselves.
Again... :rolleyes: My experience is with thousands of users all over the globe that get free support from us, slightly different perhaps.
-
Getting rid of the stored procedures is only one small step in preparing to write an application that runs on mulitple database systems, and isn't even an essential one. Writing a data access layer that is sufficiently abstract is what you have to do, and it's a lot of extra work if you are never going to make use of it. For your particular example recordset.open("insert data into table; select scope_identity() as userid") is the way that I'd work around that particular request. Most developers don't make enough use of batching SQL requests using stored procedures or not.
I can imagine the sinking feeling one would have after ordering my book, only to find a laughably ridiculous theory with demented logic once the book arrives - Mark McCutcheon
Andy Brummer wrote:
Getting rid of the stored procedures is only one small step in preparing to write an application that runs on mulitple database systems, and isn't even an essential one. Writing a data access layer that is sufficiently abstract is what you have to do, and it's a lot of extra work if you are never going to make use of it.
That makes sense, I was kinda puzzled when I read the original post, moving queries from stored procedures to code is of no use if you don't have the infrastructure in your code to support multiple databases. Personally I think you should sell the database software WITH your product. Your manager should have thought of that when he made this decision, because this way he can make profit on both the product and the database software instead of just the product. Also your better able to support the product, as you don't have to train people of the support department in all of the database products your customers use, instead you have to train them in just one.
Andy Brummer wrote:
Most developers don't make enough use of batching SQL requests using stored procedures or not.
Gotta love SQL Server. I tried to batch SQL commands in Access and it didn't work. It was one of the reasons for me to move to microsoft SQL server for even the products that were used on a single computer.
WM. What about weapons of mass-construction? "You can always try to smash it with a wrench to fix that. It might actually work" - WillemM
-
I've just recieved an email from my supervisor, asking me not to use any server-side functions, stored procedures, views or queries and to keep all database coding within the code itself - just in case we need to change databases or sell to a client that doesn't use the same database that we do. We write in-house web apps (classic ASP and C#) using SQL Server 2000 and have so far sold a total of zero applications to third parties (We are not a software house - just a small IT department serving the rest of the company). Pseudo-code for the offending stored procedure that prompted the new policy is shown below:
begin insert data into table select scope_identity() as userid endI was instructed to change it to two separate calls from within the code:recordset.open( "insert data into table" ) ... recordset.open( "select @@identity" )Any thoughts? I have mine, but I'd be interested in hearing from others...
The StartPage Randomizer | The Timelapse Project | A Random Web Page
Miszou wrote:
Any thoughts?
Find a new job :-D
only two letters away from being an asset
-
I guess anything is possible, I've never seen it be the case in practice, my apps are business apps and make *extreme* use of the power of the database engine itself and I've tested and timed and profiled with all manner of different queries and back end databases when I was researching going to a data access layer a couple of years ago and I could never find a perceptible difference between dynamic and stored procedure. Something which a lot of people said would be the case despite the common belief that stored procedures are always faster. I'm kinda intense on this issue :) and any other issue involving software development where people have beliefs that aren't validateable in the real world. I think I'm going to publish a "Heretics guide to software development" explaining why Frames are not necessarily a bad thing in web applications, why stored procedures are a bad thing in commercial application design etc etc. I can only imagine the grief people would try to give me! :)
All of our database code is in stored procedures. :~ We're using Oracle, and we have some pretty substantial stuff - a handful of our stored procedures exceed 500 lines of code. In at least one instance, we're building our query string dynamically inside the stored procedure, and then executing it to ultimately retrieve the desired record set. That particular function is freakin' huge.
"Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"...the staggering layers of obscenity in your statement make it a work of art on so many levels." - Jason Jystad, 10/26/2001 -
Rama Krishna Vavilala wrote:
I think that is a way too general statement. It's like saying that no othe language should be used except C++.
Yeah it's hyperbole I agree, I just wanted to get the juices flowing in this discussion! ;)
Rama Krishna Vavilala wrote:
I beg to differ. I did my own benchmarks and in many complex queries SPs performed a lot better than plain SQL through code. Talk about a difference of 6 hrs and 15 minutes in one case.
Woops! That's not something I would be willing to admit, there is no difference other than compilation time and the quality of the query written in the first place.
John Cardinal wrote:
there is no difference other than compilation time and the quality of the query written in the first place.
There is a third thing: data transfer from the DBMS process to the actual client process and converting them to language specific data types such as variants. In the case I am referring there were two million records which have to be all scanned due to some business requirements.
Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. -Brian Kernighan
-
Christian Graus wrote:
So, you're saying that 20 lines of SQL generate as much network traffic as a proc name ?
:rolleyes: So what? If it means more features for end users, easier to maintain code and easier to support application?
Christian Graus wrote:
and the people who had access understood that their warranty expired if they changed code themselves.
Again... :rolleyes: My experience is with thousands of users all over the globe that get free support from us, slightly different perhaps.
John Cardinal wrote:
So what?
So, I made the point, you disputed it, now you're backing down ?
John Cardinal wrote:
If it means more features for end users
It can't possibly mean that
John Cardinal wrote:
easier to maintain code
It absolutely cannot mean that, it opens the door to hideous to maintain code, although it doesn't guarentee it, and I doubt it's the case in your apps
John Cardinal wrote:
easier to support application
Only in the sense that you seem to be selling to morons and you let them walk all over you.
John Cardinal wrote:
My experience is with thousands of users all over the globe that get free support from us, slightly different perhaps.
It's true that the apps I've deployed have been local ( at least, the ones that use SQL Server, I have users all over the world with an app that doesn't use SQL ). But, I don't see how that changes the basic principle that most users should not have access to the SQL Server, and the people that do, should know that they lose their warranty if they change it. If you go in and fix a problem due to people changing procs, it gets charged at a premium rate. Do that once, and their management will make sure it doesn't happen again.
Christian Graus - Microsoft MVP - C++ Metal Musings - Rex and my new metal blog
-
I've just recieved an email from my supervisor, asking me not to use any server-side functions, stored procedures, views or queries and to keep all database coding within the code itself - just in case we need to change databases or sell to a client that doesn't use the same database that we do. We write in-house web apps (classic ASP and C#) using SQL Server 2000 and have so far sold a total of zero applications to third parties (We are not a software house - just a small IT department serving the rest of the company). Pseudo-code for the offending stored procedure that prompted the new policy is shown below:
begin insert data into table select scope_identity() as userid endI was instructed to change it to two separate calls from within the code:recordset.open( "insert data into table" ) ... recordset.open( "select @@identity" )Any thoughts? I have mine, but I'd be interested in hearing from others...
The StartPage Randomizer | The Timelapse Project | A Random Web Page
Miszou wrote:
Any thoughts?
Does your boss have pointy hair?
the last thing I want to see is some pasty-faced geek with skin so pale that it's almost translucent trying to bump parts with a partner - John Simmons / outlaw programmer
Deja View - the feeling that you've seen this post before.