Software Engineering: Latest Library is Panacea?
-
Still reading this fantastic book, which continues to be fantastic: Modern Software Engineering[^]. There is so much great content in this book, but this from Chapter 3 really strikes a chord (as someone who evaluated the use of Entity Framework numerous times over 15 years but could never feel comfortable with it). It's why many of us eschew, "Just use the latest library. It solves all your problems."
Quote:
We talk a lot about change in our industry. We get excited about new technologies and new products, but do these changes really “move the dial” on software development? Many of the changes that exercise us don’t seem to make as much difference as we sometimes seem to think that they will. My favorite example of this was demonstrated in a lovely conference presentation by “Christin Gorman.”1 In it, Christin demonstrates that when using the then popular open source object relational mapping library Hibernate, it was actually more code to write than the equivalent behavior written in SQL, subjectively at least; the SQL was also easier to understand. Christin goes on to amusingly contrast software development with making cakes. Do you make your cake with a cake mix or choose fresh ingredients and make it from scratch?
sql and stored procedures are a bore to maintain... code is better....debugging etc.....i used third party orms and way better, always avoided stored proc ....
Caveat Emptor. "Progress doesn't come from early risers – progress is made by lazy men looking for easier ways to do things." Lazarus Long
-
Its a start. I can't make it go any faster and wouldn't if I could as I'm enjoying every day getting there.
Hogan
Excellent; no-one can ask for more. However I await with interest to hear your feelings when you are tasked with a database migration on a tight schedule and find that... it's stored procedures all the way down 😉😂
-
Most libraries are rubbish. Some are a gems. Seldomly, you will find a diamond[^]. In 2014, we moved our 50 man-year legacy VB6 desktop app to a low code framework, in about 2 man-year. Our new app looks modern, up-to-date, has a wide range of new features, is multiplatform (Web, Mobile and Desktop), extremely configurable, even at runtime, looks uniform, has less bugs, displays dashboards with graphics, extendable and designable reports, even at runtime, for every view, etc., etc., etc., you name it, there it is. The app maps around 600 DB tables, some with hundreds of millions of records. All the SQL commands are built dynamically, via an ORM (XPO). Since then, I have written SQLs marginally only, basically to adjust a few old database design to todays paradigms. The new app is faster than its equivalent written in the previous good old hand written SQLs technology in VB6. Todays' source code is entirely C#. I have never been more happy to go to work since.
Interesting... I see that there is a $2,199 cost associated. Is this a development framework? Meaning...Can I generate 100s of apps for the one-time cost and those apps run standalone (can be deployed "normally") for many years without paying more? Or is this some kind of runtime I have to pay for to run each individual app? Thanks
-
Still reading this fantastic book, which continues to be fantastic: Modern Software Engineering[^]. There is so much great content in this book, but this from Chapter 3 really strikes a chord (as someone who evaluated the use of Entity Framework numerous times over 15 years but could never feel comfortable with it). It's why many of us eschew, "Just use the latest library. It solves all your problems."
Quote:
We talk a lot about change in our industry. We get excited about new technologies and new products, but do these changes really “move the dial” on software development? Many of the changes that exercise us don’t seem to make as much difference as we sometimes seem to think that they will. My favorite example of this was demonstrated in a lovely conference presentation by “Christin Gorman.”1 In it, Christin demonstrates that when using the then popular open source object relational mapping library Hibernate, it was actually more code to write than the equivalent behavior written in SQL, subjectively at least; the SQL was also easier to understand. Christin goes on to amusingly contrast software development with making cakes. Do you make your cake with a cake mix or choose fresh ingredients and make it from scratch?
raddevus wrote:
library Hibernate, it was actually more code to write than the equivalent behavior written in SQL, subjectively at least; the SQL was also easier to understand.
Of course though "subjective" is subjective. But other than that, in my experience, those that are really excited about packaged database layers are those that either have no or little experience using database(s) directly. So of course for them doing no database work would seem like an advantage. They also seem to ignore the potential design problems of ignoring the database itself as part of the architecture because when they start with the data sets are all very small.
-
Interesting... I see that there is a $2,199 cost associated. Is this a development framework? Meaning...Can I generate 100s of apps for the one-time cost and those apps run standalone (can be deployed "normally") for many years without paying more? Or is this some kind of runtime I have to pay for to run each individual app? Thanks
Just a potential warning... The comment to which you are responding to is based on a legacy system with a large persisted data set already in place which was hand-crafted over time using SQL. So either someone(s) either knew how to handle that data from the beginning or they learned over time. Then they took a tool and used it to implement the same thing. Without a background both in databases and in sizing a market attempting to use a tool to replace that knowledge might not end up well.
-
Stored Procedures are evil - they tie your work to a particular DBMS vendor, for a start. That's exactly what they are so loved by those vendors!
haughtonomous wrote:
they tie your work to a particular DBMS vendor, for a start
Unless you design and TEST your product for multiple databases from the beginning then it will be tied to that vendor regardless. Yes I have done such conversions multiple times. And you can contract out the conversion of stored procs for a new database. That is a service provided by many places. Conversely attempting to find and individually modify code in multiple places in legacy systems will be a substantial amount of work.
-
Only if you write NON-ANSI SQL and don't understand database performance and security ... and who does that?
"Before entering on an understanding, I have meditated for a long time, and have foreseen what might happen. It is not genius which reveals to me suddenly, secretly, what I have to say or to do in a circumstance unexpected by other people; it is reflection, it is meditation." - Napoleon I
Gerry Schmitz wrote:
Only if you write NON-ANSI SQL
Huh? Last time I checked there is no 'ANSI' specification for stored procedures. What exactly is your source for that?
Gerry Schmitz wrote:
don't understand database performance and security
I have written large systems. Multiple database vendors. And I have never seen a 'standard' that allows you to generically code for performance. You can impact performance at the enterprise level both by architecture and requirements but tuning for performance at the database level depend on the database.
-
Yes but... Microsoft have T-SQL and Oracle have their Pl/SQL (for example) and other database systems have their own equivalents, all of which have little in common. Using Stored procs for data queries using strict ANSI-SQL may be one thing, but most stored procs I have seen use the much more proprietary T-SQL or Pl/ SQL (etc) to move complex data-related logic as close to the database metal as possible - and that locks you in. A few stored procedures are not a big deal to migrate to another DBMS, but any decent sized database application could have several hundred stored procedures (they tend to become habitual), and that makes migrating to a different dbms an expensive proposition, which is exactly the position the vendors want to get you in.
haughtonomous wrote:
and that makes migrating to a different dbms an expensive proposition
Never seen a cheap one. Not ever. And I have done this multiple times. I have seen one system written from scratch which was intended to be database agnostic. It was a product which when delivered was more than 10 times too slow to actually meet the business requirements. And even with two dedicated employees of that company and a dedicated employee of the target company when I left the company (target) it was still 4 times too slow. They were attempting to adjust it for about two months. Note that the performance requirement was a hard limit as it involved calculating financial data that had to be processed within a fixed window.
-
In 20 plus years of development, I've never had a boss say, change to a different database platform. I feel like this is an argument for the sake of arguing instead of adding actual software value. Using stored procs and embracing a vendor's specific optimizations makes a platform faster. If we expand the idea, do you also eschew cloud computing because it creates vendor lock in?
Hogan
snorkie wrote:
In 20 plus years of development, I've never had a boss say, change to a different database platform
I have. Examples 1. The company's product supported three different databases. The company had three customers and ALL of them were on a database that the product did not support. So the customers were running the other database in their data centers just so they could use the product even though they did not have the real expertise for the database. (I started the conversion just before I left.) 2. Legacy company was running one server and they had outgrown it due to very expensive licensing. So they wanted to switch to another vender. 3. There was at least three companies involved in a contracted application. At some point it was miscommunicated what the actual targeted database vendor was. So it had to be converted mid-contract to the different vendor. 4. Legacy application had been converted to 3 different databases over time. The reason for each conversion was an attempt to increase performance. I wasn't involved in any of those conversions but I was involved in doing some clean up which only existed as an artifact of how the 2nd database system worked. 5. At one company, although it didn't happen when I was there, the new CTO in about the first week claimed that the company was going to convert all existing code from X to Y. And all existing databases from Vendor A to Vendor B.
-
Of course the client code runs on the client. The SQL still runs on the server and should be optimized there.
PIEBALDconsult wrote:
Of course the client code runs on the client.
In C# the standard 'linq' API methodology to access the database relies on libraries (and versions) that are not necessarily correctly implemented to create the expected SQL. I have seen expressions that I could see exactly how the SQL should look but instead resolved into things like the following. (No this was not a case of resolving a list incorrectly.) 1. The linq expression pulled the entire table then applied the where clauses in memory. 2. The linq expression, one expression, resolved to two different SQL calls which it then did the actual join in memory. The only way to detect this happening is by profiling each linq statement.
-
Interesting... I see that there is a $2,199 cost associated. Is this a development framework? Meaning...Can I generate 100s of apps for the one-time cost and those apps run standalone (can be deployed "normally") for many years without paying more? Or is this some kind of runtime I have to pay for to run each individual app? Thanks
With a one-time subscription[^], a dev can generate as many apps and those apps run standalone. A support center[^] with answers to your most questions. Free support forever, with a working solution to your specific question in usually less than one day. There are updates around every month, with new features for free during 12 months. Forever free updates on the versions I own, with bug corrections and security updates. Yearly renewal at 990 $ There are multi-user discounts. Source code. DevExpress XAF YouTube tutorials[^]
-
raddevus wrote:
library Hibernate, it was actually more code to write than the equivalent behavior written in SQL, subjectively at least; the SQL was also easier to understand.
Of course though "subjective" is subjective. But other than that, in my experience, those that are really excited about packaged database layers are those that either have no or little experience using database(s) directly. So of course for them doing no database work would seem like an advantage. They also seem to ignore the potential design problems of ignoring the database itself as part of the architecture because when they start with the data sets are all very small.
That's actually a perfect summary of the challenge/problem. :thumbsup: I agree, because for example, EF works for those it works for and doesn't for those who it doesn't. Most of the time EF works for those who don't know DB tech (sql, stored procs, etc) and doesn't "work" for those who would rather not have the layer between them and the DB. :thumbsup: 6 one way, half a dozen the other. :-D There are the poor unfortunate souls who begin using EF to solve all their problems & don't understand it or DB tech and then believe it is EF's fault. There really are advantages & disadvantages to both courses of action.
-
Just a potential warning... The comment to which you are responding to is based on a legacy system with a large persisted data set already in place which was hand-crafted over time using SQL. So either someone(s) either knew how to handle that data from the beginning or they learned over time. Then they took a tool and used it to implement the same thing. Without a background both in databases and in sizing a market attempting to use a tool to replace that knowledge might not end up well.
jschell wrote:
based on a legacy system with a large persisted data set already in place which was hand-crafted over time using SQL
:thumbsup::thumbsup::thumbsup::thumbsup::thumbsup::thumbsup::thumbsup::thumbsup::thumbsup: Honestly, I can see that your comments are filled with experience. I agree with this 100%. This is really the original point of the Software Engineering book too. The point that some people learn to use a hammer and use it everywhere and call it engineering. Others, learn to use the appropriate tool in the appropriate way -- Engineering!! Really enjoying your comments. Thanks
-
With a one-time subscription[^], a dev can generate as many apps and those apps run standalone. A support center[^] with answers to your most questions. Free support forever, with a working solution to your specific question in usually less than one day. There are updates around every month, with new features for free during 12 months. Forever free updates on the versions I own, with bug corrections and security updates. Yearly renewal at 990 $ There are multi-user discounts. Source code. DevExpress XAF YouTube tutorials[^]
-
That's actually a perfect summary of the challenge/problem. :thumbsup: I agree, because for example, EF works for those it works for and doesn't for those who it doesn't. Most of the time EF works for those who don't know DB tech (sql, stored procs, etc) and doesn't "work" for those who would rather not have the layer between them and the DB. :thumbsup: 6 one way, half a dozen the other. :-D There are the poor unfortunate souls who begin using EF to solve all their problems & don't understand it or DB tech and then believe it is EF's fault. There really are advantages & disadvantages to both courses of action.
Just noting also that I have never used a database layer that did not require, over time, a hack either around it or through it to achieve the necessary goal of using the database correctly. Although the scope of such work arounds should be limited over time they start to add complexity to the application because they are not fully integrated. This means, for example, that someone uses the work around because they did not understand how the API could be used to achieve the same thing. Or alternatively they failed to realize that the hack would be a better solution so they do things like multiple queries in the code and then doing in code correlations with the result.
-
Just noting also that I have never used a database layer that did not require, over time, a hack either around it or through it to achieve the necessary goal of using the database correctly. Although the scope of such work arounds should be limited over time they start to add complexity to the application because they are not fully integrated. This means, for example, that someone uses the work around because they did not understand how the API could be used to achieve the same thing. Or alternatively they failed to realize that the hack would be a better solution so they do things like multiple queries in the code and then doing in code correlations with the result.
These experiences that you talk about are exactly my experiences. So interesting. Thanks for sharing these stories. I always feel like it is just my limited knowledge of the API but sometimes you just put so much time into learning it at some point you got to just do the thing. Then later you discover, oh, that's not what the API creator intended, but it is what the API creator led us into. :laugh:
