Managing Database Updates
-
Hi everyone. For my current employer I've developed a few systems and deployed them. The deployment of the databases themselves has always been fairly easy. I use migratordotnet to update the schema as I continually develop these systems. That bit works great for me and I've never had any problems. The problem I am starting to have though is that as the number of releases comes down now to maybe 3 or 4 a year. The schema structures are usually vastly different and this gives me a real headache when I need to migrate the data from a previous version to the latest schema. At the moment to do this I'm writing custom migrations for each release. However the database now has over 60 tables so these migrations are getting very long and more and more bugs are starting to appear. So my question is how does everyone else tackle this problem? Are there industry standard approaches that I'm not aware of? Are the standard tools that I'm not using that help with this? Any help / links to docs would be great! Phil
-
Hi everyone. For my current employer I've developed a few systems and deployed them. The deployment of the databases themselves has always been fairly easy. I use migratordotnet to update the schema as I continually develop these systems. That bit works great for me and I've never had any problems. The problem I am starting to have though is that as the number of releases comes down now to maybe 3 or 4 a year. The schema structures are usually vastly different and this gives me a real headache when I need to migrate the data from a previous version to the latest schema. At the moment to do this I'm writing custom migrations for each release. However the database now has over 60 tables so these migrations are getting very long and more and more bugs are starting to appear. So my question is how does everyone else tackle this problem? Are there industry standard approaches that I'm not aware of? Are the standard tools that I'm not using that help with this? Any help / links to docs would be great! Phil
The basic problem with your predicament, is that even if you could find a third party tool to do the migrations for you, you would still always have to do the mapping and possible conversions, which I suppose are the main part of the work anyway. I think your best bet is to either start designing your database preemptively, or employ an add-and-recycle strategy when writing applications (i.e. as necessary, just only add new tables and/or columns, and as much as possible rename and/or convert existing tables and/or columns). This way you can just leave the existing databases be, and maybe clean them up a little every once in a while when you have time. Admittedly, I may not have understood the full depth of your plight.
My advice is free, and you may get what you paid for.
-
The basic problem with your predicament, is that even if you could find a third party tool to do the migrations for you, you would still always have to do the mapping and possible conversions, which I suppose are the main part of the work anyway. I think your best bet is to either start designing your database preemptively, or employ an add-and-recycle strategy when writing applications (i.e. as necessary, just only add new tables and/or columns, and as much as possible rename and/or convert existing tables and/or columns). This way you can just leave the existing databases be, and maybe clean them up a little every once in a while when you have time. Admittedly, I may not have understood the full depth of your plight.
My advice is free, and you may get what you paid for.
No I think you've hit the nail on the head. I've asked a few friends who work in different businesses what they do and they say the same that basically i have to write a migration or only add to the schema (which I hadn't thought of as an approach so thanks for that). Oh well I'll struggle with these queries then. Thanks Phil
-
No I think you've hit the nail on the head. I've asked a few friends who work in different businesses what they do and they say the same that basically i have to write a migration or only add to the schema (which I hadn't thought of as an approach so thanks for that). Oh well I'll struggle with these queries then. Thanks Phil
This may or may not be of use to you, but for example our SAP system uses a control database. In other words, you could make one database that just contains information about the schemas of your other application databases. This can help you keep better track of what different columns and formats, etc, you have, it can help you make development charts, upgrade / migration plans, and you can use it to run database checks (see if an application database has the correct schema, and such), and you can use it to build and store migration queries (or templates) there. Another nice advantage is, if you ever have to build multiple language GUIs (english and spanish for example), you can use it to store common translations for columns and controls. Good luck with it.
My advice is free, and you may get what you paid for.
-
Hi everyone. For my current employer I've developed a few systems and deployed them. The deployment of the databases themselves has always been fairly easy. I use migratordotnet to update the schema as I continually develop these systems. That bit works great for me and I've never had any problems. The problem I am starting to have though is that as the number of releases comes down now to maybe 3 or 4 a year. The schema structures are usually vastly different and this gives me a real headache when I need to migrate the data from a previous version to the latest schema. At the moment to do this I'm writing custom migrations for each release. However the database now has over 60 tables so these migrations are getting very long and more and more bugs are starting to appear. So my question is how does everyone else tackle this problem? Are there industry standard approaches that I'm not aware of? Are the standard tools that I'm not using that help with this? Any help / links to docs would be great! Phil
Phillip Donegan wrote:
maybe 3 or 4 a year
I would think if you are deploying a major schema change (vastly different) every 3-4 months then you must have an extremely dynamic business model or somebody got the design wrong, and continues to get it wrong. I usually work on single project contracts and have almost always used the migrate method, a PITA but then the business does not expect to repeat this, ever. However I am currently in a position where I am using the incrementing schema as the changes are fairly minor over a long period. The problem I find is the business does not necesarily keep up with the changes, be prepared for, why is this info not being up dated anymore type questions.
Never underestimate the power of human stupidity RAH
-
Phillip Donegan wrote:
maybe 3 or 4 a year
I would think if you are deploying a major schema change (vastly different) every 3-4 months then you must have an extremely dynamic business model or somebody got the design wrong, and continues to get it wrong. I usually work on single project contracts and have almost always used the migrate method, a PITA but then the business does not expect to repeat this, ever. However I am currently in a position where I am using the incrementing schema as the changes are fairly minor over a long period. The problem I find is the business does not necesarily keep up with the changes, be prepared for, why is this info not being up dated anymore type questions.
Never underestimate the power of human stupidity RAH
You know how it is, a request for a change or enhancement comes in, that requires some new tables. Do this 5 or 6 times and you spot a better way of doing it so decide to change the schema, then your old data is screwed to you have to write a monster migration!!
-
You know how it is, a request for a change or enhancement comes in, that requires some new tables. Do this 5 or 6 times and you spot a better way of doing it so decide to change the schema, then your old data is screwed to you have to write a monster migration!!
I now find I have quite a few "attribute" tables in my schemas. Customer now wants 2nd mobile - add a phone attribute linked to the customer table. I have one app that is ALL attributes, it tends to get nasty when trying to retrieve data, monstrous number of joins. Which goes to prove you can go too far down that path.