"Dumb" question for all the DBAs...
-
Dumb question... if you are a DBA and I came up to you and said "Here's a physical beastly box... 24 cores, ~140GB of RAM, plenty of flash storage... I need you to set up SQL 2016 on it and migrate this 10TB database from SQL 2008 R2" Assuming of course proper security and best practices, etc. Let's even "complicate" it and assume this 10TB database includes spatial data. How long would it take you? I have ZERO SQL admin experience other then slamming a few SQLs onto local machines or VMs and I think I could realistically get it setup and running in a day or two with plenty of breathing room. Yet, our supposedly experienced DBAs claim its going to take them 3 months!!! LOL... Unbelievable.
Depends really. Other considerations are How it is being used Compatible issues. Migration window Integrations Connectivity issues How the data is being migrated Testing migrations Etc etc Depending on the complexities it can take the amount of time suggested.
-
Dumb question... if you are a DBA and I came up to you and said "Here's a physical beastly box... 24 cores, ~140GB of RAM, plenty of flash storage... I need you to set up SQL 2016 on it and migrate this 10TB database from SQL 2008 R2" Assuming of course proper security and best practices, etc. Let's even "complicate" it and assume this 10TB database includes spatial data. How long would it take you? I have ZERO SQL admin experience other then slamming a few SQLs onto local machines or VMs and I think I could realistically get it setup and running in a day or two with plenty of breathing room. Yet, our supposedly experienced DBAs claim its going to take them 3 months!!! LOL... Unbelievable.
Not a DBA, but the first question is whether this is for internal use or will be exposed externally. If the former, why not just do it yourself? If the latter, then it will take time, with the actual data import not taking much time at all, but security, configuration, etc. taking a while. Plus they are probably overloaded with work.
-
Not a DBA, but the first question is whether this is for internal use or will be exposed externally. If the former, why not just do it yourself? If the latter, then it will take time, with the actual data import not taking much time at all, but security, configuration, etc. taking a while. Plus they are probably overloaded with work.
We are a B2B data provider, so the database is behind a firewall and the web services access it. We devs aren't allowed to touch the databases anymore. The DB team lead locked them all down. I agree on that, you shouldn't have people randomly do stuff to production databases. Due to the slowness of the database and ETL groups, most of us devs just sit around all day. Nothing to do while we wait on those groups. What's sad is that even with all this time we give them, the databases still suck. They go down all the time, have lots of timeouts, etc.
-
We are a B2B data provider, so the database is behind a firewall and the web services access it. We devs aren't allowed to touch the databases anymore. The DB team lead locked them all down. I agree on that, you shouldn't have people randomly do stuff to production databases. Due to the slowness of the database and ETL groups, most of us devs just sit around all day. Nothing to do while we wait on those groups. What's sad is that even with all this time we give them, the databases still suck. They go down all the time, have lots of timeouts, etc.
Throwing hardware at the problem may speed things up, but rethinking how the data is organized is also a way to get a good boost. I still stay set it up on your own, show it works and then have it deployed into the data center.
-
We are a B2B data provider, so the database is behind a firewall and the web services access it. We devs aren't allowed to touch the databases anymore. The DB team lead locked them all down. I agree on that, you shouldn't have people randomly do stuff to production databases. Due to the slowness of the database and ETL groups, most of us devs just sit around all day. Nothing to do while we wait on those groups. What's sad is that even with all this time we give them, the databases still suck. They go down all the time, have lots of timeouts, etc.
Quote:
the databases still suck. They go down all the time, have lots of timeouts, etc.
I suspect they are living in the DBAs' natural habitat -- an ivory tower where they can formulate their ideal environment. The result being unusable. We have that too.
-
Quote:
the databases still suck. They go down all the time, have lots of timeouts, etc.
I suspect they are living in the DBAs' natural habitat -- an ivory tower where they can formulate their ideal environment. The result being unusable. We have that too.
We have 2 SQL devs that write the SPs. One of them is fairly OK, but tbh, I think his skills are antiquated and he has no interest in learning new stuff. He only knows Microsoft SQL and tends to develop SPs using tons of temp tables and tends to "work around" issues rather then fix them. The other dev is awful. Us devs suspect he is a former C/C++ developer who picked up a "SQL for dummies" book one weekend and decided to become a SQL developer. Just due to the way he writes SQL code. The first project they gave him, he released an SP that took 30 minutes to run!!! He actually released it like that. They've restructured everything many times over and have got it down to 5 to 10 seconds now which still doesn't meet the requirements LOL. 30 minutes to 5 to 10 seconds is a huge improvement obviously, but it just shows how bad the SQL is at my company.
-
We have 2 SQL devs that write the SPs. One of them is fairly OK, but tbh, I think his skills are antiquated and he has no interest in learning new stuff. He only knows Microsoft SQL and tends to develop SPs using tons of temp tables and tends to "work around" issues rather then fix them. The other dev is awful. Us devs suspect he is a former C/C++ developer who picked up a "SQL for dummies" book one weekend and decided to become a SQL developer. Just due to the way he writes SQL code. The first project they gave him, he released an SP that took 30 minutes to run!!! He actually released it like that. They've restructured everything many times over and have got it down to 5 to 10 seconds now which still doesn't meet the requirements LOL. 30 minutes to 5 to 10 seconds is a huge improvement obviously, but it just shows how bad the SQL is at my company.
On the bright side -- you've found your problem.
-
Dumb question... if you are a DBA and I came up to you and said "Here's a physical beastly box... 24 cores, ~140GB of RAM, plenty of flash storage... I need you to set up SQL 2016 on it and migrate this 10TB database from SQL 2008 R2" Assuming of course proper security and best practices, etc. Let's even "complicate" it and assume this 10TB database includes spatial data. How long would it take you? I have ZERO SQL admin experience other then slamming a few SQLs onto local machines or VMs and I think I could realistically get it setup and running in a day or two with plenty of breathing room. Yet, our supposedly experienced DBAs claim its going to take them 3 months!!! LOL... Unbelievable.
If it was me, the SQL 2008 already was a VM, so I would shut it down, duplicate the VM and then update the SQL itself to 2016 - probably a day work, with long breaks for lunch and beer...
Skipper: We'll fix it. Alex: Fix it? How you gonna fix this? Skipper: Grit, spit and a whole lotta duct tape.
-
Dumb question... if you are a DBA and I came up to you and said "Here's a physical beastly box... 24 cores, ~140GB of RAM, plenty of flash storage... I need you to set up SQL 2016 on it and migrate this 10TB database from SQL 2008 R2" Assuming of course proper security and best practices, etc. Let's even "complicate" it and assume this 10TB database includes spatial data. How long would it take you? I have ZERO SQL admin experience other then slamming a few SQLs onto local machines or VMs and I think I could realistically get it setup and running in a day or two with plenty of breathing room. Yet, our supposedly experienced DBAs claim its going to take them 3 months!!! LOL... Unbelievable.
sounds like the bottle neck is not the setup but the policy environment IE corporate and convulated. And yes a day or 2 would be reasonable if it was done in isolation. For us to get a new server up and running takes a minimum of 6 months, I have simple change requests, I mean add 3 fields to an SSIS export, that is over 12 months old and the business considers the requirement "critical". We all work within the constraints of our environment and your, and mine, sucks!!!
Never underestimate the power of human stupidity RAH
-
Dumb question... if you are a DBA and I came up to you and said "Here's a physical beastly box... 24 cores, ~140GB of RAM, plenty of flash storage... I need you to set up SQL 2016 on it and migrate this 10TB database from SQL 2008 R2" Assuming of course proper security and best practices, etc. Let's even "complicate" it and assume this 10TB database includes spatial data. How long would it take you? I have ZERO SQL admin experience other then slamming a few SQLs onto local machines or VMs and I think I could realistically get it setup and running in a day or two with plenty of breathing room. Yet, our supposedly experienced DBAs claim its going to take them 3 months!!! LOL... Unbelievable.
You can make anything last three months if you try hard enough. Whether or not that means your efforts are pointed in the right direction is another matter entirely. Push for an "as built" migration, where it is performed with an absolute minimum of changes, and no "improvements", being made to the data/tables/etc, along the way. Use the words "stable", "stability", and "solid foundation" a lot, when discussing it, and keep pushing the "one thing at a time" concept -- i.e. it's less risky to move it from A to B and get it working right before implementing anything new than it is to try to do both at once.
I wanna be a eunuchs developer! Pass me a bread knife!
-
sounds like the bottle neck is not the setup but the policy environment IE corporate and convulated. And yes a day or 2 would be reasonable if it was done in isolation. For us to get a new server up and running takes a minimum of 6 months, I have simple change requests, I mean add 3 fields to an SSIS export, that is over 12 months old and the business considers the requirement "critical". We all work within the constraints of our environment and your, and mine, sucks!!!
Never underestimate the power of human stupidity RAH
Yup. My favorite dev story is that at one company I worked at, there was a change control board. You weren't allowed to touch code without their approval, so basically: *user or QA reports bug *bug goes to CCB *CCB reviews bug and decides if its worth doing anything about it *assuming CCB approved bug, it gets sent to dev group router *router routes it to proper mgr *assuming router got it right (1/2 the time they didn't) the mgr distributed it to developer for RESEARCH ONLY *there was no ownership of code, so a bug might get routed to a person who has never touched, looked at or even heard of that component *developer would research it, it usually took quite a while to research it, because you had to track down the original developer and ask them all sorts of questions to get up to speed *developer would report back the finding to the CCB *CCB would either approve, deny or request more info *if they want more info, rinse/repeat *only after approval could you fix it -- problem was, you general got approval MONTHS after you researched it, so you completely forgot everything about it -- no point in shelving the changes or anything, since they literally had thousands of branches, and you'd never be able to find your branch. So anyways, due to all the red tape, it actually took me 6 months just to change a bool from true to false :).
-
You can make anything last three months if you try hard enough. Whether or not that means your efforts are pointed in the right direction is another matter entirely. Push for an "as built" migration, where it is performed with an absolute minimum of changes, and no "improvements", being made to the data/tables/etc, along the way. Use the words "stable", "stability", and "solid foundation" a lot, when discussing it, and keep pushing the "one thing at a time" concept -- i.e. it's less risky to move it from A to B and get it working right before implementing anything new than it is to try to do both at once.
I wanna be a eunuchs developer! Pass me a bread knife!
I've learned not to waste my time chasing after people here. If somebody important comes asking me for status, I just say "so & so is road blocking the project" and let them worry about it. Sometimes they ask me to follow up with the road blocker to which I usually respond "I've already sent them 4 or 5 emails on it with no response, I can send a 6th email if you want..." sometimes they want me to and sometimes they don't.
-
Dumb question... if you are a DBA and I came up to you and said "Here's a physical beastly box... 24 cores, ~140GB of RAM, plenty of flash storage... I need you to set up SQL 2016 on it and migrate this 10TB database from SQL 2008 R2" Assuming of course proper security and best practices, etc. Let's even "complicate" it and assume this 10TB database includes spatial data. How long would it take you? I have ZERO SQL admin experience other then slamming a few SQLs onto local machines or VMs and I think I could realistically get it setup and running in a day or two with plenty of breathing room. Yet, our supposedly experienced DBAs claim its going to take them 3 months!!! LOL... Unbelievable.
-
Yup. My favorite dev story is that at one company I worked at, there was a change control board. You weren't allowed to touch code without their approval, so basically: *user or QA reports bug *bug goes to CCB *CCB reviews bug and decides if its worth doing anything about it *assuming CCB approved bug, it gets sent to dev group router *router routes it to proper mgr *assuming router got it right (1/2 the time they didn't) the mgr distributed it to developer for RESEARCH ONLY *there was no ownership of code, so a bug might get routed to a person who has never touched, looked at or even heard of that component *developer would research it, it usually took quite a while to research it, because you had to track down the original developer and ask them all sorts of questions to get up to speed *developer would report back the finding to the CCB *CCB would either approve, deny or request more info *if they want more info, rinse/repeat *only after approval could you fix it -- problem was, you general got approval MONTHS after you researched it, so you completely forgot everything about it -- no point in shelving the changes or anything, since they literally had thousands of branches, and you'd never be able to find your branch. So anyways, due to all the red tape, it actually took me 6 months just to change a bool from true to false :).
Thankfully I work in a departmental IT group, the core team deal with that sort of crap all the time. For me a user gets up walks over and bitches in my ear, if I own it it usually gets the job done in minutes, days at the outside. If another dev owns the app I get in his ear if the user has not gone direct to him. We get to treat our development like a really small dev shop. It is only when we need access to the core systems and infrastructure that we hit a brick wall.
Never underestimate the power of human stupidity RAH
-
Dumb question... if you are a DBA and I came up to you and said "Here's a physical beastly box... 24 cores, ~140GB of RAM, plenty of flash storage... I need you to set up SQL 2016 on it and migrate this 10TB database from SQL 2008 R2" Assuming of course proper security and best practices, etc. Let's even "complicate" it and assume this 10TB database includes spatial data. How long would it take you? I have ZERO SQL admin experience other then slamming a few SQLs onto local machines or VMs and I think I could realistically get it setup and running in a day or two with plenty of breathing room. Yet, our supposedly experienced DBAs claim its going to take them 3 months!!! LOL... Unbelievable.
To actually do a move, probably a day or so to get the new environment (machine, memory, etc.) set up and tested. Give a day to actually copy the data over. Minimally a day or two to write, prove, and run code to demonstrate to management that the data did, in fact, make it over intact (although that can then be used for other moves, so don't lose it). Going out on the proverbial limb here ... a week?
-
Dumb question... if you are a DBA and I came up to you and said "Here's a physical beastly box... 24 cores, ~140GB of RAM, plenty of flash storage... I need you to set up SQL 2016 on it and migrate this 10TB database from SQL 2008 R2" Assuming of course proper security and best practices, etc. Let's even "complicate" it and assume this 10TB database includes spatial data. How long would it take you? I have ZERO SQL admin experience other then slamming a few SQLs onto local machines or VMs and I think I could realistically get it setup and running in a day or two with plenty of breathing room. Yet, our supposedly experienced DBAs claim its going to take them 3 months!!! LOL... Unbelievable.
day 1: get sql 2016, install. oh which version did you want? day 1-5: figure out licensing model. day 5: ok, i think i know which licenses we need. let me raise purchase request with accounting. day 10: oh, no we need a License advisor to come in and assess. day 30: license advisors came yesterday. now i can raise purchase. day 56: purchase approved day 60: acquired licenses. now can install. day 61: oh, we need that feature that was in enterprise edition. Ok, i will just install, oh, need different licenses. day 62: seek payment reimbursement from License Advisor for poor decision. day 90: all license correct. install, upgrade database. done you getting core or user license for that machine? and if core, damn, which my company that easy with money. What over $100k.
-
We have 2 SQL devs that write the SPs. One of them is fairly OK, but tbh, I think his skills are antiquated and he has no interest in learning new stuff. He only knows Microsoft SQL and tends to develop SPs using tons of temp tables and tends to "work around" issues rather then fix them. The other dev is awful. Us devs suspect he is a former C/C++ developer who picked up a "SQL for dummies" book one weekend and decided to become a SQL developer. Just due to the way he writes SQL code. The first project they gave him, he released an SP that took 30 minutes to run!!! He actually released it like that. They've restructured everything many times over and have got it down to 5 to 10 seconds now which still doesn't meet the requirements LOL. 30 minutes to 5 to 10 seconds is a huge improvement obviously, but it just shows how bad the SQL is at my company.
I feel your pain. I've recently started working on a side project of analyzing our company's database cluster for performance issues. Given this task, the company has no tools to do any kind of DB analysis , so I'm also evaluating tools to do that. During my initial analysis run, I've found that a majority of the problems are coming from bad SQL. Over half are coming from third party products the company has purchased for accounting and so forth. Fixing in-house SQL is one thing, but to get a third party company to fix their cra... um, stuff is a PITA.
-
Yup. My favorite dev story is that at one company I worked at, there was a change control board. You weren't allowed to touch code without their approval, so basically: *user or QA reports bug *bug goes to CCB *CCB reviews bug and decides if its worth doing anything about it *assuming CCB approved bug, it gets sent to dev group router *router routes it to proper mgr *assuming router got it right (1/2 the time they didn't) the mgr distributed it to developer for RESEARCH ONLY *there was no ownership of code, so a bug might get routed to a person who has never touched, looked at or even heard of that component *developer would research it, it usually took quite a while to research it, because you had to track down the original developer and ask them all sorts of questions to get up to speed *developer would report back the finding to the CCB *CCB would either approve, deny or request more info *if they want more info, rinse/repeat *only after approval could you fix it -- problem was, you general got approval MONTHS after you researched it, so you completely forgot everything about it -- no point in shelving the changes or anything, since they literally had thousands of branches, and you'd never be able to find your branch. So anyways, due to all the red tape, it actually took me 6 months just to change a bool from true to false :).
Joe, is that you? I didn't know the Change Control Board ever let people out of their dungeon! Or did you dig your way out with a coffee cup and a USB key, again?
vuolsi così colà dove si puote ciò che si vuole, e più non dimandare --The answer to Minos and any question of "Why are we doing it this way?"
-
Dumb question... if you are a DBA and I came up to you and said "Here's a physical beastly box... 24 cores, ~140GB of RAM, plenty of flash storage... I need you to set up SQL 2016 on it and migrate this 10TB database from SQL 2008 R2" Assuming of course proper security and best practices, etc. Let's even "complicate" it and assume this 10TB database includes spatial data. How long would it take you? I have ZERO SQL admin experience other then slamming a few SQLs onto local machines or VMs and I think I could realistically get it setup and running in a day or two with plenty of breathing room. Yet, our supposedly experienced DBAs claim its going to take them 3 months!!! LOL... Unbelievable.
I have 2 answers. Either build it yourself, or outsource it. Now, if this was Oracle. I would say he is insane. You could restore that in oracle pretty easily and copy the config settings over pretty quickly. (restoring 10TB of data will take a while). With that said, it always depends. Is it one schema or 200? Do they all talk to each other. When they are done, do they have to sync the stored procedures from a repository? Is it all data or data and code? You have to cut them a little slack. 3 months seems long. Too long. Okay, way too long. I would agree with Days. I might give them a week. But I would certainly consider hiring an outside resource to come in and do it, and also AUDIT your DB guys for best practices. I find a good Audit shakes things up a bit, and puts people on notice. It also brings in a second pair of eyes which can be very helpful, and management gets around the competing groups complaining. Hard to defend the DB guys when the outside guys say they suck! I have done a few IT audits and it is always scary what you find. The great thing is that SOME stuff gets fixed while you are there (like default passwords on wireless routers, OMG). BTW, it looks like you are throwing hardware at your performance problems. Usually a good sign of a design flaw.
-
Dumb question... if you are a DBA and I came up to you and said "Here's a physical beastly box... 24 cores, ~140GB of RAM, plenty of flash storage... I need you to set up SQL 2016 on it and migrate this 10TB database from SQL 2008 R2" Assuming of course proper security and best practices, etc. Let's even "complicate" it and assume this 10TB database includes spatial data. How long would it take you? I have ZERO SQL admin experience other then slamming a few SQLs onto local machines or VMs and I think I could realistically get it setup and running in a day or two with plenty of breathing room. Yet, our supposedly experienced DBAs claim its going to take them 3 months!!! LOL... Unbelievable.
If the install was clean and easy - an hour or so If the DB files could simply be detached, XCOPY'd, and reattached, and if the data could be copied over gigabit ethernet or USB3 or something fast then it's probably a day just to copy the data over, and anything from 5 mins to hours depending on how many physical database files you had. If you were doing a straight upgrade, all security settings were the same (and scripted to allow easy setting up) then add another half hour of faffing around to get that setup (depending on how many databases) But no. Not three months. Unless that's "2 months and 28 days waiting in our TODO queue because we have a ton on our plate, and 2 days to get it done".
cheers Chris Maunder
