DELETE SysAdmin FROM Company
-
So today I've experienced what a lot of people are complaining about... Insufficient privileges! Up to now I've always been a sysadmin on all servers and databases, just because we had small teams where everyone did everything (and I did deployments, mostly). I now work for a company that has developers and admins and (somewhat) clear roles for both of them. So I deploy an application (using VSTS, which, apparently, has more rights than me). And it doesn't work... Ask an admin if he can check out a log file for me, yes there, no not there, that other folder, yeah, that one. Then ask if he can change a config for me, yes that config, that value, no, with an A, not an E, yeah, like that. Then ask if he can check that same log file for me, yes that folder, over there. Then ask if he can install a certificate for me, yes I really need it, it's from a third party we need to run our business, yes really, on the local computer, no not the user account, yeah, like that. Then ask if he can, again, check that log file for me. Then ask if he can run a SQL script for me, that database, no that one, seems the schema is new, just type CREATE SCHEMA, no just do it, you need to type GO after a CREATE SCHEMA, no GO, yeah there, try again... Log file again. Things are getting messy, turned out the IIS app pool didn't have sufficient rights for the certificate. Great, that works... Next application :sigh: I'm not complaining that the sysadmin doesn't know how to do his job, but this took an entire afternoon while it should've taken about an hour. I'm a developer, I've written the software, designed the database, deployed it to our test environment... Just let me do my job on other environments as well :sigh: Computers are my job, I kind of know how they work... X|
Best, Sander Continuous Integration, Delivery, and Deployment arrgh.js - Bringing LINQ to JavaScript Object-Oriented Programming in C# Succinctly
Sounds like a process issue to me. Probably just need a layer of management in-between to approve the changes to be deployed. Because, of course, they'd understand the changes and process, be easily accessible and be anything more than a rubber stamp. ;) What if, rather than add layers of bureaucracy, we hired smart people, and trusted them to do the right things (which includes trusting them to know when they're out-of-scope and get help).
-
So today I've experienced what a lot of people are complaining about... Insufficient privileges! Up to now I've always been a sysadmin on all servers and databases, just because we had small teams where everyone did everything (and I did deployments, mostly). I now work for a company that has developers and admins and (somewhat) clear roles for both of them. So I deploy an application (using VSTS, which, apparently, has more rights than me). And it doesn't work... Ask an admin if he can check out a log file for me, yes there, no not there, that other folder, yeah, that one. Then ask if he can change a config for me, yes that config, that value, no, with an A, not an E, yeah, like that. Then ask if he can check that same log file for me, yes that folder, over there. Then ask if he can install a certificate for me, yes I really need it, it's from a third party we need to run our business, yes really, on the local computer, no not the user account, yeah, like that. Then ask if he can, again, check that log file for me. Then ask if he can run a SQL script for me, that database, no that one, seems the schema is new, just type CREATE SCHEMA, no just do it, you need to type GO after a CREATE SCHEMA, no GO, yeah there, try again... Log file again. Things are getting messy, turned out the IIS app pool didn't have sufficient rights for the certificate. Great, that works... Next application :sigh: I'm not complaining that the sysadmin doesn't know how to do his job, but this took an entire afternoon while it should've taken about an hour. I'm a developer, I've written the software, designed the database, deployed it to our test environment... Just let me do my job on other environments as well :sigh: Computers are my job, I kind of know how they work... X|
Best, Sander Continuous Integration, Delivery, and Deployment arrgh.js - Bringing LINQ to JavaScript Object-Oriented Programming in C# Succinctly
The one time I had to work in that kind of environment, we complained until we got the permissions we needed to do our job. Otherwise, I've been responsible for everything, including debugging or working around Microsoft or other large company bugs, because the only support we could get was like that situation you just described, where we knew what the problem was, but couldn't get past the support person to the next level, assuming we could even understand that person, given their heavy accent. So, I've asked for a clay tablet and stylus for a Christmas present this year.
-
So today I've experienced what a lot of people are complaining about... Insufficient privileges! Up to now I've always been a sysadmin on all servers and databases, just because we had small teams where everyone did everything (and I did deployments, mostly). I now work for a company that has developers and admins and (somewhat) clear roles for both of them. So I deploy an application (using VSTS, which, apparently, has more rights than me). And it doesn't work... Ask an admin if he can check out a log file for me, yes there, no not there, that other folder, yeah, that one. Then ask if he can change a config for me, yes that config, that value, no, with an A, not an E, yeah, like that. Then ask if he can check that same log file for me, yes that folder, over there. Then ask if he can install a certificate for me, yes I really need it, it's from a third party we need to run our business, yes really, on the local computer, no not the user account, yeah, like that. Then ask if he can, again, check that log file for me. Then ask if he can run a SQL script for me, that database, no that one, seems the schema is new, just type CREATE SCHEMA, no just do it, you need to type GO after a CREATE SCHEMA, no GO, yeah there, try again... Log file again. Things are getting messy, turned out the IIS app pool didn't have sufficient rights for the certificate. Great, that works... Next application :sigh: I'm not complaining that the sysadmin doesn't know how to do his job, but this took an entire afternoon while it should've taken about an hour. I'm a developer, I've written the software, designed the database, deployed it to our test environment... Just let me do my job on other environments as well :sigh: Computers are my job, I kind of know how they work... X|
Best, Sander Continuous Integration, Delivery, and Deployment arrgh.js - Bringing LINQ to JavaScript Object-Oriented Programming in C# Succinctly
1/2 day? ... How about 6 months waiting for a DBA to "approve" your design. What ever happened to IBM and SQL Server and Oracle "DBA"'s? Did they ascend into heaven? Oh, right; desktop database engines.
"(I) am amazed to see myself here rather than there ... now rather than then". ― Blaise Pascal
-
So today I've experienced what a lot of people are complaining about... Insufficient privileges! Up to now I've always been a sysadmin on all servers and databases, just because we had small teams where everyone did everything (and I did deployments, mostly). I now work for a company that has developers and admins and (somewhat) clear roles for both of them. So I deploy an application (using VSTS, which, apparently, has more rights than me). And it doesn't work... Ask an admin if he can check out a log file for me, yes there, no not there, that other folder, yeah, that one. Then ask if he can change a config for me, yes that config, that value, no, with an A, not an E, yeah, like that. Then ask if he can check that same log file for me, yes that folder, over there. Then ask if he can install a certificate for me, yes I really need it, it's from a third party we need to run our business, yes really, on the local computer, no not the user account, yeah, like that. Then ask if he can, again, check that log file for me. Then ask if he can run a SQL script for me, that database, no that one, seems the schema is new, just type CREATE SCHEMA, no just do it, you need to type GO after a CREATE SCHEMA, no GO, yeah there, try again... Log file again. Things are getting messy, turned out the IIS app pool didn't have sufficient rights for the certificate. Great, that works... Next application :sigh: I'm not complaining that the sysadmin doesn't know how to do his job, but this took an entire afternoon while it should've taken about an hour. I'm a developer, I've written the software, designed the database, deployed it to our test environment... Just let me do my job on other environments as well :sigh: Computers are my job, I kind of know how they work... X|
Best, Sander Continuous Integration, Delivery, and Deployment arrgh.js - Bringing LINQ to JavaScript Object-Oriented Programming in C# Succinctly
Sander Rossel wrote:
but this took an entire afternoon while it should've taken about an hour.
...and so if the production machine goes down or gets hacked who is the person that is on the hook for that? Myself it has been a long time since I would even accept credentials on a production machine when there were actual ops personnel around. And I discourage handing out such access to others. If there is a problem then figuring out the source is easier when there are far few fingers in the mix.
-
Sander Rossel wrote:
but this took an entire afternoon while it should've taken about an hour.
...and so if the production machine goes down or gets hacked who is the person that is on the hook for that? Myself it has been a long time since I would even accept credentials on a production machine when there were actual ops personnel around. And I discourage handing out such access to others. If there is a problem then figuring out the source is easier when there are far few fingers in the mix.
But actually getting anything done becomes pretty much impossible.
Best, Sander Continuous Integration, Delivery, and Deployment arrgh.js - Bringing LINQ to JavaScript Object-Oriented Programming in C# Succinctly
-
So today I've experienced what a lot of people are complaining about... Insufficient privileges! Up to now I've always been a sysadmin on all servers and databases, just because we had small teams where everyone did everything (and I did deployments, mostly). I now work for a company that has developers and admins and (somewhat) clear roles for both of them. So I deploy an application (using VSTS, which, apparently, has more rights than me). And it doesn't work... Ask an admin if he can check out a log file for me, yes there, no not there, that other folder, yeah, that one. Then ask if he can change a config for me, yes that config, that value, no, with an A, not an E, yeah, like that. Then ask if he can check that same log file for me, yes that folder, over there. Then ask if he can install a certificate for me, yes I really need it, it's from a third party we need to run our business, yes really, on the local computer, no not the user account, yeah, like that. Then ask if he can, again, check that log file for me. Then ask if he can run a SQL script for me, that database, no that one, seems the schema is new, just type CREATE SCHEMA, no just do it, you need to type GO after a CREATE SCHEMA, no GO, yeah there, try again... Log file again. Things are getting messy, turned out the IIS app pool didn't have sufficient rights for the certificate. Great, that works... Next application :sigh: I'm not complaining that the sysadmin doesn't know how to do his job, but this took an entire afternoon while it should've taken about an hour. I'm a developer, I've written the software, designed the database, deployed it to our test environment... Just let me do my job on other environments as well :sigh: Computers are my job, I kind of know how they work... X|
Best, Sander Continuous Integration, Delivery, and Deployment arrgh.js - Bringing LINQ to JavaScript Object-Oriented Programming in C# Succinctly
Sounds like you need some sort of UAT environment between DEV and PROD. On our team, when you make a change to UAT, you document it with enough detail that it can be applied to PROD, but someone else has to make the PROD changes based on your documentation alone. You fix the documentation until the PROD deployment succeeds. What happens if you need to stand up the same app for a different site, department, client, or Disaster Site? If you want to test an application upgrade path starting with an identically built machine... If someone wants to upgrade the OS under your app? 3 years from now it will be a lot harder to remember all of the little details that got it working. It is also likely that various things were tried on the DEV machine that did not quite work correctly... Throw DEV away after a bit and clone the UAT environment for the next round of DEV.
-
Sounds like you need some sort of UAT environment between DEV and PROD. On our team, when you make a change to UAT, you document it with enough detail that it can be applied to PROD, but someone else has to make the PROD changes based on your documentation alone. You fix the documentation until the PROD deployment succeeds. What happens if you need to stand up the same app for a different site, department, client, or Disaster Site? If you want to test an application upgrade path starting with an identically built machine... If someone wants to upgrade the OS under your app? 3 years from now it will be a lot harder to remember all of the little details that got it working. It is also likely that various things were tried on the DEV machine that did not quite work correctly... Throw DEV away after a bit and clone the UAT environment for the next round of DEV.
That COULD work if all software I deployed to PROD worked. More specifically, getting the software to run on PROD wasn't the issue, getting it to run PROPERLY was. We do have a DTAP street, but production is always a little different. For example, it uses SSL (I don't know why we don't use that for everything, even internal services), it should sent out actual emails to actual people, it uses different certificates (because we can't connect to production services from a non-production environment), the firewall is configured differently, etc. Sometimes you just need to do some testing on production, especially when third party services are involved (which is the case here). But when you need someone to open a text file for you that process becomes very difficult. I do agree that we SHOULD document the deployment process for every bit of software though (but we don't).
Best, Sander Continuous Integration, Delivery, and Deployment arrgh.js - Bringing LINQ to JavaScript Object-Oriented Programming in C# Succinctly
-
So today I've experienced what a lot of people are complaining about... Insufficient privileges! Up to now I've always been a sysadmin on all servers and databases, just because we had small teams where everyone did everything (and I did deployments, mostly). I now work for a company that has developers and admins and (somewhat) clear roles for both of them. So I deploy an application (using VSTS, which, apparently, has more rights than me). And it doesn't work... Ask an admin if he can check out a log file for me, yes there, no not there, that other folder, yeah, that one. Then ask if he can change a config for me, yes that config, that value, no, with an A, not an E, yeah, like that. Then ask if he can check that same log file for me, yes that folder, over there. Then ask if he can install a certificate for me, yes I really need it, it's from a third party we need to run our business, yes really, on the local computer, no not the user account, yeah, like that. Then ask if he can, again, check that log file for me. Then ask if he can run a SQL script for me, that database, no that one, seems the schema is new, just type CREATE SCHEMA, no just do it, you need to type GO after a CREATE SCHEMA, no GO, yeah there, try again... Log file again. Things are getting messy, turned out the IIS app pool didn't have sufficient rights for the certificate. Great, that works... Next application :sigh: I'm not complaining that the sysadmin doesn't know how to do his job, but this took an entire afternoon while it should've taken about an hour. I'm a developer, I've written the software, designed the database, deployed it to our test environment... Just let me do my job on other environments as well :sigh: Computers are my job, I kind of know how they work... X|
Best, Sander Continuous Integration, Delivery, and Deployment arrgh.js - Bringing LINQ to JavaScript Object-Oriented Programming in C# Succinctly
I've been on both sides of this issue. The problem is too many companies have been burned by developers releasing code that hasn't been properly tested and confirmed to work in the target environment. Just because it works on my [development] system and server doesn't mean it will work on the CEO's.
-
I've been on both sides of this issue. The problem is too many companies have been burned by developers releasing code that hasn't been properly tested and confirmed to work in the target environment. Just because it works on my [development] system and server doesn't mean it will work on the CEO's.
Well, I guess not giving developers permissions to their target platform solves that issue as no software gets released anymore! But seriously, if untested software makes it to production then your issue is with testing, not releasing. If anything, the chances that software doesn't work as expected is only bigger if people who know nothing about the software do the releasing.
Best, Sander Continuous Integration, Delivery, and Deployment arrgh.js - Bringing LINQ to JavaScript Object-Oriented Programming in C# Succinctly
-
So today I've experienced what a lot of people are complaining about... Insufficient privileges! Up to now I've always been a sysadmin on all servers and databases, just because we had small teams where everyone did everything (and I did deployments, mostly). I now work for a company that has developers and admins and (somewhat) clear roles for both of them. So I deploy an application (using VSTS, which, apparently, has more rights than me). And it doesn't work... Ask an admin if he can check out a log file for me, yes there, no not there, that other folder, yeah, that one. Then ask if he can change a config for me, yes that config, that value, no, with an A, not an E, yeah, like that. Then ask if he can check that same log file for me, yes that folder, over there. Then ask if he can install a certificate for me, yes I really need it, it's from a third party we need to run our business, yes really, on the local computer, no not the user account, yeah, like that. Then ask if he can, again, check that log file for me. Then ask if he can run a SQL script for me, that database, no that one, seems the schema is new, just type CREATE SCHEMA, no just do it, you need to type GO after a CREATE SCHEMA, no GO, yeah there, try again... Log file again. Things are getting messy, turned out the IIS app pool didn't have sufficient rights for the certificate. Great, that works... Next application :sigh: I'm not complaining that the sysadmin doesn't know how to do his job, but this took an entire afternoon while it should've taken about an hour. I'm a developer, I've written the software, designed the database, deployed it to our test environment... Just let me do my job on other environments as well :sigh: Computers are my job, I kind of know how they work... X|
Best, Sander Continuous Integration, Delivery, and Deployment arrgh.js - Bringing LINQ to JavaScript Object-Oriented Programming in C# Succinctly
Not speaking of you directly, but generally, developers in many companies do not have have the security or legal compliance mindset. This has been the case at most every company I have worked at, and unfortunately is the case at my current company. A few months back, the new Senior Manager that is over both my team and the applications (development) team came up to me and informed me that he was making me the "gatekeeper" over all security aspects of our internal AS/400 system. I am not an AS/400 person. I have very limited experience with it as a user. I basically know how to unlock an account, reset a password, and restart a stuck print queue. Any of the AS/400 administration has been handled by the applications team for decades. So, why am I now the security "gatekeeper"? He explained that the developers would just create a new service account with full sysadmin access for every application and every robot and every agent. Whenever anyone with a Supervisor or above title asked for access to something, they granted that access without questions asked. My Sr. Manager had discovered that one of our warehouse workers had access to just about every system, including Accounts Receivable and Payable (because his supervisor wanted him to be able to check to see if some customers were current on their invoices before shipping out). I had to audit user access across the board. I took away access that was not properly approved and documented. Because people could not do their jobs the (wrong) way they were used to doing it, production ground to a screeching halt for almost a full week while access was straitened out, and people were re-trained to do their jobs the correct way. It cost the company thousands upon thousands of dollars. Granted, if something serious had happened and we had been found to be out of legal compliance, it could have cost millions. I still do not have the know-how to administer the AS/400 (although I am slowly learning). So, how do we make this work? The developers must submit security requests to me for review. I properly log the requests, ensure that all the required parties have reviewed the request and approved it, then I create a work order and send it back to the developer/admins to actually execute the security change. If they are found to have made security changes without following the procedure, they are subject to disciplinary action. (It only had to happen once.) - - - - - - I'll go out on a limb and say that most (I did not say all) developers are focus
-
But actually getting anything done becomes pretty much impossible.
Best, Sander Continuous Integration, Delivery, and Deployment arrgh.js - Bringing LINQ to JavaScript Object-Oriented Programming in C# Succinctly
Sander Rossel wrote:
But actually getting anything done becomes pretty much impossible.
That would then be a management problem. But I haven't experienced that situation myself. Operations that fails to meet requests do not last long. Just as any other position that doesn't do their job doesn't last.
-
Not speaking of you directly, but generally, developers in many companies do not have have the security or legal compliance mindset. This has been the case at most every company I have worked at, and unfortunately is the case at my current company. A few months back, the new Senior Manager that is over both my team and the applications (development) team came up to me and informed me that he was making me the "gatekeeper" over all security aspects of our internal AS/400 system. I am not an AS/400 person. I have very limited experience with it as a user. I basically know how to unlock an account, reset a password, and restart a stuck print queue. Any of the AS/400 administration has been handled by the applications team for decades. So, why am I now the security "gatekeeper"? He explained that the developers would just create a new service account with full sysadmin access for every application and every robot and every agent. Whenever anyone with a Supervisor or above title asked for access to something, they granted that access without questions asked. My Sr. Manager had discovered that one of our warehouse workers had access to just about every system, including Accounts Receivable and Payable (because his supervisor wanted him to be able to check to see if some customers were current on their invoices before shipping out). I had to audit user access across the board. I took away access that was not properly approved and documented. Because people could not do their jobs the (wrong) way they were used to doing it, production ground to a screeching halt for almost a full week while access was straitened out, and people were re-trained to do their jobs the correct way. It cost the company thousands upon thousands of dollars. Granted, if something serious had happened and we had been found to be out of legal compliance, it could have cost millions. I still do not have the know-how to administer the AS/400 (although I am slowly learning). So, how do we make this work? The developers must submit security requests to me for review. I properly log the requests, ensure that all the required parties have reviewed the request and approved it, then I create a work order and send it back to the developer/admins to actually execute the security change. If they are found to have made security changes without following the procedure, they are subject to disciplinary action. (It only had to happen once.) - - - - - - I'll go out on a limb and say that most (I did not say all) developers are focus
It's not so much about doing everything yourself, it's just that it would be really nice and no-risk if I just had access to a log file. I don't know much about security, IIS, reverse proxies, and user management. But I do know how to open a log file and run a query on a database. I consider the latter to be a part of my job, and those are the things I cannot currently do. The former can be handled by someone else.
Best, Sander Continuous Integration, Delivery, and Deployment arrgh.js - Bringing LINQ to JavaScript Object-Oriented Programming in C# Succinctly
-
Not speaking of you directly, but generally, developers in many companies do not have have the security or legal compliance mindset. This has been the case at most every company I have worked at, and unfortunately is the case at my current company. A few months back, the new Senior Manager that is over both my team and the applications (development) team came up to me and informed me that he was making me the "gatekeeper" over all security aspects of our internal AS/400 system. I am not an AS/400 person. I have very limited experience with it as a user. I basically know how to unlock an account, reset a password, and restart a stuck print queue. Any of the AS/400 administration has been handled by the applications team for decades. So, why am I now the security "gatekeeper"? He explained that the developers would just create a new service account with full sysadmin access for every application and every robot and every agent. Whenever anyone with a Supervisor or above title asked for access to something, they granted that access without questions asked. My Sr. Manager had discovered that one of our warehouse workers had access to just about every system, including Accounts Receivable and Payable (because his supervisor wanted him to be able to check to see if some customers were current on their invoices before shipping out). I had to audit user access across the board. I took away access that was not properly approved and documented. Because people could not do their jobs the (wrong) way they were used to doing it, production ground to a screeching halt for almost a full week while access was straitened out, and people were re-trained to do their jobs the correct way. It cost the company thousands upon thousands of dollars. Granted, if something serious had happened and we had been found to be out of legal compliance, it could have cost millions. I still do not have the know-how to administer the AS/400 (although I am slowly learning). So, how do we make this work? The developers must submit security requests to me for review. I properly log the requests, ensure that all the required parties have reviewed the request and approved it, then I create a work order and send it back to the developer/admins to actually execute the security change. If they are found to have made security changes without following the procedure, they are subject to disciplinary action. (It only had to happen once.) - - - - - - I'll go out on a limb and say that most (I did not say all) developers are focus
-
It's not so much about doing everything yourself, it's just that it would be really nice and no-risk if I just had access to a log file. I don't know much about security, IIS, reverse proxies, and user management. But I do know how to open a log file and run a query on a database. I consider the latter to be a part of my job, and those are the things I cannot currently do. The former can be handled by someone else.
Best, Sander Continuous Integration, Delivery, and Deployment arrgh.js - Bringing LINQ to JavaScript Object-Oriented Programming in C# Succinctly
I agree with you. If they had it set up properly, you would have at lease read access to the logs, unless there is something else in those logs that would be considered confidential. In that case, they should be parsing the logs, and giving you access to the appropriate portions. Money makes the world go round ... but documentation moves the money.
-
So today I've experienced what a lot of people are complaining about... Insufficient privileges! Up to now I've always been a sysadmin on all servers and databases, just because we had small teams where everyone did everything (and I did deployments, mostly). I now work for a company that has developers and admins and (somewhat) clear roles for both of them. So I deploy an application (using VSTS, which, apparently, has more rights than me). And it doesn't work... Ask an admin if he can check out a log file for me, yes there, no not there, that other folder, yeah, that one. Then ask if he can change a config for me, yes that config, that value, no, with an A, not an E, yeah, like that. Then ask if he can check that same log file for me, yes that folder, over there. Then ask if he can install a certificate for me, yes I really need it, it's from a third party we need to run our business, yes really, on the local computer, no not the user account, yeah, like that. Then ask if he can, again, check that log file for me. Then ask if he can run a SQL script for me, that database, no that one, seems the schema is new, just type CREATE SCHEMA, no just do it, you need to type GO after a CREATE SCHEMA, no GO, yeah there, try again... Log file again. Things are getting messy, turned out the IIS app pool didn't have sufficient rights for the certificate. Great, that works... Next application :sigh: I'm not complaining that the sysadmin doesn't know how to do his job, but this took an entire afternoon while it should've taken about an hour. I'm a developer, I've written the software, designed the database, deployed it to our test environment... Just let me do my job on other environments as well :sigh: Computers are my job, I kind of know how they work... X|
Best, Sander Continuous Integration, Delivery, and Deployment arrgh.js - Bringing LINQ to JavaScript Object-Oriented Programming in C# Succinctly
Of course, if your application had been deployed in UAT or some other environment with representative security - then you could have worked through the issues on systems you had access to. Instead of futzing around in production. Then your install/deployment would have catered for all of the issues that arose. I would also ask about whether all of the custom things you did made it back into the systems documentation - so that in five years time when someone else needs to re-install that they know to repeat the things you did to make it work. Not suggesting that this is representative of your situation - just that there are two sides to the coin. Separation of duties is painful, but sometimes the pain is in forcing the documentation and awareness into the process. Does not sound like they way it worked for you though.
