Lack of backups in the real world
-
[Planet Explorers :: Group Announcements](https://steamcommunity.com/games/237870/announcements/):
As you know by now, if you're still playing PE, our lobby server had an issue where all the code base got deleted from its server. After trying to fix it for the last month, we've arrived at the unfortunate conclusion that it's a lost cause unless we completely rewrite the code from scratch. We used a software called U-link for PE, and that software is now defunct. Even if we had the original U-link code, we still don't have the configurations and additional code we wrote back in 2013. Under the circumstances, we just don't have the resources to rewrite the multiplayer code for PE. We're truly sorry for this.
...but the devops slideshow I flicked through from clickbait times while doing a number 2 said I didn't need to have a sysadmin, what could possibly have gone wrong. :doh:
Did you ever see history portrayed as an old man with a wise brow and pulseless heart, weighing all things in the balance of reason? Is not rather the genius of history like an eternal, imploring maiden, full of fire, with a burning heart and flaming soul, humanly warm and humanly beautiful? --Zachris Topelius Training a telescope on one’s own belly button will only reveal lint. You like that? You go right on staring at it. I prefer looking at galaxies. -- Sarah Hoyt
I have a client whose business is hosted on a server in his office. He takes backups of the (Sql2005) database every day, with off-site copies taken monthly... and stores it on the server's C: drive. The server is running Win2008, and the application is written in a mix of Classic ASP and ASP.Net running on Framework 2.0. It uses TLS 1.1 protocols for SSL. In the event of a failure of his server's hard drive, or a disaster in the office, even if he had up-to-date copies of the application (there are no backups as such, each file just gets renamed to ..._Back or ..._copy or something - no naming scheme - and a newer version gets copied in), there's no way he'd find an external hosting company with a setup that could be quickly configured, and if replacing his server I'm not even sure I could easily get my hands on the server op system. So it would be a major upgrade / reconfigure at a time of maximum urgency. I've explained this to him many times but he still says he's trying to "justify the expenditure". :wtf:
-
Our source code is hosted online, so if someone manages to hack MS and delete it it would be gone. But as part of a larger team we still have local copies and could piece things back together. If that fails we could decompile some of exe files, although it won't work that well for the web front end. Guess it wouldn't hurt to have another full source code backup stored somewhere else.
For each build, we create a ZIP of the workspace (minus temporary files) and store it on a separate network drive that replicates to the DR site. (different than the build server and source control server) As long as you have a laptop or workstation with the correct IDE, you can unzip the workspace and perform a patch in minutes if needed.
-
For each build, we create a ZIP of the workspace (minus temporary files) and store it on a separate network drive that replicates to the DR site. (different than the build server and source control server) As long as you have a laptop or workstation with the correct IDE, you can unzip the workspace and perform a patch in minutes if needed.
-
[Planet Explorers :: Group Announcements](https://steamcommunity.com/games/237870/announcements/):
As you know by now, if you're still playing PE, our lobby server had an issue where all the code base got deleted from its server. After trying to fix it for the last month, we've arrived at the unfortunate conclusion that it's a lost cause unless we completely rewrite the code from scratch. We used a software called U-link for PE, and that software is now defunct. Even if we had the original U-link code, we still don't have the configurations and additional code we wrote back in 2013. Under the circumstances, we just don't have the resources to rewrite the multiplayer code for PE. We're truly sorry for this.
...but the devops slideshow I flicked through from clickbait times while doing a number 2 said I didn't need to have a sysadmin, what could possibly have gone wrong. :doh:
Did you ever see history portrayed as an old man with a wise brow and pulseless heart, weighing all things in the balance of reason? Is not rather the genius of history like an eternal, imploring maiden, full of fire, with a burning heart and flaming soul, humanly warm and humanly beautiful? --Zachris Topelius Training a telescope on one’s own belly button will only reveal lint. You like that? You go right on staring at it. I prefer looking at galaxies. -- Sarah Hoyt
-
Holy crap, I'm amazed at the "we lost teh codez during an outage" story, but even more amazed at the community who gave it 204 likes and were all like "Too bad, but thanks for the great times!" Where's the poison and the hate? :wtf: I'm not sure if this restores my faith in humanity or if they've let me down yet again :laugh:
Best, Sander sanderrossel.com Continuous Integration, Delivery, and Deployment arrgh.js - Bringing LINQ to JavaScript Object-Oriented Programming in C# Succinctly
-
I have a client whose business is hosted on a server in his office. He takes backups of the (Sql2005) database every day, with off-site copies taken monthly... and stores it on the server's C: drive. The server is running Win2008, and the application is written in a mix of Classic ASP and ASP.Net running on Framework 2.0. It uses TLS 1.1 protocols for SSL. In the event of a failure of his server's hard drive, or a disaster in the office, even if he had up-to-date copies of the application (there are no backups as such, each file just gets renamed to ..._Back or ..._copy or something - no naming scheme - and a newer version gets copied in), there's no way he'd find an external hosting company with a setup that could be quickly configured, and if replacing his server I'm not even sure I could easily get my hands on the server op system. So it would be a major upgrade / reconfigure at a time of maximum urgency. I've explained this to him many times but he still says he's trying to "justify the expenditure". :wtf:
If it might help light a fire under your clients seat, Jan 14 2020 is Server 2008's end of life which means that early next year his system is almost certain to get pwnd.
Did you ever see history portrayed as an old man with a wise brow and pulseless heart, weighing all things in the balance of reason? Is not rather the genius of history like an eternal, imploring maiden, full of fire, with a burning heart and flaming soul, humanly warm and humanly beautiful? --Zachris Topelius Training a telescope on one’s own belly button will only reveal lint. You like that? You go right on staring at it. I prefer looking at galaxies. -- Sarah Hoyt
-
Our source code is hosted online, so if someone manages to hack MS and delete it it would be gone. But as part of a larger team we still have local copies and could piece things back together. If that fails we could decompile some of exe files, although it won't work that well for the web front end. Guess it wouldn't hurt to have another full source code backup stored somewhere else.
That's more or less what we have. Code is in hosted repos - mostly Bitbucket - with psuedo-backups in the form of local copies on the machines of whoever's working on the projects. I'm not aware of anything backing up outside of Bitbucket; but it's possible someone has set something up. I'll have to remember to ask about it when I'm back in the office on Monday.
Did you ever see history portrayed as an old man with a wise brow and pulseless heart, weighing all things in the balance of reason? Is not rather the genius of history like an eternal, imploring maiden, full of fire, with a burning heart and flaming soul, humanly warm and humanly beautiful? --Zachris Topelius Training a telescope on one’s own belly button will only reveal lint. You like that? You go right on staring at it. I prefer looking at galaxies. -- Sarah Hoyt
-
If it might help light a fire under your clients seat, Jan 14 2020 is Server 2008's end of life which means that early next year his system is almost certain to get pwnd.
Did you ever see history portrayed as an old man with a wise brow and pulseless heart, weighing all things in the balance of reason? Is not rather the genius of history like an eternal, imploring maiden, full of fire, with a burning heart and flaming soul, humanly warm and humanly beautiful? --Zachris Topelius Training a telescope on one’s own belly button will only reveal lint. You like that? You go right on staring at it. I prefer looking at galaxies. -- Sarah Hoyt
-
At a former position, we discovered that the new way of doing backups produced empty backups. Oops. Fortunately, problem fixed before they were needed. As somebody once told me, "Nobody cares about backups. Restores, on the other hand...".
I've recalled this story before, but briefly - was called in as a consultant to help a customer whose PC's hard drive had failed. It was OK, they backed everything up nightly. I loaded the (floppy) disk and found ... the application code. Not a single copy of the data existed anywhere. Amazingly, the company survived by rebuilding their database from the employees' memory (they were a cleaning service company).
-
[Planet Explorers :: Group Announcements](https://steamcommunity.com/games/237870/announcements/):
As you know by now, if you're still playing PE, our lobby server had an issue where all the code base got deleted from its server. After trying to fix it for the last month, we've arrived at the unfortunate conclusion that it's a lost cause unless we completely rewrite the code from scratch. We used a software called U-link for PE, and that software is now defunct. Even if we had the original U-link code, we still don't have the configurations and additional code we wrote back in 2013. Under the circumstances, we just don't have the resources to rewrite the multiplayer code for PE. We're truly sorry for this.
...but the devops slideshow I flicked through from clickbait times while doing a number 2 said I didn't need to have a sysadmin, what could possibly have gone wrong. :doh:
Did you ever see history portrayed as an old man with a wise brow and pulseless heart, weighing all things in the balance of reason? Is not rather the genius of history like an eternal, imploring maiden, full of fire, with a burning heart and flaming soul, humanly warm and humanly beautiful? --Zachris Topelius Training a telescope on one’s own belly button will only reveal lint. You like that? You go right on staring at it. I prefer looking at galaxies. -- Sarah Hoyt
Back when doing a job as a desktop maintenance monkey, we came in to work at a new client company, went through several machines doing updates, and while there, emptied the trash. Got dragged in front of the boss and told: "never empty the trash, that's where some of those people store their working files". We all looked at each other, he looked at us and said, "I know, I know. I can't fix them. They're idiots. Just don't empty the trash."
_____________________________ A logician deducts the truth. A detective inducts the truth. A journalist abducts the truth. Give a man a mug, he drinks for a day. Teach a man to mug...
-
Back when doing a job as a desktop maintenance monkey, we came in to work at a new client company, went through several machines doing updates, and while there, emptied the trash. Got dragged in front of the boss and told: "never empty the trash, that's where some of those people store their working files". We all looked at each other, he looked at us and said, "I know, I know. I can't fix them. They're idiots. Just don't empty the trash."
_____________________________ A logician deducts the truth. A detective inducts the truth. A journalist abducts the truth. Give a man a mug, he drinks for a day. Teach a man to mug...
If I had a nickle for every time someone stored important files in the trash/recycle bin ... I got called in to a manager's office to help him recover important files that disappeared. When I asked him what folder they had been in, he pointed to the recycle bin. Apparently, he did this in the physical world as well. He put things he did not want to deal with now into his recycle bin under his desk. If he didn't need to look at it for over a month, it would get dumped. If he needed it, he would dig through the recycle bin to find it. You just can't fix stupid, no matter how hard you try. Money makes the world go round ... but documentation moves the money.
-
[Planet Explorers :: Group Announcements](https://steamcommunity.com/games/237870/announcements/):
As you know by now, if you're still playing PE, our lobby server had an issue where all the code base got deleted from its server. After trying to fix it for the last month, we've arrived at the unfortunate conclusion that it's a lost cause unless we completely rewrite the code from scratch. We used a software called U-link for PE, and that software is now defunct. Even if we had the original U-link code, we still don't have the configurations and additional code we wrote back in 2013. Under the circumstances, we just don't have the resources to rewrite the multiplayer code for PE. We're truly sorry for this.
...but the devops slideshow I flicked through from clickbait times while doing a number 2 said I didn't need to have a sysadmin, what could possibly have gone wrong. :doh:
Did you ever see history portrayed as an old man with a wise brow and pulseless heart, weighing all things in the balance of reason? Is not rather the genius of history like an eternal, imploring maiden, full of fire, with a burning heart and flaming soul, humanly warm and humanly beautiful? --Zachris Topelius Training a telescope on one’s own belly button will only reveal lint. You like that? You go right on staring at it. I prefer looking at galaxies. -- Sarah Hoyt
I've been at 2 client sites where a mainframe application had been in production for decades, but the source code was lost. There were no changes to the program (obviously) so other programs were written to manipulate the data to address necessary updates. In both cases the original specs were lost as well, so no one knew for sure what the MF did or how it did it. At another site we had good backups -- but when we needed a file restored, the help desk informed us that it would take a week to extract a file from the backup. It was faster to rewrite the file from the spec, done in a day. The help desk sent us the file from the backup 3 weeks later ... Periodically I zip a copy of the project and store it on an external HD and/or burn to DVD. Are DVDs still relevant? Consider that a finalized DVD is ransomware-proof and draw your own conclusion.
