News for hackers. Stuff that matters.
Re:Game developer loses multiplayer service code by honkycat on Sunday September 08, 2019 @05:59PM (#20913710)
Man, that is really unfortunate. My heart goes out to these developers. I've worked with a few of these "do whatever works" cowfolk before. It is exhausting. And, as we see here, can ruin a company.

Personally, I'm getting really sick of the "software doesn't matter, do whatever you want, ship it" attitude in the industry. It creates a lot of needless toil because people do not want to learn how to do things properly.

Your software is your product. The primary cost of software is NOT the initial development: It is the ongoing maintenance. When you cut corners, you are taking out a loan against your future sanity and time. Higher quality code bases and better practices lead to stability, ease of extension, and more money.

___

A while ago I was working for a start-up using Google Cloud SQL Server with Postgres.

I was relying on the built-in database backup functionality, sleeping peacefully at night knowing I effectively delegated that task to a higher power.

Then, one day, I deleted the dev database. Whoops! Hmm... I cannot find the backups...

So I go onto the google cloud slack:

Me: "Hey, where are these backups stored?

Them: "The backups are deleted with the database."

I felt a bit like Wile E Coyote running in mid-air after running straight off a cliff.

Needless to say, I dropped all current projects and took a week to developing a backup, restore, and testing solution for our product. WHEW. Terrifying stuff!

Re:Game developer loses multiplayer service code by neffy on Sunday September 08, 2019 @07:06PM (#20913949)
Things I always do a few weeks after starting a new job...

Request the backup copy of a semi-randomly chosen file. It's been enlightening.

Re:Game developer loses multiplayer service code by H8crilA on Sunday September 08, 2019 @08:05PM (#20914172)
Related: organizations that would rather pay cryptolocker ransoms than restore from backups.
Re:Game developer loses multiplayer service code by ww520 on Sunday September 08, 2019 @08:52PM (#20914350)
A backup that is deleted when the original data is deleted is not a backup. It's called a replicating clone. Backup can be restored even if the original data is deleted. Either they misrepresent or they don't understand what a backup is.
Re:Game developer loses multiplayer service code by rovr138 on Sunday September 08, 2019 @09:27PM (#20914480)
I think they deleted the database, not the content, and that’s what triggered the backups to be deleted.

Crazy still that they’re tied like that.

Re:Game developer loses multiplayer service code by Rotten194 on Monday September 09, 2019 @02:26AM (#20915659)
> Your software is your product. The primary cost of software is NOT the initial development: It is the ongoing maintenance. When you cut corners, you are taking out a loan against your future sanity and time. Higher quality code bases and better practices lead to stability, ease of extension, and more money.

I want to agree with this attitude, I really do. But with how crowded the indie game market is right now, I understand the urge to move fast and loose. Games already take years to develop and are incredibly expensive, and hiring additional developers is incredibly expensive. Considering the tiny chance your game will be successful and make its money back, why would you spend tons of time frontloading work to avoid future maintenance? In all likelihood, nobody will play your game and it won't matter! And in the off chance it does get popular, you can go back and clean things up later... definitely... right?

Re:Game developer loses multiplayer service code by honkycat on Monday September 09, 2019 @11:37AM (#20919121)
I would argue it is not that much harder to do things correctly as opposed to doing things fast and loose. It just requires reading a few books and learning a tool like ansible. It is extremely boring, but the time pays for itself.

I suspect this is a case of inexperience as opposed to someone casting off established wisdom ( you should be able to rebuil your project, be constantly testing to ensure you can, enable automated backups for all servers )

Read the stardew valley chapter in "Blood, Sweat, and Pixels".

Turned out great, but it was harrowing for a while.

Re:Game developer loses multiplayer service code by cosmotic on Sunday September 08, 2019 @10:02PM (#20914635)
I had a similar experience with a web host long ago. I requested a restore and they said they couldn't do it. After pressing and requesting a refund on the additional fees we were paying for the backup, they declined to refund because the backups were running fine as evidenced by a cron job executing like clock work. It turns out they never checked the result code or logs of the cron job it hadn't been running successfully at all.
Re:Game developer loses multiplayer service code by thedanbob on Sunday September 08, 2019 @07:08PM (#20913957)
How did they think that was ok? A backup that gets automatically deleted when the original is deleted is not a backup.
Re:Game developer loses multiplayer service code by plasma on Sunday September 08, 2019 @07:17PM (#20913995)
Yeah thats pretty... awful.
Re:Game developer loses multiplayer service code by slugabed on Sunday September 08, 2019 @07:40PM (#20914086)
In some cases, AWS RDS snapshots work that way too.
Re:Game developer loses multiplayer service code by dyeje on Sunday September 08, 2019 @09:11PM (#20914420)
Yep, have been on a project that was burned by this.
Re:Game developer loses multiplayer service code by euske on Sunday September 08, 2019 @08:53PM (#20914353)
Although I agree with your sentiment, just preaching it wouldn't change them. That's the real problem that we need to discuss. And we're yet to know how to systematically educate ourselves and our colleagues to prevent a similar accident in future.