News for hackers. Stuff that matters.
Re:Code Spaces data and backups deleted by hackers by bagosm on Wednesday June 18, 2014 @02:47PM (#7911878)
All I can say is this:

If you can delete it with a single control panel, it doesn't count as an offsite back. Fire the devops

Re:Code Spaces data and backups deleted by hackers by dijit on Wednesday June 18, 2014 @03:04PM (#7911993)
DevOps means different things to different people.

most people use the term to mean 'a coder who can configure apache'

this whole incident reeks of a lack of OpSec and general Ops knowledge.

Lack of offsite backups is one thing, but if you're using cloud infrastructure you never place all eggs in one basket.

your instances have to be disposable and reproducible, at the first hint of a break in my default action is to torch and rebuild.

Re:Code Spaces data and backups deleted by hackers by robryk on Wednesday June 18, 2014 @03:43PM (#7912328)
> your instances have to be disposable and reproducible, at the first hint of a break in my default action is to torch and rebuild.

Or rather shutdown, preserve, start replacements. IMHO your automation should be able to deal with the old preserved instances being left alone.

Re:Code Spaces data and backups deleted by hackers by toomuchtodo on Wednesday June 18, 2014 @06:25PM (#7913296)
DevOps here.

There are some things you aren't going to expect (compromise of your AWS console). This could have been solved by having MFA enabled, as well as having the app push backups in realtime, versioned with delete protection, to S3 buckets under the control of another account (write access, but no delete access).

Show of hands how many people here are doing it this way.

Re:Code Spaces data and backups deleted by hackers by mentat on Wednesday June 18, 2014 @08:02PM (#7913658)
Seriously, if your root account and all full admin accounts aren't using MFA you're just asking for it. Also if you're not using purpose specific access keys, you're just asking for it. If the first thing you do isn't calling AWS support, wow...
Re:Code Spaces data and backups deleted by hackers by guiambros on Thursday June 19, 2014 @12:01AM (#7914416)
Couldn't agree more. Everything under a single platform, no MFA, no (real) offsite backup, and on top of that they spent 12 hours corresponding with the attacker, instead of immediately calling Amazon to ask their help to shut down everything, while they still had time?

I'm sorry, but this is a succession of things not to do in terms of system operations. Probably the team never managed mission critical platforms before, and hopefully they now learned the lesson.

Re:Code Spaces data and backups deleted by hackers by jacquesm on Thursday June 19, 2014 @07:15AM (#7915538)
How many companies have not yet learned that lesson? There are probably a lot of codespaces on AWS. My reasoning is that if you make it so that a developer can set up a virtual datacenter but does not have the background of actually running such an installation then you're going to have to assume that it is probably quite fragile.

Software people tend to make all kinds of assumptions about hardware that do not work out in practice.

Re:Code Spaces data and backups deleted by hackers by guiambros on Thursday June 19, 2014 @11:49PM (#7919454)
That the problem with the recent "DevOps" trend. Lots of people coming from a "Dev" background, but no real "Ops".

And now that spinning up a couple of servers on AWS and creating snapshots on-the-fly are so easy, it gives the false impression that you don't need much to act as a sysadmin.

Re:Code Spaces data and backups deleted by hackers by maxlybbert on Wednesday June 18, 2014 @06:16PM (#7913252)
Not to be too flippant, but the company's closing shop. So, yeah, the DevOps are fired, along with everybody else.

As for the rest of us: AWS is a great one-stop shop. Unfortunately, using just AWS puts you in the "all the eggs in one basket" scenario that we were warned against as children.

Re:Code Spaces data and backups deleted by hackers by judk on Wednesday June 18, 2014 @06:26PM (#7913301)
Two-factor authentication is a second basket.

Sending a copy to Glacier is a second basket.

Does Amazon not have 30-day undelete for bulk storage? Seems crazy.

Re:Code Spaces data and backups deleted by hackers by MaulingMonkey on Thursday June 19, 2014 @04:52AM (#7915247)
> Two-factor authentication is a second basket.

Not in my books. Recommended to be sure, but this has far too many single points of failure. To name a few:

- Software corrupts data

- Hardware corrupts data

- Social engineering bypasses 2FA

Re:Code Spaces data and backups deleted by hackers by JohnTHaller on Monday June 23, 2014 @09:26AM (#7931789)
> Two-factor authentication is a second basket.

Two-factor authentication is a second basket is a better carrying strap but it's still on a single basket.