News for hackers. Stuff that matters.
Re:My $500M Mars rover mistake by cachvico on Tuesday November 28, 2023 @09:15PM (#38454615)
Verbatim from my current code:

    if strings.Contains(dbname, "prod") {
        panic("Refusing to wipe production database!")
    }
    Truncate(db)
Re:My $500M Mars rover mistake by devsda on Tuesday November 28, 2023 @10:22PM (#38455092)
Ours are not named with a common identifier and this also needs constant effort to maintain while refactoring and there's still scope for a mistake.

*ideally* devs should not have prod access or their credentials should only have limited access without permissions for destructive actions like drop/truncate etc.

But in reality, there's always that one helpful dba/dev who shares admin credentials for a quick prod fix with someone and then those credentials end up in a wiki somewhere as part of an SOP.

Re:My $500M Mars rover mistake by masklinn on Wednesday November 29, 2023 @04:38AM (#38457380)
That‘s why you do credentialing via ssh keys, and keys are explained and map to a user, and non-dba keys should expire.

If you need access for a quick prod fix, your key gets added to the machine with that explanation and a week (or lees) lifetime.

Re:My $500M Mars rover mistake by csomar on Tuesday November 28, 2023 @10:24PM (#38455105)
This is bad because if someone forgot to add prod or for whatever reason the code executed beyond the panic, you’ll wipe out the db.

There is no code that will protect your db/data. Only replication to a read-only storage will help in such situations.

Re:My $500M Mars rover mistake by saagarjha on Tuesday November 28, 2023 @11:35PM (#38455619)
If code is executing past a panic, I think it is unlikely that you can trust the integrity of your database anyways.
Re:My $500M Mars rover mistake by Andrex on Tuesday November 28, 2023 @11:14PM (#38455488)
But what if you have a chron job that auto replicates and then deletes everything after you forward it?
Re:My $500M Mars rover mistake by amluto on Tuesday November 28, 2023 @10:07PM (#38454967)
I also have a table with one row in it indicating whether the database is prod.
Re:My $500M Mars rover mistake by ericbarrett on Tuesday November 28, 2023 @10:27PM (#38455128)
Everybody replying to you that this is fragile is missing the point. This kind of code isn't the first line of defense—it's the last.
Re:My $500M Mars rover mistake by RankingMember on Tuesday November 28, 2023 @11:13PM (#38455485)
Exactly- it's layers of prevention rather than being just one screwup away.
Re:My $500M Mars rover mistake by jerf on Wednesday November 29, 2023 @10:17AM (#38460402)
And when your last line of defense fires... you don't just breath a sigh of relief that the system is robust. You also must dig in to how to catch it sooner in your previous lines.

For instance, test code shouldn't have access to production DB passwords. Maybe that means a slightly less convenient login for the dev to get to production, but it's worth it.

Re:My $500M Mars rover mistake by toasted-subs on Tuesday November 28, 2023 @10:16PM (#38455033)
Yup, I have 3 prompts if you want to wipe anything.

One of the reasons I put interactions between databases behind a cli.

Re:My $500M Mars rover mistake by stylepoints on Wednesday November 29, 2023 @12:26AM (#38455892)
#metoo
Re:My $500M Mars rover mistake by augustk on Wednesday November 29, 2023 @05:26AM (#38457705)
And then it turns out that the order of the parameters was mixed up... just kidding.
Re:My $500M Mars rover mistake by jve on Wednesday November 29, 2023 @05:46AM (#38457833)
Just yesterday, I did C# Regex.Match with a supersimple regex: ^\d+ And it seemed not to work. I asked ChatGTP and he noted that I had subtle mistake: the parameters were other way around... :facepalm:
Re:My $500M Mars rover mistake by augustk on Wednesday November 29, 2023 @06:30AM (#38458135)
That's indeed a drawback of function call syntax compared to method call syntax where the object comes before the name of the method.
Re:My $500M Mars rover mistake by manfre on Wednesday November 29, 2023 @09:15AM (#38459561)
I've added a similar safety to every project. It's not perfect, but this last line of defense has saved team members from themselves more than once.

For Django projects, add the below to manage.py:

    env_name = os.environ.get("ENVIRONMENT", "ENVIRONMENT_NOT_SET")
    if env_name in TEST_PROTECTED_ENVIRONMENTS and "test" in sys.argv:
        raise Exception(f"You cannot run tests with ENVIRONMENT={env_name}")
Re:My $500M Mars rover mistake by hunterrr on Wednesday November 29, 2023 @12:53PM (#38462632)
I think runtime checks like this using environment variables is great however what has burned me in the past is that when debugging problems, not knowing what happened at runtime when the logs were produced was problematic. So when test protected environments environment variable needed to be updated I might have a hard time back tracking to it