I'll tell you how this happens:

Colleague #1: "What password shall we set?"

Colleague #2: "Just leave it default for now as we're still testing, we will change it later".

Colleague #3: "Sounds good to me. We're behind the firewall and the NIC used for Dell iDRAC or HP iLO is on an isolated network unique to the physical datacenter. Remote access for our techs is managed through a secured bridge that requires all sorts of security hoops on our company intranet, and remote access for general internet traffic is not available due to the firewall restrictions. There's no way hackers will get through that in the first place."

Colleague #4-20: Build various integrations to database, all with their own ways of storing credentials.

Colleague #2: "It's really past due time to change the database password, but first we have to make sure all critical systems can still access the database."

Which is why forward planning and prompt action is worth so much.

I know I'm stating the obvious, but I've seen some worrying attitudes of "just in time" that seem to go hand in hand with a misunderstanding of Scrum Sprints or Kanban. Where people concentrate on the tree and ignore the vast interconnected forest around them.

Hence the old adage: days of work can save you hours of planning.

manager: why is it taking so long to deploy a simple cluster. back in my day we could code a whole ...

Source: https://news.ycombinator.com/item?id=21312609

nice meme

Excellent use of copy paste from another thread. Upvote!

Good ol' "temporary permanent" solution.

https://stackoverflow.com/a/778275

Thank you for that funny read, lots of great comments!

You would be shocked at how nonchalant and downright negligent people can be about security at even the largest companies in the US. I did consulting work at a large insurance company that had the contact information, ssn, and PHI of pretty much everyone in the America (and I mean everyone). I lost track of the number of times people checked in the production password into git. In fact our production cassandra instance still was using the default cert password 'changeit' when I left. Unsurprisingly, this company was filled with contract workers and H1B workers that were barely able (if at all) to get their work done.

Colleague #437: "So whoever first set this up has left, so I'll just follow the documentation they left to figure out what they did... Oh. ...eh, I got a deadline."

Security is a cost and nuisance. It's the first thing to be cut.

To keep high security at all times you need:

1) Process aka bureaucracy. Mandatory checklists. Checklists are returned and inspected by others. Anything missing or uncertain is checked again and fixed.

2) People who are responsible for security are independent from other concerns. They can have adversarial relationship with people responsible for getting things done if there is conflict of interest. People responsible for security must have status and power to enforce it.

Consider a scenario where you need to take the system down and fix something quickly. It's completely reasonable to allow dummy password few hours when people are around fixing the problem until the system is back online.

But if there is no process in place to remove security temporarily and then restore it something is always forgotten. People who would order password to be changed is not using it and forgets the whole thing. People who use it don't say anything and it becomes new normal.

You need to mandate checklists. You force people to use them and return them. It's costly and makes things slower.

I'd love to see a list of organisations at which these practices are the norm.

Dealing with standard practices, most especially violations of your #2, have contributed in large part to my getting off this ride.

True. The market cannot reward what it cannot see.

Pressure from the higher ups to release to production can force you to dump all the IT security knowledge you accumulated over the years.

It sounds like you aren't lazy and negligent -- not everyone is like you, unfortunately.

Usually they rely on some other mechanism for security. Like you can only access the portal admin page from the intranet or a few IP addresses. That has failed, not the fact that they didn't change the password.

Security only really works in layers. So yes, the password should have been changed too.

I'd say that both of those mechanisms failed.

This happens when a 3rd party is asked to install a system and the first party never takes ownership of securing it.

What passwords you used?

People in wealthy gated communities often don't lock their front doors. Similar principle.