News for hackers. Stuff that matters.
Re:The day my script killed 10k phones in South America by quenix on Tuesday October 19, 2021 @09:42AM (#28917991)
This feels incredibly negligent. It seems obvious that generating thousands of random phone numbers and locking them without any other checks would lock some real phones. I'm puzzled that the author did not consider this when writing the script, let alone testing it or running it on live prod servers.
Re:The day my script killed 10k phones in South America by treesknees on Tuesday October 19, 2021 @10:22AM (#28918386)
Not defending him, but he does state that unlike the other mobile providers he's worked with, this K-Pop company's backend system would lock _any_ phone number, not just the ones they're responsible for. Perhaps he didn't know this. I bet normally he'd generate random numbers not associated with the provider, check they show up as "locked" in the database, under the assumption that the backend actually doing the locking signal would reject the numbers due to them being under control of another operator.

It's similar to our networking lab where we generate bogus traffic using random src/dst IP addresses under the assumption that our null routes and firewall will block them from the real Internet. In this case, the provider just decided no firewall was needed. We can't use "fake" IP addresses (yes there are IP ranges for internal use but that doesn't work for certain testing scenarios.)

Re:The day my script killed 10k phones in South America by polartx on Tuesday October 19, 2021 @11:46AM (#28919427)
The fact that he was running production code and not writing the result to any kind of log file, which would only take a couple more lines of code, one more dependency (if he wanted to skip some time rolling his own), and a couple of megs of storage, is so absurdly irresponsible, I’m surprised he wrote this post up at all—most would be too embarrassed.
Re:The day my script killed 10k phones in South America by cycomanic on Tuesday October 19, 2021 @10:10AM (#28918255)
Too me this is another example showing how much behind "software engineering" is behind most otherl engineering disciplines. Processes in other disciplines are build very much into the practice that these things don't happen because people die if a plane crashes or a bridge collapses. Also that somehow software gets away with absolving itself of any liability (i.e. the usual "this software is provided as is...") is quite remarkable.
Re:The day my script killed 10k phones in South America by treesknees on Tuesday October 19, 2021 @10:35AM (#28918539)
You're just not looking in the right places, and your comment is a bit disingenuous. You're cherry-picking examples that don't represent an entire industry.

NASA for example has an _extremely_ tight tolerance on software specifications, testing, and proofs [1]. Medical devices and other core infrastructure also has very robust development cycles to ensure bugs don't bring down the power grid or overcharge someones heart in a pacemaker.

Meanwhile you can look at the auto industry and find example after example of shoddy engineering or manufacturing processes putting hundreds of thousands of drivers and families at risk. [2] [3] [4]

1 - https://www.nasa.gov/isd-robust-software-engineering 2 - https://news.yahoo.com/chevy-bolts-stressful-recall-16170005... 3 - https://www.consumerreports.org/car-recalls-defects/takata-a... 4 - https://www.cars.com/articles/explaining-the-toyota-floormat...

Re:The day my script killed 10k phones in South America by 6gvONxR4sf7o on Tuesday October 19, 2021 @12:39PM (#28920163)
It seems pretty clear that software is behind the auto industry in build quality/reliability too.
Re:The day my script killed 10k phones in South America by RHSeeger on Wednesday October 20, 2021 @12:39AM (#28926832)
People die all the time because things for buildings are changed "in production". It's common for the builder (or supplier... the people in charge of taking the engineers plans and actually building the thing) to swap out parts at their own discretion. And there's tons of stories of lots of people dying because of it.

There's also stories of architects designing buildings using "new and interesting techniques" with little to no testing. There's a story of one building that was lighting things across the street on fire because it was basically a big parabolic mirror.

And.. well.. the Boeing 737 Max.

So sure, engineers and related jobs are _better_ about it, but they're certainly not perfect. The still do stupid things. It's just less likely because the fields have been around for a long time and there's been time to learn the right ways to do things... and to learn the things that get people killed.

As the expression goes, regulations are written in blood... because new regulations come about when people die because of a gap.

Re:The day my script killed 10k phones in South America by travisgriggs on Tuesday October 19, 2021 @11:08AM (#28918906)
It’s like trying to breathe underwater. Just because a person has the capability and know how to breathe, they can’t breathe if not given the proper environment to do so. Yelling at them “why didn’t you take a breathe?!?!” As you watched them drown underwater is just ridiculous and naive.

But every time proper engineering doesn’t happen, we see a subset of cries like this, like it’s some moral failing in the engineer. There are surely some chronically stupid engineers out there, but 99% of the time it’s a culture that has subverted the doers below the discussers and the deciders that leads to this shitshow of “how did this happen??”

Re:The day my script killed 10k phones in South America by heavenlyblue on Tuesday October 19, 2021 @10:09AM (#28918235)
I don't understand how people can defend this developer in the first place.

I could understand if they were about 16 years old and actually abused by their employer, but generating random numbers to test the locking of phones? Basic maths should tell you that the probability of matching some real phone number is actually quite high.

Moreover, what exactly were they testing? i.e. if they were testing nonexistent numbers, then how would they even know the phones were blocked in the end?

Re:The day my script killed 10k phones in South America by shaan7 on Tuesday October 19, 2021 @10:22AM (#28918380)
> Basic maths should tell you that the probability of matching some real phone number is actually quite high.

Yeah well, the thing about pressure (of the sort the author has describing) is that you make mistakes - the kind where some times 3+3 = 9.

Source: I queued jobs to nuke the whole production database (relax, we had automated backups+verification so we recovered fast enough; and then added some gating so make this more difficult in the future)

Re:The day my script killed 10k phones in South America by jihadjihad on Tuesday October 19, 2021 @10:50AM (#28918690)
I pass no judgment, but merely take the developer at his word: https://new.pythonforengineers.com/blog/confessions-of-a-1x-...
Re:The day my script killed 10k phones in South America by svrourke on Tuesday October 19, 2021 @04:30PM (#28923366)
This link is the cure to imposter syndrome, this is an actual imposter.

I don't think they're even actually interested in programming and I cannot comprehend why they do it as a job or why they got hired.

Re:The day my script killed 10k phones in South America by polartx on Tuesday October 19, 2021 @11:51AM (#28919497)
Why would he write and _publish_ this?!? Is this some kind of humiliation fetish? These are things that should maybe be admitted in a confessional, or therapy.
Re:The day my script killed 10k phones in South America by throwdecro on Tuesday October 19, 2021 @10:32AM (#28918497)
Being somewhat generous, I don't think it's because they didn't understand the probabilities, it's because they didn't think about the impact of what they were doing at all, beyond the problem they needed to solve.

I.e., the only thing they were considering was whether the number entered at the start of the test would show up in the right places at the other end of their system. They simply didn't consider that an actual phone would be shut off along the way.