News for hackers. Stuff that matters.
Re:I don’t belong in tech by Slartie on Saturday November 26, 2016 @12:42PM (#13044866)
Totally seconding this.

Just trying to "quick fix" apparent bugs, without having understood the complex problem in its entirety beforehand, is a pretty bad approach in quite some areas of the tech world. This is true for most server-side work, for example - you just can't put a band-aid type solution on a problem created by a weird race condition between different requests processed in parallel, because you most likely either ruin the performance or don't catch all the cases in which the bug appears, or even worse: create new bugs, deadlock situations or similar catastrophes as a result.

I personally work on cash register software. In that area, it's basically this way for the server and client side of the equation, because if this highly business-critical piece of technology fails, the user really has a problem and typically can't do any business anymore. And the software release cycles on those machines also don't allow for continuous-deployment-we'll-just-test-the-fix-in-production-so-it-doesn't-matter-if-your-first-attempts-fail-miserably-style of work, so you have to get your bugs fixed thoroughly, which requires fully understanding the nature of the problem first.

Re:I don’t belong in tech by Illniyar on Saturday November 26, 2016 @12:53PM (#13044927)
"You just can't put a band-aid type solution on a problem created by a weird race condition"

Oh yes you can, and do, often, in payment processing code. I've done it (with a simple sleep for x seconds I think), and I can assure you the higher ups couldn't have been happier.

Re:I don’t belong in tech by Slartie on Saturday November 26, 2016 @02:02PM (#13045305)
Oh, so maybe THAT is the reason why one of our customers had funny cases of thousands of his customers being double-billed on card payments by the payment processing provider!

As a developer (of the cash register software, which wasn't at fault in this case), I was kinda amused by the whole thing. But I can assure you, the higher-up business people weren't amused at all.

Re:I don’t belong in tech by Illniyar on Saturday November 26, 2016 @03:56PM (#13045802)
If your using Java, there's a popular apache http framework[0] that automatically retries a failed request unless you specifically instruct it not to.

That bit us in the A$$ with double payments a few times.

0- https://hc.apache.org/httpcomponents-client-ga/httpclient/ap...

Re:I don’t belong in tech by magnetic on Saturday November 26, 2016 @08:23PM (#13047164)
If you rely on http to ensure the one-and-only-one type of paradigm, I have bad news for you (because you won't be able to distinguish failure vs success case when you hit a timeout: did it not receive the request or did I not receive the ack?).

You should embed a transaction ID to ensure idempotence of your API call. Then you can make the request 1, 2 or 3 times without any risk. The receiving end will realize the transaction ID already transitioned state and won't cause multiple payments to occur.

Re:I don’t belong in tech by Illniyar on Sunday November 27, 2016 @12:01AM (#13047814)
Obviously. But youd be surprised, some gateways don't let you send your own transactionId. Or at least they ignored it for determining duplicate requests. Though that was several years ago.
Re:I don’t belong in tech by Illniyar on Saturday November 26, 2016 @03:47PM (#13045759)
Heh. We had tons of double billing from payment processors, we never could get a straight answer from the gateway as to why it happened.

Most of them came from.a gateway called CyberSource.

Also, lots of customers have weird banks where they see both an authorization (I.e. hold) and the actual capture, and they mistake it for two captures. Usually it helps to educate them about what is an authorization.

Re:I don’t belong in tech by chris_wot on Saturday November 26, 2016 @08:13PM (#13047111)
Man, a bank that doesn't know the difference is a bank I'd be withdrawing my funds from!
Re:I don’t belong in tech by Illniyar on Sunday November 27, 2016 @05:43AM (#13048612)
The bank knows the difference, the reports they send out to customers (or maybe what the customer's see on the site) don't make the distinction clear enough.

Most banks don't even bother showing the customer authorizations of any kind.

Re:I don’t belong in tech by kalmi10 on Saturday November 26, 2016 @02:13PM (#13045366)
I always wondered why cards get processed so slowly in some countries...
Re:I don’t belong in tech by nathan_f77 on Saturday November 26, 2016 @03:08PM (#13045587)
You absolutely shouldn't need to do that often in payment processing code. Did you go back and fix it properly later?
Re:I don’t belong in tech by Illniyar on Saturday November 26, 2016 @03:44PM (#13045746)
The sleep was as good a solution as we were willing to do.

The race condition resulted in an error syncing between two datacenters, so when it happens it can be dealt with manually.

Also, to be clear, it did not affect customer waiting time as the delay was in a webhook which we used as a backup to the main payment process.

Re:I don’t belong in tech by arca_vorago on Saturday November 26, 2016 @01:32PM (#13045152)
"Just trying to "quick fix" apparent bugs, without having understood the complex problem in its entirety beforehand, is a pretty bad approach in quite some areas of the tech world."

And a million sysadmins across the world cried aloud in agreement and despair.

The point is that I think there are business side problems more than there are tech problems in this industry as a whole. I think the phrase "the perfect is the enemy of the good" is a dangerous one for so many managers to latch onto. More on point to the article though, we need more magnum opuses, more results of this kind of long term thinking, put into action. Not more band-aids on problems.

I do think though, that big picture thinkers of this ilk do find themselves trapped into a twist on the Bertrand Russell saying, "The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt."

What we need then, are some of the intelligent to gain a bit of the cocksuredness, and the stupid to gain a bit of doubt.

Re:I don’t belong in tech by chris_wot on Saturday November 26, 2016 @01:08PM (#13045014)
I'm not entirely sure that she's right in tarring the whole industry with the same brush.

Taking a bug out to tea is fantastic, because it really is about understanding the problem. But she conflates maintenance coding and program writing. There is a lot of half-baked software out there, but to say that this isn't like any other industry - like journalism - is really patently absurd. Despite best intentions, journalism is absolutely like programming. A sub-editor is really just like your QA department in many ways. And just like she made a mistake in a script on NPR unintentionally, so too do programmers.

This piece is really a mass of contradictions. Take the following:

But here’s the thing. No one will ever remember that number. No one remembers it now, and I’m sure no one noticed it when it happened. But I knew it happened, that it was an easily preventable mistake, and, in journalism, being wrong in that way is absolutely unacceptable.

So... what is it? Was it unacceptable that the figure got into the story to the extent that she got fired? No, of course not - I'm assuming NPR ran a correction, and forgave her for her mistake.

I really doubt most programmers are going out of their way to slap together code that has deliberate bugs. Yeah, they exist but to say any software development house that writes email clients, job seeking sites or any other software are intentionally encouraging this sort of behaviour seems to be an insult to the vast majority of founders and software developers who work hard to hone their craft and take care to produce quality software.

This essay seems overly emotionally wrought and with a perspective that is out of kilter with reality. For me, despite her protestations that she is no better than all the others who work at software development, there's an implicit undertone of superiority masked by claims of victimisation and isolation. It's not particularly fair, and it seems rather manipulative to me.

Re:I don’t belong in tech by dcole2929 on Saturday November 26, 2016 @02:19PM (#13045396)
I think the difference is in how the two industries she are comparing here fundamentally treat mistakes. In journalism, you might not get fired immediately for a mistake like that but certainly you're going to get reprimanded and told how unacceptable that is. Your company is absolutely going to run a correction at the first opportunity they get. Any mistake is treated as a major embarrassment by staff of a respectable news company. The viewers/readers might not notice/care but the company does.

This is totally not how things work in tech though. First of all there are a lot more mistakes/bugs, and because of that how bugs are handled is completely different. If you're not doing something where lives/money are at stake, it's ok to release with bugs so long as they aren't showstoppers. When I write something that has a bug in it, I don't even get talked to. We just write it up and toss it in the list with all the other bugs.

There are of course situations in both industries where things are different but I think in general there is definitely a major difference in how tech treats mistakes as compare to how pretty much every other industry.

Re:I don’t belong in tech by chongli on Sunday November 27, 2016 @11:14AM (#13049853)
You should try reading science journalism. It's absolutely riddled with errors and oversimplifications that never get corrected!

Moreover, sure NPR might fire you for repeatedly using incorrect numbers for which they then have to publish retractions but that doesn't stop you from using essentially correct statistics in highly misleading ways, a sin all major media sources are guilty of.

Re:I don’t belong in tech by chris_wot on Saturday November 26, 2016 @03:42PM (#13045738)
A bit hard to respond at the moment, I've been rate limited!