News for hackers. Stuff that matters.
Re:The Decline of Computers as a General Purpose Technology ... by makeitdouble on Saturday October 21, 2023 @10:02PM (#37972204)
There's many situations where manually dealing with 100+ records is a decent tradeoff. I feel people in our field overestimate the actual complexity of many dumb looking tasks.

From your bank employee example, it looks like your solution would be to open the bank's internal tool to her excel instance, have it somewhat find the right records and inject the right value, while checking the listed codes. That looks like a series of APIs if they care about data exposure, or huge CSV exports that they need to then track ?

And as it seems to be important enough to warrant verification, it means your program is not some random batch, it needs significant validation.

Then picture the internal tool changing its output format. Reports getting categorized differently because of organizational change. New fields being added to follow new legislations. The tool could break at any of these changes, and the lady's left with a broken system except a lot of money has been invested, it's supposed to have solved her problem, she might not have the same access to data she had before the integration, and overall the situation could be worse than before.

That feels like a bleak scenario, but it happens all the time as well.

Re:The Decline of Computers as a General Purpose Technology ... by ethbr1 on Saturday October 21, 2023 @10:08PM (#37972236)
We were able to fix it up with some automation in a couple days.

Critically, still running visually on her machine, and with internal resources for lifecycle support if the process changes.

Re:The Decline of Computers as a General Purpose Technology ... by makeitdouble on Saturday October 21, 2023 @10:40PM (#37972367)
> Critically, still running visually on her machine, and with internal resources for lifecycle support if the process changes.

That's such an elegant and perfect solution. She keeps track of what's happening, it all runs through her GUI so worse* case scenario she does it herself again, and she can get help if/when she needs it.

* actual worse case would be the script going berserk, but as she's not the one who wrote it, she should be shielded from the blame if that ever happens

Re:The Decline of Computers as a General Purpose Technology ... by troupe on Saturday October 21, 2023 @10:22PM (#37972290)
Do you have any guess how long it will take to break even? Presumably you charged some amount to spend a few days automating it and hopefully it frees her up to do something more valuable for the company.

Sometimes the reason things aren't automated is because there isn't anything more valuable for that person to do so automation doesn't have a positive return on investment.

Re:The Decline of Computers as a General Purpose Technology ... by ethbr1 on Saturday October 21, 2023 @10:46PM (#37972395)
It was an internal citizen developer initiative that I was supporting externally. Don't know her salary, so couldn't say.

> because there isn't anything more valuable for that person to do

I've seen an interesting shift post-COVID where (remaining) ops workers are often more coordinators of / responsible for their area of responsibility.

Which is a distinction because in that sort of job, not having Task X leads directing into finding something else you can be doing to improve your area.

Re:The Decline of Computers as a General Purpose Technology ... by concordDance on Sunday October 22, 2023 @01:03AM (#37972964)
Can't speak for them, but I've done office automation work with a breakeven measured in days.
Re:The Decline of Computers as a General Purpose Technology ... by criddell on Sunday October 22, 2023 @06:56AM (#37974336)
I’ve automated things where the breakeven will occur decades from now. It was fun though, the chance of errors is decreased, and I learned a bunch of new stuff.

Sometimes it’s about the journey and not the destination.

Re:The Decline of Computers as a General Purpose Technology ... by ethbr1 on Sunday October 22, 2023 @09:27AM (#37975173)
Calculating breakeven ROI is part of the push from a lot of companies to start citizen developer initiatives.

The point being that the "I" is also flexible -- an existing employee fooling around to learn automation whenever they have downtime is much cheaper than hiring a consultant.

Sure, the automation is worse and delivered more slowly. But you can get automations from a lot of people in parallel, plus you're upskilling your workforce.

Re:The Decline of Computers as a General Purpose Technology ... by johnmaguire on Saturday October 21, 2023 @10:17PM (#37972272)
Finding internal resources to fix the tool when things change is very prescient.

Growing up, my mom performed a job where she would essentially copy car crash data from one system into another, row by row. As a teenager learning to program, it was obvious to me this could be easily automated. She (perhaps rightfully) didn't want to run anything I cooked up on her company's computer!

A year or so later she was laid off a long with many others in the company performing similar roles.

And this is the problem, isn't it? Once the job is automated, the analyst is no longer necessary. You really need to "learn to fish."

Re:The Decline of Computers as a General Purpose Technology ... by jiggawatts on Saturday October 21, 2023 @10:07PM (#37972233)
It’s basically a JOIN, the most fundamental data manipulation operator. If you can’t automate this, then you don’t “get” computers at all.

I once had two guys arguing next to me about how it was impossible to produce a simple report about VM storage utilisation in under a month. I produced the report, printed it out, and shoved it in their hands just to shut them up while they were still busy arguing. A “month” of work required five joins.

Re:The Decline of Computers as a General Purpose Technology ... by johnmaguire on Saturday October 21, 2023 @10:14PM (#37972256)
> It's basically a JOIN

The comment you're replying just explained why this is not "basically a JOIN." There are two unrelated systems (not SQL databases) which don't expose any APIs to the analyst.

Re:The Decline of Computers as a General Purpose Technology ... by tuatoru on Saturday October 21, 2023 @10:40PM (#37972366)
It's still basically a join. If it's on a computer, it can be converted to text files, and then it can be JOINed, one way or another.
Re:The Decline of Computers as a General Purpose Technology ... by ethbr1 on Saturday October 21, 2023 @10:49PM (#37972409)
My sibling from another mother. That's exactly what we did.
Re:The Decline of Computers as a General Purpose Technology ... by csomar on Saturday October 21, 2023 @11:57PM (#37972703)
In these cases, the work is usually about connecting the systems together than the data modification itself.
Re:The Decline of Computers as a General Purpose Technology ... by TeMPOraL on Sunday October 22, 2023 @05:37AM (#37974029)
No, that's just made up work that's not required to solve a problem. Systems are already connected enough by virtue of being accessible by the same person on the same computer at the same time. Turning this into software project and building Proper Integration is only creating a tight coupling between the systems, which will only keep creating bugs and requires ongoing maintenance.
Re:The Decline of Computers as a General Purpose Technology ... by c0pium on Sunday October 22, 2023 @04:27AM (#37973754)
No they didn’t. They made up out of whole cloth a bunch of nonsense and then acted smug about how other people don’t get it. Then OP posted that it was in fact just a join.

Stop being so helpless.

Re:The Decline of Computers as a General Purpose Technology ... by nicbou on Sunday October 22, 2023 @05:46AM (#37974058)
This is a pointlessly mean and unpleasant comment. It was polite discussion up until that point.
Re:The Decline of Computers as a General Purpose Technology ... by bloak on Sunday October 22, 2023 @04:24AM (#37973736)
I think you're right. I've sometimes been in the situation of having to maintain a script that depends on APIs or data formats that change from time to time without warning. A fair amount of skill and effort is required. (There is skill involved in writing the script so that it is maintainable and not too fragile but still detects when something has gone wrong.) If the script is doing a task that has to be completed on a particular day every month, rather than whenever the script hacker is next available, then you'd have to choose between paying the hacker a reasonable retainer or having someone around who you know can still do the task manually, which means you might have to get them to do the task manually from time to time to stay in practice.

A dilemma I keep facing in practice is converting a text that someone else has produced in their own special way, like the ghastly cluttered HTML from a word processor. For example, do I attempt to use a Perl script to automatically convert some of the mark-up, or do I throw away all the mark-up and put back the italics by hand, with the risk of making a mistake? (If anyone knows of a good configurable or modifiable tool for converting cluttered HTML into sane HTML/XML I'd be interested to hear about it. My Perl scripts for doing it are not great.)

Re:The Decline of Computers as a General Purpose Technology ... by danielbln on Sunday October 22, 2023 @05:17AM (#37973945)
Sounds like a good usecase for an LLM.
Re:The Decline of Computers as a General Purpose Technology ... by ben_w on Sunday October 22, 2023 @11:01AM (#37975942)
Perhaps, but LLMs while may be slightly better (or slightly worse) than average humans, if you have 20 instances of the same LLM will have correlated errors whereas 20 humans will be… well, not entirely[0] uncorrelated, but less correlated, in their errors.

[0] We'll still sometimes get an entire office of telephone help-desk people who are all convinced that $1/megabyte is the same as $0.01/megabyte, or whatever that story was a few decades back (I can't Google it any more to find it, there was a recording of the phone call).

Re:The Decline of Computers as a General Purpose Technology ... by genewitch on Sunday October 22, 2023 @01:18PM (#37977218)
it was 0.01¢ and the rep / software was billing it as $0.01 - saying 0.01 of a penny is $0.01.

I may be off by a factor of 10.

Re:The Decline of Computers as a General Purpose Technology ... by ethbr1 on Sunday October 22, 2023 @09:23AM (#37975142)
Over the years, I've internalized a few rules to soften what you point out.

* Understand both the cost of making a mistake and of the process not running. Be aggressive on things that are 'nice to have' or have downstream manual re-validation. Be conservative on things that are 'must have' and immediately effect an outcome.

* First, do not harm -- program defensively and break execution on any violated expectations

* Understandable error messages at the user level are gold. The user has no idea what "schema mismatch" means. They can understanding "Column 3 had unexpected value. Was expecting 'X'. Saw: 'Y'"

--

On ugly HTML, that's a bad one, because of how ugly it can get.

I've only done toy work with it, but I hear good things about Beautiful Soup (Python). https://beautiful-soup-4.readthedocs.io/en/latest/