From what I have seen, most of the jobs that LLMs can do are jobs that didn't need to be done at all. We should turn them over to computers, and then turn the computers off.

They're good at processing text. Processing text is a valuable thing that sometimes needs to be done.

We still use calculators even though the profession we used to call "computer" was replaced by them.

But here reliability comes in again. Calculators are different since the output is correct as long as the input is correct.

LLMs do not guarantee any quality in the output even when processing text, and should in my opinion be verified before used in any serious applications.

> Calculators are different since the output is correct as long as the input is correct.

That isn't really true.[0] The application of calculators to a subject matter is something that does need to be considered in some use cases.

LLMs also have accuracy considerations, and although it may be to a different degree, the subject matter to which they're applicable has a broad range of acceptable accuracies. While some textual subject matter demands a very specific answer, some doesn't: For example, there may be hundreds or thousands of various ways to summarize a text that could be accurate for a particular application.

0: example: https://www.reddit.com/r/calculus/comments/upjdn4/why_do_all...

I think your point stands, but your example shows that anyone using those calculators daily should not be concerned. Those that need precision to the 6+ decimal places for complex equations should know not to fully trust consumer-grade calculators.

The issue with LLMs is that they can be so unpredictable in their behaviour. Take the following prompt that asks GPT-4 to validate the response to "calculate 2+3+5 and only display the result":

https://beta.gitsense.com/?chat=6d8af370-1ae6-4a36-961d-2902...

GPT-4o mini contradicts itself, which is not something one would expect for something we believe to be extremely simple. However, if you ask it to validate the response to "calculate 2+3+5," it will get it right.

https://beta.gitsense.com/?chat=43221de5-bff6-487a-8c0f-48ca...

By adding "and only display the result," GPT-4o mini was thrown for a loop; examples like this should give us pause.

Well, not every tool is a hammer and not every problem is a nail.

If I ask my TI-89 to "Summarize the plot in Harry Potter and the Chamber of Secrets" it responds "ERR"! :D

LLMs are good text processors, pocket calculators are good number processors. Both have limitations, and neither are good at problem sets that are outside of their design strengths. The biggest problem with LLMs aren't that they are bad at a lot of things, it's that they look like they are good at things they aren't good at.

I agree LLMs are good at text processing and I believe they will obsolete jobs that really should be obsoleted. Unless OpenAI, Anthropic and other AI companies come up with a breakthrough on reliability, I think it will be fair to say they will only be players and not leaders. If they can't figure something out, it will be Microsoft, Amazon and Google (distributors of diverse models) that will benefit the most.

I've personally found it is extremely unlikely for multiple good LLMs to fail at the same time, so if you want to process text and be confident in the results, I would just run the same task across 5 good models and if you have a super majority, you can be confident that it was done right.

Neither are humans, that's why we have proofreaders and editors. That doesn't make them any less useful. And a translator will not write the same exact translation for a text longer than a couple of sentences, that does not mean translation is a dead end. Ironically, it's LLMs that made translation a dead end.

> Previous generations of neural nets were kind of useless. Spotify ended up replacing their machine learning recommender with a simple system that would just recommend tracks that power listeners had already discovered.

“Previous generations of cars were useless because one guy rode a bike to work.” Pre-transformer neural nets were obviously useful. CNNs and RNNs were SOTA in most vision and audio processing tasks.

Language translation, object detection and segmentation for autonomous driving, surveillance, medical imaging... Indeed plenty fields where NNs are indispensable

Yeah, give 'em small constrained jobs where the lack of coherent internal representation is not a problem.

I was involved in ANN and equivalent based face recognition (not on the computational side, on the psychophysics side) briefly. Face recognition is one of these bigger more difficult jobs, but still more constrained than the things ANNs are useful for.

As far as I understand none of the face recognition algorithms in use these days are ANN based, but are instead computationally efficient versions of the brute force the maths implementations instead.

> LLMs today can already automate many desk jobs.

No they can't because they make stuff up, fail to follow directions, need to be minutely supervised, all output checked and workflow integrated with your companies shitty over complicated procedures and systems.

This makes them suitable at best as an assistant to your current worker or more likely an input for your foo as a service which will be consumed by your current worker. In the ideal case this helps increase the output of your worker and means you will need less of them.

An even greater likelihood is someone dishonest at some company will convince someone stupid at your company that it will be more efficacious and less expensive than it will ultimately be leading your company to spend a mint trying to save money. They will spend more than they save with the expectation of being able to lay off some of their workers with the net result of increasing workload on workers and shifting money upward to the firms exploiting executives too stupid to recognize snake oil.

See outsourcing to underperforming overseas workers because the desirable workers who could have ably done the work are A) in management because it pays more B) in country or working remotely for real money or C) cost almost as much as locals once the increased costs of doing it externally are factored in.

> No they can't because they make stuff up, fail to follow directions, need to be minutely supervised, all output checked and workflow integrated with your companies shitty over complicated procedures and systems.

What’s the difference between what you describe and what’s needed for a fresh hire off the street, especially one just starting their career?

> What’s the difference between what you describe and what’s needed for a fresh hire off the street, especially one just starting their career?

The fresh hire has the potential that after training and working for a while to become a much more valuable and reliable senior.

> has the potential

Good choice of wording! Definitely not a given though.

The bell curve is much wider for humans than LLMs, I don't think this needs to be said.

The difference is that a LLM is like hiring a worst-case scenario fresh hire that lied to you during the interview process, has a fake resume and isn't actually named John Programmer.

Entirely disagree.

Real talk? The human can be made to suffer consequences.

We don't mention this in techie circles, probably because it is gauche. However you can hold a person responsible, and there is a chance you can figure out what they got wrong and ensure they are trained.

I can’t do squat to OpenAI if a bot gets something wrong, nor could I figure out why it got it wrong in the first place.

boy do I love being in the same industry as people like you… :) while you are writing silly stuff like this us that do shit have automated 40-50% of what we used to do and not have extra time to do more amazing shit :)

> Spotify ended up replacing their machine learning recommender with a simple system that would just recommend tracks that power listeners had already discovered.

Do you have a source on this? Spotify also seems to employ a few different recomendation algorithms, for example Discover Weekly vs. continuing to play after a playlist ends. I'd be surprised if Discover Weekly didn't employ some sort of ML as it does recommend songs I have never heard before many times.

It's from the book by Carlsson and Leijonhufvud. Perhaps Spotify uses ML today, but the key insight from the book was that no ML was needed to build a recommender system. You can just show people songs from custom playlists curated by powerusers. So when your playlist ends you find other high quality playlists that overlap with the music you just listened to. Then you blend those playlists and enqueue new tracks. This is from memory so I might have gotten the details wrong, but I remember that this approach worked like magic and solved the issues with the ML system (bland or too random recommendations). No reason to use ML when you already have millions of manually curated playlists.

gizmo says>LLMs today can already automate many desk jobs.

I call: show me five actual "desk jobs" that LLMs have "already automated". Not merely tasks, but desk jobs - jobs with titles, pay scales, retirement plans, etc. in real companies.

But it is not all or nothing here. We replaced real programmers (backend, frontend, embedded) with it, but obviously (I guess) not all. We just require 1/5th of those roles since around beginning this year. There are a lot more 'low level' jobs in tons of companies where we see the same happening because suddenly the automation is trivial to make instead of 'a project'. It will take time for the bigger ones and it won't 'eliminate' all jobs of the same type (maybe it will in time), but it will eliminate most people doing that job as now 1 people can do the work of 5 or more.

I guess we will see the actual difference in 5-10 years in the stats. Big companies are mostly still evaluating and waiting. Maybe it will remain just a few blibs and it'll fizzle out, or maybe, and this is what I expect, the effect will be a lot larger, moving many to other roles and many completely out of work.

On a small (we see many companies inside, but many is relative, of course), but real life examples I see are translators, programmers, seo/marketing writers, data entry (copying content from pdf to excel, human webscraping etc) being replaced now.

Imagine the face of a guy who needs to do the work of 5 solo now... He is probably the happiest employee now and his salary raised 5-fold, surely yeah?

Hopefully that'll finish off big companies

I know an immigration agent who simply stopped using professional translators because ChatGPT is more than good enough for his purposes. In many ways it is actually better, especially if instructed to use the specific style and terminology required by the law.

If you think about it, human calculators (the job title!) were entirely replaced by digital electronic calculators. Translators are simply "language calculators" that perform mechanical transformations, the ideal scenario for something like an LLM to replace.

That’s professional negligence. Have the LLM prepare a draft for a human translator to review, sure. But taking the human out of the loop and letting in undetectable hallucinations? In a legal proceeding?

These are bold claims.

> This time it's different. The naysayers are wrong.

This has been said every time. What is fundamentally different this time?

> LLMs today can already automate many desk jobs.

Which desk jobs existed 2 years ago that don’t exist today because LLMs have automated them away?

If you had to bet a large amount of your own money on a scenario where you have a 3200 word text and you ask ChatGPT to change a single sentence, would you bet on or against that it would change something other than what you asked it to change? I would bet that it would, every time (even with ChatGPT's new document feature). There aren't a lot of employers who are okay with persistent randomness in their output.

If there's a job that can be entirely replaced by AI, it was already outsourced to an emerging market with meager labor costs (which at this point, is likely still cheaper than a fully automated AI).