News for hackers. Stuff that matters.
Re:Meta AI Unleashes Megabyte, a Revolutionary Scalable Mode... by epups on Wednesday May 24, 2023 @01:18AM (#36054357)
By what criteria? In several professional capacities, including coding, LLM's are far superior to an average human already.
Re:Meta AI Unleashes Megabyte, a Revolutionary Scalable Mode... by slowmovintarget on Wednesday May 24, 2023 @01:26AM (#36054405)
No they're not. They're superior at specific tests when prompted.

But asking for a complete application nearly always fails, when an average human that knows programming should be able to make one that works.

Re:Meta AI Unleashes Megabyte, a Revolutionary Scalable Mode... by CamperBob2 on Wednesday May 24, 2023 @01:34AM (#36054455)
But asking for a complete application nearly always fails, when an average human that knows programming should be able to make one that works.

"This talking dog is a dumbass. The risotto recipe he gave me sucked, and the C++ code he wrote is full of security holes. I don't see what all the hype is about."

Hint: ML will get better. Humans will not. ("Slow moving target," indeed.) What we're learning is just how many aspects of our vaunted 'intelligence' are really just features of our languages and the graphs they encode.

Re:Meta AI Unleashes Megabyte, a Revolutionary Scalable Mode... by riffraff on Wednesday May 24, 2023 @02:16AM (#36054707)
This might be true but it's irrelevant to the claim "LLMs are already better than humans".
Re:Meta AI Unleashes Megabyte, a Revolutionary Scalable Mode... by CamperBob2 on Wednesday May 24, 2023 @03:00AM (#36054942)
You are fixated on the current state of the art, when the first couple of time derivatives are what actually matter. The assertion in question is already true in a limited sense, and it's only going to go in one direction from here.
Re:Meta AI Unleashes Megabyte, a Revolutionary Scalable Mode... by ben_w on Wednesday May 24, 2023 @12:02PM (#36059991)
Google Translate, which IIRC is also a Transformer model like GPT these days, knows more natural languages than I can remember the names of, to a higher standard than I know my best non-native language, and I moved to Germany 5 years ago.

GPT-3 knows most programming languages better than I do, even though I literally learned to read with the Commodore 64 user manual back in the 80s and haven't stopped being a nerd since; and while GPT code isn't always correct or even compilable, it's not like I don't still make mistakes that cause compilation to fail a few times a day, and there's a reason we all insist on testing code rather than just assuming it will work when a dev stops typing.

Again, I acknowledge GPT-3.5 (and I assume 4 but have not used it) is not perfect: like others I'd call 3.5 a "junior developer" (and it isn't even that good in every subject!); yet, despite that, the only domains where I can regularly beat it at those where my perception of reality is fundamentally different from its perception (how words sound and how numbers are composed, so it's relatively bad at arithmetic and rhyme), or where it has been forced to be bad (ask it to write about conflict, my experience is everyone reconciles and lives happily ever after).

But in most cases it would take me years to get as good as it already is. And there are more of those subjects than I can remember the names of, too.

Re:Meta AI Unleashes Megabyte, a Revolutionary Scalable Mode... by p-e-w on Wednesday May 24, 2023 @01:46AM (#36054526)
> when an average human that knows programming should be able to make one that works.

Very few people who "know programming" are actually capable of creating a complete application that serves a specific purpose. Many professional programmers struggle to implement basic algorithms like prime number testing. Coding AIs absolutely outperform the average programming professional, because the average programming professional can barely program. The software industry's demand for programmers is far too great for every coding position to be filled with top-class full stack engineers.

Re:Meta AI Unleashes Megabyte, a Revolutionary Scalable Mode... by shreyshnaccount on Wednesday May 24, 2023 @02:50AM (#36054892)
Who is this average programmer you're talking about? We did primality testing in literally the first sem at uni
Re:Meta AI Unleashes Megabyte, a Revolutionary Scalable Mode... by pedrosorio on Wednesday May 24, 2023 @04:21AM (#36055389)
Take a look at this old, commonly referenced article, might be before your time:

https://blog.codinghorror.com/why-cant-programmers-program/

Re:Meta AI Unleashes Megabyte, a Revolutionary Scalable Mode... by TeMPOraL on Wednesday May 24, 2023 @05:07AM (#36055634)
> We did primality testing in literally the first sem at uni

There's a lot of things you did during university classes. I bet you don't even remember half of them, and of the half you do, you couldn't actually do most of them from memory right now.

The advantage LLMs have over programmers is that they've seen much more code than any human ever would, remember pretty much all of it - not necessarily verbatim, but also as learned concepts, and that this knowledge does not decay. It's all equally accessible. Whether you ask LLM for basic or optimized primality testing algorithms, or how to use some obscure part of Microsoft's DCOM that approximately nobody on the planet wrote new code for in the last decade - they'll perform equally well. Not perfect, but not worse than an average software dev that does remember the things they're asked to work with.

Re:Meta AI Unleashes Megabyte, a Revolutionary Scalable Mode... by shreyshnaccount on Wednesday May 24, 2023 @08:37AM (#36057163)
Yeah, no contest, llms can remember things. Big deal. You're comparing memory with intelligence. It's a part of it sure, but there's also cognition, reasoning and i don't even know what more
Re:Meta AI Unleashes Megabyte, a Revolutionary Scalable Mode... by TeMPOraL on Wednesday May 24, 2023 @10:25AM (#36058575)
> You're comparing memory with intelligence. It's a part of it sure, but there's also cognition, reasoning and i don't even know what more

Yes, and LLMs show all of the specific things you've listed.

Re:Meta AI Unleashes Megabyte, a Revolutionary Scalable Mode... by shreyshnaccount on Wednesday May 24, 2023 @12:08PM (#36060097)
Ah, here is our disagreement. I see what you mean but I'm not totally convinced by that.. reasoning isn't just being able to verbalize the steps you take, which llms can do but more so the steps themselves, and in my experience llms can fail in that department. Maybe they are somewhere between intelligent and not intelligent? In my opinion that's far more likely, as most things are not usually binaries but spectrums in our world.
Re:Meta AI Unleashes Megabyte, a Revolutionary Scalable Mode... by TeMPOraL on Wednesday May 24, 2023 @08:05PM (#36065572)
To be clear, I'm not claiming LLMs are already better than humans in general sense, across many domains of cognition. I do believe however that, in the aspects you mention, SOTA LLMs (primarily GPT-4) fairly close to humans on the spectrum - they're "playing the same game", so to speak, in the sense in which prior computer systems, and most life forms on Earth, are not.
Re:Meta AI Unleashes Megabyte, a Revolutionary Scalable Mode... by qup on Wednesday May 24, 2023 @02:43AM (#36054861)
The LLM knows it doesn't work, too, we just don't run it long enough to let it try it out. It would try, get an error, and know how to handle the error.

https://github.com/drifting-in-space/botsh

Check out some agents that probably can already do more than you knew about.