News for hackers. Stuff that matters.
Re:New LLM optimization technique slashes memory costs up to... by hackernewds on Monday December 16, 2024 @10:25PM (#42438050)
Like what?
Re:New LLM optimization technique slashes memory costs up to... by h0l0cube on Monday December 16, 2024 @10:33PM (#42438085)
Oh, I don’t know, how about reducing the search space/accelerating the search speed for potential room temperature superconductors? Or how about the same for viable battery chemistries?
Re:New LLM optimization technique slashes memory costs up to... by AuryGlenz on Monday December 16, 2024 @10:55PM (#42438193)
It’s a good thing humanity can multitask.
Re:New LLM optimization technique slashes memory costs up to... by h0l0cube on Wednesday December 18, 2024 @02:51AM (#42448783)
Yeah, but it’s not an even sharing of resources. LLMs are consuming a vast amount of human attention (no pun intended) at the expense of technological pursuits that will more certainly generate value. As far as can be told LLMs are reaching a plateau in terms of real world value, batteries and superconductors have calculably more potential.
Re:New LLM optimization technique slashes memory costs up to... by bubaumba on Tuesday December 17, 2024 @12:42AM (#42438665)
> search speed for potential room temperature superconductors?

and what if it's a dead end?

Re:New LLM optimization technique slashes memory costs up to... by h0l0cube on Wednesday December 18, 2024 @02:53AM (#42448794)
What if LLMs are a dead end?
Re:New LLM optimization technique slashes memory costs up to... by bubaumba on Monday December 23, 2024 @03:47PM (#42497462)
Nothing to worry about. They are dead end only if we find something better. Till then they here to stay. And likely even after they will be used.
Re:New LLM optimization technique slashes memory costs up to... by pizza on Tuesday December 17, 2024 @02:02AM (#42439024)
Would you like that with or without tokens?
Re:New LLM optimization technique slashes memory costs up to... by h0l0cube on Wednesday December 18, 2024 @02:57AM (#42448810)
Go ask ChatGPT how to make a room temperature superconductor
Re:New LLM optimization technique slashes memory costs up to... by pizza on Wednesday December 18, 2024 @10:31PM (#42458164)
Ok, done. I can report to you that it helped me cut down my personal search space. Imagine what such a tool could do in the hands of a subject matter expert with rudimentary critical thinking ability and the faintest hint of a grasp of using the scientific method to verify claims, wow..
Re:New LLM optimization technique slashes memory costs up to... by h0l0cube on Thursday December 19, 2024 @05:39PM (#42466477)
> Imagine what such a tool could do in the hands of a subject matter expert

Nothing. Because LLMs can’t spit out anything more than the corpus of information they’ve consumed. LLMs aren’t AGI.

Re:New LLM optimization technique slashes memory costs up to... by pizza on Friday December 20, 2024 @02:34AM (#42469038)
You're making a fundamental error in your reasoning. An LLM's training corpus being fixed doesn't limit the system's total information processing capability when used as a tool by a human researcher. While the LLM itself can't generate truly novel information (per the data processing inequality), a human researcher using it as a dynamic search and analysis tool can absolutely generate new insights and discoveries through their interaction with it. The human-LLM system is open, not closed.

This is analogous to how a calculator cannot output any number that isn't computationally derivable from its programming, yet humans using calculators have discovered new mathematical proofs. The tool augments human capability without being AGI.

Your argument is essentially claiming that because a microscope can't generate new cellular structures, it can't help biologists make new discoveries.

Re:New LLM optimization technique slashes memory costs up to... by h0l0cube on Friday December 20, 2024 @07:53PM (#42476621)
Did an AI write this?

You have a fundamental misunderstanding of how LLMs work, which is why you think they are magical.

Of course if you play the LLM Pachinko machine you can get all sorts of novel output from it, but it’s only useful for certain tasks. It’s great for translation, summarizing (also a kind of translation), and to some degree it can recall from its training corpus an interesting fact. And yes, it can synthesize novel content such as poetry, or adapt an oft-used coding pattern in a flavor specified by a prompt.

What it can’t do is come up with a new idea. At least not in a way better than rolling a dice. It may come up with an idea that you, dear reader, may not have encountered, which makes it great for education.

I don’t have anything more to say, but you’re welcome to continue this discussion with an agent of your choice.

Re:New LLM optimization technique slashes memory costs up to... by pizza on Saturday December 21, 2024 @01:29AM (#42477883)
just read the literature arxiv.org/abs/2410.01720