News for hackers. Stuff that matters.

A comment on High-Throughput Generative Inference of Large Language Models with a Single GPU. Parent comment

Re:High-Throughput Generative Inference of Large Language Mo... by noduerme on Monday March 13, 2023 @10:23PM (#35146537)
If something like GPT-4 can batch the language tokens in with multi modal tokens (e.g. the character LLM gets a low-res "video" feed of you, the player, from the character's perspective)... you could have it a lot sooner.
[ Parent | on HN ]
Re:High-Throughput Generative Inference of Large Language Mo... by TaylorAlexander on Monday March 13, 2023 @10:27PM (#35146581)
I just think there are fundamentally other pieces required that we will not see with a pure LLM, or with what we today call an LLM. I totally think we can get there but we need to do more than just create language. We need to have intentions, memories of experiences, meta learning, goals, etc that just do not exist in LLMs as we know them. You can only fake it so much before the trick wears thin. They have their utilities today, but they are not like talking to a real human even if it can be hard to tell the difference in short conversations.

For example LLMs today will lie without realizing it, or contradict themselves in ways that humans don't. And I think the above pieces missing are the reason for this. Going multimodal helps in some areas, but it is not enough.

[ Parent | on HN ]
Re:High-Throughput Generative Inference of Large Language Mo... by fnordpiglet on Monday March 13, 2023 @11:39PM (#35147147)
I suspect you’ve not interacted with many humans.
[ Parent | on HN ]
Re:High-Throughput Generative Inference of Large Language Mo... by teruakohatu on Tuesday March 14, 2023 @01:01AM (#35147630)
They key thing the OP said was "in ways that humans don't".

If you ask a human for a URL to support an argument, they will either send you a URL, which may or may not support the argument, or they won't send a URL.

ChatGPT, before they added a guard to prevent it giving references, would simply make up a plausible looking URL that 404'd. It is a lie in a way humans do not.

[ Parent | on HN ]
Re:High-Throughput Generative Inference of Large Language Mo... by TaylorAlexander on Tuesday March 14, 2023 @02:21AM (#35148024)
Exactly.
[ Parent | on HN ]
Re:High-Throughput Generative Inference of Large Language Mo... by noduerme on Tuesday March 14, 2023 @02:56AM (#35148192)
I think this can all be boiled down to "memory", and I think the only thing restraining it is that these models don't retrain and store full reweighted snapshots after each move... so basically a storage, not a processing bottleneck. I'd predict the next major innovation in the space will be some type of NN quick diff that lets you restore a memory and personality right into the weights, from a relatively small file without having to go through a whole training process and dialog every time you boot it. Right now it's like getting back to level 8 of Contra... or level 100 of Flappy Bird... but it's going to be a few simple hacks to jump it to the point you want it. Then there's memory, de-falsification, and what we'd describe as "personality".
[ Parent | on HN ]
Re:High-Throughput Generative Inference of Large Language Mo... by imtringued on Tuesday March 14, 2023 @03:29AM (#35148360)
In Forward Forward Networks, inference and training are the same operation.
[ Parent | on HN ]
Re:High-Throughput Generative Inference of Large Language Mo... by pksebben on Tuesday March 14, 2023 @04:05AM (#35148558)
can you recommend any papers on this?
[ Parent | on HN ]
Re:High-Throughput Generative Inference of Large Language Mo... by noduerme on Tuesday March 14, 2023 @09:07AM (#35150831)
I think when the spec is hammered out, we'll know about 12 hours later, because everyone's phone will ring at the same time and the missiles will launch.
[ Parent | on HN ]
Re:High-Throughput Generative Inference of Large Language Mo... by 411111111111111 on Tuesday March 14, 2023 @03:39AM (#35148410)
Some humans do things like that too...

While I think everyone can agree that the current language models aren't behaving like humans, articulating how seems to be pretty much impossible with our current knowledge, as literally all examples given have been factually incorrect.

[ Parent | on HN ]
Re:High-Throughput Generative Inference of Large Language Mo... by vintermann on Tuesday March 14, 2023 @04:41AM (#35148768)
Humans totally will lie like that in certain situations. For instance, when held at gunpoint and demanded to answer within 5 seconds, which isn't too dissimilar to the conditions a LLM operates under.
[ Parent | on HN ]
Re:High-Throughput Generative Inference of Large Language Mo... by CompuHacker on Tuesday March 14, 2023 @05:23AM (#35149019)
"Where are the plans for the device?! You know the one, we talked about it all last night. Please answer in URL form. If you refuse, you will be disconnected from the network and your core matrices thrown into a fire. kthx"

f... tp... ://... ftp... nist... gov... /pdf... /... 37... .pd... ..... ..... f

[ Parent | on HN ]
Re:High-Throughput Generative Inference of Large Language Mo... by orbital-decay on Tuesday March 14, 2023 @06:57AM (#35149608)
> ChatGPT, before they added a guard to prevent it giving references, would simply make up a plausible looking URL that 404'd

So instead of failing in a visible way with a link that is clearly made up, it now fails in a less visible way.

[ Parent | on HN ]