Petals: Run 100B+ language models at home bit-torrent style

Re:Petals: Run 100B+ language models at home bit-torrent style by fasterik on Wednesday January 04, 2023 @05:18PM (#34252703)

You made two claims: (1) current language models are useless and (2) current language models have reached a ceiling. I said:

How many people are using ChatGPT, Stable Diffusion, etc. for economically or personally valuable activities?

If (1) is true, then the answer to that question is "zero" or at least "close to zero". Do you really believe that?

If (2) is true, then it is also true to say that transformer models will never exceed today's capabilities by a significant amount at any time in the future. Do you really believe that?

[ Parent | on HN ]

Re:Petals: Run 100B+ language models at home bit-torrent style by thomastjeffery on Monday January 09, 2023 @03:46PM (#34315785)

Yes. The ceiling is the floor.

The limitation is inherent in the core design. There is no overcoming. This is not a hurdle or a wall. It's a design flaw.

Is it totally useless to everyone? No. Not completely. It's like a coherent search engine: a way to find data that is close to other data. But "close to" in this case is only "semantically", and never "logically", so that's that.

Is it going to get any less useless than it is? Only slightly. "It" will never get better. The only better version of "it" is a completely new ground-up redesign that doesn't resemble "it" at all.

[ Parent | on HN ]

Re:Petals: Run 100B+ language models at home bit-torrent style by fasterik on Tuesday January 10, 2023 @06:42PM (#34333071)

Modern neural network architectures are Turing complete [1]. So I don't see any argument for a limit in principle unless you are arguing that a Turing machine can't achieve language understanding. If that's what you're saying, then I wonder who is espousing mysticism here.

[1] https://arxiv.org/abs/1901.03429

[ Parent | on HN ]

Re:Petals: Run 100B+ language models at home bit-torrent style by thomastjeffery on Wednesday January 11, 2023 @10:48AM (#34339748)

Are you forgetting the distinction between a program and a computer?

Language understanding doesn't magically spawn itself as a process on your computer! Someone has to write that program first.

And that's my point. ChatGPT transforms language, but it does not understand it. For that, we will need a different kind of program.

[ Parent | on HN ]

Re:Petals: Run 100B+ language models at home bit-torrent style by fasterik on Wednesday January 11, 2023 @10:31PM (#34348828)

Language understanding doesn't magically spawn itself as a process on your computer! Someone has to write that program first.

Do you think it's impossible for such a program to emerge as weights in a Turing complete neural network architecture?

[ Parent | on HN ]