News for hackers. Stuff that matters.
Re:Ilya Sutskever's SSI Inc raises $1B by hn_throwaway_99 on Wednesday September 04, 2024 @12:12PM (#41447416)
Lots of comments either defending this ("it's taking a chance on being the first to build AGI with a proven team") or saying "it's a crazy valuation for a 3 month old startup". But both of these "sides" feel like they miss the mark to me.

On one hand, I think it's great that investors are willing to throw big chunks of money at hard (or at least expensive) problems. I'm pretty sure all the investors putting money in will do just fine even if their investment goes to zero, so this feels exactly what VC funding should be doing, rather than some other common "how can we get people more digitally addicted to sell ads?" play.

On the other hand, I'm kind of baffled that we're still talking about "AGI" in the context of LLMs. While I find LLMs to be amazing, and an incredibly useful tool (if used with a good understanding of their flaws), the more I use them, the more that it becomes clear to me that they're not going to get us anywhere close to "general intelligence". That is, the more I have to work around hallucinations, the more that it becomes clear that LLMs really are just "fancy autocomplete", even if it's really really fancy autocomplete. I see lots of errors that make sense if you understand an LLM is just a statistical model of word/token frequency, but you would expect to never see these kinds of errors in a system that had a true understanding of underlying concepts. And while I'm not in the field so I may have no right to comment, there are leaders in the field, like LeCun, who have expressed basically the same idea.

So my question is, has Sutskever et al provided any acknowledgement of how they intend to "cross the chasm" from where we are now with LLMs to a model of understanding, or has it been mainly "look what we did before, you should take a chance on us to make discontinuous breakthroughs in the future"?

Re:Ilya Sutskever's SSI Inc raises $1B by thefounder on Wednesday September 04, 2024 @12:16PM (#41447445)
I think the plan is to raise a lot of cash and then more and then maybe something comes up that brings us closer to AGI(i.e something better than LLM). The investors know that AGI is not really the goal but they can’t miss the next trillion dollar company.
Re:Ilya Sutskever's SSI Inc raises $1B by petulla on Wednesday September 04, 2024 @12:26PM (#41447549)
Ilya has discussed this question: https://www.youtube.com/watch?v=YEUclZdj_Sc
Re:Ilya Sutskever's SSI Inc raises $1B by hn_throwaway_99 on Wednesday September 04, 2024 @01:11PM (#41448121)
Thank you very much for posting! This is exactly what I was looking for.

On one hand, I understand what he's saying, and that's why I have been frustrated in the past when I've heard people say "it's just fancy autocomplete" without emphasizing the awesome capabilities that can give you. While I haven't seen this video by Sutskever before, I have seen a very similar argument by Hinton: in order to get really good at next token prediction, the model needs to "discover" the underlying rules that make that prediction possible.

All that said, I find his argument wholly unconvincing (and again, I may be waaaaay stupider than Sutskever, but there are other people much smarter than I who agree). And the reason for this is because every now and then I'll see a particular type of hallucination where it's pretty obvious that the LLM is confusing similar token strings even when their underlying meaning is very different. That is, the underlying "pattern matching" of LLMs becomes apparent in these situations.

As I said originally, I'm really glad VCs are pouring money into this, but I'd easily make a bet that in 5 years that LLMs will be nowhere near human-level intelligence on some tasks, especially where novel discovery is required.

Re:Ilya Sutskever's SSI Inc raises $1B by JamesSwift on Wednesday September 04, 2024 @01:21PM (#41448214)
Watching that video actually makes me completely unconvinced that SSI will succeed if they are hinging it on LLM...

He puts a lot of emphasis on the fact that 'to generate the next token you must understand how', when thats precisely the parlor trick that is making people lose their minds (myself included) with how effective current LLMs are. The fact that it can simulate some low-fidelity reality with _no higher-level understanding of the world_, using purely linguistic/statistical analysis, is mind-blowing. To say "all you have to do is then extrapolate" is the ultimate "draw the rest of the owl" argument.

Re:Ilya Sutskever's SSI Inc raises $1B by pajeets on Wednesday September 04, 2024 @02:01PM (#41448626)
I actually echo your exact sentiments. I don't have the street cred but watching him talk for the first few minutes I immediately felt like there is just no way we are going to get AGI with what we know today.

Without some raw reasoning (maybe Neuro-symbolic is the answer maybe not) capacity, LLM won't be enough. Reasoning is super tough because its not as easy as predicting the next most likely token.

Re:Ilya Sutskever's SSI Inc raises $1B by machiaweliczny on Wednesday September 04, 2024 @02:45PM (#41449103)
They might never work for novel discovery but that probably can be handled by outside loop or online (in-context) learning. The thing is that 100k or 1M context is a marketing scam for now.
Re:Ilya Sutskever's SSI Inc raises $1B by og_kalu on Wednesday September 04, 2024 @03:28PM (#41449688)
>All that said, I find his argument wholly unconvincing (and again, I may be waaaaay stupider than Sutskever, but there are other people much smarter than I who agree). And the reason for this is because every now and then I'll see a particular type of hallucination where it's pretty obvious that the LLM is confusing similar token strings even when their underlying meaning is very different. That is, the underlying "pattern matching" of LLMs becomes apparent in these situations.

So? One of the most frustrating parts of these discussions is that for some bizzare reason, a lot of people have a standard of reasoning (for machines) that only exists in fiction or their own imaginations.

Humans have a long list of cognitive shortcomings. We find them interesting and give them all sorts of names like cognitive dissonance or optical illusions. But we don't currently make silly conclusions like humans don't reason.

The general reasoning engine that makes neither mistake nor contradiction or confusion in output or process does not exist in real life whether you believe Humans are the only intelligent species on the planet or are gracious enough to extend the capability to some of our animal friends.

So the LLM confuses tokens every now and then. So what ?

Re:Ilya Sutskever's SSI Inc raises $1B by hn_throwaway_99 on Wednesday September 04, 2024 @04:20PM (#41450307)
You are completely mischaracterizing my comment.

> Humans have a long list of cognitive shortcomings. We find them interesting and give them all sorts of names like cognitive dissonance or optical illusions. But we don't currently make silly conclusions like humans don't reason.

Exactly! In fact, things like illusions are actually excellent windows into how the mind really works. Most visual illusions are a fundamental artifact of how the brain needs to turn a 2D image into a 3D, real-world model, and illusions give clues into how it does that, and how the contours of the natural world guided the evolution of the visual system (I think Steven Pinker's "How the Mind Works" gives excellent examples of this).

So I am not at all saying that what LLMs do isn't extremely interesting, or useful. What I am saying is that the types of errors you get give a window into how an LLM works, and these hint at some fundamental limitations at what an LLM is capable of, particularly around novel discovery and development of new ideas and theories that aren't just "rearrangements" of existing ideas.

Re:Ilya Sutskever's SSI Inc raises $1B by og_kalu on Wednesday September 04, 2024 @05:10PM (#41450869)
>So I am not at all saying that what LLMs do isn't extremely interesting, or useful. What I am saying is that the types of errors you get give a window into how an LLM works, and these hint at some fundamental limitations at what an LLM is capable of, particularly around novel discovery and development of new ideas and theories that aren't just "rearrangements" of existing ideas.

ANN architectures are not like brains. They don't come pre-baked with all sorts of evolutionary steps and tweaking. They're far more blank slate and the transformer is one of the most blank slate there is.

Mostly at best, maybe some failure mode in GPT-N gives insight to how some concept is understood by GPT-N. It rarely will say anything about language modelling or Transformers. GPT-2 had some wildly different failure modes than 3, which itself has some wildly different failure modes to 4.

All a transformer's training objective asks it to do is spit out a token. How it should do so is left for transformer to figure along the way and everything is fair game.

And confusing words with wildly different meanings but with some similarity in some other way is something that happens to humans as well. Transformers don't see words or letters(but tokens). So just because it doesn't seem to you like two tokens should be confused doesn't mean there isn't a valid point of confusion there.

Re:Ilya Sutskever's SSI Inc raises $1B by otabdeveloper4 on Wednesday September 04, 2024 @01:32PM (#41448314)
> but I'd easily make a bet that in 5 years that LLMs will be nowhere near human-level intelligence on some tasks

I wouldn't. There are some extraordinarily stupid humans out there. Worse, making humans dumber is a proven and well-known technology.

Re:Ilya Sutskever's SSI Inc raises $1B by Satam on Wednesday September 04, 2024 @03:26PM (#41449663)
To clarify this, I think it's reasonable that token prediction as a training objective could lead to AGI given the underlying model has the correct architecture. The question really is if the underlying architecture is good enough to capitalize on the training objective so as to result in superhuman intelligence.

For example, you'll have little luck achieving AGI with decision trees no matter what's their training objective.

Re:Ilya Sutskever's SSI Inc raises $1B by slashdave on Thursday September 05, 2024 @12:35PM (#41458245)
My objection is more about the data used for training, assuming we are talking about unsupervised learning. Text alone just won't cut it.
Re:Ilya Sutskever's SSI Inc raises $1B by jmugan on Wednesday September 04, 2024 @01:30PM (#41448301)
He doesn't address the real question of how an LLM predicting the next token could exceed what humans have done. They mostly interpolate, so if the answer isn't to be found in an interpolation, the LLM can't generate something new.
Re:Ilya Sutskever's SSI Inc raises $1B by maximinus_thrax on Wednesday September 04, 2024 @12:33PM (#41447634)
> On the other hand, I'm kind of baffled that we're still talking about "AGI" in the context of LLMs.

I'm not. Lots of people and companies have been sinking money into these ventures and they need to keep the hype alive by framing this as being some sort of race to AGI. I am aware that the older I get the more cynical I become, but I bucket all discussions about AGI (including the very popular 'open letters' about AI safety and Skynet) in the context of LLMs into the 'snake oil' bucket.

Re:Ilya Sutskever's SSI Inc raises $1B by nilkn on Wednesday September 04, 2024 @12:37PM (#41447707)
The argument about AGI from LLMs is not based on the current state of LLMs, but on the rate of progress over the last 5+ years or so. It wasn't very long ago that almost nobody outside of a few niche circles seriously thought LLMs could do what they do right now.

That said, my personal hypothesis is that AGI will emerge from video generation models rather than text generation models. A model that takes an arbitrary real-time video input feed and must predict the next, say, 60 seconds of video would have to have a deep understanding of the universe, humanity, language, culture, physics, humor, laughter, problem solving, etc. This pushes the fidelity of both input and output far beyond anything that can be expressed in text, but also creates extraordinarily high computational barriers.

Re:Ilya Sutskever's SSI Inc raises $1B by hn_throwaway_99 on Wednesday September 04, 2024 @12:55PM (#41447947)
> The argument about AGI from LLMs is not based on the current state of LLMs, but on the rate of progress over the last 5+ years or so.

And what I'm saying is that I find that argument to be incredibly weak. I've seen it time and time again, and honestly at this point just feels like a "humans should be a hundred feet tall based on on their rate of change in their early years" argument.

While I've also been amazed at the past progress in LLMs, I don't see any reason to expect that rate will continue in the future. What I do see the more and more I use the SOTA models is fundamental limitations in what LLMs are capable of.

Re:Ilya Sutskever's SSI Inc raises $1B by nilkn on Wednesday September 04, 2024 @01:20PM (#41448205)
Expecting the rate of progress to drop off so abruptly after realistically just a few years of serious work on the problem seems like the more unreasonable and grander prediction to me than expecting it to continue at its current pace for even just 5 more years.
Re:Ilya Sutskever's SSI Inc raises $1B by hn_throwaway_99 on Wednesday September 04, 2024 @02:45PM (#41449098)
The problem is that the rate of progress over the past 5/10/15 years has not been linear at all, and it's been pretty easy to point out specific inflection points that have allowed that progress to occur.

I.e. the real breakthrough that allowed such rapid progress was transformers in 2017. Since that time, the vast majority of the progress has simply been to throw more data at the problem, and to make the models bigger (and to emphasize, transformers really made that scale possible in the first place). I don't mean to denigrate this approach - if anything, OpenAI deserves tons of praise for really making that bet that spending hundreds of millions on model training would give discontinuous results.

However, there are loads of reasons to believe that "more scale" is going to give diminishing returns, and a lot of very smart people in the field have been making this argument (at least quietly). Even more specifically, there are good reasons to believe that more scale is not going to go anywhere close to solving the types of problems that have become evident in LLMs since when they have had massive scale.

So the big thing I'm questioning is that I see a sizable subset of both AI researchers (and more importantly VC types) believing that, essentially, more scale will lead to AGI. I think the smart money believes that there is something fundamentally different about how humans approach intelligence (and this difference leads to important capabilities that aren't possible from LLMs).

Re:Ilya Sutskever's SSI Inc raises $1B by svnt on Wednesday September 04, 2024 @03:23PM (#41449599)
Transformers in 2017 as the basis, but then the quantization-emergence link as a grad student project using spare time on ridiculously large A100 clusters in 2021/2022 is what finally brought about this present moment.

I feel it is fair to say that neither of these were natural extrapolations from prior successful models directly. There is no indication we are anywhere near another nonlinearity, if we even knew how to look for that.

Blind faith in extrapolation is a finance regime, not an engineering regime. Engineers encounter nonlinearities regularly. Financiers are used to compound interest.

Re:Ilya Sutskever's SSI Inc raises $1B by calf on Wednesday September 04, 2024 @03:04PM (#41449354)
Could it be argued that transformers are only possible because of Moore's law and the amount of processing power that could do these computations in a reasonable time? How complex is the transformer network really, every lay explanation I've seen basically says it is about a kind of parallelized access to the input string. Which sounds like a hardware problem, because the algorithmic advances still need to run on reasonable hardware.
Re:Ilya Sutskever's SSI Inc raises $1B by janalsncm on Thursday September 05, 2024 @02:53PM (#41459456)
I don’t see why it’s unreasonable. Training a model that is an order of magnitude bigger requires (at least) an order of magnitude more data, an order of magnitude more time, hardware, energy, and money.

Getting an order of magnitude more data isn’t easy anymore. From GPT2 to 3 we (only) had to scale up to the internet. Now? You can look at other sources like video and audio, but those are inherently more expensive. So your data acquisition costs aren’t linear anymore, they’re something like 50x or 100x. Your quality will also dip because most speech (for example) isn’t high-quality prose, it contains lots of fillers, rambling, and transcription inaccuracies.

And this still doesn’t fix fundamental long-tail issues. If you have a concept that the model needs to see 10x to understand, you might think scaling your data 10x will fix it. But your data might not contain that concept 10x if it’s rare. It might contain 9 other one-time things. So your model won’t learn it.

Re:Ilya Sutskever's SSI Inc raises $1B by machiaweliczny on Wednesday September 04, 2024 @02:40PM (#41449030)
Happy to review this in 5 years
Re:Ilya Sutskever's SSI Inc raises $1B by ldjkfkdsjnv on Wednesday September 04, 2024 @01:14PM (#41448155)
10 years of progress is a flash in the pan of human progress. The first deep learning models that worked appeared in 2012. That was like yesterday. You are completely underestimating the rate of change we are witnessing. Compute scaling is not at all similar to biological scaling.
Re:Ilya Sutskever's SSI Inc raises $1B by ldjkfkdsjnv on Wednesday September 04, 2024 @01:13PM (#41448138)
If its true that predicting the next word can be turned into predict the next pixel. And that you could run a zillion hours of video feed into that, I agree. It seems that the basic algorithm is there. Video is much less information dense than text, but if the scale of compute can reach the 10s of billions of dollars, or more, you have to expect that AGI is achievable. I think we will see it in our lifetimes. Its probably 5 years away
Re:Ilya Sutskever's SSI Inc raises $1B by nilkn on Wednesday September 04, 2024 @01:19PM (#41448198)
I feel like that's already been demonstrated with the first-generation video generation models we're seeing. Early research already shows video generation models can become world simulators. There frankly just isn't enough compute yet to train models large enough to do this for all general phenomena and then make it available to general users. It's also unclear if we have enough training data.

Video is not necessarily less information dense than text, because when considered in its entirety it contains text and language generation as special cases. Video generation includes predicting continuations of complex verbal human conversations as well as continuations of videos of text exchanges, someone flipping through notes or a book, someone taking a university exam through their perspective, etc.

Re:Ilya Sutskever's SSI Inc raises $1B by wubrr on Wednesday September 04, 2024 @12:40PM (#41447745)
> the more that it becomes clear that LLMs really are just "fancy autocomplete", even if it's really really fancy autocomplete

I also don't really see AGI emerging from LLMs any time soon, but it could be argued that human intelligence is also just 'fancy autocomplete'.

Re:Ilya Sutskever's SSI Inc raises $1B by hn_throwaway_99 on Wednesday September 04, 2024 @12:51PM (#41447884)
> but it could be argued that human intelligence is also just 'fancy autocomplete'.

But that's my point - in some ways it's obvious that humans are not just doing "fancy autocomplete" because humans generally don't make the types of hallucination errors that LLMs make. That is, the hallucination errors do make sense if you think of how an LLM is just a statistical relationship between tokens.

One thing to emphasize, I'm not saying the "understanding" that humans seem to possess isn't just some lower level statistical process - I'm not "invoking a soul". But I am saying it appears to be fundamentally different, and in many cases more useful, than what an LLM can do.

Re:Ilya Sutskever's SSI Inc raises $1B by wubrr on Wednesday September 04, 2024 @02:45PM (#41449099)
> because humans generally don't make the types of hallucination errors that LLMs make.

They do though - I've noticed myself and others saying things in conversation that sound kind of right, and are based on correct things they've learned previously, but because memory of those things is only partial and mixed with other related information things are often said that are quite incorrect or combine two topics in a way that doesn't make sense.

Re:Ilya Sutskever's SSI Inc raises $1B by slashdave on Thursday September 05, 2024 @12:37PM (#41458271)
> but it could be argued that human intelligence is also just 'fancy autocomplete'.

Well, no. Humans do not think sequentially. But even if we were to put that aside, any "autocomplete" we perform is based on a world model, and not tokens in a string.

Re:Ilya Sutskever's SSI Inc raises $1B by tim333 on Wednesday September 04, 2024 @03:37PM (#41449792)
>"We’ve identified a new mountain to climb that’s a bit different from what I was working on previously. We’re not trying to go down the same path faster. If you do something different, then it becomes possible for you to do something special."

Doesn't really imply let's just do more LLMs.

Re:Ilya Sutskever's SSI Inc raises $1B by sirspacey on Wednesday September 04, 2024 @11:29PM (#41453193)
Two interesting points to consider

1. If it’s really amazing autocomplete, is there a distinction between AGI?

Being able to generalize, plan, execute, evaluate and learn from the results could all be seen as a search graph building on inference from known or imagined data points. So far LLMs are being used on all of those and we haven’t even tested the next level of compute power being built to enable its evolution.

2. Fancy autocomplete is a bit broad for the comprehensive use cases CUDA is already supporting that go way beyond textual prediction.

If all information of every type can be “autocompleted” that’s a pretty incredible leap for robotics.

* edited to compensate for iPhone autocomplete, the irony.