But then huge revenue streams for Scale basically disappear immediately.
Is it worth Meta spending all that money just to stop competitors using Scale? There are competitors who I am sure would be very eager to get the money from Google, OpenAI, Anthropic etc that was previously going to Scale. So Meta spends all that money for basically nothing because the competitors will just fill the gap if Scale is turned-down.
I am guessing they are just buying stuff to try to be more "vertically integrated" or whatever (remember that Facebook recently got caught pirating books etc).
But probs. it just makes sense on paper, Scale's revenue will pay this for itself and what they could do is to give/keep the best training sets for Meta, for "free" now.
Zuck's not an idiot. The Instagram and WhatsApp acquisitions were phenomenal in hindsight.
I worked at Outlier and it was such a garbage treatment
what about the whole metaverse thing and renaming the whole company to meta?
Even if it turns out to be wasted money, which I doubt, he's still sitting on almost 2 trillion. Not an L on my book.
This seems possible, and it just sounds so awful to me. Think about the changes to the human condition that arose from the smartphone.
People at concerts and other events scrolling phones, parents missing their children growing up while scrolling their phones. Me, "watching" a movie, scrolling my phone.
VR/AR makes all that sound like a walk in the park.
If it does come, it will likely come from the gaming industry, building upon the ideas of former mmorpgs and "social" games like Pokemon Go. But recent string of AAA disasters should obviously tell you that building a good game is often orthogonal to the amount of funding or technical engineering. It's creativity, and artistic passion, and that's something that someone who spends their entire life in the real world optimizing their TC for is going to find hard to understand.
It's this number: 2,000,000,000,000.
Unless we watered-down the definition of super-intelligent AI. To me, super-intelligence means an AI that has an intelligence that dwarfs anything theoretically possible from a human mind. Borderline God-like. I've noticed that some people have referred to super-intelligent AI as simply AI that's about as intelligent as Albert Einstein in effectively all domains. In the latter case, maybe you could get there with a lot of very, very good data, but it's also still a leap of imagination for me.
Similarly, "deeper insight" may be surfaced occasionally simply by making a low-intelligence AI 'think' for longer, but this is not something you can count on under any circumstance, which is what you would expect from something claimed to be "super intelligent".
In general, I agree that these models are in some sense extremely knowledgeable, which suggests they are ripe for producing productive analogies if only we can figure out what they're missing compared to human-style thinking. Part of what makes it difficult to evaluate the abilities of these models is that they are wildly superhuman in some ways and quite dumb in others.
I have to disagree because the distinction between "superficial similarities" and genuinely "useful" analogies is pretty clearly one of degree. Spend enough time and effort asking even a low-intelligence AI about "dumb" similarities, and it'll eventually hit a new and perhaps "useful" analogy simply as a matter of luck. This becomes even easier if you can provide the AI with a lot of "context" input, which is something that models have been improving at. But either way it's not superintelligent or superhuman, just part of the general 'wild' weirdness of AI's as a whole.
I think you're basically agreeing with me. Ie, current models are not superintelligent. Even though they can "think" super fast, they don't pass a minimum bar of producing novel and useful connections between domains without significant human intervention. And, our evaluation of their abilities is clouded by the way in which their intelligence differs from our own.
I wonder if the comparison is actually original.
The sorts of useful analogies I was mostly talking about are those that appear in scientific research involving actionable technical details. Eg, diffusion models came about when folks with a background in statistical physics saw some connections between the math for variational autoencoders and the math for non-equilibrium thermodynamics. Guided by this connection, they decided to train models to generate data by learning to invert a diffusion process that gradually transforms complexly structured data into a much simpler distribution -- in this case, a basic multidimensional Gaussian.
I feel like these sorts of technical analogies are harder to stumble on than more common "linguistic" analogies. The latter can be useful tools for thinking, but tend to require some post-hoc interpretation and hand waving before they produce any actionable insight. The former are more direct bridges between domains that allow direct transfer of knowledge about one class of problems to another.
These connections are all over the place but they tend to be obscured and disguised by gratuitous divergences in language and terminology across different communities. I think it remains to be seen if LLM's can be genuinely helpful here even though you are restricting to a rather narrow domain (math-heavy hard sciences) and one where human practitioners may well have the advantage. It's perhaps more likely that as formalization of math-heavy fields becomes more widespread, that these analogies will be routinely brought out as a matter of refactoring.
Like the prompt "How can a simplicial complex be used in the creation of black metal guitar music?" https://chatgpt.com/share/684d52c0-bffc-8004-84ac-95d55f7bdc...
It is really more of a value judgement of the utility of the answer to a human.
Some kind of automated discovery across all domain pairs for something that a human finds utility in the answer seems almost like the definition of an intractable problem.
Superintelligence just seems like marketing to me in this context. As if AGI is so 2024.
If you have a cloud of usually, there may be perfectly valid things to do with it: study it, use it for low-value normal tasks, make a web page or follow a recipe. Mundane ordinary things not worth fussing over.
This is not a path to Einstein. It's more relevant to ask whether it will have deleterious effects on users to have a compliant slave at their disposal, one that is not too bright but savvy about many menial tasks. This might be bad for people to get used to, and in that light the concerns about ethical treatment of AIs are salient.
First, comfort isn't a great gauge for truth.
Second, many of us have seen this metaphor and we're done with it, because it confuses more than it helps. For commentary, you could do worse than [1] and [2]. I think this comment from [2] by "dr_s" is spot on:
> There is no actual definition of stochastic parrot, it's just a derogatory
> definition to downplay "something that, given a distribution to sample
> from and a prompt, performs a kind of Markov process to repeatedly predict
> the most probable next token".
>
> The thing that people who love to sneer at AI like Gebru don't seem to
> get (or willingly downplay in bad faith) is that such a class of functions
> also include thing that if asked "write me down a proof of the Riemann
> hypothesis" says "sure, here it is" and then goes on to win a Fields
> medal. There are no particular fundamental proven limits on how powerful
> such a function can be. I don't see why there should be.
To go a step further, some counter-arguments take the following shape: "If a transformer of size X doesn't have capability C, wait until they get bigger." I get it: this argument can feel unsatisfying to the extent it is open-ended with no resolution criteria. (Nevertheless, increasing scale has indeed shown to make many problems shallow!) So, if you want to play the game honestly, require specific, testable predictions. For example, ask a person to specify what size X' will yield capability C.
[1]: https://www.lesswrong.com/posts/HxRjHq3QG8vcYy4yy/the-stocha...
[2]: https://www.lesswrong.com/posts/7aHCZbofofA5JeKgb/memetic-ju...
Isn't stochastic parrot just a modern reframing of Searle's Chinese room, or am I oversimplifying here?
It's a smart purchase for the data, and it's a roadblock for the other AI hyperscalers. Meta gets Scale's leading datasets and gets to lock out the other players from purchasing it. It slows down OpenAI, Anthropic, et al.
These are just good chess moves. The "super-intelligence" bit is just hype/spin for the journalists and layperson investors.
Which is kind of what I figured, but I was curious if anyone disagreed.
By whom? The fact that there is a list of competitors means Meta has no monopoly in AI. And Scale AI has no monopoly in labelled data.
It’s anticompetitive. But probably not to an illegal extent. Every “moat” is, after all, a measure in anticompetitiveness.
Wouldn’t Scale’s board/execs still have a fiduciary duty to existing shareholders, not just Meta?
Meta, Google, OpenAI, Anthropic, etc. all use Scale data in training.
So, the play I’m guessing is to shut that tap off for everyone else now, and double down on using Scale to generate more proprietary datasets.