diedyesterday
diedyesterday on Hacker News
All these open source projects and the ideas in them will also help Anthropic/OpenAI's next iteration of models/harnesses, as it has done before. In the long run only access to the the raw compute power will determine relative advantage.
Give it time. The Great Reconfiguration will take a great deal of time (years, decades). As it did with the printing press, electricity, computers, internet, etc. And this one (AI) is more fundamental than all the others.
Yes, there is no "collapse". Just intentional (in the case of measurement) "narrowing" of the wavefunction (=state in the basis of some operator)or a tighter distribution of eigenvalues, driven by the specific experiment setup (e.g. we set up the experiment to measure momentum or angular momentum along-Z or ...)
Every state is a superposition. Even eigenstates (of some operator, e.g. X) are superpositions (of eigenstates of some other operator, e.g. P). So everything is a superposition and being superposition or not is relative.
The fundamental claim of QM is essentially a claim about the shape and size of the state space: That states were not limited to the edges (axes) of that high (or infinite-dimensional) space (as classical mechanics thought) and that the whole interior points were available. This is essentially the central claim of QM (imagine it as not being limited to the edges of a very high dimensional cube and that you can choose points in its interior as your states).
There will always be special demand for the "best possible" and "fasted around" inference and they will pay. "Good enough" models are useful for many tasks but cannot replace the whatever frontier is at the time. That's why the top 2 labs are rake in ~80% of total inference income.
Yes in fact. From the blog "To make something transparent...Alternatively (and this is the approach we'll cover in this post), we could try to change the refractive index of the water inside to match the embedded media. "
It's a kind of impedance matching.
I too, believe personality cults (NK, Nazi Germany, Shiite Iran, etc.being just the recent ones) are one of most evil social (power) structures.
On the virtue of scarcity, being mortal, being NOT omniscient, NOT omnipotent, being Finite, etc. (thus arising the need to optimize, etc.):
[Fig Tree] "By end of the story Gilgamesh is human; He is Us; We see the Deep; That's our gift as humans; We fight to understand something no god could ever could (emergence and richness from scarcity); The gods don't need to fear death so they don't have to live an impassionate life; It means nothing to them to run out of time; "Seeing the deep" is the apex of human potential, not a transgression into godhood; It's something that only humans can do."
Yes, Many of the the philosophical concepts which we previously thought were contingent on consciousness (like knowledge, in the philosophical sense, will, intelligence, agency, ...) have almost nothing to do with it, and they look far more important and fundamental (now with LLMs/AI agents, more than before). This is one of the contributions of generative AI to philosophy (with many many more to come).
Will has almost nothing to do with consciousness (as does knowledge).
Will is just result of a very complex yet deterministic (unconscious)
computation by an agent which guides their future action (it's that orientation/aboutness toward action which distinguishes it from other computation). A PS of that computation is sent (projected) into that agent's consciousness if they have one (and is what we think as our "will").
1. The initial stage of adoption of AI will be more of a shift (some plus + here and some minus there) and reorientation and restructuring than net exponential growth.
2. Also a good portion of contribution of AI is where it's not taken into account by metrics like GDP. I have seen an explosion of FOSS projects especially in my own area and I'm sure it's pretty much are the case for many other areas.
3. Also there will be a sink-like effect with effects of AI on net wealth production. Like Earth's own oxygenation event which took billions for the produced oxygen to (after saturating all the sinks like Earth' iron reserves and turning them to Iron ore) ultimately find its way into the atmosphere.
4. Also I'm not sure what is exactly being counted on as contribution of AI to economy. Is the data-center build-outs and growth of chip companies,etc. included in this metric in AI's favor?
...
Syria became a hell for its citizen exactly because Obama run away from enforcing the very red line (chemical weapons) he himself had drawn (for himself). He basically allowed the massacre to escalate.
Your simplistic recipe shows you don't understand the situation and way for improvement in either case.
You obviously seem to have no idea how mafia-style dictatorships like those of Iran, Russian and Venezuela work. No fault of you. Most people don't.
And even their own citizens come to the realization after a long time living under them; partly because they get caught in the constant propaganda campaign which is one hallmark of these regimes. They always live in the propaganda mode.
Google is still far far better than the competition which is crawling with manipulated results and fake sites and phishing scams. The other are so much easier to take advantage of and manipulate (e.g. search ranking). No security based filtering ,etc.
I have been personally bitten by results on the likes of Bing or DDG (fake browser addons on top, crypto phishing sites on top, etc.)
Also user experience varies somewhat and for me with the same search prompt ("midjourney"), the intended site is the first result.
To me its function looks similar to a sponge or a tampon: An additional piece that absorbs the external influence and then is subtracted away (you remain dry:)))
Regarding the conclusion about language-invariant reasoning (conceptual universality vs. multilingual processing) it helps understanding and becomes somewhat obvious if we regard each language as just a basis of some semantic/logical/thought space in the mind (analogous to the situation in linear algebra and duality of tensors and bases).
The thoughts/ideas/concepts/scenarios are invariant states/vector/points in the (very high dimensional) space of meanings in the mind and each language is just a basis to reference/define/express/manipulate those ideas/vectors. A coordinatization of that semantic space.
Personally, I'm a multilingual person with native-level command of several languages. Many times it happens, I remember having a specific thought, but don't remember in what language it was. So I can personally sympathize with this finding of the Anthropic researchers.
So Erdogan wants to be Putin of Turkey. It's up to the Turks if he can succeed.
...until they distill that "11x efficient training" again ...
Don't forget that this model probably has far less params than o1 or even 4o. This is a compression/distillation, which means it frees up so much compute resources to build models much powerful than o1. At least this allows further scaling compute-wise (if not in the amount of, non-synthetic, source material available for training).
Reminds me of how Google's AlphaGo learned to play the best Go that was ever seen. And this somewhat seems a generalization of that.
Sam Altman and OpenAI are following the example of Celebrimbor it seems. And I love what may come next...
A linear graph with a log scale on the vertical axis means the original graph had near exponential growth.
A linear graph with a log scale on the the horizontal axis means the original graph had law of diminishing return kick it (somewhat similar to logarithmic but with a vertical asymptote).
"Safe super-intelligence" is an oxymoron.
Exactly. That's what I also thought the last time someone brought this up in these forums (couple of years ago). It's a hint to the structure of the brain and also the mind (we can observe it at the consciousness level in ourselves and others, too). Algorithmic (von Neumann) computers/programs do not usually exhibit this; but neural networks do.
If we are good readers, a book helps up achieve better generalization (in the machine learning sense). That's its main contribution not the specific facts it contained.
Yeah, but evolution's most "clever" invention is consciousness, human brain and its highest state philosophy. It's all the work of evolution.
Evolution's most "clever" turn and invention is it achieving self-consciousness (we are its "self consciousness")
> "Is it really possible that this space is so rich as to have a single point in it encapsulate a whole novel?"
Not with this GPT. The context size would not allow keeping attention to the total meaning of more than 2048 tokens (as reflected in the transformed embedding of that context's last token).
For a substantial part of a novel, it would require a much larger context size with then presumably will need a higher dimensional embedding/semantic space.
Do not be fooled by the simplicity; The magic itself is in the many Q, K and V matrices (each of which is huge) which are learned and depend on the language(s). This is just the form of the application of those matrices/transformations: Making the embedding for the last token of a context "attend to" (hence attention) all information (at all layers of meaning and not just syntactic or semantic meaning but logical, scientific, poetic, discoursal, etc. => multi-head attention) contained in the context so far.
Any complex function can be made to look simple in some representation (e.g its Fourier series or Taylor series, etc.).