My problem is that GHotz is a shitty writer, has puddle deep knowledge of anything that isn't iPhone firmware, and watching him play at being a freelance tech journo and produce this kind of hollow pseudo-philosophy that is meant to sound epiphanal but comes across as deeply naïve to anyone with any actual experience in these topics.
At the same time, he’s out there talking and writing, I’m sure if he continues he’ll get good. Nobody’s born a good writer and you only become one by writing a lot.
And that's putting aside his recent Twitter drama ...
It just seems he hasn't grown a jot since then. I've been guilty of writing stuff like this even in recent years, but I've grown enough to check myself and hold a higher bar for the quality of my writing. Assuming he did read this back to himself, the fact that he saw it as publishable is a bit sad.
Suolely scoops up and ushers into my inner cloak pocket -- "This will make a fine addition to my collection...."
Competition can do that if it is fair competition and a good game design. That is why rules and regulations are important in any game or it becomes Calvinball where the only rule is there are no rules and "it can't be played the same way twice" [1], which sounds great until you start losing to cheats. Though even that has some aspect of competition, the changing of rules in your favor while others change the rules in their favor.
The flaw in the thinking is that there will be "fair" competition. Any game design with good regulations and caps on game theory advantages (especially the cheat) "fair" competition is attainable. In game theory, if the other side cheats and your side keeps cooperating, you will lose every time. There is a great little game theory game that highlights it here called The Evolution of Trust. [2]
In a market with collusion, or excessive advantage, this may not be possible to retain fairness. Even right now we can see with overweight/top heavy wealth players in capitalism if there is collusion or one player gets too big, and there is no anti-trust or "blue shell", then that player will win every time. AI needs an anti-trust or "blue shell" to knock down any player that is too advanced, but that might not be possible. Doing that is barely possible in a market run by humans now.
The market is a garden, you have to help the seeds and cull back the overgrowth at the top. This is so the whole garden can thrive, lower seeds, middle plants and large production. Right now the large overgrowth gets all the benefits, policy control, water and nutrients, taking over the garden and even harming themselves with the overgrowth.
Most real world game theory and design would be horrible game design where the larger player always wins. Now imagine a game that the larger player controls the game design, you'd never be able to nerf them.
If one player can get the "powder keg" everyone, we need the game to be able to "blue shell" the bigger and potentially colluding/cheating player.
[1] Calvinball https://calvinandhobbes.fandom.com/wiki/Calvinball
[2] The Evolution of Trust https://ncase.me/trust/
Why does what OP concludes require fairness? Wouldn't the same logic apply if every player defects, in game theory terms?
I was arguing that competition if it is a more open market with fair rules/regulations, where competition is fair, that is true. Though many times it is a fixed market, or a cheat that stifles competition that might keep it in check.
Fixed markets happen more and more where the concentration is high and efficient players will game the system.
No products or markets are better where big fish solely control everything. That is why anti-trust or regulation need to expand to funding level not just surface company level. If you own entire industries across many companies, that is still oligarchy/monopoly if it is controlled by the same funding/sources.
Concentration needs to be broken up, for competition and better market and quality of life for everyone. Most people definitely don't want to make the authoritarian systems wealthier than open markets.
Concentration starts to take us away from a fair market and more towards a fixed/gamed market.
Everything in that gets so tuned that competition is very hard to enter. Very little margin and too much optimization/efficiency is bad for resilience. Couple that with private equity backed near leverage monopolies that control necessary supply and you have trouble.
HBS is even realizing too much optimization/efficiency is a bad thing. The slack/margin is squeezing out an ability to change vectors quickly.
The High Price of Efficiency, Our Obsession with Efficiency Is Destroying Our Resilience [1]
> Superefficient businesses create the potential for social disorder.
> A superefficient dominant model elevates the risk of catastrophic failure.
> *If a system is highly efficient, odds are that efficient players will game it.*
> sometimes power becomes so concentrated that political action is needed to loosen the stranglehold of the dominant players, as in the antitrust movement of the 1890s.
Couple massive wealth and concentration, even leaning authoritarian by then as tends to happen with fixed/controlled markets, with AI/AGI and you no longer have a "blue shell" because those players control the rules and thus the game.
The text, dated August 10, 2023, discusses the development and limitations of artificial intelligence (AI) and the concept of a sudden "hard takeoff" or "FOOM" (Fast Onset of Overwhelming Might) in AI capabilities.
Historical Perspective: The author begins by referencing Elon Musk's 2014 comment about AI being like "summoning the demon." They highlight the rapid advancements in AI, such as beating human players in Go, Chess, and Shogi, and playing Atari games.
Complexity Comparison: The author argues that the universe's complexity is not just a matter of scale compared to a Go game but a difference in kind. They illustrate this by imagining the universe tiled with tiny Go boards, emphasizing that predicting the universe requires understanding at the atomic level, not just the "stone level" of Go.
Dynamics Models: Modern self-play systems like MuZero and GPT-4 are described as dynamics models that predict the next state based on the current state and action. The author connects intelligence with prediction and compression, suggesting that feeding the entire internet into a model could theoretically "win the universe."
Practical Limitations: The author challenges the idea that AI can easily dominate complex systems like the stock market. They use the example of hedge funds, explaining that dominating the market requires more compute power than the entire market, which is unrealistic.
Inclusion of Computers: In Go, the model doesn't need to include other computers, but in modeling the universe, computers must be included. The author argues that unless there's a staggering advantage, understanding them from self-play is unlikely.
Preventing FOOM: The author warns against capping FLOPS in training runs, as it could create a dangerous situation if one person breaks the restriction. They advocate for preventing a 51% attack on compute to avoid FOOM.
Efficiency and Innovation: The text dismisses the idea that a single group could achieve a 1e20x efficiency increase, arguing that more intelligence leads to new tricks, but they become harder to find.
Revolution and Gradual Change: The author predicts an information revolution that will transform intelligence, similar to how the industrial revolution transformed energy. They stress that this change won't happen overnight but will follow a gradual exponential curve.
Conclusion: The author concludes that unless a "terrifying powder keg" is built, there will be no sudden FOOM. They emphasize the complexity of the universe compared to games and call for letting the markets evolve naturally. The closing statement, "the singularity is nearer," hints at a belief in the eventual convergence of human and machine intelligence, but not in an abrupt or catastrophic manner.You may notice this is all nonsense. Somehow it's impossible to amass power without being smarter than everyone else put together despite humans doing it all the time. Does Hotz think Stalin had a brain the size of a skyscraper that he ordered airbrushed out of all photographs? Somehow Hotz can confidently predict how future technologies will work despite us having no idea what they'll look like even in principle. Somehow having gods fighting amongst themselves will work out just fine for the humans in the crossfire.
This "debate" is just people summarizing science fiction at each other. The score is one Atlas Shrugged and nine Star Wars to three Terminators and one I Have No Mouth and I Must Scream.
People seem to think that because ChatGPT appears to be intelligent that somehow we are on the cusp of a new AGI world. And that we need to serious thought leaders like George Hotz and other arrogant tech-bros to rescue us. It's just crypto all over again.
The real thought leaders are going to be those in academia, government etc. Boring policy people who know the subtleties of the real world far better.
Independent thought doesnt get tied down by bureaucratic organizations.
Its always been like this and always will.
Behind the appearance of boring personalities there are... boring personalities
Probably one of the few given the majority of the comments are directly addressing and dissecting each point of the post.
It is not jumbled, but it is 1) the way Hotz usually writes, 2) unconventional, 3) cross-domain laterally connected, and 4) somewhat heavy for a sunday morning read.
Hotz generally doesn't do "barely coherent semi-thoughts." I think the label is needlessly harsh. He's just addressing the contemporary nonsense by bridging them together and calling their bullshit out.
It's sad to see so many of the replies to your comment are people attacking him for no good reason.