jgord
We need to ensure we dont outsource thinking itself - yes its great to have a lot of our knowledge encoded in books, and in the weights of LLMs [easier to access ]
Do we even 'have' a technology if it doesn't reside actively in some human minds ?
Some of math is handed down to apprentice mages, and not encoded well in books [ perhaps books dont encode intuition well, Ive had many cases where someone has explained something not in the textbook, a trick they learned from someone else ]
Apprentice mages only become mages by grappling with hard problems, asking questions, going on long detours, discussing/arguing with master mages, trying lots of tough thought experiments - all valuable "training".
Training using the AI might lead to a golden age where more mages learn wider and deeper, and the subject flourishes and more humans understand the tech.
The danger is using the AI makes us lazy - we dont grapple with programming syntax, draw math diagrams slowly by hand, check edge cases, try to prove things ourselves the slow way and build intuition in the process.
Another issue is ownership and access. Already RAM GPUs and soon CPUs are harder to get hold of due to competition with the hyperscalar datacenter buildout. This is an example of economic inequality effectively stealing resources from the mainstream.
Will math remain open or will it be owned by the frontier models ? With Math and science journals Elselvier almost had a stranglehold, bullying universities to pay massive bulk subscriptions - fortunately Arxiv solved some of that. Can I even own a copy of a music track, tv show or movie these days, when there it is so much more profitable for companies to rent it out pay-per-view or monthly ? There is every financial incentive to slurp up the whole of Arxiv, github and every science textbook into a frontier model and rent out access via pay per token. I no longer need to go to stack overflow to look up a simple algo, google can give me a pithy summary, so its likely that the LLMs will steal traffic from github, or github could be drowned in the volume of vibe code. Is there any point buying a textbook or going to university when the AI covers the material ? If no universities then we dont have a physical space for science and tech to be discussed. That activity might go online .. or may disappear altogether - we already have less socializing in person due to the rise of social media.
We have several "alignment" problems : humans arent aligned with each other, frontier models are aligned with profit-making which may differ from the common good, the race is on and so frontier models are more aligned to short term shareholder / VC value than to longterm benefit to humanity as a whole. These mis-alignments may have always been there, but are multiplied by the power and reach of new AI technology.
From which it then follows that _all_ technology and science become magical incantations dictated by a black-box oracle, whom everyone trusts and no one understands.
That could happen in a couple of decades.
It requires so much extra discipline to think for yourself and not use it, you will be forced to use it in your job or be seen as a Luddite and your ability to think for yourself in the domain will diminish over time - your domain thinking muscle will gradually atrophy from lack of use.
Why would any of us bother thinking hard, when there is no financial incentive to doing so ?
Your analogy fails with shovels and excavators .. because General synthetic intelligence is not a single purpose narrow 'tool'. near-future AI is equivalent to an alien intelligent species competing for resources, or equivalent to a cheaper smarter non-human 'outsource worker'.
Yeah, considering how hard it is to food-diet or social-media-diet .. I think it requires immense resolve to LLM-diet or use it in a non-binge way that develops ones own depth of understanding.
Its going to require just far too much mental energy to push back against an LLM that is correct 92% of the time.
edit : removed air quotes around AI psychosis.. that might have read as sarcasm, which was not intended
Regardless of whether it is 'conscious' or not, Roman Yampolski makes the point that AI may or may not have human-kinds best interests at heart [ hell, even humans aren't aligned ]
Can we control or trust a counter-party that has intelligence at a level near or exceeding the smartest humans in all domains ?
I dont even think we can even have a safe sandbox in which to run an AI .. and frontier model training labs are certainly not air-gapped.
The current slew of excellent LLMs will probably cause a decades long dip in human cognitive ability.
If social media on smartphones had an impact of ability of students to concentrate, LLM AI will likely have a much greater negative impact on education.
There are trillions of dollars of investment pushing for humans to trust whatever an LLM says, and very little money behind encouraging people to tackle hard problems themselves and train themselves to think.
I have to say honestly, that AI looks like an apocalypse for human thinking [ Im glad Tao is warning people, I just dont think many people will listen ]
> The fix isn't to abandon AI. The fix is to use it differently. To use it in a way that sharpens us instead of softening us. To use it in a way that makes us think harder, not less.
maybe the fix is to _not_ use AI in your domain of expertise, or at all.
or, the fix could be to use narrowly trained AI - as Roman Jampolsky argues. Rather than train an AGI / ASI on the whole of human experience on the internet, perhaps train the models for narrow use - on narrow focused domains like microbiology / nano-materials etc
or the fix could be to make training AGI illegal - a regulated munition.
That might be a way to avoid creating a new intelligent species, which is competing with humans for compute and other resources... while making us dumber via outsourcing all our thinking in the short term.
Ctrl-Alt-R in firefox
Using it in a web-browser 360 panorama viewer enabled pretty substantial latency and storage improvements.
close to 4x iirc.
ditto large "xray" style orthographic images of building scans generated from pointcloud laser scan data.
avif was even better compression for some cases, but seems to take a long time to generate, and less widespread browser support. Will look at jpeg-XL .. but Im guessing it will be a while until it has widespread support
Math that only resides in the weights of models, or arcane forms such as a long lean proof or even an unread textbook .. is not the math that we should be striving for.
Likewise all other technology [ and culture ].
LLMs and AI / AGI / ASI could lead to a new renaissance of math discussion and expansion of human math and science. Or the opposite, where we outsource all our thinking to the AI, and no new generation of artisans is trained by doing hard problems, and in a generation we have killed off human math.
Likewise all of the fields of human intellect. We need to make sure we protect future generations of doctors, biologists, software developers, architects, engineers, librarians, musicians, artists ...
A moratorium on AI development might be the only way to achieve this preservation of human culture.
This is not that - although it is a summary of some of the social cliques / trends, and maybe a warning against groupthink.
However, math and science and engineering and biology/medicine have profound implications - those useful parts of that 'culture' - which we call 'technology' - extend human lifespan, healthspan, prosperity, safety.
The main practical value of investing in maintaining a population of math artisans is that they are/were needed as part of a well-trained science community needed to discover and develop new useful technology.
I think its a rational view to say LLM generated "AI-math" is a net positive if and only if, it results in _more_ high quality "human-math". By human math I mean well digested math residing in human minds, being discussed between humans and being actively re-discovered by humans, including a pool of new human student devotees.
A very bad outcome is where we lose the next generation of scientists/lawyers/mathematicians/engineers/doctors/researchers/authors because we let the AI do it all. In this dark future we outsource all our 'thinking', and avoid the years long training of grappling with hard-to-understand aspects of reality. All our culture is sucked into the event horizon of an AI black box.
AI could be a new renaissance of human math- and science- culture, or it could be the death of it.
AI isnt going away, the financial incentives and high current economic inequality, geopolitics guarantee that it will proceed as fast as possible.
AI has given us the structure of nearly every protein, which may well solve Alzheimers and cancers. AI may well find a solution to stable plasma for nuclear fusion, unlocking vast cheap energy and help us humans halt global warming.
So, how do we make sure that the economic windfalls of AI are reinvested back into human-culture, funding more human-math and human science .. resulting in a deeper pool of well educated scientists, engineers and researchers ?
I don't think that will happen by default - we are likely to have an AI-assisted dumbing-down rather than an AI enabled age of enlightenment.
Most of the general public are happy enough not to learn any math - when it doesn't make you much money and university is such a debt burden anyway. School and university students can just get the LLM to do their assignments. Why should they pay tax to fund science at universities when the cost of living is so high, and the LLMs can do all the research anyway ?
I think part of the solution is a policy to tax the windfalls of AI to mitigate the downsides of AI.
Governments should have taxed the carbon polluters and used those funds to mitigate climate change effects and research new forms of clean energy, but didnt.
We should tax LLM/AI profits and use that to fund science research - not just research to mitigate the effects of runaway AI, but to fund general science and math research and teaching, to guarantee that there is a net increase in human-math and human-science.
It is a cautionary tale that fewer students know their times-tables by heart, because a cheap calculator can do that ... but it gives me some hope that there are still people enjoying playing chess, even though chess programs are super-human.
This is true of almost any field - if we have black-box technology that only the AI can understand / fix / augment / extend, then we will lose the ability for humans to learn and understand that tech.
Its similar to outsourcing manufacturing, you lose the ability to manufacture. If you don't exercise a muscle, it atrophies. If some people dont learn to write code [ or do math or write music or books ] the hard slow way, we will lose that deep skill.
If young people don't grapple with hard problems in science, math, economics, creative writing, drawing, music making .. they will not be trained, their cognitive skills will not develop with use. We will have no doctors or engineers or musicians or lawyers in a decade or two.
We don't want to become a race of fatties who never exercise, or dummies who just scroll social media feeds without exercising their minds.
This is real.
This is possibly more exciting and of more future value than solving Navier Stokes.
It also shows us a way, when it seems we are lost.
- inequality / taxation
- demographics / aging population
- climate change mitigations
- affect of social media esp on young people
- safety of self-driving cars
- economic impacts of LLMs
- educational "" ""
- security "" ""
- surveillance of in-home devices
- data collection and sharing
- right to repair
If the technorati are woefully unprepared for the age of AI, which is being rapidly thrust upon us by the big money, then the wider public dont stand a chance of being informed / prepared.
Apart from my own desire to read the internet to find out what actual people think .. Im worried about a kind of bit-rot, where any decade now, gen-pop reader has no idea what content came from humans and what came from AI.
Im assuming the people who _train_ AI [ LLMs ] actually need to separate the two, and avoid the feedback loop of training the next LLM on the output of the previous LLM.
Presumably, this would lead to a kind of reversion to the mean, akin to making photocopies of photocopies back in the day, resulting in copy degradation.
We already have a kind of bit-rot from moving away from physical media - many documentaries on TV, and audio / video / movies / games are being lost as they are not moved to permanent archive storage.
Old out of print books should be scanned and made public and permanently available online, as they are part of our cultural heritage [ not least for the purposes of training current and future AI as the defacto archives / oracles ]
and sql examples.
I guess we are in the age of "Ill get my people to talk to your people" or "Ill get my AI to summarize your AI output"
The benefit of a guide would be it tells me what the main changes are, why they matter, with pithy illustrative usage examples.
I guess I have an aversion to idiomatic [ ego-massaging, feelgood ] LLM-speak.
I found this tour unreadable.
Here's the official release notes page : https://www.postgresql.org/docs/19/release-19.html
which unsurprisingly does have a list of important changes near the top.
Academics and founders who might work on developing practical products using NN / ML / RL techniques to solve a realworld problem in engineering/logistics/medicine are not getting investment money. VCs and most people are blind to the fact there is AI outside of LLMs, despite the fact that we have seen AlphaGo and AlphaFold as evidence of non-LLM AI progress in hard domains.
This is perhaps a sub-problem of a larger issue - hyper-inequality means that capital is not allocated to talent [ capital is localized, talent is more widely spread throughout the population ].
We are not getting money to things that will grow our future such as :
- small innovative startups
- university science research
- people who are young enough to have kids, being able to afford them
- new garage bands / authors / musicians / photographers
- public works / infrastructure / libraries
- local retail : bookshop, artisanal bakery, cafe
AI could be a golden age of human flourishing - but thats not where we are heading, what we are seeing is a territory rush by the megacorps.
The fact that RAM and GPU prices have risen so fast, is evidence of supply and demand effect where inequality steals resources from the commons [ middle of the economy ].
Can a talented garage inventor / math or arts student afford a Ryzen AI dev platform, let alone a DGX spark on which to create the next important technology innovation ?
The graphs I see all have ridiculous ranges, either +1.5C hopium or +6C hellscape - whereas +2.6C within 20 to 30 years seems a most likely [and unpleasant] scenario.
Are advances like AlphaFold likely to speed up progress ?
As we are at or nearing +1.5C now, with global emissions at an all time high [ perhaps peaking soon ] theres a good chance we are headed toward +2.6C by ~2060.
Keep in mind the CO2 stays up there for a long time, centuries, so when we get to net-zero its max-CO2 and therefore max-heat.