andrewgleave

I like to think of technology as packaged-up knowledge intended to solve a particular problem.

The glib epigrams and aphorisms it shoehorns in to all prose is by far the worst regression of any model I can recall in terms of simply generating simple, clear output. I cannot think of another model that's gotten worse at writing plainly while being explicitly instructed to do so. Infuriating.

I've switched mostly to Sol and if I have to use Opus, the first task once the code is written is to ask Sol to strip and re-write (from the code as ref) all documentation Opus wrote.

Yes. The irritating epigram / aphorism style it now uses is such a regression compared to previous Anthropic models. Probably is the case that this is due to watermarking - though hardly subtle if it is.

My Lindy alarm has gone off!

Looks like MLX is not a supported backend in Ollama so the numbers for the Mac could be significantly higher in some cases.

It would be interesting to swap out Ollama for LM Studio and use their built-in MLX support and see the difference.

For any stamp collectors here, the Isle of Man Post Office [1] has just issued an official set of 6 Roger Dean and Rick Wakeman stamps [2]:

[1] https://iomstamps.com/collections/wakeman [2] https://www.bbc.co.uk/news/articles/clyqe679gqno

Theory building is the secret sauce, and all variants of "this is how to use AI effectively" I've seen are inferior to the epistemologically sound theory Naur outlines in his paper.

Feynman said, "The first principle is that you must not fool yourself - and you are the easiest person to fool" when talking about science, but it also applies to the properties of LLM output.

Not cosmological but yesterday Apple released an interesting protein folding model with 3B param transformer-based arch which runs on M-series hardware and is competitive with state-of-the art models. [1] Code [2]

[1] https://arxiv.org/pdf/2509.18480 [2] https://github.com/apple/ml-simplefold

Yes. Juniors have a lack of knowledge about how to build coherent mental models of problems whose solution will ultimately be implemented in code, whereas seasoned engineers do.

Seniors can make this explicit to models and use them to automate "the code they would have written," whereas a junior doesn’t know what they would have written nor how they would have solved it absent a LLM.

Same applies to all fields: LLMs can be either huge leverage on top of existing knowledge or a crutch for a lack of understanding.

For anyone interested in the history of Sellafield and its role in reprocessing, "Britain's Nuclear Secrets: Inside Sellafield" on BBC 4 at the moment is worth a watch. Presented by Jim Al-Khalili.

https://www.bbc.co.uk/programmes/b065x080

Reminds me of Brett Victor's demo of projected AR turbulence around a toy car at Dynamicland. Only a short clip, but you get the idea: https://youtu.be/5Q9r-AEzRMA?t=47

> “There's kind of like two different ways you could describe what's happening in the model business right now. So, let's say in 2023, you train a model that costs 100 million dollars. > > And then you deploy it in 2024, and it makes $200 million of revenue. Meanwhile, because of the scaling laws, in 2024, you also train a model that costs a billion dollars. And then in 2025, you get $2 billion of revenue from that $1 billion, and you spend $10 billion to train the model. > > So, if you look in a conventional way at the profit and loss of the company, you've lost $100 million the first year, you've lost $800 million the second year, and you've lost $8 billion in the third year. So, it looks like it's getting worse and worse. If you consider each model to be a company, the model that was trained in 2023 was profitable.” > ... > > “So, if every model was a company, the model is actually, in this example, is actually profitable. What's going on is that at the same time as you're reaping the benefits from one company, you're founding another company that's like much more expensive and requires much more upfront R&D investment. And so, the way that it's going to shake out is this will keep going up until the numbers go very large, the models can't get larger, and then it will be a large, very profitable business, or at some point, the models will stop getting better. > > The march to AGI will be halted for some reason, and then perhaps it will be some overhang, so there will be a one-time, oh man, we spent a lot of money and we didn't get anything for it, and then the business returns to whatever scale it was at.” > ... > > “The only relevant questions are, at how large a scale do we reach equilibrium, and is there ever an overshoot?”

From Dario’s interview on Cheeky Pint: https://podcasts.apple.com/gb/podcast/cheeky-pint/id18210553...

Yeah. Read “Programming as Theory Building” by Naur [1] to understand why you need to still need to develop a theory of the problem and how to model it yourself lest the LLM concoct (an incorrect) one for you.

[1] https://gwern.net/doc/cs/algorithm/1985-naur.pdf

I recently built a quick SwiftUI app to pin quotes and posts from X to my home screen.

I found I was liking/bookmarking insightful content on X I rarely saw again and wanted a way to resurface them somewhere I would see multiple times per day.

Can import from X via share sheet or manually enter them. It's minimal, but I've found having:

"i hate how well asking myself "if i had 10x the agency i have what would i do" works"

there every time I unlock my phone, was worth the development effort.

https://apps.apple.com/gb/app/lumatta/id6740705796

Going to leave this here: https://plato.stanford.edu/entries/popper/#GrowHumaKnow

This is very similar to when I used OpenCV to read the angle of a cardboard knob I pinned on the wall in my office to change the volume on Spotify!

"""But what Richard hated, or at least pretended to hate, was being asked to give advice. So why were people always asking him for it? Because even when Richard didn't understand, he always seemed to understand better than the rest of us. And whatever he understood, he could make others understand as well. Richard made people feel like a child does, when a grown-up first treats him as an adult. He was never afraid of telling the truth, and however foolish your question was, he never made you feel like a fool."""

This is why he is spoken of with such reverence and why his insights have profoundly impacted both scientists and non-scientists alike. Few Nobel laureates have achieved such popular influence.

Unsurprisingly, this pessimistic and shortsighted take emanates from a newspaper in heart of the EU...

And by the third slit simulation they seem to have refuted their own hypothesis.

You may find Constructor Theory interesting. An attempt to express physical laws solely in terms of possible and impossible transformations.

“These include providing a theory of information underlying classical and quantum information; generalising the theory of computation to include all physical transformations; unifying formal statements of conservation laws with the stronger operational ones (such as the ruling-out of perpetual motion machines); expressing the principles of testability and of the computability of nature (currently deemed methodological and metaphysical respectively) as laws of physics; allowing exact statements of emergent laws (such as the second law of thermodynamics); and expressing certain apparently anthropocentric attributes such as knowledge in physical terms.”

https://arxiv.org/abs/1210.7439

Perversely, this "settled knowledge" nonsense actively inhibits the growth of knowledge and just leads to the "justified true belief" swamp...

All we have is our current best explanations/understanding. We use them to make progress until we find a problem with it – something it fails to account for, something it makes an incorrect prediction for etc.

Then, we attempt to conjecture new explanations which solves the new problem while still accounting for all the useful aspects of the old theory.

That is knowledge creation, and also progress.

[Edit]

It's just blind prophecy and therefore a fiction.

There is no way to predict the growth of future knowledge. If there was it would be equivalent to already having said knowledge.

Whether people do or don't create some specific piece of knowledge is solely down to what problems they have, and whether we choose (individually or as a species) to attempt solve them or not.

Our fallible knowledge is the byproduct of us solving problems.

An epistemologically flawed article.

Anyone interested in Dynamicland may want to check out Paper Programs.

It's been about 5 years since I last played with it.

It’s great fun and should be much more interactive with 5+ years of browser and hardware development.

https://github.com/janpaul123/paperprograms

David Deutsch’s “Physics Without Probability” covers the history of probability, it’s legitimate and misconceived uses and concludes that according to MWI there is no such thing in reality - it’s basically that probabilities correspond to how measures of the multiverse proportion themselves as differentiation occurs.

I watched it a few years ago so may be misremembering bits but I think that is the gist…

Worth a watch especially if you balk at this idea just to to see a strong counter argument.

https://m.youtube.com/watch?v=wfzSE4Hoxbc&t=0s

Edit: link

Bret Victor's interactive version of this in Dynamicland:

https://twitter.com/worrydream/status/977058412990152704

The Logic of Experimental Tests, Particularly of Everettian Quantum Theory:

https://arxiv.org/pdf/1508.02048.pdf

Followed by Deutsch: All evils are caused by insufficient knowledge.