rhdunn

It's fairly straightforward to write a function that detects looping output -- I wrote one for promptfoo in [1].

For training you would need to hook it up to a LoRA training harness, identify the looping weight direction, and adjust the weights accordingly. I've not (yet) looked into how to do this.

[1] https://news.ycombinator.com/item?id=48555634

The JetBrains IDE, Visual Studio, and VS Code all support http files as well, which can be checked into an application.

See e.g.:

[1] https://www.business-standard.com/technology/tech-news/what-...

[2] https://venturebeat.com/infrastructure/5-gpu-utilization-the...

[3] https://venturebeat.com/infrastructure/ai-bubble-update-79-o...

[4] https://www.techradar.com/pro/5-utilization-is-a-math-fail-m...

[5] https://www.datacenterdynamics.com/en/news/microsoft-has-ai-... [Nov 2025]

Sam Altman's argument about RAM utilization is reasonable on a first pass, but misses several key points:

1. people aren't using (GPU) RAM [1] for just running LLMs (in the local model sense), they are using it for other things... games, system displays, other models (Text-to-Speech, Speech Recognition, etc.);

2. being able to run models locally allows them to be used when the internet is down, use them on private data (e.g. medical records, proprietary company data, etc.), including training on that data;

3. the big AI companies are already storing unused GPUs on shelves waiting for their roll-out in data centres, as they can't build the capacity fast enough.

With point (3) that goes against Sam Altman's argument about utilization. The big companies have the hardware, but aren't utilizing it themselves.

[1] And if the argument extends to RAM in general you have all sorts of applications, especially on developer hardware -- editing and compiling software; running local servers; running CAD and other applications; etc.

I don't buy that analogy. Brains do grow over time, but what's more important is the continual learning and interaction with the environment.

You need to have multiple layers in an LLM to process and refine the information. The larger the token count, etc. are the larger each layer is. The activation density of these networks is low compared to the smaller parameter models.

It may be that you need the large networks in order to learn complex tasks, but to me it seems like the big companies are throwing compute (more data, more hardware, more parameters, etc.) at the problem because that is currently easier and simpler than doing things like creating new architectures.

AFAICT, all the architecture advancements (rope, mixture of experts, reasoning, etc.) have all come from outside of the big AI companies. I think those are where we'll see the better gains, some of which are likely to make these easier to run on smaller hardware.

AFAICT there are several things that affect the size of the model:

1. the number of transformer layers -- e.g. each layer adds the ability to provide more information to the residual stream [1], [2]; each layer is `tokens * features * active context`

2. the number of tokens -- the more tokens there are the more word fragments and Unicode characters you can represent

3. the number of features assigned for a token -- these are used to associate trained semantic information for the token; with the active context this can encode contextual information for that token (the idea being that the model can differentiate senses of a word like "bank" depending on its context, so "bank (river)" will have different weights than "bank (building for money)")

4. the active context -- this is the number of `token * features` to include when predicting the next token

One of the key features of LLMs and transformer architectures is that training across different languages helps the model learn the underlying concepts and associations, so training on English only would lose that.

A better approach may be something like a byte architecture or a model where the tokenization is trained with the model (similar to vision and audio) instead of as a separate step. Or maybe training smaller models longer to take advantage of double descent [3], [4].

[1] https://arxiv.org/abs/1512.03385 "Deep Residual Learning for Image Recognition"

[2] https://www.youtube.com/watch?v=QgH9sr7G13Q "The most cited paper of the century is a brilliant hack"

[3] https://arxiv.org/abs/2403.10459 "Understanding the Double Descent Phenomenon in Deep Learning"

[4] https://www.youtube.com/watch?v=z64a7USuGX0 "What the Books Get Wrong about AI [Double Descent]"

It's interesting how all the recent SOTA models are in the 1T-3T range. This even has a large number of active parameters (A52B). It is also likely that OpenAI, Anthropic, et. al. have similar sized models.

This has two effects: 1) it prevents effective use of these models on consumer-grade hardware; and 2) it pushes the requirements to train and run these models on existing and future data centres, making them more expensive to run.

It would be nice to have more slimmer models and more active research that isn't just "let's throw more compute at the problem". Maybe they'll train smaller models from the large model like Qwen have done in the past.

It's easier to call someone a bot or other names than to engage with them or to try and understand their point of view.

We're increasingly pushed into different groups and sides with little to no toleration of each other. It's been progressing more over the past 10-15 years in the UK, US, and possibly elsewhere.

I think it's a combination of both...

Before AI people would still try asking questions on SO despite the hostility because it was the only/main place to get answers. Over this time, SO popularity was deteriorating.

After AI people who had bad experiences with asking questions on SO would ask the AI instead. This is because the AI wouldn't mark your question as a duplicate, wouldn't question its validity [1], etc. It would just answer your question. You could even ask it follow-up questions if the answer wasn't quite what you were looking for. All of this in real time, not waiting to get an answer.

[1] Unless the question triggered the safety gates that the AI companies have in place.

The picture element [1], [2] supports controlling which image is used via a media selector, allowing you to serve smaller images on smaller devices and on things like the viewport orientation.

[1] https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/...

[2] https://caniuse.com/picture

And with light/dark mode support and CSS variables the colours in a print mode should be easy to define and support.

So you're primarily left with hiding content that shouldn't be printed (like navigation or menu buttons), defining page breaks, and some other quality of life changes for printing.

Printing also covers things like exporting to PDF, or rendering in an eBook format (EPUB, Kindle, etc.) for reading the content offline while travelling, etc.

Probably some form of ablation [1] e.g. [2]. This is typically done by identifying the direction that the refusals/political alignment is represented within the model's weights and then reversing that direction.

[1] https://en.wikipedia.org/wiki/Ablation_(artificial_intellige...

[2] https://arxiv.org/abs/2603.04355 -- Efficient Refusal Ablation in LLM through Optimal Transport

I like a lot of Scala but dislike implicits. I feel like it is the wrong abstraction in that it is similar to implicit type conversion in C/C++, making it harder to reason about what the code is doing and navigating to the types/methods being called.

I prefer extension methods for what Scala implicits do [1], [2]. The only part that is lacking with extension methods is to be able to extend a type to implement an interface, e.g. Comparable, but you can create extension methods that return anonymous objects [3].

[1] https://kotlinlang.org/docs/extensions.html#anonymous-extens...

[2] https://learn.microsoft.com/en-us/dotnet/csharp/programming-...

[3] https://discuss.kotlinlang.org/t/extensions-function-to-impl...

There are three return states that a parser (or partial parser) can return:

1. an item (`T`) for a successful parse;

2. an error (`ParseError`) for an unrecoverable parse error;

3. a partial item (`T with ParseError`) for a recoverable parse error or warning (known statically).

You may also want to differentiate between:

1. leaf nodes (numbers, identifiers, string literals, etc.);

2. composite/nested nodes (addition, multiplication, variable assignment, etc.).

The A3B part of that model means that each "expert" in the mixture of experts (MOE) model use 3B parameters. At 4-bit quant, you can fit 2 parameters per byte. That gives you 1.5GB per expert (plus a bit for overhead), which makes it usable on an 8GB card.

It's all about trade-offs: how much of the model you want to load into VRAM; how much VRAM you want to reserve for context; how much quantization you/the model can tolerate; how many tokens/second you can tolerate; whether you want to use the VRAM for anything else (graphics, Text-to-Speech, etc.); etc.

There's a Minecraft YouTuber called TangoTek who's building a deck building game called Decked Out 3 in Minecraft. That is using a token ring system to control events and messages sent in the game to avoid conflicts [1].

[1] https://www.youtube.com/watch?v=R18APaR7mTQ at 28:54.

How so? Can you link to the numbers?

Antropic's revenue is up [1], [2], [3] and if you calculate adjusted operating income (by excluding most expenses like training costs) then by that measure it is profitable. However, if you factor in all the numbers it is operating at a loss.

[1] https://finance.yahoo.com/technology/ai/articles/anthropic-v...

[2] https://fortune.com/2026/09/29/anthropic-leaked-ipo-prospect...

[3] https://www.cnbc.com/2026/10/15/anthropics-third-quarter-rev...

And CoreWeave's stocks are trending down due to concerns over its debts.

[1] https://www.wsj.com/livecoverage/stock-market-today-dow-sp-5...

[2] https://www.tradingview.com/news/seekingalpha:173ffc32a094b:...

[3] https://www.cnbc.com/2026/10/15/coreweave-credit-default-swa...

Stonking is British slang for large/huge/etc. For example, "Labour won by a stonking majority."

Some of the things that annoy me re: snaps are:

1. the update process is separate from the main system update -- "close this program to update, or will forceably update in X days" notifications;

2. breaking desktop file links where an update can create a duplicate file and after the old snap is removed it removes your pins to the taskbar;

3. possibly breaking other things due to the way the snap directory for the app is not stable.

Yeah, it also suffers from the over-powered protagonist trope found in a lot of fanfics. It has good elements (of what I've read thus far). I think the story could have been better if it focused on a few crossovers and technologies instead of going "here's a completely unrelated power/ability from a new setting", but the premise (earning rewards for writing that result in random abilities from different franchises) doesn't really lend itself to that.

Like, you could still have the different members of the forge and lean into a single source of the tech. Warhammer 40K could fit, given Tetra and some of the initial abilities/power. So you could keep most/a lot of the summit meeting events.

Just like with L Ron Hubbard's work being a foundation for Scientology.

My Immortal?

If you're looking for a job without a stable source of income every expenditure matters. You need to cover essentials -- rent/mortgage, clothing, food, utilities, etc. Those give you a baseline burnthough rate: how long you can survive on just your savings. Any additional expense decreases that and makes it harder to build back up if doing things like saving for a house.

I think it's only a matter of time, like the build-up to the financial crash in 2008 or the dot-com bubble. There's currently a huge amount of money being bet on the big AI tech companies without any real sense of return. And the companies like Oracle that are funding data centre build-out are sitting on unused GPU stock.

Recently, the Australian AI firm failed to secure an IPO [1], [2].

OpenAI is looking to go for an IPO despite shaky revenue [3], [4], [5], resulting in a stock drop for companies like Oracle invested in OpenAI [6].

Anthropic is in a similar situation: looking to IPO despite shaky finances [7], [8], [9].

The companies involved are all investing in each other and trading money between them leading to circular investment [10].

[1] https://www.wsj.com/finance/firmus-ipo-ai-trade-f490c240

[2] https://www.abc.net.au/news/2026-10-10/how-firmus-the-bigges...

[3] https://techcrunch.com/2026/10/08/openais-revenue-is-reporte...

[4] https://www.inc.com/chris-morris/what-openais-leaked-financi...

[5] https://futurism.com/future-society/ai-financial-bubble-open...

[6] https://www.cnbc.com/2026/10/08/open-ai-revenue-nvidia-oracl...

[7] https://www.investors.com/news/technology/anthropic-ipo-fili...

[8] https://www.newconstructs.com/wp-content/uploads/2026/10/Ant...

[9] https://www.tradingkey.com/analysis/stocks/us-stocks/2622154...

[10] https://graceblakeley.substack.com/p/the-ai-circular-economy

It's a data centre being build specifically to run LLMs (and possibly other ML models). As such, it will be focused on NVIDIA servers with large numbers of inter-connected GPUs (so they can run the 2T-4T parameter models from OpenAI, Anthropic, etc.) [1].

It's a different build-out to a data centre that can host video/YouTube content (fast CPU, no GPU, large number/size of hard drives), or that can run web servers (fast CPUs, low to average hard drive capacity).

[1] https://www.nvidia.com/en-gb/data-center/

Yeah, like what happened in the 2008 financial crisis where governments had to step in to save the banks as people defaulted on housing payments due to high mortgages.

Oracle is already close to being classified as junk and reaching a tipping point. If/when that happens it is unclear what affect that will have on the markets on companies invested in AI datacenters with Oracle, and other companies that Larry Ellison and other Oracle investors own shares in.

Listening to Ed Zitron on various podcasts, there are several issues with Oracle stocks:

1. they are one place above junk;

2. they are being treated as effectively junk;

3. if they get classified as junk, any pension portfolios holding Oracle stock must sell as they can't hold junk stocks.

If that happens then there will be a cascade of effects. Not just in the AI/datacentre sector but elsewhere, as Larry Ellison holds a stake in Paramount/helped finance the Paramount/WB merger.

[1] https://www.youtube.com/watch?v=rHaPfdnRoZ4 -- Mostly Economics Podcast #55: Who pays when the AI bubble bursts with Ed Zitron (12:36 - 23:53)

The Cask of Amontillado is great for that. And the classic poem The Raven describes existential dread so well.

I found with King the scarier aspects were the more mundane and psychological parts of his writing. I also found in some cases the adaptations to be better in some ways -- The Shawshank Redemption adaptation is great. And the poem in the TV adaptation of The Tommy Knockers felt more impactful than the one in the book.