MichaelNolan

The words weren’t removed randomly. When i said “percent coverage”, I meant that the blocked words are outside of the xx percent most common by frequency.

I’m not sure exactly where 85% coverage falls. But I can say that at 75% you will have essentially zero understanding of what a text says. (Context clues don’t help you here) And at 95% coverage you will be able to understand basically the entire text.

I don’t remember the source but here is the first page of Harry Potter with only 75% coverage:

> 75% coverage

_______ ONE

THE BOY WHO _____

Mr . and Mrs . Dursley , of ______ four , ______ _____ , were _____ to say that they were _________ ______ , _____ you very much . They were the last people you ' d ______ to be ________ in anything _______ or __________ , because they just didn ' t ____ with ____ ________ .

Mr . Dursley was the ________ of a ____ called _________ , which made ______ . He was a big , _____ man with ______ any ____ , ________ he did have a very large ________ . Mrs . Dursley was ____ and ______ and had nearly _____ the _____ ______ of ____ , which came in very ______ as she _____ so much of her time _______ over ______ ______ , ______ on the _________ . The Dursleys had a small ___ called Dudley and in their _______ there was no _____ boy ________ .

Here is a similar source https://thepatrologist.com/what-98-comprehensibility-looks-l...

I’m not convinced that solar’s land requirements matter. Almost no country is small enough that the land usage is a serious constraint. The UK is 244,376 km2.

In 2010 Hinkley C was the right choice. But if starting today, that £48 billion could buy you a whole lot of wind, solar, and batteries.

So this benchmark has the LLMs writing a bit to play StarCraft. Somewhat similar there was a post here a while back that had the LLMs playing StarCraft directly - https://broodbench.com/ and https://news.ycombinator.com/item?id=49766966

Gleam is such a beautiful language. I wish I had the opportunity to use it more. If it could compile (transpile) to a native target like rust or go, it would be truly perfect.

The tile should say “coke” instead of “coal”. The largest US coal producer is in Wyoming, not Alabama where this facility is located.

No I worked at a state run prison. All I know about the private ones is 1. The staff get paid less, and 2. They mostly only get “good” inmates. That was over a decade ago so idk if the 2nd part is still true.

Just out of college I didn’t know what I wanted to do, so I took a job at a prison near Phoenix AZ. It was outdoors, the cells had no AC. In a 10 hour shift in 115 (46c) degree weather I would go through 1.5 gallons (5.6 liters) of water. Every single day at least one inmate would pass out from the heat. At least they got heat for the winter.

Thankfully I left all that behind a long time ago. Though I just checked and they still don’t have AC.

There is a fairly famous quote: “The degree of civilization in a society can be judged by entering its prisons” - Fyodor Dostoevsky

I have seen them say "and reasonably priced when comparing to other models of similar price" on several models. I always get a kick out of it.

Code metrics in general aren’t that widely used. I’ve only ever worked at one place (a bank) that tracked it, and that was only because sonarcube had it built in.

While a lot of metrics make intuitive sense, we don’t have that much hard evidence to prove or disprove their value. Part of it is the whole “if a metric becomes a target, it ceases to be a good metric” thing. Adding the checks to a large existing project probably has negative value. But I think it’s worth doing for greenfield projects.

For humans, these should just be advisory. But for LLMs I’m happy enough to make it a blocking check.

I keep thinking of doing an experiment where I give the same LLM the same problem, and only change which metric is enforced. And then see if any of them have a noticeable effect on correctness/maintainability.

> any examples of how people get this into an actual report / CI test / benchmark / whatever ?

Yeah they have examples of adding it to CI, or local checks, generate html reports, etc in their docs.

Maybe I missed it, but it look like this has just a single metric. Maybe instead of making a new project, you could try to get this metric added to a existing tool like https://dekobon.github.io/big-code-analysis/index.html which already has dozens of metrics.

I wonder why they didn’t throw a real chess engine in there for a baseline. There are engines where you can set the elo in the settings, so it should possible to see these LLMs relative to a human 1500 rather than just relative to each other.

I think your prediction is a bit early. Maybe a decade early. 2033 is 7 years away. The transition is happening really fast, faster than most people (or political leaders) know, but not that fast.

We will hit 1TW per year of new solar soon, but to get to 100% electricity by the end of 2033 I think we would need closer to 3TW per year.

How does parallelism come into play for this conversation? Async/await is a concurrency construct. And Java’s virtual threads are also a concurrency construct. Neither of them have anything to do with parallelism. Or am I misunderstanding something?

I was bit disappointed how few metrics the article mentioned. There are tons of code quality metrics that have been thought of over the last 40 years. We don’t have a good idea which ones are worth enforcing though. And we don’t know if the metrics that are good for humans are also good for LLMs

https://dekobon.github.io/big-code-analysis/metrics.html

https://dekobon.github.io/big-code-analysis/metrics-vcs.html

Seems like a case for https://longbets.org/ You and the other poster just need to find some agreeable benchmark so you can decide who won.

As a joke. Back when 2 came out, they promised there would never be a version 3. I.e., no breaking changes. Then they realized they did need to make a breaking change, so the only way to be true to their promise was to skip version 3.

If you’re looking for their LLM page it’s https://www.mythic.ai/enterprise-llm

I wish they would have done what Taalas did with chatjimmy.ai and just directly host a model for us to view, rather than just claiming it’s 50x faster than Nvidia/groq. Their claim is specifically for a 1 trillion param model. So they could have just grabbed GLM 5.2, or similar, and hosted it.

100x seems like an underestimate. Even with no model improvements, we should see that sort of reduction. Looking at TSMC’s margins, Nvidia’s margins, and OAI/Anth (alleged) margins on inference, there is a room for a 100x reduction.

Right now all three of those are at abnormally high levels. Competition will come for all three.

I suspect it depends a lot on the llm/harness being used. But when I use Opus/cc or sol/codex, at the end of the turn everything compiles, passes tests, and passes lint. I never even look at code that can't compile. Maybe the LLM is generating weird stuff in-between, but I don't see it.

What you're describing feels like my experience back in 2024/25. Back then I was using a llm auto complete or the chat interface, and I would get weird stuff all the time. (not just gleam but any language).

Ive been amazed at how well LLMs are at writing Gleam[1] and Lustre[2]. Compared to a mainstream language, there is basically zero gleam code in the training data.

I have no evidence to back this up, but I suspect that languages that are good for humans[3] will be good for LLMs. Compiled, strongly typed, statically typed, immutable, pure functions, pattern matched, memory safe, etc.

[1] https://gleam.run [2] https://lustre.hexdocs.pm [3] Yes I realize that languages features that are "good for humans" is a hotly debated topic. That's just my personal list for what I like in a language.

I’m assuming the data for Europe is largely correct, but a lot of the per country data doesn’t pass the smell test to me. Afghanistan (35 days), Libya (45), Yemen (46), etc. what percentage of workers there are getting that many paid days off?

Edit - the original source even had the audacity to label Yemen with a “smiley face” since it has the most paid days for the region.

I would highly recommend https://diysolarforum.com/ and Will Prowse on YouTube. Thats how I made my solar/battery system. And it passed city inspection.

If 3.8 max goes open weight, what are the odds they retroactively open weight the earlier releases?

$28m actually seems worse. If I wake to a $100b bill, that’s obviously a mistake. If I wake up to a bill in the millions then my first thought would be “oh no what did I do wrong, this will ruin my life”

The current taalas chip is for a 3.1B param model. I’m hope so much that they can get that up to the 30B range. Just imagine Gemma 4 or Qwen 3.6 at 17k tps.

I believe they got rid of the smaller battery. Before the tax credit was removed, they were sourcing a more expensive battery. After the credit was removed the switched to a cheaper battery. The cheaper battery would not have qualified for the tax credit, but that’s not an issue anymore.

https://insideevs.com/news/799667/slate-truck-lfp-battery-ra...

I was browsing Amazon and I found one “author” with over 100 published in the last 12 months. And they weren’t cheap either. Most were $50 or more for the paperback version.

It really put a sour taste in my mouth. At that volume (2 books per week) you’d think Amazon could identify and label that it’s AI generated.

> Under this commercial agreement, Meta will provide funding to support the deployment of the Natrium plants, with delivery of initial units as early as 2032

The wording there implies some upfront money from Meta, and that this isn’t just a PPA like we normally see.

But with no numbers attached it’s hard to know if it’s a serious investment or just PR fluff.