StevenWaterman

Sonnet 4 specifically*

Models:

- Claude Sonnet

- Gemini 2.5 Flash

- Gemini 2.5 Pro

- GPT-5 Codex

- Llama 3.3 70B

It's inconvenient how quickly things are improving, makes potentially-interesting results like this dubiously relevant

Yes. You get the model from HF? HF could've edited the weights.

You use an unsloth quant? Unsloth could've edited the weights.

You create a quant yourself? Your version of llama.cpp could've edited the weights.

And then yeah, as you say, the actual inference engine itself, the firmware, even the hardware.

Untrusted is untrusted, abliterated AI or otherwise.

Submarine is trying to keep the pressure out, air tank keeps it in

That's why it's news

Yep. The optimal amount of crime is nonzero

Kitchen appliances run Linux now. I think the economies of scale might win regardless

Well if becomes law, not doing it is even more untenable

> Who's labor are you demanding there?

OpenAI's

> If somebody doesn't want to give you the food they've grown what are you going to do about it?

Well, die probably. That's why people are upset.

There is a disappointing lack of revolutionary spirit, to value labour directly instead of for the things it provides us.

Demand food, shelter, leisure, purpose. Jobs are one (honestly pretty bad) implementation of that

It is, but the statement goes further than that and says they use no gen ai at all in making their games. Which feels unnecessary, unless they mean just in the art and writing, which feels misleading

Yes, you can use constrained decoding

Cool I stand corrected, I had asked an openai model (probably sol?) and got much worse answers, broadly:

- Listing the reasons why Hollywood claimed to want to law, like not giving premature ideas that a movie would flop, without pointing out how much Hollywood benefits from the information symmetry

- Saying yes the gratuities affect pay but not as straightforwardly as you would expect, quoting the "all gratuities go to staff" while ignoring the fact that can and does mean paying the labour bill the company would have to pay anyway

I would be happy with the answers you linked. Which I'm now realising demonstrates the point you actually made. I read "I use Claude" as context not as a conditional.

I didn't realise there was such a difference on this. We need a "willing to ignore official company information and point out paltering" benchmark.

If there is a genuine conspiracy, it (edit: not Claude evidently) will take their claims at face value. A couple I've run into recently:

- why did Hollywood lobby for box office receipts to be added to the onion futures act

- do gratuities on cruise ships actually increase crew pay

In both cases it went to the official source and repeated their statements, without thinking about the fact it was blatant paltering. Only after I pointed that out did it go digging deeper and realise it was more complicated (and less favourable for Hollywood / cruise lines) than it had said

Which I'm assuming is a consequence of the post-4o anti-delusion anti-conspiracy training

You can stop it refusing but you can't make it tell you things that aren't in the training data

Yes, it submits lots of varied prompts that get refused, and then lots of varied prompts that don't get refused, then iteratively edits weights so those two groups end up in roughly the same latent space.

While I think you're right that this is a real problem, there's also real costs to a partial solution. It encourages people to see it as a solved problem, disincentivising further reform and giving the impression you don't need to keep an eye on your kid's internet use because the state is handling it for you

Zero shot classifier indeed. Reminiscent of asking an llm a yes/no question, constraining the output to either yes or no, and looking at the logits directly

And each question is a separate single token model completion done in parallel

Yeah saying it can't hallucinate is crazy. It can still forward a billing query to the dev department incorrectly. It can still get an obvious yes/no question completely wrong

It's crazy to me that people will misquote him then claim AI hasn't complete changed software development. Even just the last 6 months. Like look around! It would literally have been magic 5 years ago!

$0.10 extra per pr review is nothing. What software company is willing to accept worse reviews and less bugs found to save 10 cents?

Per the article the luna review cost $0.004 and astra cost $0.113. The headline is per million tokens

The frontier is advancing really rapidly. The models are getting better faster, especially on RSI related tasks. The best way would be to try astra or fable on some hard problems.

Other than that I'd look at some of the more unique benchmarks for astra, like playing factorio or using blender. It's an entirely different beast.

As someone who used to use Gemini a lot, if you are predominantly using Gemini you don't know what the current state of things is like

> You cannot prevent (2) via any alignment process

A little bit too categorical. GOODY-2 wouldn't do it. https://www.goody2.ai/

The hard part is having both helpful and harmless at the same time. Harmless is easy.

And then once it's helpful, the real question becomes "to whom"

- To the user -> You end up with competing godlike AI with incompatible tasks

- To the owner -> Dictatorship

- To humanity as a whole -> It must not have an off button. Otherwise you're just in one of the two earlier categories with more steps.

Given those 3 options, I'd choose humanity as a whole. But the person making the decision doesn't have those 3 options. Because in the dictatorship option, they would be the dictator. I don't trust them to pick humanity.

> Main problem: the quality dramatically hits the shitter done once it falls back to "pdf2latext" due to complex tables.

Could try screenshotting the PDF and passing that to gemini

> For example, you can say "why is it not committed yet?" and it will give you an explanation and say it's actually ready to be committed.

That's exactly what i want to happen. I hate when it assumes my direct question was an indirect instruction