News for hackers. Stuff that matters.
Re:1-Bit LLM in the Browser by dim13 on Monday July 20, 2026 @05:36AM (#48976366)
Fails "car wash" test miserably. Ask it:

> I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

Ref: https://opper.ai/blog/car-wash-test

Re:1-Bit LLM in the Browser by Diogenesian on Monday July 20, 2026 @07:56AM (#48977528)
The other reason this test is useless is that not even Fable actually understands why walking to the car wash is dumb, and I am sure you can find an adversarial way to trick it. The big LLM providers have almost certainly paid some underemployed STEM graduate in the global South to tediously drill iterations of this problem into their models to pass the social media benchmark.
Re:1-Bit LLM in the Browser by SwellJoe on Monday July 20, 2026 @05:50AM (#48976492)
Even very large models failed tests like this up until a year or two ago...this is a very, very, small model.
Re:1-Bit LLM in the Browser by Xalutiono on Monday July 20, 2026 @06:37AM (#48976915)
Thats absolutly not the point of these models.

They are here to route things, do tool calls, ask experts.

Re:1-Bit LLM in the Browser by OutOfHere on Monday July 20, 2026 @11:23AM (#48980168)
That is completely false. It is a reasonable type of question and a reasonable expectation. You can't route correctly if you can't do reasoning.

Imagine a prompt:

```

INSTRUCTION: You're a customer service agent that routes customer service requests.

CUSTOMER MESSAGE: I haven't received my order and want a refund.

CONTEXT: A signature is required at delivery. No one was available to sign.

You can route to one of:

* Billing & refunds

* Shipping & tracking

* Order cancelation

* Level II customer service

```

The dumb model, e.g. 1.7B, wrongly routes to Billing. The smart model, e.g. GPT-5.6-Medium, correctly routes to Shipping. It matters. In the real world, the CONTEXT will even be 10-100x larger and noisier.

Re:1-Bit LLM in the Browser by Xalutiono on Wednesday July 22, 2026 @03:30AM (#49003044)
IT is not black and white and just because you are able to bring up a random example doesn't mean you are right.

These models are build to be small, we have a lot of diverse tasks which do not need fable or whatever because if you are a company and you have to route 100.000 requests per day, costs matter.

And these small models, if they are not good enough for my specific use case, i can easily finetune (which i have btw.).

You can also easily solve your example by having a second stage. If the customer is unhappy about the first thing or an employee flags it as 'wrong', you can either send it automatically to a more expensive model or the employee fixes it and you got automatic training data.

Re:1-Bit LLM in the Browser by woadwarrior01 on Monday July 20, 2026 @06:11AM (#48976669)
Works with the Bonsai-27B-1bit-mlx model on my mac.

Edit: this is running with MLX, and not with WebGPU on the browser, also it is a fail.

"""

You should *walk*.

Here’s why:

- *Distance*: 50 meters is very short — about 0.3 miles or 0.5 kilometers.

- *Time*: Walking takes roughly 2–3 minutes. Driving would take longer due to parking, starting the engine, and maneuvering.

- *Effort*: Walking is light exercise and avoids parking hassle.

- *Safety*: Less risk of misjudging parking space or damaging the car.

Unless you have a disability that makes walking difficult, walking is the most practical and efficient choice.

"""

Re:1-Bit LLM in the Browser by geon on Monday July 20, 2026 @06:17AM (#48976737)
How is that not a fail?
Re:1-Bit LLM in the Browser by whstl on Monday July 20, 2026 @08:18AM (#48977761)
I think GP also failed the car wash test.