0xbadcafebee

It's not really exaggeration. It's doing something similar to abliteration, but more. First it finds the part of the weights that does a thing. But then the difference is, while abliteration deletes stuff, this is adding/changing things. It does this by doing semantic analysis of the weights, changing a little bit, see what happens. It's not super complex or advanced; it's essentially a form of statistical analysis and experiment.

This means we might not need to fine-tune with thousands of passes and tons of GPU. We could just tell a frontier model to edit some weights and iterate until there's better results. This could cut the cost of fine-tunes by 100x. Makes no difference to the normal end-user, but pretty huge improvement for LLM research and model tuning.

It's less an industry thing and more a war thing. US wants to fight more wars and for that you need a plentiful supply of portable energy that other countries can't block. For physical wars that's petroleum fuels. By controlling "Greater North America" they can secure their cheap fuel supply and hunker down here. Investing in green energy would de-emphasize the infrastructure you need to process and use massive amounts of oil for cheap.

The coordination is organic. Simply start with this assumption: "each device will check [something] every 60 seconds". Within a few years, all systems will be synced.

When the power goes out on the grid, everything is turned off at the same time. When the power comes back on, everything turns back on at the same time. Every machine that had a 60 second timer is now operating from the same starting time.

That ship sailed a long time ago. Science decided that killing mice for experiments (regardless of the experiment) is fine. In fact, a large amount of the mice killed are just redundant (not needed for a particular experiment)

In computer/systems/distributed engineering this is called request smoothing or traffic shaping, and it's implemented by adding jitter, which is a random time offset/pad added to the interval.

The funny thing is, modern electrical codes already account for this phenomenon. In order to implement Net Metering (selling your power to the grid), modern standards implement two kinds of jitter:

- Sandia Frequency Shift: a minute shift in the power frequency generated by inverters. If the grid is connected, this is effectively eliminated; if the inverter is disconnected from the grid, it detects the frequency has shifted and disconnects the inverter from the grid.

- Slip-Mode Frequency Shift: basically the same thing, but phase shift (which I don't understand).

Both of these are used for anti-islanding (don't create an island of power). When the inverter detects it's isolated and stops outputting current, it then has to check every 5 minutes, with jitter, for whether the grid is restored or not. It has to add random jitter, because otherwise all the mini battery/solar installs will come online at the same time and swamp the grid, causing another outage.

The tech will never be there with LLMs. They can't learn, they hallucinate, lie, cheat, steal, destroy things, break the law, etc. You can't train them out of these behaviors, it's how LLMs work.

An LLM is just a really fancy nail gun to replace hammers. Much faster nailing, but you need to hold it right to not shoot yourself in the foot.

[delayed]

Unfortunately it wouldn't fix anything. It would kill gig work because the model isn't sustainable with regular taxes and minimum wage (if it were, businesses would've already been doing that instead of independent contracting). It also wouldn't fix the income inequality that created gig work. People would be back to dying of thirst, without a thimble of water.

The real fix is to make it so their main job is hydrating enough, or the thing half the country hates the idea of (public fountains). IRS can't do either. Independent contracting is still a valuable mode of work, when the contractors are paid enough. So rather than ban forms of gig work, we need to provide good paying jobs and social services, and get the rich to pay their fair share.

Physician salary is only 9% of total healthcare cost. The rest of the labor pool is administrative - clerks, management, etc.

Physicians also make 2-3x more than physicians in other countries. Why? Their education cost is also 2-3x higher than in other countries.

Hospitals and procedures are expensive, in part because of insurance. Insurance is high because our society is too litigious. It's also expensive because of consolidated private healthcare systems that can control and mark up prices due to lack of competition.

Other countries spend half what we do on healthcare. And their people are often healthier.

The entire 'gig economy' model is exploitation. It only works because of income inequality. It requires a large pool of workers who don't make enough money to support themselves (due to insane healthcare and housing costs, and low wages and benefits) and thus will take any money they can get, despite the poor pay, long hours, and lack of benefits. It's taking a person slowly dying of thirst and offering them a thimble of water in exchange for knitting a sock.

Any bends or crimps in the wire create localized stress points which fail early. For crazy high loads you don't want any of that, so the zinc method is the standard. But I think this particular implementation will be rethought.

The pictures on page 25 seem to show how this socket design fails. First because they didn't evenly distribute the wires in the socket (seriously??)... And second, the shear on the zinc from the compressed wire bundle is essentially a narrow rectangle in the direction of the cable with almost all the force concentrated in a narrow forward bit, not really distributed evenly through the socket, because of the wire ends splaying outward and not being equally distributed. I guess the idea is the rest of the zinc towards the back provides a counter force on the front part, but there's a whole lot more zinc in the back pulling on a smaller amount in the front. The tight bundle and the conical shape is great for a tight install, but doesn't help distribution of load. Compare to the Golden Gate Bridge wire termination (https://c7.alamy.com/comp/2RY9PKW/golden-gate-bridge-plans-i...). The zinc creep is definitely bad, but it seems made worse by an uneven distribution of load in the socket, in addition to all the other problems...

tl;dr once the outer wires break in the socket (in part due to zinc creep, but also potentially more load on them), the "glue" keeping the wire bundle connected to the socket is broken, the former wedge keeping the core in place becomes a straight line, and you have a core ready to pop out.

So someone used a small model with many requests & a massive amount of tokens, to eventually beat a large model with a single request & tiny amount of tokens. .....Cool? Were they not expecting that to work?

The most terrifying part of this is all the devices that might be affected. Ignore servers and desktops. Linux 4.4 came out nearly 11 years ago. While most Linux devices' firmware is actually more ancient than that (Linux 2.x and 3.x isn't uncommon), any device that did ship with Linux 4.4+ is going to get hacked, because nobody patches those devices. Any smart device in the world that isn't a server, desktop, or microcontroller, is probably running Linux.

As part of the Software Building Code (that we don't have, and desperately need), one of the things we should have is mitigation for these abandoned internet safety risks. Here's how I would define that section:

  Section 4    Connections to Public Internet
  Section 4.1  Device Support
    Devices connected to the public internet WILL require one of the following:
      a) Guaranteed LTS support for a minimum of 10 years
      b) Guaranteed rolling release support for a minimum of 10 years
    Devices without this support are not permitted to be connected to the public internet.
  Section 4.2  Vendor Responsibility
    When releasing devices, vendors must determine whether this device is certified for
    public internet use. Support must be maintained according to S4.1. Failure to comply will
    result in fines of up to $5,000 per affected device.
  Section 4.3  Installer Responsibility
    Any device connected to the public internet which is not certified for public internet
    use is in violation, and will be subject to fines of up to $500 per affected device.
  Section 4.4  Device Owner Responsibility
    Any installed devices will be replaced once they reach their End Of Life period,
    or after 10 years, whichever comes first. Installed devices in use that exceed this
    period are in violation, and subject to fines of up to $250 per affected device.

(These fines are tiny compared to the fines for things like fire code violations, but enough money that a company, contractor, or individual would rather fix their device than pay the fines)

[delayed]

> probably pretty good protections

In my career I've seen maybe two companies have pretty good protections. Both had government contracts so they had no choice. Otherwise there's no reason for companies to secure their stack. They make money easier without it.

The buildings are on fire because they're being built out of straw because it's cheaper. Only a government regulation to build out of brick is going to stop the fires.

I don't think they're purposefully bloated, any more than literally all software today is. It all is bloated, but I don't think anyone set out for that. Despite that, the bloat is now the business case. If one of these AI CEOs announced "our main goal is to make tiny models for consumer devices", their stock would crash.

I get that logically you would think that a very wealthy company would improve their technology once they had the resources to do that. But companies do not run by logic. They run by humans, making human decisions, motivated by almost every conceivable thing except logic.

> When you're a startup, expediency and funding prevent this, but by the time you're selling "solutions" and baking designs into ASICs you

.... are still run by the same startup people who think "working like a startup" was a good idea in the first place.

Right now I work with a bunch of people who came up working for startups. They are responsible for the products that make the company the most money. They are still on PHP 7.4.

Everything is enabled by default in Linux, it's just their way. Privilege escalation exploits for Linux are found every year. If you can RCE a Linux box, assume you can also get root.

This is strategic. By making it so consumer grade hardware can't serve it, you have to pay them for access to it, and that funds the server hardware market, which keeps the AI market afloat. If they made smaller models, we would all be running local AI in a few years, nobody would pay them, and they would go bankrupt (and a worldwide recession would begin). It's going to be up to us (FOSS community) to train and publish our own small models. (Also because the US and China are going to make releasing open weights illegal or highly regulated soon)

Of course you can use it for commercial purposes. You just can't serve it as an inference SaaS. https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B/raw...

Open weight is still the correct term because it's not a licensing thing, it's an availability thing. OpenAI/Anthropic are Closed weight because you can't download them at all. Open Source is different because its original use had to do with licensing requirements. Not every phrase with "Open" in it is going to have the same requirements.

When it comes to words, popular usage defines (and redefines) the meaning.

I use this tool every day at work to run different official agent harnesses with subscriptions. The features are plentiful and very useful, upstream development is fast, they accept changes quickly, stable/reliable operation, lightweight. Best of all: it's just a traditional FOSS project, they're not trying to sell you something.

It is created with AI, but I have had no issues using it for months. I'm kind of amazed at how good it is.

It's a model designed to be tiny and still functional, unlike other larger models that are just scaled down and then run into problems (doom loops, intelligence/reasoning failure, tool call failure). You use it when you want a fine-tuned tiny model processing text/audio/video on multiple platforms. (I don't work for them but this is my takeaway)

I love how nobody asks me to give evidence or examples. You don't want to find out you're wrong.

The US is far more hostile than China, also relies on slave labor, and is so increasingly authoritarian that our country's domestic & foreign policy now resembles a 3rd world dictatorship.

Doesn't matter though. The big mistake was allowing manufacturing to be exported from the US to begin with. Nearly any advanced technology we use or wish to produce is now dependent on China. They control not only the advanced manufacturing process, but also the refinement of rare earths (we can mine it but not refine it) and have cheaper labor. Not only can't we make tech without them, we can't sell it at competitive prices. Things will soon get worse as robotics and vertical integration makes Chinese products even cheaper and better quality. And we can't compete on robots because China controls that production chain too.

The smart play would be to build up our own green energy sector to gain cost savings, while investing in its manufacturing, so we can eventually compete and support ourselves (which Biden and Obama passed funding for). Trump eliminated all that when he came back, because the oil companies wanted more profits, they literally don't believe climate change is real, and they have no idea how diplomacy works. We could've had cheap battery, wind, and solar manufacturing in the US. Not anymore.

> and have fun doing it, for better or worse

Like how "the boys" in engineering would made jokes about having sex with the female office manager in the middle of meetings. It was only a great time for certain people.

I kind of forgot about the shared tenancy attack model. I think I was assuming/hoping the software was magically isolating my tokens, when in reality that's impossible for shared tenants. There's whole classes of attacks on things like shared cache, before we get to the LLM-specific features and potential bugs. Enterprises with highly sensitive data should really be on non-shared hosts.

Fwiw, if you have to explain your product's name to the customers who just want to use it, that's a sign the products are confusing.

Tiny/2026-11. Medium/2026-6. Big/2026-10. XL/2026-01. It may not be sexy, but people know exactly what it is and what to use.