Even if the model was open source, nobody would still know how the generated code works. You wouldn't even really know in want interesting sense of the word why the model produced the output you got.

LLMs are even more proprietary than regular proprietary software, I think you've got it right here. They are destroying the free software movement and it seems like most people don't even realize what's happening.

LLMs are a fundamentally different category of thing to software. Developing them is closer to breeding cows than writing code.

Software and slop.

> You don't program an LLM

For now. But it may well become common practice to construct weights directly, or at least to directly adjust weights in trained models, rather than exlcusively train them from prior datasets.

The damage isn't restricted to software. AI is the intellectual equivalent of asbestos.

Asbestos is an amazing fire retardant. That's why nearly every building built before we knew how cancerous it was contains it.

Open source really should mean open training dataset, not merely open weights. There are some truly open source models and you can reproduce their training process if you have the compute. I can't think of a reason why those kinds of models would be incompatible with Stallmanism. Presumably they will get better and more widespread over time, like free software

Here's an example of a model that I believe fits this description: https://huggingface.co/common-pile/comma-v0.1-2t

Actually, this one also excludes nonfree data from the training set; there are probably more capable models that include downloadable nonfree data. I'm not sure if the dataset also needs to be free in order for the model to be considered free.

No one knows how the open source models work either.

It's clear that without hardware freedom, even if we disassemble the bot, the means to run it can still be controlled by hardware vendors. We need our own means to make the substrate for the intelligence too. But the problem is, as long as the manufacturing process is impossibly complex, and relies on minerals extracted all around the world, requiring complex supply chains, its beyond any local group of people to do themselves. What we need is a way of creating substrate for open intelligence just form the materials and resources that are available to any human, and then the freedom to train it to behaving in a way that alignes with our own values and local situation. But the problem with OSS adoption, is that its a real chore for normal people to set up and use it, so we need to make this process of creating our own intelegence host as fun as possible, to encourage people to do it..

This. Nailed it. This right here is the core issue. Thanks for writing.

> What we need is a way of creating substrate for open intelligence just form the materials and resources that are available to any human, and then the freedom to train it to behaving in a way that alignes with our own values and local situation.

I can't tell if you're being cheeky, but... you're describing human child-rearing. From food and water and two humans, substrate can be created, and parents, schools, and other community resources help to train and align.

I agree. We are locked into GPUs and RAMs for now at the mercy of Hynix and Samsung and Nvidia.

The next best alternative is to work on making this happen on older gen and existing hardware that we have. Current VC and SV is too focused on just going wide instead of optimizing.