Well, it should be possible to crowdsource training a FOSS version, right? There should be a SETI-at-home for training neural networks. I would donate some GPU power for sure.

SETI@home achieved 50 times the computing power of the world's largest supercomputer [0], so it might actually be the only way to train the future GPT4 or GPT5.

[0]: https://en.wikipedia.org/wiki/SETI@home#Statistics

At the moment crowd-sourcing ML training doesn't work very well because updates to the model's weights have to be shared between all of the nodes in the computation periodically. Giant models tend to be trained on clusters with very fast interconnects for this reason.

Distributed GPT training doesn't really work (sadly). Maybe someone will get a good solution to that, though.

Says the person who likely owns a washing machine, sink connected to plumbing, microwave, stove, lighters, clothes made with a sewing machine, ect. ect.

GPT-3 will take way less time to make a good substitute that costs the power of compute than other historical time saving technologies. Unlike other historic technologies, they pretty much spell out exactly how to do it, and own no patents related to its creation. I have trouble seeing the downside.

I see your point, but aren't you making me into a bit of a straw man? When did I say that I was some open-access Luddite who won't use any technology that they can't build themselves. I just like the current state of programming, where I can productively build new and exciting things without having to rely on a lot of proprietary libraries or tools. Nothing more, nothing less.

Luddite: a person opposed to new technology or ways of working.

Seems like you came up with the perfect word to describe your view on this. :P

I defintiely do think there is reason to worry about the future you're imagining happening, but as someone who has read through the papers on gpt, this is in no way going to end up being an exclusive proprietary API, so it'll very soon have open source and likely free or at compute cost options.

And well, if it doesn't make us better, no one will use it. If it does, you'll have to adjust to remain competitive, just like millions of professionals for going back millenia who encounter new technology in their chose and vocation. Ultimately, if it makes us better, and doesn't enslave us to the whims of a monopolistic holder (which the point of my post was to conjecture that it likely wouldn't) then it'll probably (though not defintiely) be better for us long term.

Hopefully it doesn't erode programmers abilities like spell check seems to have eroded people's spelling abilities but I fear that's a likely side effect.

Fair enough, it seems like we mostly agree on how technology generally affects the world. I can't say I currently possess the domain knowledge to confirm your views on GPT, so I'll just have to hope you are right.

There is a FOSS version of GPT-3 being worked on by EleutherAI, but they are only at 6 billion parameters so far, while the largest model of GPT-3 is 175 billion (more parameters is better (usually)). EleutherAI is getting more compute from CoreWeave to actually train the GPT-3-like FOSS model, so that's something to look forward to eventually :). Github Copilot uses Codex though, which seems to be a GPT model trained on just code by OpenAI. It wouldn't be too hard to train a FOSS version of Codex on the open-source code of Github and other sources.

it is absurdly ironic that OpenAi is turning out to be one of the biggest opennes perpetrators in the near-future/present.