Holy crap that is dark. I like learning about ML for fun, and now I have to assume that their model is intentionally misinforming me to sabotage my learning? It is absolutely bananas that somebody decided that was ok behavior.
But I think you could have a full open source training software pipeline that's set up to work with Wikipedia, Common Crawl, Books3, Library Genesis, Anna's Archive, and whatever other useful data sets people can name. There would just be a step where you have to provide your own copy of Library Genesis (or whatever subset of it you have managed to obtain).
https://corecursive.com/coding-machines-with-don-and-krystal...
This comment is not entirely on point with your comment, it circles around and above it looking for lift though.
If you're not doing work that requires your code to stay in home nation data centres, Claude for Deepseek, Deepclaude (https://github.com/aattaran/deepclaude) is a great way to get better at using Claude like tools for software development. It even does a pretty good job of putting together cover letters for job applications...
Using Deepclaude is very much cheaper than using claude... For hobby projects, I've found it useful. A recipe (for cooking) management app I've made took a couple of hours to put together and cost $US 0.5. Claude is far more expensive.
The downsides of Deepclaude for many are:-
- DeepSeek is a Chinese corporation so the Chinese Communist Party may ask for data if it wants it.
- DeepClaude isn't as fast as normal Claude, though it's still pretty fast and I think fast enough (YMMV).
- DeepClaude might not be as optimised for various code issues that Claude may be able to solve more quickly or effectively.
- The same safeguards are probably on DeepSeek, but you won't be "wasting" as much money as you might on using Claude.
Inference focused hardware (https://www.youtube.com/watch?v=nvPqHoVSenE, AI generated speech) may in the medium future cause a large enough cost/energy reduction for LLM tools like Claude to make local LLMs more attractive.
Inference focused hardware would make running Open Source models like DeepSeek on local machines far cheaper and control over safeguards would return to the end user.
Hopefully this leads to a localised LLM provision market where local businesses provide varieties of these "local" LLM services. Here, local could mean on premise through to state or nationally based LLM services. Eventually, government orgs outside of the US may demand this kind of LLM use, in the same way governments legally require data to be stored within national borders for many critical government functions.
A bloke can dream I guess...
...Could affordable inference focused hardware also cause the bottom to fall out of these stock market bending valuations for AI corps and their datacentre obsessions?... Not to mention the societal costs caused by the AI super corps building these data centres. At the moment, they're nearly making a profit... They seem almost like speculative companies... Is that a term?
Like did they break through the productivity seal? Or are they willing to spend that much more on it since they see their failure as a like existential threat to humanity. I doubt it our boss sees your software the same way.
Isn't this the classic "dev wants to do start-up, has no skills ouside dev, do builds a dev tool" trap?
Realistically, nobody intellectually honest really knows.
Plenty of people say that by 2030 we have AGI, others estimate 10-20 years.
I personally say 5-15 years.
AI 2027 estimates for 2027: "OpenBrain automatescoding"
No one said 2 years ago, that AGI is coming in 2026.
But lets be honest, I do not know, thats true.
I ahven't seen technology like this which affects me directly. But from past we know what disruptive technology looks and feels like. The weaving chair/loom disrupted an industry, energy, steam engine, internet.
Now we have the most generic technology an LLM/AI at a time were every major problem was solved:
We have the internet which allows for fast communication, we have very fast hardware, we have a supply chain which can react/act very fast globally and we have the richest companies on our planet investing into this technology unseen amounts of money. We have a local race between the richest companies on the world and a global race between the biggest world powers on our planet.
And we have the smartest people on our planet involved in this too. A lot of peple from academy went to the industry, some gave up their tenure for AI.
It would be very ignorant to assume that all of this can't lead to significant and fast change in our society.
It might not, but its like looking at a huge wildfire 50km away and just going to bed.
Even if that were 100% true, it only collapses the coding effort to near zero. Anyone who's built and shipped a real product should know that coding is maybe 50% of the work, and on a mature product it can be much less.
i dont write code by hand anymore but shipping something people want is as hard (or maybe harder?) as its ever been
i also run loops that comb through slack / github to auto-propose a fix & have another agent auto-review, but you need the human to stamp, and they fail in subtle ways architecturally
But even with this toy project, and the target market being someone I should know very well (me), I often struggle to figure out what I want the app to do. When I go through brainstorming or grilling sessions it'll often ask me a question about how the product ought to work and I'm just like ¯\_(ツ)_/¯ give me suggestions and I'll let you know.
Sometimes it takes hours of discussion and tracking down decision-makers just to figure out what the intended behavior is.
Apart from many other issues with this, heavily subsidized subscription plans won't last forever, and if you start burning your own money on tokens in this way, you'll soon realize it's terribly inefficient.
People think otherwise with AI partly because Anthropic kept telling us that they didn't have to write code or review code any more for most of their work. Their agent swarms just comb through their github, slack and wikis to figure out what to do next, and another swarm of agents just review, test, merge, deploy, A/B test, and revert the code. Boris alone merged nearly 300 PRs in the past week (or two?). So the top research labs seem have broken the productivity seal.
And then they talk about this recursively self-improving AI that is so powerful, so autonomous that they advocate that every company should be prepared to "pause" the effort. And their Fable/Mythos has this specific restriction as mentioned in their model card[1] that they are going to reject requests to tune and train models because, well you guess it, the models are too powerful to be used by mere mortals.
[1] We’ve implemented new interventions that limit Claude’s effectiveness for requests targeting frontier LLM development (for example, on building pretraining pipelines, distributed training infrastructure, or ML accelerator design). Using Claude to develop competing models already violates our Terms of Service, but enforcing this restriction through our safeguards avoids accelerating the actors most willing to violate these terms. Unlike our interventions for cybersecurity, biology and chemistry, and distillation attempts, these safeguards will not be visible to the user. Fable 5 will not fall back to a different model. Instead, the safeguards will limit effectiveness through methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning (PEFT).