But what happens with generations that will grow up with AI readily available? There is a good chance that there will be a generational skill atrophy in the future, as less people will be inclined to develop the experience required to use AI as a helper, but not depend on it.
However we were born post invention of photography and look at the havoc it's wreaking with post-truth.
The answer to that lies in reforming the education system so that we teach kids digital hygiene.
How on earth we still teach kids Latin in some places but not python? It's just an example, extrapolate python to everything tech that is needed for us to have a healthy relationship with tech.
Perhaps that's also a reason why - tech is so large, there's no time in a traditional curriculum to teach all of it. And only teaching what's essential is going to be tricky because who gets to decide what's essential? And won't this change over time?
There is no perfect solution, but most imperfect attempts are superior to doing nothing.
So it's teaching them a language they can't use to augment their work between or pass their work to other non-techies.
Pyinstaller will produce PE, ELF, and Mach-O executables, and
Py2wasm will produce wasm modules that will run in just about any modern browser.
(I'm guessing that's what you were hinting at.)
If we're teaching everyone some language, we could very much decide that this language ought to be installed in the "normal person computing environment".
I definitely don't want people to learn to write code from JavaScript as it has way too many issues to be deemed representative of the coding experience.
Now imagine if we’d managed to do that with JS or Python instead.
Also digital literacy is a fantastic skill - I'm all for it. And I think that digital (and cultural) literacy leads me to wonder if AI is making the human experience better, or if it is primarily making corporations a lot of money to the detriment of the majority of people's lives.
Replace lying with LLM and all I see is a losing battle.
Current parents, though, aren't going to teach kids how to use it, kids will figure that out and it will take a while.
I thought it was cute when we had the "anniversary" for Back to the Future's timestamp, but for that one ... "too soon, man"
Now, compare that with our world: even if thing X is obviously harming the kids, there is nothing we can do.
I wanted to highlight this assumption, because that's what it is, not a statement of truth.
For one, it doesn't really look like the current techniques we have for AI will scale to the "much better" you're talking about -- we're hitting a lot of limits where just throwing more money at the same algorithms isn't producing the giant leaps we've seen in the past.
But also, it may just end up that AI provider companies aren't infinite growth companies, and once companies aren't able to print their own free money (stock) based on the idea of future growth, and they have to tighten their purse strings and start charging what it actually costs them, the models we'll have realistic, affordable access to will actually DECREASE.
I'm pretty sure the old fashioned, meat-based learning model is going to remain price competitive for a good long while.
Perhaps some rare open source rebels will hold the line, and perhaps it'll be legal to buy the hardware to run them, and maybe the community will manage to keep up with feature parity with the commercial models, and maybe enough work can be done to ensure some concept of integrity in the training data, especially if some future advance happens to reduce the need for training data. It's not impossible, but it's not a sure thing, either.
In the super long run this could even grow into the major problem that AIs have, but based on how slow humanity in general has been to pick up on this problem in other existing systems, I wouldn't even hazard a guess as to how long it will take to become a significant economic force.
So yeah, the current AI companies are making it very difficult for public alternatives to emerge.
I wanted to draw attention to Moore's Law and the supercomputer in your pocket (some of them even ship with on-board inference hardware). I hear you that the newest hottest thing will always require lighting VC money on fire but even today I believe one could leverage the spot (aka preemptable) market to run some pretty beefy inference without going broke
Unless I perhaps misunderstood the thrust of your comment and you were actually drawing attention to the infrastructure required to replicate Meta's "download all the web, and every book, magazine, and newspaper to train upon petabytes of text"
None of this includes hardware optimizations either, which lags software advances by years.
We need 2-3 years of plateauing to really say intelligence growth is exhausted, we have just been so inundated with rapid advance that small gaps seem like the party ending.
It's a bit similar with the brain, learning and AI use. Except when it comes to gaining and applying knowledge, the muscle that is trained is judgement.
"This is the worst form of web there will ever be; it will only get better."
I for one can't wait to be force fed ads with every answer.
I think learning and critical thinking are skills in and of themselves and if you have a magic answering machine that does not require these skills to get an answer (even an incorrect one), it's gonna be a problem. There are already plenty of people that will repeat whatever made up story they hear on social media. With the way LLMs hallucinate and even when corrected double down, it's not going to make it better.
That's absolutely not the case, paper maps don't have a blue dot showing your current location. Paper maps are full of symbols, conventions, they have a fixed scale...
Last year I bought a couple of paper maps and went hiking. And although I am trained in reading paper maps and orientating myself, and the area itself was not that wild and was full of features, still I had moments when I got lost, when I had to backtrack and when I had to make a real effort to translate the map. Great fun, though.
3D Army Land Navigation Courses - https://news.ycombinator.com/item?id=43624799 - April 2025 (46 comments)
> There is a good chance that there will be a generational skill atrophy in the future, as less people will be inclined to develop the experience required to use AI as a helper, but not depend on it.
I don't how to care for livestock or what to do to prepare and can a pig or a cow. I could learn it. But I'll keep using the way of least resistance and get it from my butcher. Or to be more technological: I'd have to learn how to make a bare OS capable of starting from a motherboard, it still does not prevent me from deploying k8s clusters and coding apps to run on it.
You'd sing a different tune if there was a good chance from being poisoned by your butcher.
The two examples you chose are obvious choices because the dependencies you have are reliable. You trust their output and methodologies. Now think about current LLMs-based agents running your bank account, deciding on loans,...
We already see this today: a lot of young people do not know how to type in keyboards, how to write in word processors, how to save files, etc. A significant part of a new generation is having to be trained on basic computer things same as our grandparents did.
It's very intersting how "tech savvy" and "tech compentent" are two different things.
https://www.cato-unbound.org/2006/01/08/jaron-lanier/gory-an...
Typing on a keyboard, using files and writing on a word processor, etc. are accidental skills, not really essential skills. They're like writing cursive: we learned them, so we think naturally everybody must and lament how much it sucks that kids these days do not. But they don't because they don't need to: we now have very capable computing systems that don't need files at all, or at least don't need to surface them at the user level.
It could be that writing or understanding code without AI help turns out to be another accidental skill, like writing or understanding assembly code today. It just won't be needed in the future.
I will lament that professionals with desk jobs can't touch-type. But not out of some "back in my day" bullshit. I didn't learn until my 20s. I eventually had an "oh no" realization that it would probably pay major dividends on the learning investment. It did. And then I knew.
I was real good at making excuses to never learn too. Much more resistant than the student/fresh grads I've since convinced to learn.
So if anything, we're going back to the past, when typing need only be learned by specialists who worked in certain fields: clerical work, data entry, and maybe programming.
Writing cursive may not be the most useful skill (though cursive italic is easy to learn and fast to write), but there's nothing quite like being able to read an important historical document (like the US Constitution) in its original form.
And so the people who are aiming to go into that kind of work will learn these skills.
Academia is a tiny proportion of people. "Business" is larger but I think you might be surprised by just how much of business you can do on a phone or tablet these days, with all the files shared and linked between chats and channels rather than saved in the traditional sense.
As a somewhat related example, I've finally caved into to following all the marketing staff I hire and started using Canva. The only time you now need to "save a picture" is... never. You just hit share and send the file directly into the WhatsApp chat with the local print shop.
And this is exactly what is meant by generational skill atrophy. You no longer own your own files or manage your own data, it's all handled in cloud solutions outside of your control, on devices you barely understand how they work and in channels controlled by companies looking to earn a profit.
When any of those links break, you are suddenly non-functional. You can no longer access your files, you can no longer work on your device. This skill atrophy includes the ability to correctly analyze and debug problems with your devices or workflow in question.
Just like there is already generational gap with developers who don't understand how to use a terminal (or CS students who don't understand what file systems are).
AI will ensure there are people who don't think and just outsource all of their thinking to their llm of choice.
So I just went to DeepSeek instead and finished like 25% of my project in a day. It was the first time in my whole life that programming was not fun at all. I was just accomplishing work - for a side project at that. And it seems the LLMs are already more interested in talking to me about code than my dad who's a staff engineer.
I am going to use the time saved to practice an instrument and abandon the "programming as a hobby" thing unless there's a specific app I have a need for.
On point of discussing code.. a lot of cloud frameworks are boring but good. It usually isn't the interesting bit and it is a relatively recent quirk that everyone seems to care more about the framework compared to the thing you actually wanted to achieve. It's not a fun algorithm optimization, it's not a fun object modeling exercise, it's not some nichey math thing of note or whatever got them into coding in the first place. While I can't speak for your father I haven't met a programmer who doesn't get excited to talk about at least one coding topic this cloud framework just might not have been it.
I only read your comment after I posted mine, but my take is basically the same as yours: the GP thinks the IT learning-treadmill is fun and his dad doesn't.
It's not hard to see the real problem here.
And learning new technologies in pursuit of resume-driven-development is fun?
I gotta say, if learning the intricacies of $LATEST_FAD is "fun" for you, then you're not really going to have a good time, employment-wise, in the age of AI.
If learning algorithms and data structures and their applicability in production is fun, then the age of AI is going to leave you with very in-demand skills.
Nothing to do with employment. I was just doing a "home-cooked app"[0] thing for fun that served a personal usecase. Putting it on my resume would be a nice-to-have to prove I'm still sharpening my skills, but it isn't the reason I was developing the app to begin with.
What I think at least is the administration and fault monitoring of lots of random machines and connected infrastructure in the cloud might be left somewhat untouched by AI for now, but if it's just about slinging some code to have an end product, LLMs are probably going to overtake that hobby in a few years (if anyone has such a weird hobby they'd want to write a bunch of code because it's fun and not to show to employers).
You're right that a lot less people will be writing code like a lot less people are sewing these days but people still do it for fun.
P.S.: I really enjoyed reading moonbound :)
I tried using an LLM to help with a small hardware project and ended up throwing out all the code halfway down the line and doing it myself. I don't have a good time with LLMs and mostly only use it for work.
Damn kids, you were supposed to be teasing me for not knowing how the new tech works by now.
Spot on. Look at the stark difference in basic tech troubleshooting abilities between millennials and gen z/alpha. Both groups have had computers most of their lives but the way that the computers have been "dumbed down" for lack of a better term has definitely accelerated that skill atrophy.
- Recruitment processes are not AI-aware and will definitely won't be able to identify the more capable individual hence losing out on talent
- Police departments are not equipped to deal with the coming wave of complaints regarding cyberfraud as the tech illiterate get tricked by anonymous LLM systems
- Universities and schools are not equipped to deal with students submitting coursework completed by LLM hence missing their educational targets
- Political systems are not equipped to deal with subversive campaigns using unethical online entertainment platforms (let's not called them social media please) such as FB and they are definitely not equipped to deal with those campaigns when they boost their effectiveness with LLM at scale
Yes, and it seems to me that at least democracies haven't really figured out and evolved to deal with the Internet after 30 years.
So don't hold your breath !
Worse yet many educators are not being supported by their administration since enrollments are falling and the admin wants to keep the dollars coming regardless of if the students are learning.
It's worse than just copying Wikipedia because plagarism detectors aren't as effective and may never be.
It's an arms race and right now AI cheating has structural advantages that will take time to remove.
But that does nothing for homework or long term projects where you can't control the student's physical location for the duration of the work.
You could do a detailed interview after the work is completed, to verify the student actually understands the work they supposedly produced. But that adds to the time spent between instructors and students making it harder to scale classes to large sizes. Which may not be a completely bad thing.
It is for those who manage those educational establishments. And should they fall as a result of cheating directly or indirectly, the adjacent economies will fall or be existentially damaged. The repercussions of all this are not trivial.
This.
It will in a sense just further boost inequality between people who want to do things, and folks who just want to coast without putting in the effort. The latter will be able to coast even more, and will learn even less. The former will be able to learn / do things much more effectively and productively.
Since good LLMs with reasoning are here, I've learned so many things I otherwise wouldn't have bothered with - because I'm able to always get an explanation in exactly the format that I like, on exactly the level of complexity I need, etc. It brings me so much joy.
Not just professional things either (though those too of course) - random "daily science trivia" like asking how exactly sugar preserves food, with both a high-level intuition and low-level molecular details. Sure, I could've learned that if I wanted too before, but this is something I just got interested in for a moment and had like 3 minutes of headspace to dedicate to, and in those 3 minutes I'm actually able to get an LLM to give me an excellent tailor-suited explanation. This also made me notice that I've been having such short moments of random curiosity constantly, and previously they mostly just went unanswered - now each of them can be satisfied.
I disagree. I get egregious mistakes often from them.
> because I'm able to always get an explanation
Reading an explanation may feel like learning, but I doubt it. It is the effort of going from problem/doubt to constructing a solution - and the explanation is a mere description of the solution - that is learning. Knowing words to that effect is not exactly learning. It is an emulation of learning, a simulacrum. And that would be bad enough if we could trust LLMs to produce sound explanations every time.
So not only getting the explanation is a surrogate of learning something, you also risk internalizing spurious explanations.
> Reading an explanation may feel like learning, but I doubt it. It is the effort of going from problem/doubt to constructing a solution - and the explanation is a mere description of the solution - that is learning. Knowing words to that effect is not exactly learning. It is an emulation of learning, a simulacrum. And that would be bad enough if we could trust LLMs to produce sound explanations every time.
Every person learns differently, and different topics often require different approaches. Not everybody learns exactly like you do. What doesn't work for you may work for me, and vice versa.
As an aside, I'm not gonna be doing molecular experiments with sugar preservation at home, esp. since as I said my time budget is 3 minutes. The alternative here was reading about it on wikipedia or some other website.
I'd rather just skip the hassle and keep using known good sources for 'learning about' things.
It's fine to 'learn about' things, that is the extent of most of my knowledge. But from reading books, attending lectures, watching documentaries, science videos on youtube or, sure, even asking LLMs, you can at best 'learn about' things. And with various misconceptions at that. I am under no illusion that these sources can at best give me a very vague overview of subjects.
When I want to 'learn something', actually acquire skills, I don't think that there is any other way than tackling problems, solving them, being able to build solutions independently and being able to explain these solutions to people with no shared context. I know very few things. But I am sure to keep in mind that the many things I 'know about' are just vague apprehensions with lots of misconceptions mixed in. And I prefer to keep to published books and peer reviewed articles when possible. Entertaining myself with 'non-fiction' books, videos etc is to me just entertainment. I never mistake that for learning.
I am not a physicist and I will most likely never require to do anything related to quantum physics in my daily life. But it's fun to be able to have a quick mental model to "have an idea" about who was Max Planck.
Funny you should mention him, I am very interested in his conceptions about the nature of reality:
'Planck said in 1944, "As a man who has devoted his whole life to the most clear headed science, to the study of matter, I can tell you as a result of my research about atoms this much: There is no matter as such. All matter originates and exists only by virtue of a force which brings the particle of an atom to vibration and holds this most minute solar system of the atom together. We must assume behind this force the existence of a conscious and intelligent spirit [orig. geist]. This spirit is the matrix of all matter."'
The Wikipedia might lack the interactivity I enjoy from the technology ? I mean, I'm all up for local LLMs, but currently this just seems overpriced...
I've been finding that ChatGPT is helpful when taking a "first dive" into an unfamiliar topic. But, after studying the topic at greater depth through primary sources, I'll start to see many subtle errors, or over-simplifications, or claims stated as facts which are actually controversial among experts, in the ChatGPT answers. Overall, I'd say ChatGPT can provide a good approximation of truth, which can speed up research by providing instant context. But it should not by any means be the final destination when researching a topic.
My biggest barrier to EVERYTHING is not knowing the right word or term to search. LLMs ftw.
A proper LLM would let me search all of my work's artifacts when I ask about some loose detail I half remember. As it is, I know of a topic and I simply can't find the _exact word_ to search so I can't find the right document or slack conversation
I will note however, that it has expanded his capabilities. Some of the tools he use are scriptable and he can now prompt his way into getting these scripts. Something he'd previously would have needed a programmer for. In this aspect his capabilities now overlap mine, but he's still not the slightest more interested in actually learning programming.
Well said. Textbook problem that has the answer everywhere.
The question is, would you create similar neural paths if reading the explanation as opposed to figuring it out on your own?
Excelent point, and I believe the answer is a resounding negative.
Struggling with a problem generates skills and knowledge which you then possess and recall more easily, while reading an answer merely acquires some information that competes with a whole host of other low-effort information that you need to remember.
Plato might have been wrong about the ills of cyberization cognitive skill such as memory. I wonder if two thousand years later from then, we will be right about the ills of cyberization of a cognitive skill such as reasoning
I agree. I don't really feel like I know something unless I can go from being presented with a novel instance of a problem in that domain and work out a solution by myself, and also explain that to someone else - not just happen into a solution.
> Plato might have been wrong about the ills of cyberization cognitive skill such as memory.
How so? From the dialogue where he describes Socrates discussing writing I get a pretty nuanced view that lands pretty much where you did above: access to writing fosters a false sense of understanding when one can read explanations and repeat them but not actually internalize the reasoning behind it.
You will still need the textbook because llms hallucinate just as much as a teacher can be wrong in class. There is no free lunch, llm is just a tool. You create the meaning.
THEN SAID A teacher, Speak to us of Teaching.
And he said:
No man can reveal to you aught but that which already lies half asleep in the dawning of your knowledge.
The teacher who walks in the shadow of the temple, among his followers, gives not of his wisdom but rather of his faith and his lovingness.
If he is indeed wise he does not bid you enter the house of his wisdom, but rather leads you to the threshold of your own mind.
The astronomer may speak to you of his understanding of space, but he cannot give you his understanding.
The musician may sing to you of the rhythm which is in all space, but he cannot give you the ear which arrests the rhythm nor the voice that echoes it.
And he who is versed in the science of numbers can tell of the regions of weight and measure, but he cannot conduct you thither.
For the vision of one man lends not its wings to another man.
And even as each one of you stands alone in God’s knowledge, so must each one of you be alone in his knowledge of God and in his understanding of the earth.
being a neophyte in a subject, and relying solely on 'wisdom' of LLMs seems like a surefire recipe for disaster.
If you trust symbols blindly, sure it's a hazard. But if you treat it as a plausible answer then it's all good. It's still your job to do the heavy lifting of understanding the domain of the latent search space, curate the answers and verify the information generated
There is no free lunch. LLMs isn't made to make your life easier. It's made for you to focus on what matters which is the creation of meaning.
> Sontag argues that the proliferation of photographic images had begun to establish within people a "chronic voyeuristic relation to the world."[1] Among the consequences of this practice of photography is that the meaning of all events is leveled and made equal.
This is the same with photography as with llms. The same with anything symbolic actually. It's just a representation of reality. If you trust a photograph fully that can give you a representation of reality that isn't grounded in reality. It's semiotics. Same with llm, if you trust it fully you are bound to get screwed by hallucination.
There are gaps in the logical jumps, I know. I'd recommend you take a look at Philosophize This' episodes about her work to fill them at least superficially.
However we are not talking about everyone, are we? Just people that "will cut corners on verifying at the first chance they get".
Is it you? I have no idea. I can only remain vigilant so it's not myself.
But if you're so insecure about yourself that you invest more energy into faking it than other people do in actually doing it this, this is probably a one-way street to never actually be able to do anything yourself.
Open source AI tools that you can run locally in your machines? Awesome! AI tools that are owned by a corporation with the intent of selling your things you don’t need and ideas you don’t want? Not so awesome.
"Who needs seat belts and airbags? A well-disciplined defensive driver simply won't crash."
We didn't simply avoid inventing cars because we didn't know how to make crashes safe.
I would take that advice with caution. LLM's are not oracles of absolute truth. They often hallucinate and omit important pieces of information.
Like any powerful tool, it can be dangerous in the unskilled hands.
I wouldn't be so sure. Search engine quality has degraded significantly since the advent of LLMs. I've seen the first page of Google entirely taken up by AI slop when searching for some questions.
You nailed it. LLMs are an autodidact's dream. I've been working through a physics book with a good-old pencil and notebook and got stuck on some problems. It turned out the book did a poor job of explaining the concept at hand and I worked with ChatGPT+ to arrive at a more comprehensible derivation. Also the problems were badly worded and the AI explained that to me too. It even produced that Latex document study guide! Moreover, I can belabor a topic which I would not do with a human for fear of bothering them. So for me anyway, AI is not enabling brain rot, but brain enhancement. I find these technologies to be completely miraculous.
I see the same risk when AI is understood to be a learning tool. Sure, it can absolutely be a tool for learning, but it does take some will power to intentionally learn if it is solving short-term problems.
That temptation is enormously amplified if AI is used as a teaching tool in grade school! School is sometimes boring, and it can be challenging for a teen to push through a problem-set or essay that they are uninterested in. If an AI will get them a passing grade today, how can they resist?
These problems with AI in schools exist today, and they seem destined to become worse: https://www.whitehouse.gov/presidential-actions/2025/04/adva...
If you just play a game on its own, you end up playing all the non optimal strategies and just enjoy the game the most fun way. But then someone will spend weeks with spreadsheets working out the absolute time fastest way to progress the game even if it means repeating the most mundane action ever.
Now everyone watches a YouTube guide to play the game and ignores everything but the most optimal way to play the game. Even worse is that games almost expect you to do this and make playing the non optimal route impossibly difficult.
I'll emphasize this: for generally well-understood subjects, LLMs make incredibly good tutors.
Talking to ChatGPT or whichever, I feel like I'm five years old again — able to just ask my parents any arbitrary "why?" question I can think of and get a satisfying answer. And it's an answer that also provides plenty of context to dig deeper / cross-validate in other sources / etc.
AFAICT, children stop receiving useful answers to their arbitrary "why?" questions — and eventually give up on trying — because their capacity to generate questions exceeds their parents' breadth of knowledge.
But asking an (entry-level) "why?" question to a current-generation model, feels like asking someone who is a college professor in every academic subject at once. Even as a 35-year-old with plenty of life experience and "hobbyist-level" knowedge in numerous disciplines (beyond the ones I've actually learned formally in academia and in my career), I feel like I'm almost never anywhere near hitting the limits of a current-gen LLM's knowledge.
It's an enlivening feeling — it wakes back up that long-dormant desire to just ask "why? why? why?" again. You might call it addictive — but it's not the LLM itself that's addictive. It's learning that's addictive! The LLM is just making "consuming the knowledge already available on the Internet" practical and low-friction in a way that e.g. search engines never did.
---
Also, pleasantly, the answers provided by these models in response to "why?" questions are usually very well "situated" to the question.
This is the problem with just trying to find an answer in a textbook: it assumes you're in the midst of learning everything about a subject, dedicating yourself to the domain, picking up all the right jargon in a best-practice dependency-graph-topsorted order. For amateurs, out-of-context textbook answers tend to require a depth-first recursive wiki-walk of terms just to understand what the originally answer from the textbook means.
But for "amateur" questions in domains I don't have any sort of formal education in, but love to learn about (for me, that's e.g. high-energy particle physics), the resulting conversation I get from an LLM generally feels like less like a textbook answer, and more like the script of a pop-science educational article/video tailor-made to what I was wondering about.
But the model isn't fixed to this approach. The responses are tailored to exactly the level of knowledge I demonstrate in the query — speaking to me "on my level." (I.e. the more precisely I know how to ask the question, the more technical the response will be.) And this is iterative: as the answers to previous questions teach and demonstrate vocabulary, I can then use that vocabulary in follow-up questions, and the answers will gradually attune to that level as well. Or if I just point-blank ask a very technical question about something I do know well, it'll jump right to a highly-technical answer.
---
One neat thing that the average college professor won't be able to do for you: because the model understands multiple disciplines at once, you can make analogies between what you know well and what you're asking about — and the model knows enough about both subjects to tell you if your analogy is sound: where it holds vs. where it falls apart. This is an incredible accelerator for learning domains that you suspect may contain concepts that are structural isomorphisms to concepts in a domain you know well. And it's not something you'd expect to get from an education in the subject, unless your teacher happened to know exactly those two fields.
As an extension of that: I've found that you can ask LLMs a particular genre of question that is incredibly useful, but which humans are incredibly bad at answering. That question is: "is there a known term for [long-winded definition from your own perspective, as someone who doesn't generally understand the subject, and might need to use analogies from outside of the domain to explain what you mean]?" Asking this question — and getting a good answer — lets you make non-local jumps across the "jargon graph" in a domain, letting you find key terms to look into that you might have never been exposed to otherwise, or never understood the significance of otherwise.
(By analogy, I invite any developer to try asking an LLM "is there a library/framework/command-line tool/etc that does X?", for any X you can imagine, the moment it occurs to you as a potential "nice to have", before assuming it doesn't exist. You might be surprised how often the answer is yes.)
---
Finally, I'll mention — if there's any excuse for the "sycophantry" of current-gen conversational models, it's that that attitude makes perfect sense when using a model for this kind of "assisted auto-didactic learning."
An educator speaking to a learner should be patient, celebrate realizations, neutrally acknowledge misapprehensions but correct them by supplying the correct information rather than being pushy, etc.
I somewhat feel like auto-didactic learning is the "idiomatic use-case" that modern models are actually tuned for — everything else they can do is just a side-effect.
I really agree with what you've written in general, but this in particular is something I've really enjoyed. I know physics, and I know computing, and I can have an LLM talk me through electronics with that in mind - I know how electricity works, and I know how computers work, but it's applying it to electronics that I need it to help me with. And it does a great job of that.
Some of these students are dishonest. Many aren't. Many genuinely believe the work they submit is their own and that they're learning the languages. It isn't and they aren't.
People are absolutely horrendous at this kind of attribution. They forget sources. They mistake others' ideas for their own. Your model of intention, and your distinction between those who wish to learn and those who pose, don't work. The people most inclined to seek the assistance that these tools seem to offer are the ones least capable of using them responsibly.
These tools are a guaranteed path to brain rot and an obstacle to real, actual study and learning, which require struggle and confusion without access to easy answers.
People are quite poor at this kind of attribution, especially when they're already cognitively overloaded. They forget sources. They mistake others' ideas for their own.
This attitude is common not only among students, in my experience many people behave this way.
I also see some parallels to LLM hallucinations..
If they are using LLMs to deliver final work they are all posers. Some are aware of it, many aren't.
> Many genuinely believe the work they submit is their own, that they really did do the work, and that they're learning the languages. It isn't, they didn't, and they aren't.
But I'm talking about a very specific intentionality in using LLMs which is to "help us understand what's missing in our understanding of the problem, if our solution is plausible and how we could verify it".
My model of intention and the distinction relies on that. You have a great opportunity to show your students that LLMs aren't designed to be used like that, as a proxy for yourself. After all, it's not realistic to think we can forbid students to use LLMs, better to try to incentivise the development of a healthy relationship with it.
Also, LLMs aren't a panacea. Maybe in learning languages you should stay away from it, although I'd be cautious to make this conclusion, but it doesn't mean LLMs are universally bad for learning.
In any case, if you don't use LLMs as a guide but a proxy then sure it's a guaranteed path to brain rot. But just as a knife can be used to both heal and kill, an LLM can be used to learn and to fake. The distinction lies in knowing yourself, which is a constant process.
LLM use is the absolute last thing I want to discuss with students. I can think of few worse ways I could spend my limited time with them.
Educators can, should, and must forbid students from using tools that do their work for them -- i.e. cheating.
LLMs are always bad for learning. Always. They offload and bypass mental work.
This is fascinating and compelling. My concern with using LLMs for this sort of thing is similar to my concern with using them in a business setting: it provides a direct line from a single idea to a finished product. That isn't a great way to generate a product or to learn. The messy process of discovering other avenues, hitting dead ends, working through them, and finding your way to a solution to your problem will leave you in a very, very different, and much better, position than the one you find yourself in after taking the LLM's advice. You'll have a richer understanding of the terrain of the problem (edit: even when your problem is "tell me about this problem"), and, as a result, you'll have provided yourself new, improved, or refined tools for the next problem you encounter.
The blinkered solution that the LLM gives you results in something narrower and much worse. You may gain something helpful from the sources it suggests. Its proposals may lead you to an interesting solution. But you've lost out everything else -- and I'd argue it's the "everything else" that is most essential to learning, not the specific solution or source you've chosen.
To put this another way, I've been thinking lately about what I think I'm teaching my students, and what I think experts have that amateurs and novices lack. It isn't possession of the specific knowledge or reasoning that constitutes correct understanding or a correct answer to a given question (e.g. the stuff you'd find in a good written source). It's not the ability to solve difficult problems more easily. It's not any of the concrete stuff that one associates with a certain field (e.g. a Roman historian knows a lot about Rome). It's more nebulous. It's probably best described as just fluency with the problem space, which enables a person to retrieve, use, present, reason, and creatively rework. Having a skill and knowing don't boil down to the stuff you're able to do or the facts you can trot out. Those are the superficial stuff. They're incidental to learning itself.
When you rely on LLMs, I think you're essentially shearing away the entire problem space and replacing the general problem-solving skill with the superficial stuff to which the problem-solving skill applies. You wind up with literally a trivial version of the understanding you'd have if you'd worked through the issues and problems yourself.
I don't know whether this makes sense. It's a thought I've just recently been turning over in my head and I'm not sure I can articulate it well. It started from my watching an interview with a historian of authoritarianism, marveling at the polish of her answers and realizing that her expertise has nothing to do with the specific insights or historical facts she was trotting out in answer to viewer questions. It has everything to do with a deeper, subtending faculty that one develops through years of intensive study.
Edit: here's the video -- https://m.youtube.com/watch?v=vK6fALsenmw . The historian is clearly incredibly, incredibly smart, and the questions aren't terribly difficult, but I was still bowled over by just how good her answers are -- the diverse base of knowledge that she draws on, the fluent ease with which she ties together the historical, psychological, and sociological threads.
The same way a teacher doesn't substitute the texboook, LLM won't substitute DYOR. It'll help you understand where your flaws lie. The heavy lifting is still your job.
LLM changed nothing though. It's just boosting people's intention. If your intention is to learn, you are in luck! It's never been easier to teach yourself some skill for free. But if you just want to be a poser and fake it until you make it, you are gonna be brainrot waaaay faster than usual.