> Where AI really really shine is to help an engineer get proficient in a language they don't know well.

I used GitHub Copilot in a project I started mostly to learn Go. It was amazing. I spent not so much time fiddling around with syntax, and much more time thinking about design.

So very much this. As I was learning Rust, I'd ask what the equivalent was for a snippet I could create in Java. It is funny. I look at the Java code provided by prompts and go meh. The Rust code looks great. I realize this is probably due to 1) me being that junior level in Rust or 2) less legacy crap in the training model. I'm sure it is both, with more of the former as I work from working to beautiful code.

I used to agree, but as an experienced engineer asking about rust and y-crdt, it sent me down so many wrong rabbit holes with half valid information.

I used Claude recently to refresh my knowledge on the browser history API and it said that it gets cleared when the user navigates to a new page because the “JavaScript context has changed”

I have the experience and know how to verify this stuff, but a new engineer may not, and that would be awful.

Things like these made me cancel all my AI subscriptions and just wait for whatever comes after transformers.

I think of AI like a very eager and bright 20 yr old intern who’s really into whatever tech I’m trying to learn.

Good knowledge of facts, willing to give anything a try, but not much wisdom.

> Good knowledge of facts

I haven’t seen this demonstrated in gpt-4 or Claude sonnet when asking anything beyond the most extreme basics.

I consistently get subtly wrong answers and whenever I ask “oh okay, so it works like this” I always get “Yes! Exactly. You show a deep understanding of…” even though I was wrong based on the wrong info from the LLM.

Useless for knowledge work beyond RAG, it seems.

Search engines that I need to double check are worse than documentation. It’s why so many of us moved beyond stack overflow. Documentation has gotten so good.

ChatGPT and Claude outputs are equivalent to a 20 yr old intern who goes to MIT?

But actually whenever this happens you get rich signal about the hidden didactic assumptions of your method of prompting it about things you yourself are unsure you can verify for yourself, and also of how you thought the tool worked. This is a good meta skill to hone.

In the same way going to a fortune teller can give me rich insight about my love life and work life balance, sure.

A visitor to physicist Niels Bohr's country cottage, noticing a horseshoe hanging on the wall, teased the eminent scientist about this ancient superstition. "Can it be true that you, of all people, believe it will bring you luck?"

"Of course not," replied Bohr, "but I understand it brings you luck whether you believe it or not."

s/luck/lower negative log likelihood. Besides, a participant can still think about and glean truths from their reflections about a conversation they had which contained only false statements.

What point are you trying to make?

That it’s better to be superstitious than not, because maybe it matters?

The internal mechanisms by which a model achieves a low negative log likelihood become irrelevant as it approaches perfect simulation of the true data distribution.

This is how I think about it as well. It’s like the equivalent of trying to build an ML model with only positive examples otherwise

> I have the experience and know how to verify this stuff

That was GP's point. Steve Yegge has also talked about this.

Is that not the opposite point?

Someone who does not have that experience would be much more harmed by the false information LLMs can and do spit out.

It allows you to write a little code in languages that you are not familiar with. However, it is nothing like actually internalizing a language.

That’s true. But if you are already an experienced developer who’s been around the block enough to call bullshit when you see it, these LLM thingies can be pretty useful for unfamiliar languages. But you need to be constantly vigilant, ask it the right questions (eg: “is this thing you wrote really best practice for this language? Cause it doesn’t seem that way”), and call bullshit on obvious bullshit…

…which sometimes feels like it is more work than just fucking doing it yourself. So yeah. I dunno!

Not sure I get how that would work. It seems to me that to do my job I will have to validate the semantics of the program, and that means I will have to become familiar with the syntax of Go or whatever, at a fairly sophisticated level. If I am glossing over the syntax, I am inevitably glossing over the fine points of how the program works.

It depends on the language and the libraries you will use. Python with a well known library? Sure no problem. Almost any model will crank out fairly error free boilerplate to get you started.

Terraform? Hah. 4o and even o1 both absolutely sucked at it. You could copy & paste the documentation for a resource provider, examples and all, and it would still produce almost unusable code. Which was not at all helpful given I didn’t know the language or its design patterns and best practices at all. Sonnet 3.5 did significantly better but still required a little hand holding. And while I got my cloud architecture up and running now I question if I followed “best practices” at all. (Note: I don’t really care if I did though… I have other more important parts of my project to work on, like the actual product itself).

To me one of the big issues with these LLM’s is they have zero ability to do reflection and explain their “thought process”. And even if they could you cannot trust what it says because it could be spouting off whatever random training data it hovered up or it could be “aligned” to agree with whatever you tell it.

And that is the thing about LLM’s. They are remarkably good bullshitters. They’ll say exactly what you want them to and be right just enough that they fool you into thinking they are something more than an incredibly sophisticated next token generator.

They are both incredibly overrated and underrated at the same time. And it will take us humans a little while to fully map out what they are actually good at and what they only pretend to be good at.

LLMs are a great help with terraform and devops configuration, they often invent things but at least the point at the documentation I need to look up on.

Of course everything needs double-checking but just asking the LLM: "how do I do X" will usually at least output all the names of terrraform resources and most configuration attributes I need to look up.

They are great for any kind of work that requires "magical incantations" as I like to call them.

I guess it depends on how you define "proficiency". For me, proficiency implies a fundamental understanding of something. You're not proficient in Spanish if you have to constantly make use of Google Translate.

Could code assistants be used to help actually learn a programming language?

- Absolutely.

Will the majority of people that use an LLM to write a Swift app actually do this?

- Probably not, they'll hammer the LLM until it produces code that hobbles along and call it a day.

Also, LEARNING is aided by being more active, but relying on an LLM inherently encourages you to adopt a significantly more passive behavior (reading rather than writing).

Yes! Reading some basic documentation on the language or framework, then starting to build in Cursor with AI suggestions works so well. The AI suggests using functions you didn't even know about yet, then you can go read documentation on them to flesh out your knowledge. Learned basic web dev with Django and Tailwind this way and it accelerated the process greatly. Related to the article, this relies on being curious and taking the time to learn any concepts the AI is using, since you can't trust it completely. But it's a wonderfully organic way to learn by doing.

Tell me you're a Cursor advertisement chatbot without telling me you're a Cursor advertisement chatbot.

I'll have you know that I successfully clicked on several traffic lights and motorcycles this week to get into a web site. Didn't even need Cursor!

If you don’t know the language well enough AI could lead you down the path of bad practices and you won’t recognize it.

I agree with this take. AI has essentially served me well as a more useful search engine when working in new languages, libraries, or frameworks.