Humans suffer from automation bias and other cognitive biases.
Anything that causes disengagement from a process can be challenging, especially with long term maintainability and architectural erosion, which is mostly what I actively search for to try and avoid complacency with these tools.
But it takes active effort to avoid for all humans.
IMHO, writing the actual code has always been less of an issue than focusing on domain needs, details, and maintainability.
As distrusting automation is unfortunately one of the best methods of fighting automation bias I do try to balance encouraging junior individuals to use tools that boost productivity while making sure they still maintain ownership of the delivered product.
If you use the red-green refactor method, avoiding generative tools for the test and refactor steps seems to work.
But selling TDD in general can be challenging especially with Holmström's theorem, and the tendency of people to write implementation tests vs focusing on domain needs.
It is a bit of a paradox that the better the tools become, the higher the risk is, but I would encourage people to try the above, just don't make the mistake of copying the prompts required to get from red to green as the domain tests, there is a serious risk of coupling to the prompts.
We will see if this works for me long term, but I do think beginners manually refactoring with though could be an accelerator.
But only with intentionally focusing on learning why over time.
"C makes it easy to shoot yourself in the foot; C++ makes it harder,
but when you do it blows your whole leg off". -- Bjarne StroustrupWell in modern software we stand on the shoulders of many many giants, you have to start somewhere. Some things you may never need to learn (like say learning git at a deep level when learning the concept of add, commit, rebase, pull, push, cherry pick and reset are enough even if you use a GUI to do it) and some thins you might invest in over time (like learning things about your OS so you can optimize performance).
The way you use automation effectively is to automate the things you don’t want to learn about and work on the things you do want to learn about. If you’re a backend dev who wants to learn how to write an API in Actix go ahead and copy paste some ChatGPT code, you just need to learn the shape of the API and the language first. If you’re a rust dev who wants to learn how Actix works don’t just copy and paste the code, get ChatGPT to give you a tutorial and then your write your API and use the docs yourself.
From personal experience and from the popularity of rstudio, jupyter etc. the evidence points in the other direction.
It’s because when you start out you just need to learn what code looks like, how does it run (do all the lines execute at once or sequentially), that you have these boxes that can hold values etc.
Maybe in C/C++ where build systems are some kind of lovecraftian nightmare?
Another big difference is the size of the standard library. One can hold on his brain all the information needed to program in C, but I would argue that for C++ or Java it would be too taxing and an IDE is almost a requirement, the alternative being consulting the documentation often.
People making new languages should resist this trend. Do not implement another damn package manager. Think about the problem and make it easy for people to just use someone's tarball of code.
But if you don't know how Actix works, how can you be sure that the ChatGPT-generated tutorial is going to be particularly good? It might spit out non-idiomatic, unnecessarily arcane, or even flat-out wrong information, asserted confidently, and you may not have any good way of determining that. Wouldn't you be better off "using the docs yourself" in the first place, assuming they have a halfway decent "Getting Started" section?
I know it's easy to draw an analogy between "AI-assisted coding" and autocompleting IDEs, but under the hood they're really not the same thing. Simpler autocompletion systems offer completions based on the text you've already typed in your project and/or a list of keywords for the current language; smarter ones, like LSP-driven ones, perform some level of introspection of the code. Neither of those pretend to be a replacement for understanding "how the system works." Just because my editor is limiting its autocomplete suggestions to things that make sense at a given cursor position doesn't mean I don't have to learn what those methods actually do. An LLM offering to write a function for you based on, say, the function signature and a docstring does let you skip the whole "learn what the methods actually do" part, and certainly lets you skip things like "what's idiomatic and elegant code for this language". And I think that's what the OP is actually getting at here: you can shortcut yourself out of understanding what you're writing far more easily with an LLM than with even the best non-LLM-based IDE.
> Automation bias is the propensity for humans to favor suggestions from automated decision-making systems and to ignore contradictory information made without automation, even if it is correct.
Note this 1998 NASA paper to further refute the IDE/WYSIWYG editor claims.
https://ntrs.nasa.gov/citations/19980048379
> This study clarified that automation bias is something unique to automated decision making contexts, and is not the result of a general tendency toward complacency.
The problems with automation bias has been known for decades and the studies in the human factors field is quite robust.
While we are still way to early in the code assistant world to have much data IMHO, there is evidence called out even in studies that edge towards positive results in coding assistants that point out issues with complacency and automation bias.
https://arxiv.org/abs/2208.14613
> On the other hand, our eye tracking results of RQ2 suggest that programmers make fewer fixations and spend less time reading code during the Copilot trial. This might be an indicator of less inspection or over-reliance on AI (automation bias), as we have observed some participants accept Copilot suggestions with little to no inspection. This has been reported by another paper that studied Copilot [24].
Deciding what parts of your workflow to automate is whats important.
Case in point, there is an employee in my office who, after attaining a 4-year CS degree, was unaware that "git" wasn't "GitHub". GitHub is so aggressively pushed on new programmers that they don't even need to know that git exists, much less how it works. Actions that don't have a large colorful button may as well be impossible for someone so thoroughly dependent on tools-in-the-middle.
Was this cost effective for the company?
Or not.
All hail the mashup.
It is… not a terribly effective approach to programming.
It's one thing to have the llm generate a function call for you where you don't remember all the parameters. That's a low enough abstraction where it serves as a turbo charged doc lookup. It's also probably okay to get a basic setup (toolchain etc. for an ecosystem you're unfamilar with(. But to have it solve entire problems for you especially when you're learning is a disaster.
I'm reminded of the Wat talk: https://www.destroyallsoftware.com/talks/wat
Is it worth learning the mental model behind this system? Or am I better off just shoveling LLM slop around until it mostly works?
For example if you are a frontend developer doing typescript in React you could learn how React’s renderer works or how typescript’s type system works or how the browser’s event listeners work. Over time you accumulate this knowledge through the projects you work on and the things you debug in prod. Or you can purposefully learn it through projects and studying. We also build up mental models of the way our product and it’s dependencies work.
The reason a coworker might appear to be 10x or 100x more productive than you is because they are able to predict things about the system and arrive at solution faster. Why are they able to do that? It’s not because they use vim or type at 200 wpm. It’s because they have a mental of the way the system works that might be more refined than your own.
JavaScript is weird, but show me a language that doesn’t have its warts.
False equivalence much? Languages have warts. JS is a wart with just enough tumorous growth factors to have gained sentience and started its plans toward world domination.
If you just follow a few relatively simple rules JS is actually very nice and "reliable". Those "rules" are also relatively straight forward: let/const over var, === unless you know better, make sure you know about Number.isInteger, isSafeInteger, isObject etc etc. (there were a few more rules like this - fail to recall all of them - has been a few years since i touched JS) - hope you get the idea.
Also when I looked at JS I was just blown away by all the things people built on top of it (babel, typescript, flowtype, vue, webpack, etc etc).
A mental model might be something like "JavaScript has strict and non-strict comparisons", but there are no strict less-than comparisons for example, so remembering to use === instead of == is just a one-off neat tip rather than an application of some more general rule.
If you want to learn javascript, then yes, obviously. You also need to learn the model to be able to criticise it (effectively) -- or to make the next wat.
> am I better off just shoveling LLM slop around until it mostly works?
Probably not, but this depends on context. If you want a script to do a thing once in a relatively safe environment, and you find the method effective, go for it. If you're being paid as a professional programmer I think there is generally an expectation that you do programming.
AI empowers normal people to start building stuff. Of course it won’t be as elegant and it will be bad for learning. However these people would have never learned anything about coding in the first place.
Are we senior dev people a bit like carriage riders that complain about anyone being allowed to drive a car?
Engineering is also the process of development and maintenance over time. While an AI tool might help you build something that functions, that's just the first step.
I am sure that there are people who leverage AI in such a way that the build a thing and also ask it a lot of questions about why it is built in a certain way and seek to internalize that. I'd wager that this is a small minority.
To learn effectively you need to challenge your knowledge regularly, elaborate on that knowledge and regularly practice retrieval.
Building things solely relying on AI is not effective for learning (if that is your goal) because you aren’t challenging your own knowledge/mental model, retrieving prior knowledge or elaborating on your existing knowledge.
1. Talk to people, read docs, skim the code. The first objective is to find out what we want the system to do.
2. Once we've reverse-engineered a sufficiently detailed specification, deeply analyze the code and find out how well (or more often poorly) it actually meets our goals.
3. Do whatever it takes to make the code line up with the specification. As simply as possibly, but no simpler.
This recipe gets you to a place where the codebase is clean, there are fewer lines of code (and therefore fewer bugs, better development velocity, often better runtime performance). It's hard work but there is a ton of value to be gained from understanding the problem from first principles.
EDIT: You may think the engineering culture of your organization doesn't support this kind of work. That may be true, in which case it's incumbent upon you to change the culture. You can attempt this by using the above procedure to find a really nasty bug and kill it loudly and publicly. If this results in a bunch of pushback then your org is beyond repair and you should go work somewhere else.
In contrast, if you're a frontend developer, you don't need to know C++ even though browsers are implemented in it. If you're a C++ developer, you don't need to know assembly (unless you're working on JIT).
I am convinced AI tools for software development will improve to the point that non-devs will be able to build many apps now requiring professional developers[0]. It's just not there yet.
[0] We already had that. I've seen a lot of in-house apps for small businesses built using VBA/Excel/Access in Windows (and HyperCard etc on Mac). They've lost that power with the web, but it's clearly possible.
If we are calling software developers who don’t understand how things work, and who can get away with not knowing how things work, engineers, then that’s a separate discussion of profession and professionalism we should be having.
As it stands there’s nothing fundamentally rooted in software developers having to understand why or how things work, which is why people can and do use the tools to get whatever output they’re after.
I don’t see anything wrong with this. If anyone does, then feel free to change the curriculum so students are graded and tested on knowing how and why things work the way they do.
The pearl clutching is boring and tiresome. Where required we have people who have to be licensed to perform certain work. And if they fail to perform it at that level their license is taken away. And if anyone wants to do unlicensed work then they are held accountable and will not receive any insurance coverage due to a lack of license. Meaning, they can be criminally held liable. This is why some countries go to the extent of requiring a license to call yourself an engineer at all.
So where engineering, actual engineering, is required, we already have protocols in place that ensure things aren’t done on a “trust me bro” level.
But for everyone else, they’re not held accountable whatsoever, and there’s nothing wrong with using whatever tools you need or want to use, right or wrong. If I want to butt splice a connector, I’m probably fine. But if I want to wire in a 3 phase breaker on a commercial property, I’m either looking at getting it done by someone licensed, or I’m looking at jail time if things go south. And engineering or no different.
It's because 1) most of us don't work on things that can get people jailed or killed and 2) the US leans towards not regulating language so much.
But if it makes you more comfortable, just think of the term "software engineer" as tongue-in-cheek, like some people call janitors "sanitation engineers"
The idea that wanting to become better at using something is pearl clutching is frankly why everything has become so mediocre.
LLMs are just a faster and more wrong version of the copy-paste stackoverflow workflow it’s just now you don’t even need to ask the right question to find the answer.
You have to teach students and new engineers to never commit a piece of code they don’t understand. If you stop at “I don’t know why this works” then you will never be able to get out of the famous multi hour debug loop that you get into with LLMs or similarly the multi day build debugging loop that everyone has been through.
The real thing that LLMs do that is bad for learning is that you don’t need to ask it the right question to find your answer. This is good if you already know the subject but if you don’t know the subject you’re not getting that reinforcement in your short term memory and you will find things you learned through LLMs are not retained as long as if you did more of it yourself.