Such engineers must be juniors, or have persisted in their life with a totally wrong idea of how engineering is done and engineering is built. I can't imagine thinking that one could build software systems or essay arguments by trusting an AI system you don't understand to provide answers you don't understand.
Like you say, we very well could faceplant society with this misconception that regurgitation plausible code and prose is what education and understanding looks like.
Good engineering requires good reasoning skills and GPT has exactly zero reasoning. It cannot do the thing that humans do and it cannot do what a calculator can do. I think it is neat and fun, but that is all, a novelty.
I’ve used auto-routers for PCB layout and they will do a 90% job that takes just as much work to redo to get the last 10% as it would to have done it right by hand from the start. There may be a future for operator-in-the-loop type guided AI generative models but I don’t see a lot of effort devoted to making real systems like that. Watson seemed to have this potential and failed even after a brilliant display of ingenuity on Jeopardy. I see these models headed the same way.
> All men are mortal > Socrates is a man > Is socrates mortal
To which it gave a very detailed and correct reply. I then tried:
> All cats are white > Sam is a cat > Is sam white?
To which it gave an almost identically worded response that was nonsensical.
I personally do not think it is the size of the model in question, it is that the things it does that appear to reflect the output of human cognition are just an echo or reflection. It is not a generalizable solution: there will always be some novel question it is not trained against and for which it will fall down. If you make those vanishingly small, I don’t know, maybe you will have effectively compressed all human knowledge into the model and have a good-enough solution. That’s one way of looking at an NN. But the problem is fundamentally different than chess.
I think this composed with more specialized models for things like identifying and solving math and logic problems could make something that truly represents what I think people are seeing the potential in this. Something that encodes the structure behind these concepts, is extensible, and has a powerful generative function would be really neat.
No, because the thing that makes you a 10x engineer is not generally writing code. (Let me just take the "10x" term for now as given and not dip into critique I've made elsewhere already.) It is certainly a baseline skill required to get there, but the things that make you 10x are being able to answer questions like, should this be built at all? There are 5 valid architectures I could use to solve this problem, which has the overall best cost/benefits analysis for the business as a whole? (As opposed to the 1x, who will likely run with either the first they come up with, or the one architecture they know.) If I'm working with 5 teams to solve this particular problem, what's the correct application of Conway's Law, in both directions, to solve this problem with the minimum cost in the long term? What's the likely way this system will be deprecated and can we make that transition smoother?
I am abundantly confident you could feed this AI a description of your problem in terms of what I gave above and it will extremely confidently spit out some answer. I am only slightly less confident it'll be total garbage, and most of that confidence reduction is just accounting for the possibility it'll get right by sheer luck. "The average of what the internet thinks" about these issues can't be more than a 2x engineer at best, and that's my very top-end estimate.
I'm not promising no AI will ever crack this case. I'm just saying this AI isn't going to do it. Over-reliance on it is more likely to drop you down the "Xx engineer" scale than raise you up on it.
For that matter, at least at the level I operate at most of the time, coding skill isn't about how fast you can spew it out. It's about how well you understand it and can manipulate that understanding to do things like good, safe refactorings. This tech will not be able to do refactorings. "How can you be so confident about that claim, jerf?" Because most people aren't digging down into how this stuff actually works. These transformer-based technologies have windows they operate on, and then continue. First of all, refactoring isn't a "continuation" anyhow so it's not a very easy problem for this tech (yes, you can always say "Refactor this code" and you'll get something but the nature of this tech is that it is very unlikely to do a good job in this case of getting every last behavior correct), but second of all, anything that exceeds the window size might as well not exist according to the AI, so there is a maximum size thing it can operate on, which isn't large enough to encompass that sort of task.
It really reminds me of video game graphics, and their multiple-orders-of-magnitude improvements in quality, whereas the underlying data model of the games that we are actually playing have grown much, much more slowly. Often late 1990s-era games are actually richer and more complicated than the AAA games of today. But on the surface, a modern game blows away any 1990s game, because the surface graphics are that much better. There's an analog to what transformer-based AI tech is doing here... it is really good at looking amazing, but under the hood it's less amazing an advance than meets the eye. I do not mean to slag on it, any more than I want to slag on graphics technology... both are still amazing in their own right! But part of what they're amazing at is convincing us they're amazing, regardless of what lies beneath the tech.
I also want to make clear that while this is fundamental to this particular technology, I'm not saying it's fundamental to all possible AI architectures. But it is pretty ingrained into how transformers work. I don't think it can just "evolve" past it, I think anything that "evolved" past it would be a fundamentally different architecture.
I don't know, but I'll offer my perspective.
I am an OK programmer. Something like 1x? I learned proper computer science and programming in my late 20's and got a degree in Software Engineering from an extension program at an Ivy League school. Maybe it's because I learned later in life, maybe it's because I'm not that smart after all, but my glaring weakness as a programmer is programming "in the small" - ie, coding very local algorithms in ~5-10 lines of code. I trip up with small errors in computation, state, etc. This is possibly common to many great programmers, but I think I am weaker than average in this area.
On the other hand, I am better at a higher level of programming. I like abstraction, software design, software and systems architecture. And at an even higher level, I am quite skilled at - and do quite a bit of - business analysis and UX.
I have been trying Github Copilot and I am very excited about how it elevates my weaknesses - it's a perfect fit for me. I see a future where I will be able to play my generalist role much better.
As for people who are 1x, but don't have other skills, I'm also optimistic. I think the market will push them into new exciting directions. Their 1x coding skills, coupled with AI assitance, could possibly be a huge asset in new emerging roles in the economy.
These AI models don't actually understand anything about what they're generating, not to mention the world they're supposed to building solutions for. They're just using statistics to predict the next likely output based on some input. Maybe there is a way to incorporate these things into the development process today, but I think we're still far away from seeing an AI replacing a human engineer.
What goes into the average developer's day? I think a lot of time is spent writing boilerplate, fighting with tiny CI errors, shuffling code, refactoring, writing tests and staring at code hunting bugs. As this technology matures, all of that noise might go away, and I think a lot of people are having a hard time imagining what they are left with.
Remember, we were trained in the "classical" fashion. How will budding engineers approach learning in 10 or even 50 years?
(I feel like these tools are incredibly neat to see but also likely to lead to a corrosive effect on trust as people no longer know who's real — and that'll get worse the first time someone launches a good AI assistant to handle email / chat trained on your writing style)
That one might actually be easy to solve: all we need is to normalize the idea that you're personally responsible for whatever your email/chat assistant AI says or does in your name. After all, it's you who are making a decision to reroute other people to a bot.
(Many would say just using such assistant in the first place is disrespectful towards the people trying to reach you. I personally don't agree, because in my experience, those who'd complain about it are exactly those I find even more disrespectful, as they tend to impose themselves, interrupting me with random inane nonsense. Yes, I suck at chit-chat.)
I think this idea is actually the default, so all we need is to keep it alive, to protect it from all kinds of AI assistant startups willing to dump billions of dollars on marketing campaigns trying to convince everyone that half-broken assistants are good and fine, you should use them and should not be blamed for occasional mishaps.
Ideally, we'd require labels but there's no easy way to enforce that without something like random auditing of AI implementers.
* Understanding the quality of sources
* Understanding the different between primary and secondary sources.
* Establishing chains of evidence and tracing them through large datasets.
* Integrating information from different sources and using correlation to establish validity.
Basically, it would be a crash course in how to do research, and the best possible preparation for living in a world where we drown in uncertain information and still have to make the best choices that we can.
For bonus points, at the end of the process they would have a better understanding of the subject matter than somebody who had only read about it.
That said, adult me could probably use ChatGPT to stitch together a 1000 word high school paper pretty easily (and probably evade plagiarism/LLM detectors). But, as you say, I could probably get a lot of the way there by other means as well and putting words on the page is mostly not the hard part anyway.
But it seems kind of dubious to begin with, I'm not sure why LLM output would be so easily detected.
If you play with it a bit, there are definitely patterns.
Here's one I did over the weekend: 99.98% fake.
The Battle of Agincourt was a significant event in English and French history, as it was a major victory for the English army against the French during the Hundred Years' War. The battle took place on October 25, 1415, near the town of Agincourt in northern France.
The English army, led by King Henry V, was vastly outnumbered by the French forces, which were estimated to be around 30,000 soldiers compared to the English army of just 6,000. Despite the odds, the English were able to emerge victorious thanks to their superior tactics and weaponry.
The English army was composed of mostly longbowmen, who were able to shoot arrows with great accuracy and power. The French, on the other hand, relied heavily on heavily-armored knights, who were less effective in the muddy and wet conditions of the battlefield. The English were also able to take advantage of the narrow front of the battlefield, which allowed them to focus their firepower and overwhelm the French forces.
Whereas the first couple paragraphs from a book I'm working on came out 0.02% fake.
With this view, I think chatGPT is less of a novel UX shift than an accelerant on an existing one.
I think it proves the age old belief that as technology reaches maturity it loses its fancy monikers and we will probably just look at this as “very good, very specific, google” one day
The student still had access to primary sources, textbooks, google and AI answers.
In general I am a fan of homework which forces critical thinking. Rephrasing someone elses work is a far less valuable skill than the critical thinking which is demanded for that type of work. Unrelated of any AI, students should learn how to read a text and be able to investigate its contents.
They have to learn to verify it. I think you missed point of OPs idea.
Public schools will struggle to teach kids anything, and to distinguish between A grades that were earned and those achieved largely through cheating, while parents will resist efforts to switch to grading schemes that can't be trivially gamed (just try switching to only-in-person-tests-count grading and witness the rage that follows when Little Johnny's grades plummet). Kids who lean on the motivation from graded homework to get themselves to do the work (and so, learn the material) will struggle, as well, even if that is allowed to happen.
This will enhance the reputations of schools that can buck the trend, applying more resources (mostly more teachers, for smaller class sizes, to make it easier to keep an eye on everything and really gauge & guide learning progress for each student) and an "if you don't like it, fuck off to somewhere else" attitude to parent complaints about grading practices. Which means, elite private schools will benefit.
I expect that a 4.0 from a public school will mean almost nothing, while at the same time coming in under a 4.0 will look terrible ("everyone cheats and you still couldn't get a 4.0? How dumb are you!?") which will just worsen the cheating problem.
Maybe parents will surprise me and not throw shit-fits over solutions to this, and public schools will be able to implement them. But I doubt it.
I can't imagine teachers reading through essay after essay, each a tedious cookie-cutter nugget of banal pseudo-perfection. The teachers will give up in no time.
There would be no point giving feedback, since it is not the student to who you are giving it. Likewise, the students would have little interest in feedback, since they have no personal ego investment in their output.
Besides the best essays will be generated by those with access to the better generative software. At this point there is simply nothing to grade. What would be the point?
Once essay writing and grading bots are locked in combat it's only a matter of time until that market exhausts itself and disappears in a puff of logic.
Maybe this is all a good thing. The process of writing and grading essays as a way of "measuring" knowledge has been a sham for too long. It's a throwback to school from the turn of the last century.
I am blown away by Copilot's capabilities (haven't tried ChatGPT yet, it's down right now), but I can only operate it because I have _written_ tons of code beforehand. I am getting the best results from it by writing prompts in a language that sometimes includes domain-specific language ("accounts", etc.) but still _heavily_ leans on programming constructs ("write a map with a tuple of integers as a key, and a string as a value")
Maybe there's a way of learning how to code without ever writing a genuine line by yourself. You just read the language docs, start writing prompts and create a testing loop based on expected outputs. Maybe. But it's uncharted territory.
Same question goes for good ole writing - can you produce new Hemingways with the process you describe? I know there is a certain vision where _the AI will be the new Hemingway_ but you are talking about a shorter term ~10 where it's unlikely will be the case... Will kids be able to properly write if they only train to be editors?
As an addendum, I was at GitHub conf and saw Prof. Malan of Harvard CS50 fame claiming that at present his course will _not_ allow the use of tools like Copilot. I have taken the course and they have plagiarism control systems but I don't know how they are going to control this potential explosion.
I think it can be argued that we have, seemingly overnight, made vast progress towards that world. You specifically mention code, but that concept is irrelevant in this world. Indeed, this is uncharted territory!
I mean, what do you propose, that we skip all the steps in computer science history and just start at the end? Hardly "daft, it's simply the best way we have come up to provide machines with instructions until now. And it's not like people have not tried other paradigms (ex: graphical programming, "low-code", etc.).
Also, compared to programming in assembly or binary, programming in Python or other high-level languages is a huge advance in itself. Python, at the end of the day, is nothing but a bridge between natural language and machine code.
> You specifically mention code, that concept is irrelevant in this world
Current computer systems run on trillions upon trillions of lines of code.
GPT-3 or Copilot don't change that fact, in fact they will continue to pile up lines of code.
They are systems that map natural language to code, by drawing from an infinitely massive corpus of code.
They bring the idea of expressing systems entirely in natural language one step closer, but it's still very far away - almost a pipe dream.
The output of these innovations is still code. So tell me again how code has become irrelevant, or how people who do not understand code at all will be able to leverage these advances?
I always took it in another way: learning to program a computer is an enlightening, visceral experience highlighting just how bad natural language and normal modes of thinking suck. They're heavily optimized for reality in which the most important thing was managing social dynamics in small groups, and beliefs were good or bad depending on how close they are to what everyone else beliefs. Whether or not they were true at object level did not matter at all.
We live in different times now. There's a demand for clear, logical, precise thinking. The very thinking we're not well-equipped for by default. The very thinking you call "bending our brains backwards". Because object-level truth matters much more now, there are consequences for getting things wrong - such as countless of ways of getting scammed by people who are better at this than you are.
Not to mention, this "bending our brains backwards" is what gives humanity superpowers. It's what gave us all the technology we enjoy today - energy, transportation, amenities, communication, entertainment. Building and maintaining these things requires precise thinking and precise communications. So, until a super-human AI takes all that over for us (and effectively turns out into NPCs in our own story), glorifying the natural, human mode of communication is just self-handicapping.
I only understand what a compiler is doing after having written one (albeit a tiny one) myself during my studies.
How many people need to understand how a compiler works to sling some code for a paycheck or just for fun.
I learned how compilers work purely for fun, it provides me absolutely no benefit in my day to day life, and before that I taught myself how to code for the exact same reason. If I had to go the opposite route, learn how compilers work before learning to code, I seriously doubt I’d have taken the time.
Other things are the same, I’m horrible at doing math but managed to learn enough linear algebra to finish a coding project I was interested in doing. If I would have had to code up all the hard bits instead of calling matrix1 * matrix2 that project would never have happened.
Another layer of abstraction isn’t always a bad thing, these AIs will eventually figure out how to write bug free code when you tell it to “write a function which calculates the flight velocity of a sparrow carrying a coconut”.
Beside, with ChatGPT in its current state, it's still absolutely impossible to get it to produce any useful code without knowing how code works. There is an advantage to the precise specification of a formal language. At least for the next decade, I think there will be human proof-reading AI output before generated code is ran, with catastrophic consequences when that doesn't happen.
What I've difficulties getting over right now is every time I write something (text or code) I ask myself "Could ChatGPT have written it better?"
I asked it to retry, and got this:
function calculateSparrowFlightVelocity(coconutWeight) { // Assume that a sparrow can carry a maximum of 1.5 ounces of weight if (coconutWeight > 1.5) { return 0; }
// Assume that the flight velocity of a sparrow is 5 meters per second // and the weight of a coconut is 0.5 ounces return 5 * (1 - (coconutWeight / 0.5)); }
// Example usage: console.log(calculateSparrowFlightVelocity(0.5)); // Output: 4 meters per second console.log(calculateSparrowFlightVelocity(1.0)); // Output: 3 meters per second console.log(calculateSparrowFlightVelocity(1.5)); // Output: 0 meters per second (sparrow cannot carry this weight)
Can any Monty Python fans confirm if this is correct?
Only the ones who want a good job.
One of the consequences of automation has been a widening gap between top jobs and ordinary jobs. It's plausible that AI will continue the same process. Many jobs that pay well and are intellectually and socially rewarding today may become similar to airport security checks in the future. You observe what a machine does, double-checking its conclusions and looking for anything out of ordinary. And you often fail at that, because the human brain is poorly suited for such tasks.
When the future is uncertain, its good to have a deeper and wider understanding of things than what your current job requires.
I think it's pretty important. I use my very basic knowledge of how compilers work occasionally in my mostly-interpreted code world to diagnose and debug performance issues.
I also used a low-level debugger and taint/trace method recently to diagnose Windows DLL issues that happened after X seconds of starting up a program that interfaced with specialty hardware. A couple young early-career technicians/engineers watched me do it and were incredibly flabbergasted and figured me for some sort of genius when in reality I kinda only know how it works and was just reading the stack/heap as I was paging through; knowledge I acquired from booting SoftICE a few times as a kid.
There's a real lack of simply knowing how a computer and/or a network works, which is to be expected since layers and layers of abstraction have made it very easy to teach computer literacy and even computer programming on many levels. The issue is that when things go wrong and Stack Overflow doesn't have the answers, almost all young engineers I work with have no experience at low-level analysis and debugging, very little knowledge of first principles, and don't know where to Google to start acquiring it.
This isn't me shaking my fist at the kids or anything. They're more productive than I was at their age/experience level with code, and that's due to better knowledge and integration using IDEs which I never had, plus likely better high-level understanding of what's going on. But I fear a generation of developers and technicians who largely have little understanding of how computers work is problematic.
Hmm, this is a good point. I get incredible results from Copilot but I realized just now it's primarily because I have a fairly deep understanding of computer science as it pertains to the domain of code I'm writing in, and thus my comments that prompt the AI are quite accurate, fleshed out, and detailed.
If I didn't know any of the underpinning of the algorithms and data structures (like a new graduate from a code bootcamp for example), it might be much tougher to make hay out of Copilot's AI.
Never thought of it like that.
The need to check your sources is an important lesson to learn early, AI or no.