Copilot is particularly helpful when working on React components where it makes eerily accurate predictions. I see technology like Copilot becoming an indispensable part of the programmer toolbelt similar to IDE autocomplete for many people.
I also see it changing the way that programmers document their code. With Copilot if you write a really good descriptive comment before jumping into the implementation, it does a much better job of suggesting the right code, sometimes even writing the entire function for you.
1. The AI suggestions were often less helpful than the type driven IDE autocompletions (using IntelliJ).
2. The AI plugins were very aggressive in pushing their completions to the top of the suggestions list, even when they were strictly less helpful than the defaults.
The result was it actually slowed me down.
Looking at the marketing materials for these services, they're often focused on dynamic languages like Python or JavaScript where there's far less information available for the IDE to help you with. If you've picked your language partly due to the excellent IDE support, it's probably harder for the AI to compete with hand-written logic and type system information.
If Copilot is as good as it gets but only for some languages, won’t it influence what languages will be chosen by devs or companies?
(It's also conceivable I'm put out of a job, but I'm not worried yet. So far I can only imagine AI automates away the boring part, and I super-duper-want that.)
I'm sure there's at least a few relevant XKCD strips to insert here.
Thinking about generating 10 times the code you need, just because you can generate it instead of writing (perfomant?) code.
Edit: You are honestly downvoting me for saying something that might actually happen. If copilot lives up to the hype, but for a limited number of languages, this can have a profound affect on what languages people might decide to use in the future.
Basically, if you look at what’s not yet available to the public, we have engines that can write an entire program that does what you want, a test suite, documentation, and write thoughtful comments on all the forums about why the program is good. They could make about 100,000 programs an hour on a cluster, together wih effusive praise for each, but that would make everyone supicious.
Whether it's a joke or astroturfing, this alone makes it brilliant.
> GitHub Copilot is a code synthesizer, not a search engine: the vast majority of the code that it suggests is uniquely generated and has never been seen before. We found that about 0.1% of the time, the suggestion may contain some snippets that are verbatim from the training set.
Probabilities are fun!
e.g. https://lwn.net/Articles/61292/ and most likely only one opinion.
on the other hand, it would be interesting to learn about what the copyright implications are of
a) creating a utility like copilot (it is a software program) and contains a corpus based on copyrighted material (the database that has been trained)
b) using it to create code based on the corpus and resulting in software as a work under copyright.
I say dumb because I am, perhaps chauvinistically, assuming that no brilliant algorithmic insights will be transferred via the AI copilot, only work that might have been conceptually hard to a beginner but feels routine to a day laborer.
Then again that assumption suggests there'd be nothing to sue over.
Also, Oracle v Google opens the possibility of a fair-use defense in the event that Copilot does regurgitate a small snippet of code.
[1] https://docs.github.com/en/github/site-policy/github-terms-o...
1. https://news.ycombinator.com/item?id=23093911
2. https://www.reddit.com/r/HobbyDrama/comments/gfam2y/furries_...
https://twitter.com/natfriedman/status/1409883713786241032
Basically they are building a system to find explicit copying and warn developers when the output is verbatim.
Anyone want to hire me to teach your grandma how to use the internet?
"Prediction: AI will cause the price of work that can happen in front of a computer to decrease much faster than the price of work that happens in the physical world. This is the opposite of what most people (including me) expected, and will have strange effects"
And I was like yeah I gotta start preparing for next decade.
I'm skeptical.
The envelope of "programming" will continue to shift as things get more and more complex. Your mother-in-law is not going to install Copilot and start knocking out web apps. Tools like this allow programmers to become more productive, which increases demand for the skills.
Reminds me of something I read that claimed when drum machines came out, the music industry thought it was the end of drummers. Until people realized that drummers tended to be the best people at programming cool beats on the drum machine.
Every single technological advancement meant to make technology more accessible and eliminate expertise has instead only redefined what expertise means. And the overall trend has been a lot more work opportunities created, not less.
So the winners were those that adapted earlier and the losers were those that didn't/couldn't adapt.
This translates to: If you're mindlessly doing the same thing over and over again, then it's a low value prop and is at risk. But if you're solving actual problems that require thought/expertise then the value prop is high and probably going to get higher.
We can automate the mundane but that’s usually the stuff that requires creativity, so the automated stuff becomes uninteresting in that realm. People will seek crafted experiences.
It's just concentrated into the hands of a very few super specialists, it's much harder to get to their level but their work is much much more important.
See: https://www.youtube.com/watch?v=ZSRHeXYDLko / Preventing the Collapse of Civilization / Jonathan Blow
I think the analog to programming is a bit more direct in this sense; most companies aren't going to go with something like Copilot unless it's supplemental or they're on an entirely shoestring budget; it'll be the bigger companies wanting to squeeze out that extra 10% productivity that are betting hard on this - same with where larger bands would do this to have an extremely clean studio track for an album.
So, during this period, a ~10% increase in population saw a 250% increase in drummers.
It does not appear that the drum kit killed the drummer.
Big caveats about what these surveys defined as "drummer" and that this doesn't reflect professional drummer gigs, just the number of drummers.
[1] https://m.facebook.com/Bumwrapdrums/posts/how-many-drummers-...
[2] https://www.quora.com/How-many-people-play-drums-in-the-US
By the same line of thinking, If you can get by with AI generated code did you really require a seasoned, experienced developer in the first place? If your product/company/service can get by with copy pasta to run your CRUD app (which has been happening for some time now sans the AI aspect) did you ever really need a high end dev?
I think its like anything else, 80% is easy and 20% is not easy. AI will handle the 80% with increasing effectiveness but the 20% will remain the domain of humans for the foreseeable future.
Worth considering maybe.
Nowadays everyone can make professional looking photos so the demand for photographers has shrunk, as the supply has increased.
AI will be similar -- it will not just give more tools to people already in a given field (programming, writing, whatever), but also bring new people in, and also create new fields. (I personally can't wait for the gardening of AI art to catch on. It'll be so weird[1].)
[systems] Designing data intensive applications - kleppman
[programming] SICP - sussman & abelson
Last one is an old scheme book. No other book (that I read) can even hold a candle to this one, in terms of actually developing my thought process around abstraction & composition of ideas in code. Things that library authors often need to deal with.
For example in react - what are the right concepts to that are powerful enough to represent a dynamic website & how should they compose together.
Replacement of programmers will follow these lines. New tools, like copilot (haven't tried, but will soon), new languages, libraries, better IDEs, stack overflows, Google, etc will make programming easier and more productive. One programmer will do the work that ten did. That a hundred did. You'll learn to become an effective programmer from a bootcamp (already possible - I know someone who went from bootcamp to Google), then from a few tutorials will.
Just like the secretary's role in the office was replaced by everyone managing their own calendars and communications the programmer will be replaced by one or two tremendously productive folks and your average business person being able to generate enough code to get the job done.
That doesn't mean anything. The last 20 years have seen an absurd chase of more and more stupidity in job titles to make people feel they are "executive assistants" instead of secretaries, "vice presidents" instead of whatever managerial role, etc, etc.
This would scale to support any number of economists. This would also be a simpler model and that simplicity might lead to a better product. In your model, the economists must explain to you, then you must write the code. That adds a layer where errors could happen - you misunderstand the economists or they explain poorly or you forget or whatever. If the economists could implement things themselves - less room for "telephone" type errors. This would also allow the economists to prototype, experiment, and iterate faster.
There are more "secretaries" than ever, and they get to do far more productive things than delivering phone messages.
Why would an architect bother with sending some work overseas if tools like this would enable them to crank out the code faster than it would take to do a code review?
1) AI tools increase developer productivity, allowing projects to get completed faster; and
2) AI tools offset a nonzero amount of skill prerequisites, allowing developers to write "better" code, regardless of their skill level
With those in mind, it seems reasonable to conclude that the price to e.g. build an app or website will decrease, because it'll require either fewer man-hours 'til completion and/or less skill from the hired developers doing said work.
You do make a good point that "building an app" or "building a website" will likely shift in meaning to something more complex, wherein we get "better" outputs for the same amount of work/price though.
And actually, I really don't see AI in the next decade making more of a difference than what Github did (making thousands of man-hour of works available for free). Around 2040 or 2050, maybe. But not soon, AI is still really far.
Yes, and this in turn increases demand as more people/companies/etc.. can afford it.
And even that could still be fine for programmers, as other firms will be enticed into buying the creation of software -- firms that didn't want to build software when programming was less efficient/more expensive.
But on the other hand, it also can mean that.
programmers, on the other hand, are wage laborers, individually selling their labor to employers who profit by paying them less.
industry is sitting on the opposite side of the equation here. I wonder what will replace "learn to code". whatever it is, the irony will be almost as rich as the businesses that profit from all this.
There are exceptions -- fancy weapons don't widely raise standards of living -- but the trends are strong.
On the first though, we have little reason to think tech will systematically diminish the roles people can fill. In the broad, the opposite has tended to happen throughout history -- although the narrow exceptions, like factory workers losing their jobs to robots, are real, and deserve more of a response than almost every government in the world has provided. For political stability, let alone justice.
Initially, they will appear more productive with CoPilot. Businesses will decide they do not need anybody other than those who want to work with CoPilot. This will lead to adverse selection on the quality of programmers that interact with CoPilot ... especially those who cannot judge the quality of the suggested code.
That can lead to various outcomes, but it is difficult to envision them being uniformly good.
Well like everything in life I guess it depends? The only iron rule I can always safely assume is supply and demand.
But for programming especially on the web, it seems everyone has a tendency of making things more difficult than it should be, that inherent system complexity isn't going to be solved by ML.
So in terms of squeezing out absolute efficiency from system cost, I think we have a very very long way to go.
I don't think software engineers will get much cheaper, they'll just do a lot more.
Otherwise I am struggling explaining why there is such a great demand for devs that short courses (3-6 months) are successful, the same courses that fail at teaching the fundamental of computing.
Maybe many. If the cost/benefit equation doesn't work, it makes no sense to do the project.
> I don't think software engineers will get much cheaper, they'll just do a lot more.
If they do more for the same cost, they are cheaper. You as a developer will be earning less in relation to the value you create.
Welcome to the definition of productivity increases, which is the only way an economy can increase standard of living without inflation.
Doesn't matter as long as I create 5x value and earn 2x for it. I still am earning double within the same time and effort.
It is certainly impressive to see how much the GPT models have improved. But the devil is in the last 10%. If you can create an AI that writes perfectly functional python code, but that same AI does not know how to upgrade an EC2 instance when the application starts hitting memory limits, then you haven't really replaced engineers, you have just given them more time to browse hacker news.
You're not replacing the engineer, but you're giving every engineer a tireless companion typing suggestions faster than you ever could, to be filled in when you feel it's going to add value. My experience with the alpha was eye opening: this was the first time I've interacted with an AI and felt like its not just a toy, but actually contributing.
https://blog.waymo.com/2020/10/waymo-is-opening-its-fully-dr...
I think full self driving is possible in the future, but it will likely require investments in infrastructure (smarter and safer roads), regulatory changes, and more technological progress. But for the last decade or so, we had "thought leaders" and VCs going on and on about how AI was going to put millions of drivers out of work in the next decade. I think it is safe to say that we are at least another decade away from that outcome, probably longer.
[1] https://finance.yahoo.com/news/number-american-taxi-drivers-...
But you are clearly more knowledgeable with your 0 drivers replaced comment.
That's exactly the kind of thing "serverless" hosting has done for a while now.
But over the coming decades AI could dominate coding. I now believe in my lifetime it will be possible for an AI to win almost all coding competitions!
They feed you all these algorithms in college and your brain suggests new algorithms based on those patterns.
An AI agent can interact with an environment and learn from its environment by reinforcement learning. It is important to remember that pattern matching is different from higher forms of learning, like reinforcement learning.
To summarize, I think there are real limitations with this AI, but these limitations are solvable problems, and I anticipate significant future progress
Also Generative Adversarial Networks original implementation was to pit neural networks against each other to train them , they don't need human intervention.
Some come from the other end of the process.
I want to solve that problem -> Functionally, it’d mean this and that -> How would it work? -> What algorithms / patterns are there out there that could help.
Usually people with less formal education and more hands on experience, I’d wager.
More prone to end up reinventing the wheel and spend more time searching for solutions too.
Which fits the pattern matching described by the grandparent.
A few people I know, most of which haven’t been to college, or done much learning at all, but are used to work outside of what they know (that’s an important part), tend to solve problems with things they didn’t know at the time they set out to solve said problems.
Which doesn’t really fit the pattern matching mentioned by the grandparent. At least not in the way it was meant.
(Some call me heterodox, I prefer 'original thinker'.)
This looks like a clever example of supervised learning. But supervised learning doesn't get you cause and effect, it is just pattern matching.
To get at cause and effect, you need reinforcement learning, like AlphaGo. You can imagine an AI writing code that is then scored for performing correctly. Overtime the AI will learn to write code that performs as intended. I think coding can be used as a "playground" for AI to rapidly improve itself, like how AlphaGo could play Go over and over again
AlphaGo learns a game with fixed, well-defined, measurable objectives, by trying it a bazillion times. In this autocomplete idiom the AI's objective is constantly shifting, and conveyed by extremely partial information.
But you could imagine a different arrangement, where the coder expresses the problem in a more structured way -- hopefully involving dependent types, probably involving tests. That deeper encoding would enable a deeper AI understanding (if I can responsibly use that word). The human-provided spec would have to be extremely good, because AlphaGo needs to run a bazillion times, so you can't go the autocomplete route of expecting the human to actually read the code and determine what works.
This exists: https://en.wikipedia.org/wiki/Automated_theorem_proving
It's like saying that calculators can solve complex math problems; it's true in a sense, but it's not not strictly true. We solve the complex math problems using calculators.
I would very much like GPT-f for something like SMT, then it could actually make Dafny efficient to check (and probably avoid needing to help it out when it gets stuck!)
Then we shall be reaching singularity.
Helping devs go through "boring", repetitive code faster seems like a good way to increase our productivity and make us more valuable, not less.
Sure, if AI evolves to the point where it reaches human-level coding abilities we're in trouble, but that's the case this is going to revolutionize humanity as a whole (for better or worse), not merely our little niche.
The hard part has always been writing an API that models external behavior correctly.
Knowing what code to generate with the AI: 200k/yr
Or maybe I'm thinking of tech leads. I don't know, my org is flat.
This is the problem with things like Spreadsheets, dag-drop programming, code generators.
Its not easy to tell a program what to change and where to change.
I've been saying something like that for a while, but my form was "If everything you do goes in and out over a wire, you can be replaced." By a computer, a computer with AI, or some kind of outsourcing.
A question I've been asking for a few years, pre-pandemic, is, when do we reach "peak office"? Post-pandemic, we probably already have. This has huge implications for commercial real estate, and, indeed, cities.
But yes, getting some kind of legal opinion will probably be cheaper with an AI.
https://www.suls.org.au/citations-blog/2020/9/25/natural-lan...
> I would
Well you're going to have a problem describing actual automation when you encounter it. What would you call it when NLP results are fed into an inference engine that then actually executes actions - instead of just providing summarized search results? Super-duper automation?
But I don't think AI will become capable of complex thought in the next one/two decades, so if you're training to be a software architect, project manager, data analyst I think you should be safe for some time.
Over time it will certainly do more, but it's probably quite a long time before it can be completely unsupervised, and in the meantime it's increasing the output of programmers.
I think I’ll side with his expectation, but then again, my salary depends on it.
1) AI makes really good code completion to make juniors way more productive. Senior devs benefit as well.
2) AI gets so good that it becomes increasingly hard to get a job as a junior--you just need senior devs to supervise the AI. This creates a talent pipeline shortage and screws over generations that want to become devs, but we find ways to deal with it.
3) Another major advance hits and AI becomes so good that the long promised "no code" future comes within reach. The line between BA and programmer blurs until everyone's basically a BA, telling the computer what kind of code it wants.
The thing though that many fail to recognize about technology is that while advances like this happen, sometimes technology seems to stall for DECADES. (E.g. the AI winter happened, but we're finally out of it.)
You'd definitely still need some seniors in this scenario, but it feels possible that tooling like this might reduce their value-per-cost (and have the opposite effect on a larger pool of juniors).
As another comment said here, "if you can generate great python code but can't upgrade the EC2 instance when it runs out of memory, you haven't replaced developers; you've just freed up more of their time" (paraphrased).
For context-specific questions it's even worse. The other day a stop owner that sells coffee beans insisted that we try out conversing with Google translate. I was trying to find the specific terms for natural, honey, and washed process. My Chinese is okay, but there's no way to know vocab like that unless you specifically look it up and learn it. Anyway, I felt pressured to go through with the Google translate charade even though I knew how the conversation would go. I said I wanted to know if this coffee was natural process. His reply was 'of course all of our coffees are natural with no added chemicals!' Turns out the word is 日曬, sun-exposed. AI is no replacement for learning the language.
State of the art image classification still classifies black people as gorillas [1].
I rue the day we end up with AI-generated operating systems that no one really understands how or why they do what they do, but when it gives you a weird result, you just jiggle a few things and let it try again. To me, that sounds like stage 4) in your list. We have black box devices that usually do what we want, but are completely opaque, may replicate glitchy or biased behaviors that it was trained on, and when it goes wrong it will be infuriating. But the 90% of the time that it works will be enough cost savings that it will become ubiquitous.
[1]: https://www.theverge.com/2018/1/12/16882408/google-racist-go...
Does "natural process" have a Wikipedia page? I've found that for many concepts (especially multi-word ones), where the corresponding name in the other language isn't necessarily a literal translation of the word(s), the best way to find the actual correct term is to look it up on Wikipedia, then see if there is a link under "Other languages".
[0]https://en.wikipedia.org/wiki/Coffee_production#Dry_process
When a new language / framework / library comes around, GitHub copilot won't have any suggestions for when you write in it.
Do you have a source for this re the last 20 years? It seems to me automation has been shifting the demand recently towards more skilled cognitive work.
Automation is a force multiplier. AI is a cheaper way of doing what humans do.
And the AI doesn't even need to be "true" AI. It simply needs to be able to do stuff better than what humans do.
Like protein solving? /s
I don't know, cranking out some suggestion for a function is not the same as writing a complete module / application.
Take the job of a translator, you would say the job would go extinct with all the advances in autotranslation? here it says that 'employment of interpreters and translators is projected to grow 20 percent from 2019 to 2029, much faster than the average for all occupations' [1]. You still need a human being to clear up all of the ambiguities of languages.
Maybe the focus of the stuff we do will change, though; but on the other hand, we do tend to get a lots of changes in programming; it goes with the job. maybe we will get to do more code reviews of what was cranked out by some model.
However, within a decade it might be harder to get an entry level job as a programmer. I am not quite sure if i should suggest my profession to my kids, we might get a more competitive environment in some not so distant future.
[1] https://www.bls.gov/ooh/media-and-communication/interpreters...
Only for the first time to train a model for that.
Repurposed Google-fu. We'll always have jobs :)
You assumption is that programming demand is finite, AND that all programmers are equal, both of those are false.
I must also say that actual programming is around 10-15 % of the programmer job, so the tool will make you around 10% more overall productive.
I wouldn't be surprised to see really good static analysis and automated code review tools coming out of these teams very soon.
The real strength of Windows is backwards compatibility, especially dealing with proprietary software. Its messiness is a relic of the way things were done in the past, it is also the reason why my 20+ year old binary still runs. And it does without containers or full VMs.
I much prefer developing on Linux (I still miss the Visual Studio debugger), but different platform, different philosophy.
Note: I don't know much about mainframes. I heard they really are the champions of backwards compatibility. But I don't think the same principles are applicable to consumer machines.
I don't even remember the last time Windows got in my way, in fact.
But no matter how much work you put into your Windows install it will be just as slow/fast, uncustomizable, and unoptimizable as it was out of the box.
wsl on windows 10 has been amazing to develop and work on.
Now imagine how amazing it would be just on Ubuntu ;-)
e.g. this is in my .bashrc in wsl, it writes stdout to my windows clipboard:
function cb () { powershell.exe -command "\$input | set-clipboard" }
Windows gets tons of hate in our community, but I gave it a chance a couple years ago after being frustrated with osx and it has been amazing and I think a lot of people would come around to it if they gave it a chance. I am biased towards linux though since I'm an sre, so maybe that is why I never could quite get comfortable on osx. I really disliked having to learn how to do something once on a mac, then do that same thing again on linux to get something into production.
TabNine at times was magical, but I stopped using it after Codota started injecting ads directly into my editor[1]
[0] https://betakit.com/waterloo-startup-tabnine-acquired-by-isr... [1] https://github.com/codota/TabNine/issues/342
also tabnine is useless in multi-lines completes. which is where co-pilot should be strong.
But nothing gets close to Copilot. It "understands" what you're trying to do, and writes the code for you. It makes type-based autocompletions useless.
For most backend programming jobs, the challenge is not in writing complex code but figuring out what the business wants, needs and should have in the first place and distinguishing between them. Figuring out how to integrate with existing systems, processes, fiefdoms and code. Knowing when to say yes and no, how to make code future proof, etc. This is a task fundamentally unfit for what we currently call "AI", because it's not actually intelligent or creative yet.
On the frontend, it becomes even more nebulous. Maybe Copilot can suggest common themes like Bootstrap classes for a form, CSS properties, the basic file structure and implementations for components in an SPA, etc. As I see it, the main challenge is in UX there, not the boilerplate, which again makes it about understanding the user, figuring out how to make UI feel intuitive, etc. Again: unfit for current AI.
I cannot offer any opinion on the utility for actually complex, FANG-level code, for lack of experience.
I think of it as not needing juniors for boring work, all you need as a company is seniors and AI.
It's still insanely impressive (assuming the examples aren't more cherry picked than I'd expect).
It's good for our health, too!
I feel like the current generation of AI is bringing us close enough to something that works once in a while but requires constant human expertise ~50% of the time. The self-driving industry is in a similar situation of despair where millions have been spent in labelling and training but something fundamental is amiss in the ML models.
We live in a very small corner of the space of possible universes, which is why finding a prior in program space within it is a big deal.
Edit: Check this screencast for instance: https://twitter.com/francoisz/status/1409908666166349831
I definitely see the utility in the linked screencast. But I am left to wonder whether this effectiveness is really a symptom of the extreme propensity for boilerplate code that seems to come with anything web-related.
You can actually see all the code blocks Copilot is thinking about if you want to, but that is indeed a distraction.
Based on the FAQ, it looks like some information about how you interact with the suggestions is fed back to the Copilot service (and theoretically OpenAI) to better improve the model.
So, while it may not understand "how to make code better" on its own, it can learn a bit from seeing how actual devs do make code better and theoretically improve from usage.
Do you have examples of what the line ones can do? The site doesn't really provide any of those.
https://gist.github.com/berndverst/1db9bae37f3c809e5c3f56262...
https://github.com/HackerNews/API doesn't list this as a valid endpoint. Indeed, when I try to access it via curl, I get a HTTP 405 with a "Permission denied" error (same result when I try to access nonexistent-endpoint.json).
Based on the HN search on the website, I'd expect the correct autocomplete to involve hn.algolia.com [0].
[0] https://hn.algolia.com/api points at https://hn.algolia.com/api/v1/search?query=...
To me, this points at the need for human input with a system like this. There is a Firebase endpoint, yes, and Copilot found that correctly! But then it invented a new endpoint that doesn't exist.
- the review from my peers is faster (the code is more correct) - I come back less to the code (because I have thought about all the corner cases when checking the copilot code) - As I care more about naming & inline docs (it helps copilot), the code is actually cheaper to maintain.
I think hit the Tab key more than I hit any other key ;)
Given that MS owns GitHub and how valuable this is - I imagine it will be coming soon.
Does it train a programmer for accepting less than ideal code because it was suggested? Similar to how some programmers blindly copy code from StackOverflow without modification.
Seems like there is a potential downside that's being ignored.
Maybe juniors, but I don't see this being likely for anyone else. I've been using TabNine for ages and it's closer to just a fancy autocomplete than a code assistant. Usually it writes what I would write, and if I don't see what I would write I just write it myself (until either I wrote the whole thing or it suggests the right thing). Of course, barring some variable names or whatever.
I don't have it "write code for me" - that happens in my head. It just does the typing.
Think of it as tab complete. If it's wrong, don't hit tab.
It suggests entire blocks of code - but not in every context.
Do you know in which "context" it suggests a block?
https://gist.github.com/berndverst/1db9bae37f3c809e5c3f56262...
May or may not reduce the demand for programmers, though. We'll see.
In what, proportion roughly?
It's really like pair programming, even though I'm coding alone. I have a better understanding of my own code, and I tend to give better names and descriptions to my methods. I write better code, documentation, and tests.
Copilot has made me a better programmer. No kidding. This is a huge achievement. Kudos to the GitHub Copilot team!