You are alluding to something I've been thinking about. The new AI tools are going to replace some workers and elevate others. The difference, I think, will be in how those two groups use the tools. The losers will be using them just to get a job done. Basic cut-n-paste coding. The winners will be using the tools to learn.

Jobs once said something like computers are bicycles for the mind. For me, that's an incredibly compelling idea and I've been looking for examples where it's true in my life. As a person who still really enjoys programming even after doing it for 40 years, I see LLMs as key tool for me to continue improving and learning.

Yeah, but how do you get access to 4?

You need to pay for ChatGPT Plus, or sign up for their API (there’s a waitlist).

FWIW, I got API access quite quickly. Applied March 20, got access March 27. My application was only some 2 line bullshit, their example for "why do you want access" rewritten.

I applied March 16 and I'm still waiting.

* * *

I got API access to GPT-4 almost immediately too, but I know people who have been waiting weeks. On the other hand, I don't have plugin access on ChatGPT yet, even though I applied for that as soon as it came out. There's probably quite a lot of luck involved.

I'm almost at 2 months of waiting at this point.

Strange. I applied April 6th and still don't have access

I've been waiting for over a month.

By paying $20/month with no trial.

Decide for yourself whether that's worth it based on other people's experiences. Personally, I'll pass.

There is also https://nat.dev - $5 with no expiry

Login or signup as only options, w/ zero context? Seems sketchy.

Thanks! THAT is the context I was looking for.

The site is made by Nat Friedman, Github's former CEO.

Ok... relevance?

Bing is free, and runs on 4.

Yeah, but it is weaker and provides different, more search focused, results than ChatGPT4.

I'm leveraging this to build https://codeamigo.dev. Basically it's learn to code, but with an AI assistant. The main reason people drop off of coding tutorials or stop learning is when they see their first error message. I'm already seeing people get un-stuck by using the AI as an assistant, this tech (GPT) is going to completely revolutionize learning.

Check out the new StarChat model by hugging face, might be useful for your project

Looks very cool. I am also working heavily in this space (using AI to help learning).

One minor point: In your demo, you state that "Today, almost 50% of code is written by AI, so why shouldn't you learn how to code with AI?"

Citation? I thought this might be true of developers using Copilot (based on their recent publications), but even that seems a bit of a stretch. I guess you could claim this because it's so easy to generate LOTS of code with an AI model. How much of that code is used, though? Really 50% of ALL code written now?

Thanks for checking it out!

"Today, GitHub Copilot is behind an average of 46 percent of a developers' code across all programming languages -- and in Java, that number jumps to 61 percent." - https://visualstudiomagazine.com/articles/2023/02/15/copilot...

This tracks since 90% of all Java code is boilerplate.

That post just cites an original [1] from someone at GitHub which says:

> Case in point: When we first launched GitHub Copilot for Individuals in June 2022, more than 27% of developers’ code files on average were generated by GitHub Copilot. Today, GitHub Copilot is behind an average of 46% of a developers’ code across all programming languages—and in Java, that number jumps to 61%.

I'm not sure what this means. GitHub does not have the ability to measure Copilot usage across all code written anywhere in the world, so surely it can't be that. Do they mean code generated by Copilot as a fraction of code produced by someone using an IDE with the Copilot plug-in? That number still seems high to me.

[1] https://github.blog/2023-02-14-github-copilot-now-has-a-bett...

Even if they were somehow measuring all code created in the world, considering it's even easier than copy & pasting general code to ask AI for shitloads of code without even bothering to review it, so code created would be quite a poor metric compared to code put into production at least once.

I guess it depends on how they calculate the percent. I assume it's by character count.

So it wouldn't surprise me. For repetitive code it's pretty good. And even with non repetitive code, once you start typing a bit it is good at finishing it.

Especially in a verbose language like Java. How much of copilot use is something that autocomplete would have handled anyways?

Your website looks nice, but I don't like this statement: "Today's developers didn't learn C before learning Python [...]" Makes me feel like I am a "yesterday's developer" and that it is bad thing.

How about binary before Python?

I think the parallel in that sentence doesn't work with binary and C. I would rephrase the sentence to something along the lines of: Today's developers don't use punched cards, why should you learn how to code without the most modern tools?

BTW, in some countries, you learn binary in grade school :)

And this is what you might say while you're learning. As you gain more experience, you realize that there's a wide range of solutions with some spectrum of a fit to your situation. A given solution may fall down in certain cases. What I love you Stack Overflow, is the context which comes with it. You get replies on a given proposed solution which point out where the solution may fail.

GPT4 absolutely does the same thing if you ask it to. Tell it to suggest approaches and discuss pros and cons and future pitfalls and it does, which makes sense because it is in part stack overflow.

The silver lining of LLMs impact on Stack Overflow may be that the platform focuses on facilitating the deeper type of problem solving you describe. My experience with Stack Overflow over the years has been mixed. Frequently users misunderstand the question, threads take weeks or months to resolve, users are called incompetent by frustrated experts, answers are indecipherable to a non expert, and sometimes toxic. ChatGPT frequently outperforms these weakpoints.

If they embrace a chatGPT interface as a first line of defense for users questions, it may improve the quality of real questions, reduce repetition, and filter through actually good and compelling questions. LLMs still need systems that encourage high quality data generation from humans to improve its model. I would like to imagine a world where there is a win-win for AI and the website we've relied on so much over the years.

The worst is the large amount of questions that solve an unrelated problem that's specific to the single asker's use case. Question, how do you do x? Response, oh, you can y instead and avoid x. Sorry, I want to know how to x, change the question

Step 1: Microsoft buys Stack Overflow

Step 2: They implement GPT-4 responses to questions

Step 3: The community + context provide the same feedback

Step 4: GPT-5/6/7 gets progressively better

If they train GPT-4 to post wrong answers on the internet, and then learn from the responses, we'll have the singularity in a month.

This is partially down to good promoting though.

It has seen those Stack Overflow responses and if you ask it to give you more context about a certain approach, it will.

I've learned when asking GPT-4 for help to push past the first answer, or ask it to offer more solutions. Also to ask why it choose a certain approach.

Quite often it will give me a response that doesn't fit well with my code, and I'll say "I don't want to do it this way, please give me another option, keeping in mind that [some important aspect of the code base]."

It then spits it out and it's usually way closer. Iterate on this and it's often quite good.

I think you're completely right if a person were copy pasting the first response every time, but that person is simply using it wrong in my opinion.

>you realize that there's a wide range of solutions with some spectrum of a fit to your situation

I ran into this the other day and it's made me very leery about using chatGPT for learning things I don't already know. I moved on from toy examples and tried getting it to write a program in a domain I was already intimately familiar with: I immediately realized that this thing is going to help people churn out all sorts of code that is a poor fit for the surrounding system or ecosystem.

Basically I had it write a program in Elixir as a GenServer. After that was done I asked it how I could integrate that solution into a Phoenix web application. I had to hold chatGPT's hand, so to speak, as we evolved the solution from:

(1) Spawning (and linking!) the GenServer to a controller i.e. in response to a web request.

(2) Spawning and linking the GenServer to the application but outside the supervision tree.

(3) Spawning the GenServer inside a supervision tree.

Now technically the application could have worked, more or less, if you ran it at any one of those intermediate steps. However only the last solution is the robust and idiomatic one. It felt almost like a constraint solver to me: we iteratively arrived at the right answer by adding more and more constraints to the solution. The problem, as it relates to a novice, is that they don't have a list of constraints rattling around in their head. (Language idioms, best practices, framework knowledge, system architecture knowledge, etc.)

The insidious thing here is that 1 or 2 are syntactically valid programs. If you try to beat 1 and 2 into submission you're going to have a subtly broken system, one that makes a lot of Elixir programmers very sad.

---

I'll be watching the Khan Academy guys pretty closely, because they seem to have put a lot of thought into how students could use these systems safely and effectively.[1] I think the answer is going to be we need AIs that are aimed at professionals, and AIs that are aimed at education. The hard part will be that somehow we need to direct people to use one appropriate to their skill level.

[1]: https://www.ted.com/talks/sal_khan_how_ai_could_save_not_des...

I understand the excitement feeling of using something like this to greatly reduce the learning effort. Before GPT/ChatGPT, it was definitely frustrating to "bootstrap" your knowledge in any new area where you don't have prior experience.

But still, I treasure the old ways I learned how to program. It was such a fulfilling experience by making newbie mistakes and getting corrected in hard ways, and you really only get the authentic insight by going through that process, and by reading all those important books chapter by chapter. It's probably not as "fun" as using GPT-4 though.

I've been trying to do it to some extent but I can't say it's been enjoyable (granted most of it has been GPT-3). It's more like a tutor that never calls out your mistakes and gaps. Anything you ask, it comes up with an affirmative answer, I try the answer and it ends up being a hallucination.

At this point, i am in a place where GPT gives me good solutions that might not work vs having to go through all the noise Googling stuff to find a working solution.

But it is great just to learn different concepts at a high level. Helps me clear my mind when coming up with designs

I think fun is the key difference as well! Even when it takes longer it’s a lot more engaging.

I actually often add GPT4 responses into a markdown page and build a whole library of “how to do x”