I dunno what you do for a job but if that is your hard problems or even medium problems then I think you lucked out big time.

I felt the same, SWE with 20 years experience. Absolutely none of this is relevant to anything I’ve been paid to do for my whole career.

If someone on my team was doing things this unusually they’d probably be let go.

I use ChatGPT for similar. I can spend time looking into various languages/libraries I don't use often and figure it out. Or I can ask an LLM and have an answer I can validate in the same amount of time I start browsing search results.

You're missing the point. These LLM which are replacing "junior developers" aren't what's going to be used for "hard or even medium problems" because you don't just give junior developers "hard problems" to solve in isolation. Now try to give some thought as to the types of problems junior developers are suited to solve. Now apply ChatGPT to those scenarios.

I never said these problems were hard.

But... they're things that require research. Do you know how to loop through all of your Apple Notes using AppleScript off the top of your head?

That research isn't free: it takes time.

As someone who hasn't done much work with AppleScript before, I would guess it would take me about half an hour to figure out how to do this. But there's a risk that it might take longer.

So the sensible thing is probably not to take on that project at all! I don't care enough about solving it to invest the research time.

But the time taken to chuck a prompt through GPT-4 and then test the results to see if it works is less than a minute.

I wrote more about how this is encouraging me to be more ambitious with my projects here: https://simonwillison.net/2023/Mar/27/ai-enhanced-developmen...

100% agree with this. I used ChatGPT earlier today to give me networkx code for computing the connected components of a graph and visualizing the graph. This isn't hard to do, but I don't use networkx all that often and I forget the exact API. I could go to read documentation and piece together an example myself, or I could ask ChatGPT for an example, which tends to be much faster than doing the former.

I Googled "applescript loop through apple notes" and this was the first result: https://apple.stackexchange.com/questions/225938/notes-scrip.... Looks exactly like what ChatGPT came up with for you minus the dialog.

I then Googled "applescript print to console" and got this: https://stackoverflow.com/questions/13653358/how-to-log-obje... ; admittedly this answer took a few minutes to read through, but the second most upvoted answer recommended using log, which your ChatGPT-developed solution (eventually) used.

This all took me < 5 minutes.

Yeah, those StackOverflow posts are the perfect illustration of why I find ChatGPT so much useful for this kind of thing.

Compare them to my original transcript. They don't provide me with enough information: how do I actually run that code?

I got ChatGPT to split me out a zsh script, which caused it to show me how to use "osascript -e ".

I think Simon has plenty of experience solving hard problems: https://simonwillison.net/about/

That's irrelevant, we are taking about solving hard problems with ChatGPT none of the examples are even slightly complex.

Not sure where this idea that my post was about "hard problems" came from.

I'm excited about using ChatGPT/GPT-4 because it takes the time I need to figure out how to do something "easy" down from half an hour to sub-5-minutes.

Dude, simonw invented Django web framework

They’re probably not hard and medium problems but if you can take care of these easy and medium problems with chatgpt, that saves so much bandwidth to have time to tackle the fun and “hard” problems.

Also, take a look at the problems and projects simonw actually tackles.

You don't get the point.

I work in Deep Learning Research. I don't get any help at all from ChatGPT for my core job. Copilot spews gibberish, too.

I do get enough help about peripherals. Some weeks ago, I needed help with Flask and HTML deploying a model to show it to stakeholders. (I learned Flask some years ago, but not needing it regularly, I forgot enough.)

The data cleaning, preprocessing, model training, making it better than humans were the hard tasks.

Deploying a Flask app with a simple HTML frontend was the easy task. But easy != free. It would have required 2-3x more time researching how to do exactly what I needed, which I did with Copilot and ChatGPT in ~1 hr.

sorry but these examples are not impressive at all and by no means a representative of any serious programmer's workload.

Programmers are paid not to bang out code, but rather to figure out the mess and crap of the existing codebase and how to selectively add one-two lines to change system's behavior and keep stability of the system.

Part of the trick of making the most of LLMs is figuring out what kind of things to apply them to.

I wouldn't expect much from them for figuring out gnarly changes in huge existing codebases (at least not yet).

But they've been actively encouraging me to write smaller, tighter tools - the good old Unix philosophy - for which they are extremely well suited.

This comment makes me feel as if I've been taking a wrong approach to using LLMs in my day to day work. I haven't been able to get much value out of them as a large majority of the work I do requires a good deal of context with some repo specific problems.

Our existing tooling and helpers lack modularity to begin with. I'm now thinking it would be a better approach to start with having it build smaller tools and patch them together in more useful and interesting ways myself as opposed to being upset that it can't deliver complex, context aware solutions.

Still not buying into the hype that LLMs will replace all software engineers by 2030 due to thinking the nature of our work is not to write code but to solve problems, regardless of the tools we are using, but I definitely see potential productivity gains from using the tool with a different approach to what I have previously attempted.

So basically llm is the new short shell script.

Fair enough, but i also don't really feel this is threatening anybody's job.

I think it's pretty useful to share concrete examples like this. I basically always have a ChatGPT4 CI tab open these days when I'm developing these days - it's a usually much faster/better go-to than Google or SO for looking stuff up. ChatGPT is great for all the random stuff I don't care to remember the syntax for.

* I'm always using it to munge/generated tables/csv/markdown/json - you can basically throw any copy and paste from a random PDF that's some weird gobbedlygook of tabs, spaces, newlines and get something cleanly formatted. On the one hand, it seems like a waste of computation, but on the other hand, it's way cheaper than my time and there are so many tasks that require using poorly formatted output. Even better, CI will of course write awk/sed for you if you need to do any automation.

* I'm always forgetting the syntax for named byobu sessions (it happily wrote a script to help with that) but I've also been staging some dev servers and it was able to generate the scripts to create new named session and windows, attaching/creating when necessary, handling if the processes were running, and creating the systemd units for spinning these up.

* On this same project it wrote some python scripts for managing SSH tunnels and reverse tunnels, including filtering/logging of error messages, handling jump servers, etc. This is all stuff I've done years ago (and even written lots of docs for), but it was actually way faster for ChatGPT to generate these than digging those out.

* I've been running into issues w/ some HTML5 audio output and needed to swap to websocket streaming w/ webmedia output (which I wasn't familiar with at all). ChatGPT gave me the code to swap into my FastAPI server and the frontend code I had w/o having to do any further research, great.

* I hate Docker setups, and I had issues w/ Nvidia containers and GPUs not showing up w/ my docker config. I was able pass the various error messages and get my problems fixed without spelunking/hair-pulling. Same with figuring out some cross-container network hijinx.

* There's a bunch of one-offs that I might just not have bothered doing, that I can just ask it to do as well - eg, I've previously written code for poisson distributions and the like, so I knew what to ask for, but would have been a huge PITA to dig out exactly how to do it, but took like no effort to just ask GPT4 CI to figure out a one off I just wouldn't have done otherwise: https://chat.openai.com/share/80fa7bc0-e099-4577-bad9-d026e7...

Those are good examples. I'm constantly doing weird one-off things in Ruby or Bash, and maybe I'll ask ChatGPT about more of them.

I really wanted to know what vague things OP's humans were doing, but they haven't responded to anything.

I've been reading your blog since the dark ages. Thanks for the great content over the years!