Like you said, you might have given up on F# without ChatGPT assistance, and the main way ChatGPT is able to help with F# is because of all of the example code it's been trained on. If developers rely more and more on LLM aid, then a new language without strong LLM support might be a dealbreaker to widespread adoption. They'll only have enough data once enough hobbyists have published a lot of open-source code using the language.
On the other hand, this could also leading to slowing adoption of new frontend frameworks, which could be a plus, since a lot of people don't like how fast-moving that field can be.
i.e. the advantages of an even-higher-level python that's almost like pseudo-code with assembly-level speed and rust-level safety, where some complexity can be abstracted out to the LLM.
As an example, why not write in f# and let an 'AI-compiler' optimise the code...
The AI-compiler could then make sure all the code is type safe, add in manual memory management to avoid the pitfalls of garbage collection, add memory safety etc - all the hard bits.
And then if we gave the AI-compiler those sort of responsibilities, then we can think about how this would impact language design in the longer term.
None of this is with current generation LLM's, but might be where we end up.
So far we haven't been able to close this gap fully with current compilers and interpreters (i.e. python still runs slower than C).
It seems like that gap could be closed through, for example, automated refactoring into a rust-like language during compilation, or directly into more-efficient byte-code/ASM that behaves identically.
And surely if that is a possibility, that would affect language design (e.g. if you can abstract away some complexity around things like memory management).
- A rough crash course in F#. I'll say "what's the equivalent in F# of this C# concept?". It will often explain that there is no direct concept, and give me a number of alternative approaches to use. I'll explain why I'm asking, and it'll walk through the pros/cons of each option.
- Translating about 800 lines of TypeScript JSON schema structures to F#. A 1:1 translation is not possible since TypeScript has some features F# doesn't, so ChatGPT also helped me understand the different options available to me for handling that.
- Translating psuedo-code/algorithms into idiomatic F# as a complete F# beginner. The algorithms involve regex + AST-based code analysis and pattern matching. This is a very iterative process, and usually I ask for one step at a time and make sure that step works before I move onto the next.
- Planning design at a high-level and confirming whether I've thought through all the options carefully enough.
- Adding small features or modifications to working code: I present part of the function plus relevant type definitions, and ask it for a particular change. This is especially useful when I'm tired - even though I could probably figure it out myself, it's easier to ask the bot.
- Understanding F# compiler errors, which are particularly verbose and confusing when you're new to the language. I present the relevant section of code and the compiler error and 90% of the time it tells me exactly what the problem and solution is; 5% of the time we figure it out iteratively. The last 5% tends I have to stumble through myself.
- Confirming whether my F# code is idiomatic and conforming to F# style.
- Yes it makes mistakes. Just like humans. You need to go back and forth a bit. You need to know what you're doing and what you want to achieve; it's a tool, not magic.
Note: this is the commercial product, ChatGPT-4. If you're using the free ChatGPT 3.5, you will not be anywhere near as productive.
https://chat.openai.com/share/d041af60-b980-4972-ba62-3d41e0... https://github.com/Mk-Chan/gw2combat/blob/master/generate_co...
If you try to generate code, you’ll find it underwhelming, and frankly, quite rubbish.
However, if you want an example of what I’ve seen multiple people do:
1) open your code in window a
2) open chatgpt in window b (side by side)
3) you write code.
4) when you get stuck, have a question, need advice, need to resolve an error, ask chatgpt instead of searching and finding a stack overflow answer (or whatever).
You’ll find that it’s better at answering easy questions, translating from x to y, giving high level advice (eg. Code structure, high level steps) and suggesting solutions to errors. It can generally make trivial code snippets like “how do I map x to y” or “how do I find this as a regex in xxx”.
If this looks a lot like the sort of question someone learning a new language might ask, you’d be right. That’s where a lot of people are finding a lot of value in it.
I used this approach to learn kotlin and write an IntelliJ plugin.
…
…but, until there’s another breakthrough (eg. Latent diffusion for text models?) you’re probably going to get limited value from chatgpt unless you’re asking easy questions, or working in a higher level framework. Copy pasting into the text box will give you results that are exactly as you’ve experienced.
(High level framework, for example, chain of thought, code validation, n-shot code generation and tests / metrics to pick the best generated code. It’s not that you cant generate complex code, but naively pasting into chat.openai.com will not, ever, do it)
Many times it works really well, and it surfaces the kind of example I need. Sometimes it works badly. Usually when it's bad, going to the google/sift method has similar results. Which I guess makes sense, it couldn't find much to train on, so that's why it's answer wasn't great.
One area it works really well for me is 3rd party apis where their documentation is mostly just class/function/etc. ChatGPT generally does a good job of producing an orchestrated example with relevant comments that helps me see the bigger picture.
The fact that it usually had errors didn't bother me at all -- it got much of the way there, and it did so by doing the stuff that is slowest and most boring for me: finding the right libraries / functions / API set up, structuring the code within the broader sweep.
Interesting side note: un-popular languages, but ones that have been around for a long time and have a lot of high-quality and well-documented code / discussion / projects around, are surprisingly fecund. Like, it was surprisingly good at elisp, given how fringe that is.
But it’s obvious outside of jr dev work and hobby projects there’s no way it could possibly grasp enough context to be useful.
Yeah usually it was the shittiest managers I ever met that shared this belief. It sounds like they all repeat the same thing - gpt is a better google.
Those types huh
Not saying your post is devoid of any substance, just trying to educate myself on my blindspots
Tech has, for years, seen people with cognitive/narrative intelligence as the people who are actually smart.
LLMs help the intuitive people reach the level of the cognitive/narrative people. Cognitive/narrative people can't really understand this in the same way the intuitive people are bad at syntax or database structure. The steamrolling will be slow and merciless.
I had a problem last week where I wanted to extract sheet names and selection ranges from the Numbers app for a few dozen spreadsheets. ChatGPT, came up with the idea of using Apple script and with a but of coaxing wrote a script to do it. I don't know ApplesScript and I really don't want to learn it. I want to solve my problem and its 10 lines of AppleScript did just that.
We're nowhere near LLMs being capable of writing codebases, be we are here for LLMs being able to write valuable code because those concepts are orthogonal.
1. some, most
- De-obfuscate a obfuscated JS code
- Unminify a JS code. Asked it to guess function names based on the functionality
- Work with it like a rubber duck to plan out the possible solutions to a code problem
- To suggest function names based on the functionality
- To name repos
- Modify a piece of Go code to add specific functionality to it. I don’t know to write Go; I can read it and grok the high level functionality
I mean, I don't like intellisense either (but simple autocomplete is fine). Perhaps it is because I only code to help with my job, I don't get paid to have a lot of good quality code output.
As one myself LLMs don't feel like giving up control it feels like collaboration. You probably don't like collaboration.
When I interact with an LLM it feels like I'm speaking to someone who lacks inherent conceptual understanding, but is doing their utmost best to bullshit me. This is problematic when it comes to algorithmic thinking.
I sometimes need to solve problems that involve tweaking existing algorithms in somewhat novel ways. For example "k-medoid clustering but with various capacity constraints on clusters", or "contraction heirarchies but accounting for varying vehicle dimension restrictions".
I love collaborating with smart colleagues to come up with ideas to solve these problems. Seeing chat-gpt fail to solve very simple traveling salesman type problems gives me little confidence that it can do novel conceptual tihnking.
Also what you're doing here is asking chatGPT for the answer. chatGPT is more effective via collaboration. Meaning instead of shoving the entire problem down it's throat and asking it to solve it, you ask it for advice. Ask it for bits and pieces of things.
A lot of front-end development for example does not require familiarity with any algorithms, formulae or mathematical structures. You might need to reason about large systems and have a rough idea of when hashmaps are useful, but the bulk of the work is constructing an interface according to functional business requirements. I frequently see comments here along the lines of "why am I being interviewed about algorithms when I'll never use them in my job".
A more mathematically oriented developer may be in the business of modelling and predictions. They may be confronted with a novel real world problem involving traffic, or trading, or electricity networks, that potentially no one has tried to solve before. They may be required to find a mathematical structure that closely approximates real world behaviour, implement that structure via code, and deploy a continuous model which allows their client to analyse and project.
Of course, you also have academic mathematicians using software like Maple or SageMath to assist with their research. This is another level more mathematical. Perhaps what you're getting at is that people can ask ChatGPT questions like "write me some Sage code to get the Delaunay triangulation of this set of points". I totally agree that it can probably do well at these tasks.
You also talk about things like traffic. Modelling traffic is mathematical? Sounds like a simulation to me. Man take a look at GTA. That game is almost entirely made by builder engineers creating simulations. It's the same shit and likely far more advanced then what any data scientist can come up with.
Anyway from your example and from what Ive seen it sounds like you're still doing the same thing. CS algorithms. You're just using algorithms that aren't likely very popular or very specific to data and stats. But adjusting stuff like k-means clustering still sounds like regular cs stuff to me.
There's no point in calling it more "mathematical" because it's not. The builder engineer who wrote all the systems in GTA or even say red dead redemption use a ton of "math" even and they don't term themselves more "mathematical" even though their simulations are likely more complex than anything you will ever build.
That's why when you called your self mathematically superior (again don't deny this.. we all know what you really mean here) I thought you were talking actual math. Because if you looked at a math equation it doesn't look anything like an algorithm. Math equations are written as a single expression. Math equations model the world according to a series of formula. It's very different to a cs algorithm.
Mathematical oriented programming involves largely the same thing and using algebras of mathematics.
If you're not doing this just call it data science instead of trying to call yourself more "mathematical". If you truly were more mathematically oriented you would know what I'm talking about.
Geeze some guy writing "models" and doing some applied math+stats like what every other freaking programmer out there is doing and he calls himself more "mathematically oriented."
Data science definitely forms part of what I do, as my employer stores a lot of data that we use to estimate various parameters. But there's also work on creating bespoke routines for solving vehicle routing problems in niche domains, which I wouldn't really class as data science.
Thanks for the discussion, anyway. I'm not interested in being insulted.
"Bespoke routines for vehicular routing problems" lol. I mean phrases like that reveal what you think of yourself as.
You're writing simulations. That's all. "Bespoke" lol. And those simulations have lower fidelity then a video game like GTA which likely does traffic at higher levels of fidelity and real time with a renderer.
I have a doctorate in mathematics. Prior to that I've done work cs. Doesn't mean shit. I don't name drop that crap to pretend to be superior.
I've always been more of a fastidious crafter than a "just get it built" person, but I also struggle with a blank page. I thrive on editing more than writing. Since forever, I like to get out something that works or mostly works, and then start carving on it until I like it.
LLMs have been helping me get some ink on the page, but very little of what they suggest ends up in the final product.
Supposedly Tom Robbins writes books entirely by putting one word after another starting with the first one and finishing with the last one. I don't know if that's apocryphal, but I do think that's closer to the process for some people.
But if I were a writer, I'd be squarely in the "get out a first draft, it will be like pulling teeth, but just get something down; then you can do the fun part of revising and polishing".
I love to write code, I love to modify existing code too.
I do not love to read and then fix code after someone all the time. With ChatGPT I have to read then understand then fix code after ChatGPT every time.
Also, I do not love to fix code that often contains hallucination.
Second, have you tried pasting any syntax errors you get back into the chat window? Usually, GPT will try to fix them.
Third, you can have GPT write a function for you and then write a different version of the function using a different algorithm along with a test (e.g. a fuzz test) to check whether two functions produce the same output given the same input. This makes it less likely that an error will slip through because both algorithms would have to be produce the same incorrect output for the same input.
Second, your fix is to verify code generated by LLM and then play with it trying to find a way to fix it. I`m quite sure that I will spend less time and less mental power by writing code in the first place.
Third fix is to play with chatGPT more, and read more code generated by chatGPT trying to find errors in it.
What happened with "reading code written by someone else is harder than writing your own code"?
It is not intellectually stimulating for me to think about what the syntax for a dict comprehension is, or what exactly I'm supposed to do to map over the values of an array in javascript without screwing it up, or any of a million other kinds of minutia. Computers know the answers to these uninteresting questions.
There's no question about this being the right way to use it for me, but I wonder if this could introduce something bad for someone just starting out, who hadn't first got all the reps with the drudgery over decades? Still mulling on that.
And maybe there's some as yet unexplored design space out there for which this isn't true, but my prior is that we're actually just circling around some essential complexity at the core of the problem, which will never be eliminated.
But I also think this is probably just a classic geezer's concern about kids being on my lawn, and it will probably work out just fine for these youths :)
This strikes me as the essence of expertise, the difference between talking to someone who's read a book or a review paper, vs someone who is a real expert at that thing. You can be quite competent and insightful from having done a modest amount of reading and thinking, but if you're seeing the Matrix, and know how all these forces are interacting with each other, that's just a whole other level. It makes an impression when you brush up against it.
However: it always weighs on me how this is all a matter of framing. Like, I don't know the electronics. There's a level of metal that I don't get. Some of the really high-level stuff is now beyond me, too. I catch myself saying "I should really take x {weeks, months, years} and really grok that." And yet my actual experience suggests this is a mirage.
More briefly: there are always more layers for a fuller understanding. It's hard to see how many of them are really useful. Maybe the kid who is 10x better than me at LLM collaboration will benefit more than from having a deeper stack. It's interesting to ponder how these different consequences will play out.
My messy answer is that for many projects it's either neutral or an actual hindrance - I have to remind myself to pragmatically not care about stuff that won't matter for years at the project's expected growth rate - but for other projects it's very useful. I think the way to go is to seek out those kinds of projects, in order to avoid frustration all around.
Make, docker, kubernetes, all fit this pattern. Heck, maybe I won't be so down on autotools if I run into it again now that I can attack it with LLM support.
Or html / css; I haven't written any of that in the LLM era, but maybe I'd enjoy it more now.
Not clear to me if the author actually uses LLMs to do meaningful work, or is speculating about how they might be used.
I've written about 2500 lines of F# for the first time in the past 1.5 weeks using ChatGPT-4 to guide me. It has been an constant back and forth, iterative process. My decades of development experience factored in heavily to guide the process. I would've have been at maybe a quarter the progress without ChatGPT, or given up entirely on F# as my language.
I don't think that iterative aspect will be eliminated any time soon for AI-supported complex, creative processes. It's no different from tweaking inputs to a Photoshop filter until your experienced brain decides things look right.
To that end you need to know roughly what looks "right" before you use an LLM. This will all become second nature to the average developer in the next 5-10 years.