I don't trust ml computing to be precise in the same way I don't trust humans to be precise (we can all potentially write code with bugs) because the process is fundamentally non deterministic.
The way we solve this with humans is to create tools that test verify what we make, backed by some sort of proof.
I guess once ml computing can get feedback from these tools, the situation will improve?
you are so close with the rest but..
> fundamentally non deterministic
thats not quite it.. it is non-deterministic plus the way the parts are re-ordered is flexible in ways that are not obvious, and do result in nonsense sometimes. More specifics are according to the engine(s) in use..
I have no idea if an LLM specifically is the tool for the job or not, but presumably with the input of the source code of millions of compiled executables and the compiled output, "an" AI system can learn to be a compiler. If it can then analyse the generated programs for things like performance, size or other metrics of compiler goodness, it can compile "better".
Hopefully it won't make any mistakes when you come to use it because that will be the devil's own job to track down: compiler bugs are bad enough as it is!
But then we can conceive of a compiler bug-finding AI...
The power of LLMs is the fact that they use natural language. It’s like bashing on pipe based string processing that enabled all the Unix applications in the beginning. The simplicity of the system is what powers the entire thing. You cant take it away.
Complexity is the problem. This is what humans, and now LLMs, struggle with when it comes to designing and coding solutions. Managing complexity is why we've attempted to invent paradigms such as OOP, which some claim only makes things worse.
ML is a tool for results that are just good enough. But people think they can program with it.
LLMs are a good way to get a glimpse of the right answer you still need to get deeper look in topics for:
A: validate the result of the LLM
B: get a better understanding of your topic
That is more or less the state of all plans on putting llms into anything useful. I think the lack of and education of language parsing ambiguity is the main reason why so many people are buying the hype here.
>Siri call me an ambulance.
Is not a solved problem and will never be if you're using English as your input language.
-ME: It took my requirements for a simple webapp, gave me the front-end and back-end code. Then it tweaked the code appropriately based on my comments and bug reports, without arguing back or asking why I'm so dumb. In 10 minutes I went from idea to prod potential.
-HIM: Effing thing is going to put me out of a job...
-Me: PMs who jump on the bandwagon and train people in how to write good, detailed and precise specs will be fine.
-Him: Hahahaha. No one writes good specs. I’m safe for now.
Here’s something I posted earlier:
I asked ChatGPT “Let’s say I want a program to delete unused files. Can you write a detailed specification that a developer could use?”. I was as vague as possible on purpose.
Here’s what it gave me:
“Program Name: FileCleaner
Overview: The FileCleaner program is designed to identify and delete unused files from a specified folder or directory. The program scans the directory for files that have not been accessed for a specified period of time and prompts the user to delete them. The program is intended to free up disk space and improve system performance.
Requirements:
The program must be able to scan a specified folder or directory for files that have not been accessed for a specified period of time.
The user should be able to specify the time period after which a file will be considered unused.
The program should prompt the user to delete the identified unused files.
The user should have the option to select which files they want to delete.
The program should permanently delete the selected files from the system.
The program should log all deleted files, including their location and name, in a separate file for future reference.
The program should have a user-friendly interface that is easy to navigate and understand.
Functional Specification:
The user launches the FileCleaner program.
The user is prompted to select the folder or directory they want to scan for unused files.
The user is prompted to enter the time period after which a file will be considered unused (e.g., 30 days).
The program scans the specified folder or directory for files that have not been accessed for the specified time period.
The program displays a list of unused files, including their location and name. The user is prompted to select which files they want to delete.
The program permanently deletes the selected files from the system.
The program logs all deleted files, including their location and name, in a separate file for future reference.
The program displays a message indicating that the selected files have been successfully deleted.
The user can choose to exit the program or scan another folder or directory.
Non-Functional Specification:
The program must be designed and implemented using a high-level programming language, such as Python or Java.
The program should be able to handle large files and directories without crashing.
The program should have a user-friendly interface that is easy to navigate and understand.
The program should be able to run on multiple operating systems, including Windows, MacOS, and Linux.
The program should be secure and not pose a risk to the system or user data.
The program should be well-documented, with clear and concise instructions for use and troubleshooting.
By following these specifications, a developer can create a FileCleaner program that efficiently identifies and deletes unused files from a specified folder or directory.”
This could be improved even more by having it ask questions to narrow down exactly what is needed
Then I told it I wanted a GUI, so it started writing a perfectly fine CS101-style tkinter program. I couldn't get pip to work, but chatgpt solved that for me too.
Then I told it to rewrite it in rust. It generated plausible code, but it couldn't finish the problem before getting cut off. And I was unable to get it to continue where it left off.
If you have a one-shot problem, you may be able to avoid making the actual program and just jump from problem to solution. A lot of the systems that programmers design are potentially representable instead as new iterations of that one shot problem, and therefore something no code need be written for.
This does run a risk of getting a wrong answer infinitely fast, but that has never not been the case with computers, it's just a case of now having a kind of "level 3 autonomy" where you may need to abruptly grab the wheel and get the answer yourself.
Let me give you an example of what I did just this morning. In fact, ChatGPT is generating code in another tab as I'm typing this - GPT4 is quite slow (well, at least it takes longer than my attention span which has been reduced to that of a sea urchin by years of social media and usage of internet in general), especially when you let it create 50 to 100 lines at a time.
I'm working on a specialized ETL tool for geospatial data. In fact, I've had several iterations of this tool over 5 or more years, but for the last few months I've been working on a new version that incorporates everything I've learned about the domain in that time. This conceptualization, the learning of what the tool should do and how all components fit together, is not something a computer can do. I would have to be able to describe it to get a computer to generate it, of course, but I couldn't a few years ago because I was still finding out what it should do to provide the most value. The reason I had several versions is because I kept learning more about what I need it to do.
But many of the boring machinations I absolutely can have a computer do. For example, a part of this tool is a tiny DSL which is basically a way of specifying combinations of parameters with some dress-up. I'd been dreading making a proper parser for weeks, since a) the 'design' of this DSL has fluctuated quite a bit, and b) I'd have to dreg up my knowledge of parsers from the last time I wrote one 10 years ago. So I've been getting by using a combination of regex, split() and some hand massaging of strings.
But this morning, I had ChatGPT write me the biggest part of a parser using three questions:
- "Write a parser in python for the following format: [optional identifier] [open square braces] [either a list of values, separated by commas; or a list of key-value pairs, separated by commas, the keys and values are separAted by equals signs] [close square braces]. The identifier is optionally followed by an 'at' sign, and a qualifier after that. Show me the EBNF and a parser in python that implements this grammar."
- "The qualifier should come before the square brackets, it's part of the identifier."
- "Write some unit tests for this grammar. Include corner cases for all varieties of the allowed input, so with or without identifier, with or without qualifier, with plain lists and with lists of key/value pairs. Include cases for 0, 1 and 3 values in all places where more than one element can appear."
(try it, I used GPT4 but you'll probably get similar results if you use the free version)
The result I got from this has 2 mistakes: in one case I had to swap two tokens in the EBNF. This one I found by reading the docs while waiting for the output of whatever ChatGPT suggested as a fix after me copy and pasting the error message I got. So this was fixed, by me, in a few minutes (ChatGPT's suggestion for a fix made no sense). The second one, ChatGPT suggested a correct fix after I copy and pasted the error message also.
So, ChatGPT gave me a skeleton grammar, complete with the syntax of the library which I had never heard about 2 hours ago. The second of my first three questions was because I gave it ambiguous input, I only thought of that part while I was writing the question and didn't bother to go back and edit it in in the right place where it would have been more obvious. Then I copy/pasted the raw output from ChatGPT in a new file, ran it, I got the issues fixed (in collab with ChatGPT) in 20 minutes or so. This would have taken me all morning and probably all day if I include all unit tests and all other details just a few months ago. How is this not a massive game changer?
Now, I would not have been able to do any of this hadn't I already known what an 'EBNF grammar' was, and hadn't known how to read and implement one. This tool will not replace programmers. It will however eliminate the need for the very low skill ones, and make the rest (much) more productive.
Something I'm really interested in, after we've gotten the full 32k context and multi-modal capabilities of GPT-4 and done all the Langchain/ReAct shenaniganery we can, is GPT-5, or 6. These things are coming so fast and delivering step changes in productivity, it really does feel like something new.
Wrt to productivity - I do think it's misleading to look at it like 'it used to take me 5 hours, now only 1, so that's 5x' or some other (imo naive) quantification. It's much more about removing drudgery and thus freeing up mental resources for interesting things. Say I do a task with a certain tool in the same time it took me without it, but in a much less mentally taxing way (because that's what drudgery does to me - activate bore-out mode). From the naive metric pov there is no productivity improvement. But if that leaves me capable of doing more in the rest of the day/week because I don't have to fight myself to move on with something, I'd argue it's still a productivity gain.
Does typing with a nice keyboard and in a good chair make me more 'productive'? Well, I don't think anyone will argue that I can't type on a e5 Chinese crap one while sitting on a bucket before a door on two sawhorses. I can even type and think equally fast in those circumstances. But better tools still make one more 'productive' in the larger picture, I don't think anyone with experience will deny this either. This 'productivity gain' is in that sense somewhat ethereal, in a poetic sense of that word. To me that's where these tools are at now.
- They do not transpile Rough English to Deterministic English. Infact they do not do any transpiling at all.
- LLMs learn the probabilities of words in the dataset for all contexts from that dataset (context is a ordered set of words). This is called training.
- Once training is done, LLMs can generate text given a prompt and the probabilities that it has learned. The analogy of LLMs being auto-complete on steroids is very apt.
- Whether the text generated by LLMs is factual or not is purely coincidental.
I would highly recommend watching and working though the code (if possible) of NanoGPT by Andrej Karpathy. https://www.youtube.com/watch?v=kCc8FmEb1nY
A lot of things which seem like magic about LLMs will get demystified.
Now one can argue that LLMs are showing human like reasoning/intelligence/sentience as an emergent behavior. This is hard to argue against because all these terms are extremely hard to define.
IMO, the only emergent behavior that LLMs are showing is the output they generate looks like it might have been generated by a human which should not be surprising given that LLMs like ChatGPT has trained on a large amount of human written text available on internet.
In computer science, pseudorandomness is considered non-deterministic. Determinism is a function of the inputs, not the machine state.
https://en.m.wikipedia.org/wiki/Deterministic_algorithm#:~:t....
Because full determinism is not always desirable, the researchers have implemented an explicit "temperature" parameter that you can use to inject randomness to the outputs. If you set that to 0.0 you will always receive the same output for the same input and model version.
Sometimes, I can almost catch my brain generating the next word from what I’ve said so far, or listening to what some else is saying.
No, because the probability of a word on the internet being factual is not coincidental. Factuality compresses the corpus; the truth is generally the simplest explanation for a set of observations. (The collected text of the internet is a set of observations about reality.)
> IMO, the only emergent behavior that LLMs are showing is the output they generate looks like it might have been generated by a human
The whole point of the Turing Test is to stop people from asking "yes, it acts indistinguishable from a human but is it human?" "Generating output that looks human" is in fact the entirety of AGI.
- The corpus of written language is full of ambiguity and contradictory statements.
- A lie makes it's way half way around the world before the truth can even get its pants on.
- What's thought true today will not be thought true tomorrow. This happens sometimes in the direction of veracity. Sometimes, the other way around.
- Factuality and consensus are not the same.
Never said it was easy. :P
> - The corpus of written language is full of ambiguity and contradictory statements.
Right, but the truth is the one set of information that logically cannot be contradictory. That gives it an advantage in terms of compression.
The rest is correct, but just means that the learning algorithm has a harder time discovering truth, not that it's impossible.
> Never said it was easy.
My point is that this process of determination happens over time and in a non-linear fashion, so the corpus contains tons of noise around any given truth statement.
> > The corpus of written language is full of ambiguity and contradictory statements.
> Right, but the truth is the one set of information that logically cannot be contradictory.
Please refer to Godel's incompleteness theorem.
- Another human, named Programmer, generate some solution based on his expirence.
- ProductDesigner, refines the spec to get closer to "the thing he has in mind"
- Programmer, while annoyed, tries to his best effort to write a close solution
- And we call it iteration of development
Is it really that different from your experience? Ok, maybe not your experience, but in the world of outsourcing works it's really kinda like that.
The solution to the above is never "ProductDesigner should learn to speak a deterministic language." It's "when Programmer feels the spec is too ambiguous, he should give feedback to ProductDesigner."
You can't eliminate the "vagueness of natural languages" from the equation. The only differences are, to me, 1) AI is really not that smart yet. 2) AI doesn't admit it doesn't know something. Humans are reluctant to admit it as well, but as least we do it when it's necessary.
The other day I was trying to find out what percentage of Konstantin Tsiolkovsky's works were published before the age of 60 vs after the age of 60, something that's almost impossible to find on the internet (I don't speak Russian). I spent a lot of time trying to get Bing to answer it for me (didn't succeed), and one of the times I asked it to look up all the bibliographic information it could on Tsiolkovsky, go through each file and count the dates of publication, then tell me how many were after his 60th year. It said "Sorry, that would require me to download and analyse a large number of PDFs, which is outside my current capabilities. I'm sorry I couldn't be of more assistance this time. "
They've improved a lot on the Yes Man aspect of it.
3) This is why we developed the programmer personality that loves to tell the designers how they haven't thought this thing through very well yet!
The human here is acting as a compiler for the real world. In each iteration, it's raising exceptions and errors for the AI. The iterations will continue until the AI compiles the correct output.
Yup. Maybe even harder.
Ditto requirements gathering, analysis, acceptance testing, adoption (evangelism), etc. All that squishy organizational psychology stuff.
Coding is the easy part.
A programming language!
[0] https://causalinf.substack.com/p/prompts-are-the-wands-of-gp...
In reality, Human only understand a fraction of the necessary solution (maybe even a fraction of the full problem!) when they begin to program the solution. A big part of the iteration is discovering things the Human did not know about the problem or the necessary solution. Eventually we converge on A solution, which may not look very much like the originally envisioned solution at all.
So no, I do not think the LLM is functioning as a transpiler for english. I think its a tool providing feedback in a necessary iterative cycle or discovery and development. The iteration itself is the "transpiler for english", in a crude way.
That way you're gaining productivity from the all the code generation the LLM does for you and you have final say on all the committed code, it seems like a win-win.
We already have, they’re called programming languages
I was wondering the submit pic option on GPT-4 actually would be this well and perhaps out HTML/CSS to do it - maybe not possible today, haven't tried it yet.
Or who knows - maybe it will be be a like a senior product design who will just figure it all out, but that would require a long convo / submitting a design brief / customer profiles, etc.
- Human describes the problem specs in vague English
- LLMs generate some solution based on the texts it has seen (LLMs may generate non-factual information here but assume the output is factual)
- Human refines the prompt to get closer to "the thing he has in mind"
- LLMs still tries _to the best of its effort_ generate a "close" solution
- and we loop here
Assume the specs of the problem is fixed. Why LLMs couldn't generate the output the Human "had in mind"?
Because English description of the problem was not deterministic?
What can we do? Invent a language for automation specification. Then LLMs can accurately churn up a solution in one shot.
But..that "deterministic language" is what programming languages are. So LLMs at this current state are basically a transpiler from rough-English to a deterministic-English.
But, can LLMs be thought only the specs of a programming language (like C) then generate assembly code from a valid C code? In effect, can LLMs be exact compilers not English transpilers?