This is the main bottle neck, in my kind. A lot of people are missing from the conversation because they don't understand AI fully. I keep getting glimpses of ideas and possibilities and chatting through a browser ain't one of them. On e we have more young people trained on this and comfortable with the tech and understanding it, and existing professionals have light bulbs go off in their heads as they try to integrate local LLMs, then real changes are going to hit hard and fast. This is just a lot to digest right now and the tech is truly exponential which makes it difficult to ideate right now. We are still enveloping the productivity boost from chatting.
I tried explaining how this stuff works to product owners and architects and that we can integrate local LLMs into existing products. Everyone shook their head and agreed. When I posted a demo in chat a few weeks later you would have thought the CEO called them on their personal phone and told them to get on this shit. My boss spent the next two weeks day and night working up a demo and presentation for his bosses. It went from zero to 100kph instantly.
I can benchmark the quality of one LLM's translation by asking another to critique it. It's not infallible, but the ability to chat with a multilingual agent is brilliant.
It's a new tool in the toolbox, one that we haven't had in our seventy years of working on computers, and we have seventy years of catchup to do working out where we can apply them.
It's also just such a radical departure from what computers are "meant" to be good at. They're bad at mathematics, forgetful, imprecise, and yet they're incredible at poetry and soft tasks.
Oh - and they are genuinely useful for studying, too. My A Level Physics contained a lot of multiple choice questions, which were specifically designed to catch people out on incorrect intuitions and had no mark scheme beyond which answer was correct. I could just give gpt-4o a photo of the practice paper and it'd tell me not just the correct answer (which I already knew), but why it was correct, and precisely where my mental model was incorrect.
Sure, I could've asked my teacher, and sometimes I did. But she's busy with twenty other students. If everyone asked for help with every little problem she'd be unable to do anything else. But LLMs have infinite patience, and no guilt for asking stupid questions!
Do you speak two or more languages? Anyone that does is wary of automated translations, especially across estranged cultures.
> It's a new tool in the toolbox, one that we haven't had in our seventy years of working on computers,
It's data analysis at scale, and reliant on scrapping what humans produced. A word processor does not need TB of eBooks to do it's job.
> They're bad at mathematics, forgetful, imprecise, and yet they're incredible at poetry and soft tasks.
Because there's no wrong or right about poetry. Would you be comfortable having LLMs managing your bank account?
> If everyone asked for help with every little problem she'd be unable to do anything else.
That would be hand-holding, not learning.
I'm aware that it's imperfect. It's still pretty cool that they're multilingual as an emergent property - and rather than merely translating, you can discuss aspects of another language. Of course it hallucinates - that's the big problem with LLMs - but that doesn't make it useless. Besides, while automated translators aren't perfect, they're the only option in a lot of situations.
> It's data analysis at scale, and reliant on scrapping what humans produced. A word processor does not need TB of eBooks to do it's job.
And I'm not replacing word processors with LLMs, nor did I claim that they were trained in a vacuum.
> Because there's no wrong or right about poetry. Would you be comfortable having LLMs managing your bank account?
No, which is why I don't intend to... they're probabilistic and immensely fallible. I wasn't claiming they were gods. My point was that they're far outside of what we usually use computers for (e.g. managing your bank account), and that opens a lot of possibilities.
> That would be hand-holding, not learning.
Correct. That's why we don't do it... and in normal circumstances, it'd probably be better to deeply consider the problem in order to work out why you were wrong. But a week before the exam it's excellent. I got a healthy A*, so it doesn't seem to have hurt.
What exactly are you arguing against? I'm not convinced they're a route to AGI either, and I'm not about to replace my graphics driver with an LLM, nor code written by one - but you seem to have a vendetta against them that's lead to you sidestepping every single point I made with a snide remark against claims you seem to have imagined me making.
It's a token prediction machine. We've already generated most of the ideas for it, and hardly any of them work because see below
> Getting AI to do work involves getting AI to understand what needs to be done from highly bandwidth constrained humans using mouse / keyboard / voice to communicate.
No. Gettin AI to work you need to make an AI, and not a token prediction machine which, however wonderful:
- does not understand what it is it's generating, and approaches generating code the same way it approaches generating haikus
- hallucinates and generates invalid data
> Anyone using a chatbot already has felt the frustration of "it doesn't get what I want". And also "I have to explain so much that I might as well just do it myself"
Indeed. Instead of asking why, you're wildly fantasizing about running out of ideas and pretending you can make this work through other means of communication.
The model is doing the exact same thing when it generates "correct" output as it does when it generates "incorrect" output.
"Hallucination" is a misleading term, cooked up by people who either don't understand what's going on or who want to make it sound like the fundamental problems (models aren't intelligent, can't reason, and attach zero meaning to their input or output) can be solved with enough duct tape.
But true, it's just the output.
Perhaps less than before, but still making very fundamental errors. Anything involving number I'm automatically suspicious. Pretty frequently I'd get different answers for the same question (to a human).
e.g. ChatGPT will give an effective tax rate of n for some income amount. Then when asked to break down the calculation will come up with an effective tax rate of m instead. When asked how much tax is owed on that income will come up with a different number such that the effective rate is not n or m.
Until this is addressed to a sufficient degree, it seems difficult to apply to anything that involves numbers and can't be quickly verified by a human.
But. Try this approach instead: have it generate python code, with print statements before every bit of math it performs. It will write pretty good code, which you then execute to generate the actual answer.
Simpler example: paste in a paragraph of text, ask it to count the number of words. The answer will be incorrect most of the time.
Instead, ask it to out each word in the text in a numbered list and then output the word count. It will be correct almost always.
My anecdotal learning from this:
LLMs are pretty human-like in their mental abilities. I wouldn't be able to simply look at some text and give you an accurate word count. I would point my finger / cursor to every word and count up.
The solutions above are basically giving LLMs some additional techniques or tools, very similar to how a human may use a calculator, or count words.
In the products we've built, there is an AI feature that generates aggregations of spreadsheet data. We have a dual unittest & aggregator loop to generate correct values.
The first step is to generate some unittests. And in order to generate correct numerical data for unittests, we ask it to write some code with math expressions first. We interpret the expressions, and paste it back into the unittest generator - which then writes the unittests with the correct inputs / outputs.
Then the aggregation generator then generates code until the generated unittests pass completely. Then we have the code for the aggregator function that we can run against the spreadsheet.
Takes a couple of minutes, but pretty bulletproof and also generalizable to other complex math calculations.
Programming is codified applied mathematics and involves numbers in all but the most trivial programs.
> LLMs are pretty human-like in their mental abilities.
LLM's are algorithms. Algorithms do not have "mental abilities", people do. Anthropomorphizing algorithms only serves to impair objective analysis of when they are, or are not, applicable.
> In the products we've built, there is an AI feature that generates aggregations of spreadsheet data. We have a dual unittest & aggregator loop to generate correct values.
> The first step is to generate some unittests. And in order to generate correct numerical data for unittests, we ask it to write some code with math expressions first. We interpret the expressions, and paste it back into the unittest generator - which then writes the unittests with the correct inputs / outputs.
> Then the aggregation generator then generates code until the generated unittests pass completely. Then we have the code for the aggregator function that we can run against the spreadsheet.
How is this not a classic definition of overfitting[0]?
Or is the generated code intentionally specific to, and only applicable for, a single spreadsheet?
Actually, in this case, comparing how we as humans think to how LLM's work is in fact useful. It's hard for us to eyeball a word and say how many consonants are in it, we need to count. I wouldn't ask a human to eyeball a tax return and tell me what the totals are reliably without giving them the tools to add things up. LLM's are the same way.
It's true that anthropomorphizing in general can be a trap, but when working with LLM's it can be a useful guide in pointing the way towards workable solutions.
Agreed. Contemplating the difference between what people and LLM's are is very useful IMHO. Understanding this is key to making informed decisions as to when LLM's can provide real value.
The assertion originally proffered, however, is quite different than your nuanced perspective:
> > LLMs are pretty human-like in their mental abilities.
It is this to which I object.
I don't understand that. Can you expand on it?
When does the AI or the programmer need to do math calculations?
I actually think you would, lots of time I've been surprised by how accurate people can eye-ball stuffs.
Actually this ability is at very core of human brain. Yet we humans traded most of it for the ability to speak better. Nevertheless, brain can still read very fast as well as count objects/words really fast, but as you've never really trained, this part of the brain is optimised a lot to do other stuff. Try scanning the texts diagonally, and try to come up with a random word count and what this text is meaning. On the first tries you will be making a lot of errors, but eventually, just 100 hours of training (much less when you are kid) and you will be able to scan the texts in seconds and extract the meaning and the word count very accurately. This is a real technique people use for reading fast.
The thing is – the best and most useful ability of human brain is to adapt, from the very moment it adapted to the harsh reality of being expelled from trees and living on the ground – that's how language was created in the first place, as a result of this adaptation. LLM's can't adapt today, nor doesn't need to adapt, so their mental abilities will never come close to the brain which was adapting for millions of years. Something which is not adapting will become obsolete and come to it's end, this is a fundamental law.
Yes.
Suppose someone developed a way to get a reliable confidence metric out of an LLM. Given that, much more useful systems can be built.
Only high-confidence outputs can be used to initiate action. For low-confidence outputs, chain of reasoning tactics can be tried. Ask for a simpler question. Ask the LLM to divide the question into sub-questions. Ask the LLM what information it needs to answer the question, and try to get that info from a search engine. Most of the strategies humans and organizations use when they don't know something will work for LLMs. The goal is to get an all high confidence chain of reasoning.
If only they knew when they didn't know something.
There's research on this.[4] No really good results yet, but some progress. Biggest unsolved problem in computing today.
[4] https://hungleai.substack.com/p/uncertainty-confidence-and-h...
Of course, citing sources would always trump any kind of confidence rating in my mind; sources provide provenance, while confidence rating can be fudged just like a bogus answer can.
it does 2 things. 1 tells me how deep i am in the conversation and 2 when the computation falls apart, i can assume other things in its response will be trash as well. and sometimes number 3 how well the software is working. ranges from 20$ to 35$ on average but a couple days they go deep to 45$ (4,7,9 responses in chat session)
today i learned i can knock the computation loose in a session somewhere around the 4th reply or $20 by injecting a random number in my course work and it was latching onto the number instead of computing $
I'm not keen on marketing words like "superintelligence" but boiling it down that's what in my mind the OP said. These systems are limited in ways that we do not yet fully appreciate. They are not silver bullets for all or maybe even many problems. We need to figure out where they can be deployed for greater benefit.
Totally. They are large language models, not math models.
I think the problem is that 'some people' overhype them as universal tools to solve any problem, and to answer any question. But really, LLMs excel in generating pretty regular text.
Once we have actual super intelligence there is no need for humans to innovate anymore. It is by definition better than us anyway.
I guess you could still have artisanal innovation
Computers might do some things “better” than humans, but we’re still going to do things because we _want_ to. Often the fun is in the doing.
LLM’s can vomit out code faster than me, I still enjoy writing software. They’ll vomit out a novel or a song, but I still like reading/listening to stuff a human has taken time and effort to create something, because they can.
In other words, in another area: you can take all the art made in the world before Guernica and throw it in any probabilistic network but for sure you won't get Guernica out of that.
> Human civilization is most likely not alone in the cosmos but is instead encompassed within a cosmic host...
Nah, I'm good. I don't need some guy philosophizing about AI being our pathway to join a galactic super-civilization.
When AI is actually doing something useful for humanity, call me back.
Is this because it's actually making less mistakes, or is it just because most people have used it enough now to know not to bother with anything complex?
Your paycheck depends on people believing the hype. Therefore, anything you say about "superintelligence" (LOL) is pretty suspect.
> Getting AI to do work involves getting AI to understand what needs to be done from highly bandwidth constrained humans using mouse / keyboard / voice to communicate.
So, what, you're going to build a model to instruct the model? And how do we instruct that model?
This is such a transparent scam, I'm embarrassed on behalf of our species.
2. The original post is, to my eyes, not implying anything about training a model to help use other models, so the second part seems irrelevant.
Your microwave is more capable than a typical human.
If of course your definition of capabilities is narrowly defined to computations and ignores the huge array of things humans can do that computers are no where close to.
For example?
For example, running a lemonade stand.
You'd need thousands, if not millions of dollars to build a robot machine with a computerized brain capable of doing what a 6 year old child can do - produce lemonade from simple ingredients (lemon, sugar, water) and sell it to consumers.
Same with basically all cooking/food-service and hospitality tasks, an physical therapy type tasks (massage, chiropractor, etc)...
heck, even driving on public roads still doesn't seem to be perfect, despite 10+ years on investment and research by leading tech companies, although there is also a regulatory hurdle here.
And, more to the point, they aren't indicative of intelligence. Computers have cleared the intelligence requirements to run a lemonade stand by a large margin - and the other tasks too for that matter.
One could travel between continents in minutes on an ICBM with a reentry vehicle bolted to the front but we don't because it's too expensive. It's a perfectly reasonable constraint to demand that a technology be cost effective. Otherwise it has no practical value.
By that logic, libraries, art museums and calculators are "super intelligent"
This is where I lost the plot. The techno-futurists always seem to try to co-opt Moore's law or similar scaling laws and claim it'll somehow take care of whatever scifi bugaboo du jour they're peddling, without acknowledging that Moore's law is specifically about transistor density and has nothing to do with "strides" in "AI".
> whatever super-intelligence is can't be far away.
How do you figure? Or is it just an article of your faith?
It's been the same old, tired argument for seven decades--"These darned computers can count real real fast so therefore any day now they'll be able to think for themselves!" But so far nobody's shown it to be true, and all the people claiming it's right around the corner have been wrong. What makes this time different?
https://epochai.org/blog/training-compute-of-frontier-ai-mod...
https://ourworldindata.org/grapher/test-scores-ai-capabiliti...
We're still seeing exponential upswing in compute and we appear to already be probing around human capacity in the models. Past experience suggests that once AIs are within spitting distance of human ability they will exceed what a human mind can do in short order.
These are not subtle trends.
EDIT: re: the computer time metric, by the same token shouldn't block chains have changed the world by now if computer time is the predictor of success? It makes sense for the industry proponents of LLMs to focus on this metric, because ultimately that's what they sell. Microsoft, NVidia, Google, Amazon, etc all benefit astronomically from computationally intensive fads, be it chatbot parlor tricks or NFTs. And the industry at large does as well--a rising tide lifts all boats. It's not at all obvious any of this is worth something directly, though.
Fair enough. What do you think is driving the uptick of AI performance and why don't you think it will be correlated with the amount of compute invested?
The limitations business looks like a red herring. Being flawed and limited doesn't even disqualify an AI from being super-intelligent (whatever that might mean). Humans are remarkably flawed and limited, it takes a lot of setup to get them to a point where they can behave intelligently.
> EDIT: re: the computer time metric, by the same token shouldn't block chains have changed the world by now if computer time is the predictor of success?
That seems like it would be a defensible claim if you wanted to make it. One of the trends I keep an eye on is that log(price) is similar to the trend in log(hash rate) for Bitcoin. I don't think it is relevant though because Bitcoin isn't an AI system.
Some examples to illustrate the point: supply chain risks and an ever increasing amount of dependencies (look at your average React project, though this applies to most stacks), overly abstracted frameworks (how many CPU cycles Spring Boot and others burn and how many hoops you have to jump through to get thigns done), patterns that mess up the DBs ability to optimize queries sometimes (EAV, OTLT, trying to create polymorphic foreign keys), inefficient data fetching (sometimes ORMs, sometimes N+1), bad security practices (committed secrets, anyone? bad usage of OAuth2 or OIDC?), overly complex tooling, especially the likes of Kubernetes when you have a DevOps team of one part time dev, overly complex application architectures where you have more more services than developers (not even teams). That's before you even get into the utter mess of long term projects that have been touched by dozens of developers over the years and the whole sector sometimes feeling like wild west, as opposed to "real engineering".
That's why articles like this ring true: http://www.stilldrinking.org/programming-sucks
However, the difference here is that I wouldn't overwhelm anyone who might give me money with rants about this stuff and would navigate around those issues and risks as best I can, to ship something useful at the end of the day. Same with having constructive discussions about any of those aspects in a circle of technical individuals, on how to make things better.
Calling the whole concept a "scam" doesn't do anyone any good, when I already derive value from the LLMs, as do many others. Look at https://www.cursor.com/ for example and consider where we might be in 10-20 years. Not AGI, but maybe good auto-complete, codegen and reasoning about entire codebases, even if they're hundreds of thousands of lines long. Tooling that would make anyone using it more productive than those who don't. Unless the funding dries up and the status quo is restored.
I think the testimonies often repeated by coders that use these code completion tools is that "it saved me X amount of time on this one problem I had, therefore it's great value". The issue is that these all fall into a research of n=1 test subjects. It's only useful information for the subject. It appears we don't realize in these moments that when we use those examples, even to ourselves, we are users reviewing a product, as opposed to validating if our workflow is not just different but objectively better.
The truth lies in the aggregate data of the quality and crucially the speed by which fixes and requirements are being implemented at scale across code bases.
Admittedly, a lot of code is being generated, so I don't think I can say everyone hates it, but until someone can do some real research on this, all we have are product reviews.
To me it seems very much like we're somewhere near the peak of the hype cycle: https://en.wikipedia.org/wiki/Gartner_hype_cycle
Except in the case of "AI" we get new releases that seem somewhat impressive and therefore extend the duration for which the inflated expectations can survive. For what it's worth, stuff like this is impressive https://news.ycombinator.com/item?id=41693087 (I fed my homepage/blog into it and the results were good, both when it came to the generated content and the quality of speech)
> The truth lies in the aggregate data of the quality and crucially the speed by which fixes and requirements are being implemented at scale across code bases.
Honestly? I think we'll never get that, the same way I cannot convincingly answer "How long will implementing functionality X in application Y with the tech stack Z for developer W take?"
We can't even estimate tasks properly and don't have metrics for specific parts of the work (how much creating a front end takes, how much for a back end API, how much for the schema and DB migrations, how much for connecting everything, adding validations, adding audit, fixing bugs etc.) because in practice nobody splits them up in change management systems like Jira so far, nor are any time tracking solutions sophisticated enough to figure those out and also track how much of the total time is just procrastination or attending to other matters (uncomfortable questions would get asked them, way too metrics would be optimized for).
So the best we can hope for is some vague "It helps me with boilerplate and repeatable code which is most of my enterprise CRUD system by X% and as a result something that would take me Y weeks now takes me Z weeks, based on these specific cases." Get enough of those empirical data points and it starts to look like something useful.
I think lots of borderline scams and/or bad products based on overblown products will get funded but in a decade we'll probably have mostly those sticking around that have actual utility.
> don't know what I would use a podcast like this for, but the fact that something like this can be created without human intervention in just a few minutes is jaw dropping
AI has recently gotten good at doing stuff that seems like it should be useful, but the limitations aren’t obvious. Self driving cars, LLM’s, Stable Diffusion etc are awesome tech demos as long as you pick the best output.
The issue is the real world cares a lot more about the worst outcomes. Driving better than 99% of people 24/7 for 6 months and then really fucking up is indistinguishable from being a bad driver. Code generation happens to fit really well because of how people test and debug code not because it’s useful unsupervised.
Currently balancing supervision effort vs time saved depends a great deal on the specific domain and very little about how well the AI has been trained, that’s what is going to kill this hype cycle. Investing an extra 100 Billion training the next generation of LLM isn’t going to move the needles that matter.
This is just another way of saying this technology, like Blockchain before it, is a solution in search of a problem.
Remember the old bit about the media-- the stories are always 100% infalliable except strangely in YOUR personal field of expertise.
I suspect it's something similar with AI products.
People test them with toy problems -- "Hey ChatGPT, what's the square root of 36", and then with something close to their core knowledge.
It might learn to solve a lot of the toy problems, but plenty of us are still seeing a lot of hallucinations in the "core knowledge" questions. But we see people then taking that product-- that they know isn't good in at least one vertical-- and trying to apply it to other contexts, where they may be less qualified to validate if the answer is right.
For me, the number of times where it's led me down a hallucinated, impossible, or thoroughly invalid rabbit hole have been relatively minimal when compared against the number of times when it has significantly helped. I really do think the key is in how you use them, for what types of problems/domains, and having an approach that maximizes your ability to catch issues early.
Gell-Mann amnesia:
https://en.wikipedia.org/wiki/Michael_Crichton#GellMannAmnes...
Won't that be fun.
Ehm yes. That's because it actually doesn't work as well as the hype suggests, not because it's too "high bandwidth".
Which tells me the problem may be fundamental, not a technical one. It's not just a matter of needing "more intelligence". I don't question the intelligence or skill of the people on the outsourced team I was working with. The problem was simple communication. They didn't really know or understand our business and its goals well enough to anticipate all sorts of little things, and the lack of constant social interaction of the type you typically get when everybody's a direct coworker meant we couldn't build that mind-meld over time, either. So we had to pick up the slack with massive over-specification.
Left to their own devices they often implemented requirements that were full of errors or even completely unnecessary because they did not understand the domain well enough to ask pointed questions.
the problem is really, can it learn "I need to turn this over to a human because it is such an edge case that there will not be an automated solution."
Makes sense.
I find this funny because for what I use ChatGPT for - asking programming questions that would otherwise go to Google/StackOverflow - I have a much better time writing queries for ChatGPT than Google, and getting useful results back.
Google will so often return StackOverflow results that are for subtly very different questions, or ill have to squint hard to figure out how to apply that answer to my problem. When using ChatGPT, i rarely have to think about how other people asked the question.
SOTA models are impressive, as is the idea of building AGIs that do everything for us, but in the meantime there are a lot of practical applications of the open source and smaller models that are being missed out on in my opinion.
I also think business is going to struggle to adapt and existing business is at a disadvantage for deploying AI tools, after all, who wants to replace themselves and lose their salary? Its a personal incentive not to leverage AI at the corporate level.
We talk about scaling laws, superintelligence, AGI etc. But there is another threshold - the ability for humans to leverage super-intelligence. It's just incredibly hard to innovate on products that fully leverage superintelligence.
At some point, AI needs to connect with the real world to deliver economically valuable output. The ratelimiting step is there. Not smarter models.
In my mind, already with GPT-4, we're not generating ideas fast enough on how best to leverage it.
Getting AI to do work involves getting AI to understand what needs to be done from highly bandwidth constrained humans using mouse / keyboard / voice to communicate.
Anyone using a chatbot already has felt the frustration of "it doesn't get what I want". And also "I have to explain so much that I might as well just do it myself"
We're seeing much less of "it's making mistakes" these days.
If we have open-source models that match up to GPT-4 on AWS / Azure etc, not much point to go with players like OpenAI / Anthropic who may have even smarter models. We can't even use the dumber models fully.