Can you please explain more why you think so ?
Thank you.
And just as every other hype cycle, this one will crash down hard. The crypto crashes were bad enough but at least gamers got some very cheap GPUs out of all the failed crypto farms back then, but this time so much more money, particularly institutional money, is flowing around AI that we're looking at a repeat of Lehman's once people wake up and realize they've been scammed.
Isn't that an inherent problem with pretty much everything nowadays: crypto, blockchain, AI, even the likes of serverless and Kubernetes, or cloud and microservices in general.
There's always some hype cycle where the people who are early benefit and a lot of people chasing the hype later lose when the reality of the actual limitations and the real non-inflated utility of each technology hits. And then, a while later, it all settles down.
I don't think the current "AI" is special in any way, it's just that everyone tries to get rich (or benefit in other ways, as in the microservices example, where you still very much had a hype cycle) quick without caring about the actual details.
As someone who loves to pour ice water on AI hype, I have to say: you can't be serious.
The current AI tech has opened up paths to develop applications that were impossible just a few years ago. Even if the tech freezes in place, I think it will yield substantial economic value in the coming years.
It's very different from crypto, the main use case for which appears to be money laundering.
Which has substantial economic value (for certain groups of people).
Roughly 11-12% of GDP.
In many countries, black+grey market is larger than the ‘white’ market. The US is notoriously ‘clean’ compared to most (probably top 10).
Even in the US, if you suddenly stopped 10-12% of GDP we’re talking ‘great depression’ levels of economic pain.
Honestly, the only reason Crypto isn’t bigger IMO is because there is such a large and established set of folks doing laundering in the ‘normal’ system, and those work well enough there is not nearly as much demand as you’d expect.
Correct me if I'm wrong. Doesn't the public ledger make it really bad for this purpose when enough is known about some of the actors interacting on the chain here? It's basically an exact recording that everybody can see, of who interacted with whom, based on pseudonyms, right?
Cash is superior in the ability to have intermediate transactions hidden. However it has similar issues in that bills are serialized, and I believe scanned/tracked - or at least there is the capability to do so. So in theory, when moving it out of and back into the system, it can be tied to real identities not just pseudonyms too.
Though frankly, it seems like no one really cares most of the time.
My argument is that if it's advertised as a direct precursor to AGI based on wishful thinking and people don't know any better, then it's no different to claims about how putting blockchain technology in X industry will solve all of its problems.
I use LLMs daily and don't scoff at AI generated imagery or use cases like agentic systems, but there absolutely is a similar hype cycle to every other innovation out there where people are borderline delusional in the initial stages (Kubernetes will solve all of our issues, moving to cloud and microservices will solve all of our issues, the blockchain will...), before the limitations crystallize and we know what each technology is good or bad at.
Though maybe that says more about human nature than the technology itself.
> It's very different from crypto, the main use case for which appears to be money laundering.
That's akin to saying "The main use case for AI appears to be stealing people's art and even for writers and others it seems to be firing people to replace them with soulless AI generated slop."
I'd even argue that there's nothing wrong with the technologies themselves, be it LLMs, AI for image, video, audio generation, blockchain and crypto, or whatever. The problems arise based on how the technologies are used, or in my argument above - how they're touted as the solution to all the problems. Some people profit a lot, others collide with reality and their limitations at speed.
In other words, if the technology will generate 100 billion USD of actual value but people are betting on 500 billion USD, then clearly we have a bit of an issue.
You say tomato, I say freedom from the tyranny of fiat power structures.
The question is, where will this "economic value" be? Because "economic value" and actual progress that helps society are two very different things. For example, if someone wants to hire people, they can use "AI" to sift through the applications. But people looking for a job can also use "AI" to write their applications. In the end you may have created "economic value", but its an arms race and a waste of resources at the core, more digital paperwork, a waste of compute. So the actual value is not positive, it is negative. And we see that in many places where this so called "AI" is supposed to help.
Does it mean it is entirely useless? No, but the field of applications where it actually makes sense and has an overall net benefit is way smaller than many believe.
Plus, there are different types of neural networks in use for decades already. Look at OCR, for example, where the commercial OCR software switched to neural networks around the mid 90s already. So it is not that "AI" as such is bad, just that this generative neural network stuff is overly hyped by people who have absolutely no clue about it, but have to hop on the bandwagon to not be left out and keep shareholder values up, because most of these shareholders are equally stupid about the whole issue. Its a circus that burns many resources, money that could have created way more value in other areas.
*in environments and projects where they are a good fit
> Microservices is just an excuse to build shit software.
*in environments and projects where they are a bad fit
Suppose you turn out to be wrong. What would convince you?
(I know that it's generally rude to include LLM output in HN comments, but in this case I think it's essential supporting material to elevate the discussion of LLM capabilities above "yes it is", "no it isn't".)
First, I just asked ChatGPT (GPT-4o mini, what they offer anonymous free users):
> Please diagram the sentence, "If my mother's road had been cultural, my uncle's sloth squad residue would have compiled a periodic meadow of night."
I just invented that sentence, picking several words with a random number generator, but of course I can't prove to you that I didn't copy it from some book you haven't seen. (For that, you'd have to invent a sentence yourself, or at least provide me with some words to include in it.) I hope you can at least agree that it's a highly unlikely sentence for anyone ever to have uttered before!
After talking to itself for a few paragraphs, it arrived at the following grammatical analysis:
Step 4: Diagramming the sentence
Main Clause:
Subject: "my uncle's sloth squad residue"
Verb: "would have compiled"
Direct Object: "a periodic meadow of night"
"a periodic meadow" (noun phrase)
"of night" (prepositional phrase modifying "meadow")
Conditional Clause:
Subject: "my mother's road"
Verb: "had been" (linking verb)
Predicate Adjective: "cultural"
This is exactly correct. From that it produced the following ASCII art "diagram", which looks like nonsense to me, but maybe it's in some kind of sentence-diagramming formalism I haven't seen: If
______/______
| |
my mother's had been
road |
cultural
my uncle's sloth squad residue
|
would have compiled
|
a periodic meadow of night
|
of
night
I told it the diagram looked like nonsense, and it produced this one, which looks worse: If
_______/______
| |
my mother's road had been
| |
cultural (Main Clause)
|
my uncle's sloth squad residue
|
would have compiled
|
a periodic meadow
|
of night
I asked GPT-4 (paid) the same question, and it gave me another exactly correct grammatical analysis in words: 1. **Identify the main clauses**:
- "If my mother's road had been cultural" is a dependent clause (specifically, a conditional clause).
- "my uncle's sloth squad residue would have compiled a periodic meadow of night" is the main clause.
2. **Start with the main clause**:
- Subject: "my uncle's sloth squad residue"
- "my uncle's" modifies "sloth squad," which in turn modifies "residue."
- Predicate: "would have compiled"
- This includes the modal auxiliary verb ("would"), perfect auxiliary ("have"), and main verb ("compiled").
- Direct object: "a periodic meadow of night"
- “a” is an article modifying “meadow,”
- “periodic” modifies “meadow,”
- “of night” is a prepositional phrase modifying “meadow.”
3. **Diagram the dependent clause**:
- Subject: "road"
- Modified by possessive noun phrase “my mother’s”
- Predicate: "had been"
- Complement/Adjective: "cultural"
(I feel like this is definitely "a level that you'd assume to be competitive to a human", in mschuster91's phrase; most native English speakers can't do that.)But then it came up with the following incorrect diagram:
+---------------------------------------------------+
| Main Clause |
+---------------------------------------------------+
/ | \
/ | \
/ | \
[My uncle's] [sloth squad] [residue]
\ / /
\ / /
\ / /
would have compiled -------------------
\
\
\
[a periodic meadow of night]
/ | \
/ | \
a periodic [of night]
+---------------------+
| Dependent Clause |
+---------------------+
If
|
road
___|_____
| |
my mother's had been cultural
Telling it the diagram was nonsense didn't help.But my conclusion from this is not that GPT-4 can't analyze the grammatical structure of sentences it had never seen. It's that GPT-4 gets "confused" when it tries to draw ASCII art, like a person trying to tell you how to draw it over the telephone, but not quite as bad. I've seen similarly abominable results when it tried to draw circuit schematics and guitar tablatures as ASCII art. I tried asking it for a graphviz diagram, which was syntactically correct graphviz code but didn't express the sentence structure properly.
I tried simplifying the problem with the prompt, 'Please provide a syntactic tree showing the syntactic derivation of the sentence, "My uncle's sloth squad residue would have compiled a periodic meadow of night," using graphviz syntax and with individual words at the leaf nodes of the tree.' which yielded a slightly incorrect diagram. (I'm still sure nobody has ever uttered this sentence before.) On being told, "This diagram is not quite right.", GPT-4 produced the following diagram, which I believe is correct except for an extra "s" on "uncle" (you can see this in your browser at https://is.gd/Axufjx):
digraph G {
node [shape=plaintext];
S [label="S"];
NP [label="NP"];
VP [label="VP"];
S -> NP;
S -> VP;
// Noun Phrase breakdown
DP_my [label="DP"];
DP_my -> My;
UnclePossessive [label="NP"];
UnclePossessive -> Uncles;
UnclePossessive -> ApostropheS;
NP -> DP_my;
NP -> UncleCompound;
UncleCompound [label="NP"];
SlothSquadResidue[label = "N'"];
SlothSquadResidue-> Sloth;
SlothSquadResidue-> SquadResidue;
SquadResidue[label = "N'"]
SquadResidue-> Squad;
SquadResidue-> Residue;
UncleCompound-> UnclePossessive;
UncleCompound-> SlothSquadResidue;
// Verb Phrase breakdown
ModalWould[label = "Modal"];
ModalWould-> Would;
AuxHave[label = "Aux"];
AuxHave-> Have;
VCompiled[label = "V"];
VCompiled-> Compiled;
ObjectNP[label = "NP"]; // Direct object noun phrase
ArticleA[label = "D"];
ArticleA-> A;
MeadowPhrase[label = "N'"]; // 'Meadow' modified by 'periodic'
AdjPeriodic[label = "Adj"];
AdjPeriodic-> Periodic;
MeadowWord[label = N];
MeadowWord -> Meadow;
MeadowPhrase -> AdjPeriodic;
MeadowPhrase -> MeadowWord;
PPofNight[label = PP]; // Prepositional phrase 'of night'
PrepOf[label=P]
PrepOf-> Of;
NightNounPhrase[label=N]
NightNounPhrase-> Night;
PPofNight -> PrepOf;
PPofNight -> NightNounPhrase;
ObjectNP -> ArticleA;
ObjectNP -> MeadowPhrase;
ObjectNP -> PPofNight;
VP -> ModalWould;
VP -> AuxHave;
VP -> VCompiled
VP -> ObjectNP
}
Claude produced a reasonably serviceable diagram on the first try with that prompt: digraph SyntacticTree {
node [shape=plaintext];
S -> NP1;
S -> VP;
NP1 -> DET1 [label="Det"];
NP1 -> NP2 [label="NP"];
NP2 -> NP3 [label="NP"];
NP2 -> N1 [label="N"];
NP3 -> N2 [label="N"];
NP3 -> POS [label="Pos"];
VP -> AUX1 [label="Aux"];
VP -> AUX2 [label="Aux"];
VP -> V [label="V"];
VP -> NP4 [label="NP"];
NP4 -> DET2 [label="Det"];
NP4 -> ADJ [label="Adj"];
NP4 -> N3 [label="N"];
NP4 -> PP [label="PP"];
PP -> P [label="P"];
PP -> NP5 [label="NP"];
NP5 -> N4 [label="N"];
DET1 -> "My";
N2 -> "uncle";
POS -> "'s";
N1 -> "sloth";
N1 -> "squad";
N1 -> "residue";
AUX1 -> "would";
AUX2 -> "have";
V -> "compiled";
DET2 -> "a";
ADJ -> "periodic";
N3 -> "meadow";
P -> "of";
N4 -> "night";
}
On being told, I think incorrectly, "This diagram is not quite right.", it produced a worse diagram.So LLMs didn't perform nearly as well on this task as I thought they would, but they also performed much better than you thought they would.
__
/ _)
_.----._/ /
/ /
__/ ( | ( |
/__.-'|_|--|_|
Asking for a Tyrannosaurus Rex gives you more or less the same brontosaurs: __
/ _)
_.----._/ /
/ /
__/ ( | ( |
/__.-'|_|--|_|I'm pretty sure there are part-of-speech tagging parsers using fairly shallow statistics that could also have produced an equivalently good sentence diagram. https://corenlp.run/ seems to produce a correct parse, though in a different format.
You just have to be prepared to take a karma hit for it. The audience here does not consist largely of 'hackers', but seems to skew toward the sort of fearful, resentful reactionaries that hacker culture traditionally opposes.
I wouldn't have pegged ChuckMcM as being one of them, though. That's an unpleasant surprise.
That implicit awareness is indistinguishable from knowledge, at least to the extent it yields correct results.
And the fact that it was inferred from examples rather than taught explicitly is indistinguishable from intelligence.
That said, I believe that this is going one of two ways: we use AI to make things materially harder for humans, in a scale from "you don't get this job" to "oops, this is Skynet", with many unpleasant stops in the middle. By the amount of money going into AI right now and most of the applications I'm seeing being hyped, I don't think we have have any scruples with this direction.
The other way this can go, and Cerebras is a good example, is that we increase our compute capability and our AI-usefulness to a point where we can fight cancer and stop/revert aging, both being a computational problem at this point. Even if most people don't realize it, or most people have strong moral objections to this outcome and don't even want to talk about it, so it probably won't happen.
In simpler words, I think we want to use AI to commit species suicide :-)
We are more than Broca's areas. Our intelligence is much more than linguistic intelligence.
However, and this is also an important point, we have built language models far more capable than any language model a single human brain can have.
Makes me shudder in awe of what's going to happen when we add the missing pieces.
Evolution selects for efficiency.
If token prediction could work for everything, our brains would also do nothing else but token prediction. Even the brains of fishes and insects would work like that.
The human brain has dedicated clusters of neurons for several different cognitive abilities, including face recognition, line detection, body parts self perception, 3D spatial orientation, and so on.
I think this is a poor argument here. From an evolutionary point of view, our brains are optimized to:
- Provide fine-motor control to craft weapons and tools (enhancing adaptibility and enabling us to hunt way outside our weight class)
- Communicate/coordinate effectively in small groups
- Do sensor processing and the above with a low energy budget
Our brains are *not* selected to be minimum-complexity intelligences, and a lot of what our brain does is completely useless for AGI building (motor control, sensor processing, ...).
Furthermore, the cost/complexity (from a evolutionary PoV) is a totally different beast from what complexity means to us.
Just consider flight as an example: A fruitfly is an insanely simple and straightforward beast, but to us, a biochemically fuelled, beating-wing design is still infeasibly complicated. If our approach to flight had been to ape after how nature does it in detail, we likely still would not have planes.
I do agree that todays LLMs still have clear architectural flaws that we need to overcome (online learning being a very glaring one), but, to pick up the flight analogy, we might well have the main wing structure already down, and we won't necessarily have to make those wings beat to get into the air...
Doesn't mean that there aren't some part of our brains that are needed for an AGI, and are not present in the current crop of LLM.
They receive information about photons and air vibrations and control muscles, okay. If a human brain was hooked up the way ChatGPT was, only to text input and output, would that make it not intelligent?
I am planning a masters and phd on that question, so give me a few years to answer.
I'd expect an actually "general" intelligence Thing to be able to be as versatile in intellectual tasks as a human is - and LLMs are reasonably decent at repetition, but cannot infer something completely new from the data it has.
Because this statement really makes no sense. Transformers are perfectly capable (and capable of perfectly) learning mathematical functions, given the necessary working-out space, e.g. for long division or for algebraic manipulation. And they can learn to generalise from their training data very well (although very data-inefficiently). That's their entire strength!
Perhaps your "definition" should be simply that LLMs have temporarily seen limitations in their ability to natively do math unassisted by an external memory, but are exceptionally good at very advanced math when they can compensate for their lossy short-term attention memory...
ChatGPT, while being as far from true AGI as the Elisa chatbot written in Lisp, is extraordinarily more useful, and being used for many things that previously required humans to write the bullshit, like lobbying and propaganda.
And Crypto... right now BTC is at an historical highest. It could even go higher. And it will eventually crash again. It's the nature of that beast.
It's certainly the case that an LLM alone cannot achieve AGI. As a component of a larger system though? That remains to be seen. Maybe all we need to do is duct tape a limbic system and memory onto an LLM and the result is something sort of like an AGI.
It's a little bit like saying that a ball bearing can't possibly ever be an internal combustion engine. While true, it's sidestepping the point a little bit.
You said "I would guess you're not asking a serious question here"
Related to AI given all around noise, really wanted to understand kind of contrarian view of monetary aspects.
Once again, apologies if the question seems frivolous.
[1] https://patents.justia.com/assignee/cerebras-systems-inc
This so isn't important to your overall point, but where would I begin to look into this? Sounds fascinating!
Basically the "secret sauce" of the startup recruiting me was that they were going to do wafer scale FPGAs that could be tiled together to build arbitrarily complex systems like military phased array radars and such. All very hush hush but apparently they had recruited some key talent from Xilinx which was annoying Xilinx.
Just like the dotcom bubble, AI is gonna hit, make a few companies stinking rich, and make the vast majority (of both AI-chasing and legacy) companies bankrupt. And it's gonna rewire the way everything else operates too.
This is the part that I think a lot of very tech literate people don't seem to get. I see people all the time essentially saying 'AI is just autocomplete' or pointing out that some vaporware ai company is a scam so surely everyone is.
A lot of it is scams and flash in the pan. But a few of them are going to transform our lives in ways we probably don't even anticipate yet, for good and bad.
A YouTuber watch regularly includes questions they asked Chat GPT and very single time there’s a detailed response in the comments showing how the output is wildly wrong from multiple mistakes.
I suspect the backlash from disgruntled users is going to hit the industry hard and these models are still extremely expensive to keep updated.
New models aren't being trained specifically on single answers which will only help.
The expense for the larger models is something to be concerned about. Small models with function calls is already great, especially if you narrow down what they are being used for. Not seeing their utility is just a lack of imagination.
I dont believe there are any significant academic critiques doubting this. There are a lot of armchair hot takes, and perceptions that this stuff isn't improving up to their expectations, but those are pretty divorced from any rigorous analysis of the field, which is still improving at staggeringly fast rates compared to any other field of research. Aint no wall, folks.
In the real world if you know the correct answer you don’t need to ask the question. A self driving car that needs you to pay attention isn’t self driving.
Any system can get canned response, the value of AI is completely in its ability to handle novelty without hand holding. And none of these systems actually do that even vaguely well in practice rather than providing response that are vaguely close to correct.
If I ask for a summary of an article and it gets anything wrong in the article that’s a 0 because now I need to read the article to know what it said. Arguably the value is actually negative here.
You don't have to pay attention, that's the point. You can code without reading code now. Sure you gotta tell it what the app looks like with each iteration - but again, that's temporary til the next model comes out with good enough vision to assess that itself. None of this is permanently planning on requiring human interaction - it's just early days and these are progressing through mediums one at a time.
They're not canned responses either. They're bespoke mixtures of all the various elements of the current environment/context translated to an answer. It certainly handles novelty - that's the whole point. They certainly handle plenty of novelty - like entire mediums of text and images - to expert levels. I think you're just being greedy for more, here.
As for consistency and avoiding error? There are benchmarks for that. There are error checking methods. Those are all steadily improving too, and are already well-consistent on easier topics/mediums. It would be foolish to think that's innately impossible from AI for remaining ones.
That said, I think they do a great job of exploiting this technique to create a "larger"[1] chip. And like storage it benefits from every core is the same and you don't need to get to every core directly (pin limiting).
In the early 2000's I was looking at a wafer scale startup that had the same idea but they were applying it to an FPGA architecture rather than a set of tensor units for LLMs. Nearly the exact same pitch, "we don't have to have all of our GLUs[2] work because the built in routing only uses the ones that are qualified." Xilinx was still aggressively suing people who put SERDES ports on FPGAs so they were pin limited overall but the idea is sound.
While I continue to believe that many people are going to collectively lose trillions of dollars ultimately pursuing "AI" at this stage. I appreciate the the amount of money people are willing to put at risk here allow for folks to try these "out of the box" kinds of ideas.
[1] It is physically more cores on a single die but the overall system is likely smaller, given the integration here.
[2] "Generic Logic Unit" which was kind of an extended LUT with some block RAM and register support.