Re:Show HN: less than 650 LOC trainable GPT only using NumPy...
by jimmyl02 on Friday November 17, 2023 @01:26PM
(#38307736)
One small difference is that the GPT architecture is just the decoder stack of the original transformer as opposed to the full encoder decoder stack in the original.
I agree the branding play on GPTs in general is pretty smart and strong from OpenAI though.
Re:Show HN: less than 650 LOC trainable GPT only using NumPy...
by cchance on Friday November 17, 2023 @02:12PM
(#38308489)
Honestly i feel like the fact that everyone is just calling LLM's GPT at this point doesn't really help OpenAI, ChatGPT would, but the fact is that unlike "googling" something became synonymous for searching on the internet, GPT != OpenAI-ing something, GPT just became what people call LLM's it seems like lately, the fact the term isn't the name of the company or the full name "chatgpt-ing" sort of breaks that hold i feel like.
Re:Show HN: less than 650 LOC trainable GPT only using NumPy...
by Paul-Craft on Friday November 17, 2023 @10:02PM
(#38314480)
Most of the newer LLMs that have come out are GPTs though.
Re:Show HN: less than 650 LOC trainable GPT only using NumPy...
by tverbeure on Friday November 17, 2023 @01:23PM
(#38307668)
FWIW: the GitHub project description says “GPT-like”. It’s the title here that dropped the “like”.
Re:Show HN: less than 650 LOC trainable GPT only using NumPy...
by __loam on Friday November 17, 2023 @01:36PM
(#38307907)
Regarding regulatory capture, I listened to an interview with Lena Khan, the current head of the FTC, and this exact thing came up as something regulators are worried about. I think regulators are aware of the danger of letting industry insiders regulate their own industry, so I'm hopeful for some sensible regulations that help promote rather than harm competition. The FTC also exists to prevent monopoly.
Re:Show HN: less than 650 LOC trainable GPT only using NumPy...
by PartiallyTyped on Friday November 17, 2023 @01:14PM
(#38307565)
The most interesting thing in this whole saga is that decoder only models (aka causal transformers like GPT) are as effective as they are.
Re:Show HN: less than 650 LOC trainable GPT only using NumPy...
by quickthrower2 on Friday November 17, 2023 @06:17PM
(#38312030)
Is GPT subject to trademark. It stands for Generative Pre-training Transformer?
Re:Show HN: less than 650 LOC trainable GPT only using NumPy...
by joennlae on Friday November 17, 2023 @06:22PM
(#38312085)
They are still applying: https://tmsearch.uspto.gov/bin/showfield?f=doc&state=4805:wl...
Re:Show HN: less than 650 LOC trainable GPT only using NumPy...
by Paul-Craft on Friday November 17, 2023 @10:02PM
(#38314473)
I can't see the search query or results you got from that link, but I did a search as well and found OpenAI applied for GPT-5, GPT-6, and GPT-7, which is no surprise. I'd be surprised if any of those were granted, though, because "GPT" itself is a generic term already in research papers.
Re:Show HN: less than 650 LOC trainable GPT only using NumPy...
by joennlae on Friday November 17, 2023 @06:18PM
(#38312050)
The author here: I absolutely agree with you. I went for a bit more catchy title.
Re:Show HN: less than 650 LOC trainable GPT only using NumPy...
by Paul-Craft on Friday November 17, 2023 @09:57PM
(#38314428)
Mo Gawdat has famously said that GPT-4 was something like "4300 lines of code," and that he could have written that when he was a kid. He's clearly a smart man, so, I think we could extrapolate his comments to claim that a smart college student with some CS knowledge could have written it. These sorts of "GPT in $X LOC" demos pretty much confirm it.
For comparison this would be like referring to convonets as Inception architectures back during the CV boom (or VGGnets before that)