Re:OpenAI: Start using ChatGPT instantly
by ldjkfkdsjnv on Monday April 01, 2024 @02:07PM
(#39897152)
No becuause ChatGPT was trained on the whole internet using a new algorithm. You needed perfect tagged data for the old algorithm (page rank). LLMs are a fundamental leap forward
The GPT algorithm isn't a secret.
The RLFH data — users giving a thumbs up or down, or regenerating a response, or even getting detectably angry in their subsequent messages — is.
PageRank isn't a secret.
Google's version of RLFH — which link does a user click on, do they ignore the results without clicking any, do they write several related queries before clicking a link, do they return to the search results soon after clicking a link — are also secret.
That the Transformer model is a breakthrough doesn't make it a moat; that the Transformer model is public doesn't mean people using it don't have a moat.
Hence why I'm criticising the use of "moat" as mere parroting.
This description also works for ChatGPT, what it has which others do not (at anything close to the same scale).
Google's search engine is a well understood bit of matrix multiplication on a scale many others can easily afford to replicate.
This description also works for GPT-3.