News for hackers. Stuff that matters.
Re:We Aren't Close to Creating a Rapidly Self-Improving AI by yinser on Saturday April 29, 2023 @11:54AM (#35754086)
To add another anecdote to your question: the transformer became a part of the first context aware embedding model GPT-1. Not to say it couldn’t be done with another tool but it was first done with a transformer. Previous embedding models like word2vec, GloVe and fasttext were not contextually embedding and didn’t give you a language graph that would then go on to support a language model capable of “understanding” what you were saying or asking for.

There are no comments on this comment yet.