Re:Ask HN: How does ChatGPT understand the prompts?
by skilled on Saturday July 08, 2023 @08:38AM
(#36643794)
LLMs simply process the input and generate outputs based on patterns seen during training.
Here's the process in brief:
Tokenization: The input text gets broken down into smaller chunks, or tokens. Tokens can range from a single character to a whole word.
Embedding: Tokens get translated into numerical vectors - this is how models can process them.
Processing: These vectors are then processed in the context of the others. This is done via a type of neural network called a Transformer[0] network, which handles context particularly well.
Context Understanding: The model uses patterns learned from its training to predict the next word in a sentence. It's not a human-like understanding, but rather it estimates the statistical probability of a word following the preceding ones.
Generation: The model generates a response by continuously predicting the next word until a full response is formed or it reaches a certain limit.
Here's the process in brief:
Tokenization: The input text gets broken down into smaller chunks, or tokens. Tokens can range from a single character to a whole word.
Embedding: Tokens get translated into numerical vectors - this is how models can process them.
Processing: These vectors are then processed in the context of the others. This is done via a type of neural network called a Transformer[0] network, which handles context particularly well.
Context Understanding: The model uses patterns learned from its training to predict the next word in a sentence. It's not a human-like understanding, but rather it estimates the statistical probability of a word following the preceding ones.
Generation: The model generates a response by continuously predicting the next word until a full response is formed or it reaches a certain limit.
[0]: https://huggingface.co/learn/nlp-course/chapter1/4