How ChatGPT Works

We are amazed at the way Large Language Models generate responses. LLM can generate such human-like responses and it is fine-tuned to perform various tasks, i.e., text generation, translation, image generation and response to user queries.  In this article, I will discuss the mechanics behind the ChatGPT, it works based on the Transformer Architecture with the self-attention mechanism.

It is interesting to know when a user asks a question to ChatGPT and he gets a response from ChatGPT, all types of processing happen. Let’s discuss the step-by-step mechanics behind it.

1. Tokenization – The model breaks down the user query into smaller, manageable pieces called tokens. These tokens could be either words, characters or sub-words. This tokenization process is crucial because it translates the human-readable text into a format that the machine-learning model can understand and manipulate.

2. Encoding – Each token is then sent via several encoder levels in the model following tokenization. Every token is converted into a high-dimensional vector during this stage. These vectors capture both syntactic roles and semantic meaning; these are not only numerical representations of the tokens. The model can comprehend the context, subtleties, and connections between various question components due to this encoding.

3. Decoding – The decoder's contextual base is built upon the vectors produced by the encoder. The response tokens are generated in an autoregressive fashion, one after the other, by the decoder layers. This indicates that every new token that is generated is dependent on the encoder's understanding of the full input query in addition to the tokens that came before it in the output.

4. Self-Attention Mechanism– The self-attention mechanism is one of the most innovative features of the Transformer architecture. This approach enables the model to give different weights to different sections of the input query and to the tokens it has previously created during the decoding phase. This weighted attention plays a crucial role in maintaining the resulting text's narrative coherence and contextual relevance to the query.

5. Vocabulary Probability Distribution – The model computes a probability distribution over its whole vocabulary for every new token that is to be generated. Advanced approaches can be utilized to add a degree of originality and randomness to the output, even if the token with the highest likelihood is often selected as the next in line.

6. Iterative Token Generation – The decoder iteratively cycles through decoding, self-attention mechanism and vocabulary probability distribution steps to continue generating new tokens. This process keeps going until a predetermined endpoint is reached. This could occur when you use up all of your tokens when you run across a certain end-of-sentence token, or under another circumstance.

7. Detokenization & Final Output Generation – It is a post-processing step, once the decoder has generated a complete set of tokens for the response, the next step is to convert these back into natural language text. At last detokenized text is packaged to create a comprehensive response to the user query.

We discussed the processing of the user query and output generation. The user query gets processed through a series of steps and then output is generated through decoding and self-attention mechanism. ChatGPT generates one token at a time using an autoregressive mechanism, that’s how it generates output to the user query.