Articles tagged with “self-attention”

ChatGPT's Technical Foundations: Transformers to RLHF

An examination of the five key technical innovations behind ChatGPT, from the Transformer architecture and pretraining to RLHF and its successors (DPO, GRPO), GPU hardware evolution through NVIDIA Blackwell, and tokenization.

40 min read

9/27/2025

chatgpt transformer architecture rlhf large language models machine learning artificial intelligence natural language processing self-attention ai