트랜스포머가 틱택토를 연주하는 방법

An interactive guide to the architecture behind modern language models.

Instead of predicting the next word, this Transformer predicts the next move in a game of fading Tic-Tac-Toe—making every step of the model easy to visualize and understand.

Play the game, inspect every matrix multiplication, and watch tokens flow through the network in real time.

What’s covered

• Tokenization and embeddings
• Learned positional encoding
• Self-attention (Q, K, V)
• Multi-head attention
• Causal masking and softmax
• Residual connections and layer normalization
• MLP (feed-forward network)
• Unembedding and sampling
• Model ablations (no positional encoding, no causal mask, no MLP, no residual stream)

Includes interactive visualizations for every stage of the Transformer pipeline – from input tokens to the final prediction.

https://sbondaryev.dev/articles/transformer