Software 1
NanoGPT Transformer Notebook
A hands-on PyTorch notebook that rebuilds the core pieces of a character-level GPT model on Tiny Shakespeare, from tokenization and batching through masked self-attention and generation.
What
- Loads and explores the Tiny Shakespeare dataset for language-model training.
- Builds character-level tokenization with encode/decode mappings.
- Walks through autoregressive prediction targets and context windows.
How
- Uses PyTorch tensors, batches, and train/validation splits.
- Implements causal masking so tokens cannot attend to future positions.
- Builds self-attention and multi-head attention components step by step.
Result
- Shows practical understanding of transformer internals beyond API usage.
- Connects math concepts like scaled dot-product attention to runnable code.
- Strong focused artifact for deep-learning and LLM fundamentals.