Tutorial
Hands-on AI/ML systems learning projects, curated from ai_lab. Real code, run and verified where the hardware allows, honestly reported — including bugs found and negative results — where it doesn't.
Distributed Training & ML Infra
A hands-on curriculum covering DDP, FSDP2, tensor/pipeline/expert parallelism, FP8 quantization, and fault tolerance — implemented from scratch, numerically verified against single-device references.
Not a tutorial-following exercise: every phase ends with a script that runs, a metric that was measured, and a short writeup of what broke and why — including a macOS-specific FSDP2 device-mesh crash, a gloo backend limitation, and an honest negative result where naive vs. block-wise FP8 quantization barely differed in a toy run.
View on GitHub →Transformer Training & Interpretability Basics
Small, readable scripts for training, inference, and interpretability: causal self-attention from scratch, a KV-cache speedup measurement, and a logit-lens walk through a real GPT-2's layers.
A ~150-line minimal GPT (causal multi-head attention, pre-norm, generation), a char-level training run, a serial-inference-vs-training-forward comparison, and a layer-by-layer logit-lens readout using TransformerLens.
View on GitHub →Compute-Pause Verification — Briefing
A self-contained technical map of public research on verifiable AI slowdown: datacenter primer, governance proposals, cryptographic verification schemes, and open follow-up problems.
Diagrams plus who/what/maturity/lineage for the main papers and proposals in this space — the Cankaya Plan A/B, zkLLM, VerInf, FlexHEG — scoped to what's actually checkable under low trust, not a product pitch.
View on GitHub →