DeepSeek-V3: Diagrammed
Multi-head latent attention and mixture of experts, made visible.
Notes & explorations
Archive
Deep learning, diagrams, and the mathematics underneath.
↗︎
Diagrams Explained (Quickly)
A short guide to reading the wires, operations, and structure of neural circuit diagrams.
What's so Hard About Representing Deep Learning Models?
Making the axes, sizes, and broadcasting of deep learning operations explicit.
Case Study: Shortfalls of Attention is All You Need
A closer look at what the original Transformer description leaves implicit.
Preprints for Applied Category Theory Conference 2024
Functor string diagrams and diagrammatic negative information.
Understanding Mixtral-8x7b
A diagrammatic tour of the Mixtral-8x7B mixture-of-experts architecture.
Residual Networks and Information Theory
Skip connections, memory, and an information-theoretic view of residual networks.
Wasserstein GANs and the unexpected mathematics of a new field
The unexpected mathematics behind a new approach to generative modelling.
Entropy and Thermodynamics
An exploration of entropy, probability, and the foundations of statistical mechanics.
A Poster for Neural Circuit Diagrams
A visual introduction to neural circuit diagrams and the mathematics behind them.