Labs

These labs build a conceptual understanding of large language models from the ground up, starting with the simplest possible model and working toward the systems you interact with every day.

TinyLM

Build the simplest possible language model: one that learns by counting which words tend to follow which other words in a text corpus, then generates new text by sampling from those observed frequencies. Using a tongue-twister as the training corpus, you will watch the model make decisions step-by-step, inspect what it has "learned," and understand both why the approach works and where it breaks down.

Matrices

Rebuild TinyLM using matrices and NumPy. The model produces the same outputs, but representing it as matrix operations reveals the computational structure that underlies all neural networks. You will learn how matrix multiplication works, how a one-hot vector selects a row, and how this framework generalizes far beyond counting words.

Embeddings

Discover that words have geometry. When a model is trained to predict neighboring words, it learns a dense vector for each word—an embedding—and words with similar meanings end up near each other in that space. You will explore pre-trained GloVe embeddings, then extend TinyLM to learn its own embeddings from scratch, tracing how the model adjusts millions of numbers over many training passes to make better and better predictions.

Local Models

Run a large language model directly on your own computer. Using Ollama, you will host an open-weight LLM, write Python code to interact with it, and give it tools—Python functions it can call to fetch real-world information like the current time or the weather. You will also learn how quantization and distillation shrink billion-parameter models down to sizes that fit on consumer hardware, and browse the landscape of available models to judge what will run on your device.