The Math Behind Machine Learning

Watch

• The Strange Math That Predicts (Almost) Anything ↗Veritasium
• Large Language Models explained briefly ↗3Blue1Brown
• But what is a neural network? | Deep learning chapter 1 ↗3Blue1Brown

Read

• Introduction to Linear Algebra ↗An indispensable primer, this text demystifies the linear algebra underpinning data transformations and algorithm mechanics, making complex concepts accessible.Gilbert Strang
• All of Statistics: A Concise Course in Statistical Inference ↗A rigorous yet remarkably compact survey of statistical inference, providing the essential probabilistic toolkit for understanding machine learning's uncertainties.Larry Wasserman
• Convex Optimization ↗The authoritative treatise on convex optimisation, this work is crucial for grasping the algorithmic engines that drive most modern machine learning models.Stephen Boyd and Lieven Vandenberghe
• A Few Useful Things to Know About Machine Learning ↗This concise essay distils the core principles and common pitfalls of machine learning, offering invaluable insights into its mathematical and practical challenges.Pedro Domingos
• The Elements of Statistical Learning: Data Mining, Inference, and Prediction ↗A comprehensive and mathematically rigorous exposition of statistical learning methods, this volume is essential for those seeking to understand the theoretical underpinnings of predictive modelling.Trevor Hastie, Robert Tibshirani, Jerome Friedman

Voices to follow

• Yoshua Bengio ↗A foundational figure in deep learning, his work has elucidated the mathematical principles underpinning modern neural networks and their capacity for complex pattern recognition.Professor of Computer Science, University of Montreal; Scientific Director, Mila (Quebec AI Institute)
• Geoffrey Hinton ↗A key architect of deep learning, his insights into neural network architectures and learning algorithms have profoundly shaped the mathematical landscape of artificial intelligence.Emeritus Professor, University of Toronto; Engineering Fellow, Google
• Yann LeCun ↗Renowned for his pioneering work on convolutional neural networks, his research bridges theoretical mathematics with practical applications in computer vision and machine learning.Professor, New York University; Chief AI Scientist, Meta
• Andrew Ng ↗A leading educator and practitioner, he excels at demystifying the mathematical underpinnings of machine learning for a broad audience, making complex concepts accessible.Adjunct Professor, Stanford University; Founder, DeepLearning.AI
• Cynthia Rudin ↗Her work champions interpretable machine learning, emphasizing the development of models whose mathematical logic is transparent, crucial for high-stakes applications.Professor of Computer Science, Duke University

Glossary

• AccuracyHow often a machine learning model makes correct predictions compared to the total number of predictions it makes. It tells you how good the model is at its job. For example, if a spam filter correctly identifies 95 out of 100 emails as spam or not spam, its accuracy is 95%.
• AlgorithmA set of step-by-step instructions that a computer follows to solve a problem or complete a task. Think of it like a recipe for a computer. For example, the steps you follow to bake a cake, like "mix flour and sugar," "add eggs," and "bake at 180°C," form an algorithm.
• DataRaw facts, numbers, images, or any information that a computer uses to learn from. It's the fuel for machine learning. For example, all the photos you've ever taken, the songs on your playlist, or the scores from your last five math tests are all types of data.
• FeatureA specific, measurable characteristic or piece of information about the data that the model uses to learn. Think of them as clues. For example, if you're trying to predict if a fruit is an apple, its color, size, and shape would be features.
• ModelThe "brain" that a machine learning system creates after it has learned from data. It's like a trained expert that can make decisions or predictions. For example, after studying many pictures of cats and dogs, a machine learning system creates a model that can tell the difference between a new cat picture and a new dog picture.
• Neural NetworkA type of machine learning model inspired by the human brain, made of many interconnected "neurons" that process information. It's good at recognizing complex patterns. For example, the technology that allows your phone to recognize faces in photos or understand your voice commands often uses neural networks.
• PredictionWhat the trained machine learning model guesses or decides about new, unseen data. It's the answer the model gives based on what it has learned. For example, if a weather app uses past weather data to guess if it will rain tomorrow, that guess is a prediction.
• Supervised LearningA type of machine learning where the model learns from data that has already been labeled with the correct answers. It's like learning with a teacher who provides the solutions. For example, if you show a computer thousands of pictures, each clearly marked "cat" or "dog," and it learns to tell them apart, that's supervised learning.
• TrainingThe process where a machine learning model learns from a lot of data, like a student studying for an exam. During training, the model adjusts itself to find patterns and relationships in the data. For example, when you practice throwing a basketball repeatedly to get better, you are "training" your muscles and brain to improve your aim.
• Unsupervised LearningA type of machine learning where the model tries to find patterns and structures in data that doesn't have any pre-existing labels or correct answers. It's like exploring a new topic without a guide. For example, if a computer looks at a list of all your friends and groups them into different cliques based on their shared interests without being told who belongs where, that's unsupervised learning.

Threads 7

Where this connects to other fields, and why it's worth knowing.

• Political Ideologies Political Science

  Ask thousands of people hundreds of unrelated questions, on taxes, guns, healthcare, and their answers mostly line up along a single left-to-right line. A whole tangle of opinions squishes down to one number. That squishing is a real math trick called principal component analysis, and 'left versus right' is literally its top result.

• Law, Inequality & Social Power Law

  Train a crime-predicting AI on old arrest records and it learns where police already patrolled, not where crime actually is. So it sends cops back to the same neighborhoods, generating more arrests there, which 'confirms' the AI, a loop. The bias from the past gets laundered into cold, official-looking math that people trust more than they should.

• Language Acquisition Literature

  A three-year-old picks up grammar from a few thousand messy sentences. An AI needs to read a huge chunk of the internet to do the same. That gap has a name, the 'poverty of the stimulus,' and it's the same puzzle machine learning still can't solve: how does a tiny brain learn so much from so little?

• How Learning Works Education

  Cramming all night lets you nail tonight's practice questions but flop on the real exam, because you memorized instead of learning. Machine learning calls that 'overfitting.' Spaced-out practice feels slower and less productive, yet it's the version that actually sticks for the test, the same fix, called regularization, that stops an AI from just memorizing.

• Philosophy of Science Philosophy

  The philosopher Hume asked a nagging question: why should the past predict the future at all? Math gives a hard answer called the no-free-lunch theorem: no learning system can generalize without some assumption baked in first. There's simply no bias-free way to learn from data, whether you're a scientist or an AI, you have to guess something to start.

• AI Art Arts & Design

  When an AI 'invents' a new image, it's really wandering through a giant map of everything it has already seen, picking a spot between familiar points, half-cat, half-cloud. Its 'creativity' is geometry: a stroll across a squished-down atlas of past pictures. The surprise is real, but it's blended from old ingredients, not conjured from nothing.

• Vaccines Health

  A vaccine works exactly like teaching an AI. You show a computer one labeled example of a cat so it can spot cats it's never seen; a vaccine shows your immune system one harmless sample of a virus so it can recognize the real thing, even new variants, later.