Optimization

Watch

• How your brain's executive function works -- and how to improve it | Sabine Doebel ↗TED
• Bayes theorem, the geometry of changing beliefs ↗3Blue1Brown
• Differential equations, a tourist's guide | DE1 ↗3Blue1Brown

Read

• Convex Optimization ↗An indispensable guide for anyone seeking to master the art of finding optimal solutions within constrained systems, offering both rigorous theory and practical algorithms.Stephen Boyd and Lieven Vandenberghe
• Numerical Optimization ↗The definitive reference for practitioners and theoreticians alike, providing a comprehensive treatment of algorithms that underpin modern computational optimization.Jorge Nocedal and Stephen J. Wright
• Introduction to Operations Research ↗A foundational text that demystifies the practical application of mathematical models to complex decision-making, essential for any aspiring operational strategist.Frederick S. Hillier and Gerald J. Lieberman
• Nonlinear ProgrammingA seminal work that introduced the Karush-Kuhn-Tucker conditions, providing the bedrock for understanding optimality in constrained non-linear problems.H.W. Kuhn and A.W. Tucker
• The Theory of Dynamic ProgrammingIntroduced a revolutionary approach to sequential decision-making, laying the groundwork for algorithms that optimize multi-stage processes across diverse fields.Richard Bellman

Voices to follow

• Stephen Boyd ↗A luminary in convex optimization, his work has made complex problems tractable across engineering and data science.Professor of Electrical Engineering, Stanford University
• Dimitris Bertsimas ↗A pioneer in robust optimization and analytics, he bridges theoretical rigour with practical applications in decision-making.Professor of Operations Research, MIT
• Yuri Nesterov ↗His foundational contributions to convex optimization algorithms, including accelerated gradient methods, have profoundly shaped the field's efficiency.Professor of Mathematical Engineering, Université catholique de Louvain
• Michael I. Jordan ↗A foundational figure in machine learning, his insights often illuminate the underlying optimization challenges and solutions in artificial intelligence.Professor of Electrical Engineering and Computer Sciences, UC Berkeley

Glossary

• ConstraintsThese are the limits, rules, or conditions that you must follow when trying to find the best solution. They tell you what you can and cannot do. For example, if you're baking cookies, a constraint might be that you only have 2 cups of flour, so you can't make an endless number of cookies.
• Feasible RegionThis is the set of all possible choices or solutions that satisfy all the given constraints. Any point within this region is a valid option, even if it's not the best one. For example, if you can only spend between 1 and 3 hours studying, and between 0 and 2 hours playing, the feasible region would be all the combinations of study and play hours that fit those rules.
• Linear ProgrammingThis is a specific mathematical method used for optimization problems where both the objective (what you want to maximize or minimize) and the constraints (the rules) are expressed as simple straight-line equations or inequalities. For example, a bakery might use linear programming to figure out how many cakes and cookies to bake to maximize profit, given limits on flour, sugar, and oven time.
• MaximizeTo maximize means to make something as large or as much as possible. You're trying to get the highest value for your objective. For example, a company tries to maximize its profits by selling more products at a good price.
• MinimizeTo minimize means to make something as small or as little as possible. You're trying to get the lowest value for your objective. For example, you might try to minimize the amount of time you spend on chores so you have more free time.
• Objective FunctionThis is the goal or target you're trying to make as good as possible (either the biggest or the smallest). It's usually a mathematical rule that tells you how well you're doing. For example, if you're trying to earn the most money from selling cookies, your objective function would be the total profit you make.
• Optimal SolutionThis is the single best answer or choice you find after considering all the variables and constraints. It's the point where your objective function is at its maximum or minimum value. For example, if you're trying to find the fastest route to school, the optimal solution would be the specific path that gets you there in the least amount of time.
• OptimizationOptimization is finding the best possible way to do something, like getting the most out of a situation or solving a problem in the most efficient way. It's about making choices to achieve the best outcome. For example, if you want to pack the most clothes into your suitcase, you're optimizing your packing strategy.
• Trade-offA trade-off happens when you have to give up one thing to get another, because resources or choices are limited. It's about balancing different goals. For example, if you spend more time playing video games, you might have to trade off some study time.
• VariableA variable is something you can change or adjust in a problem to see how it affects the outcome. It's like a dial you can turn. For example, if you're deciding how many hours to study for two different subjects, the number of hours for each subject would be your variables.

Threads 11

Where this connects to other fields, and why it's worth knowing.

• The World Wars History

  The math that tells Amazon which warehouse ships your order and routes a thousand planes without crashes was born in wartime. Scientists invented it to sneak supply ships past hidden U-boats and aim anti-aircraft guns at fast planes. They called it operations research, and it's now the quiet engine behind modern logistics.

• Space & Aerospace Engineering Technology

  To catch a space station that's ahead of you, you'd think you should just speed straight toward it. But in orbit, firing your engine drops you to a lower path where you actually move faster and swing around to meet it. It proves a deep point: when the rules of the game are curved, the smartest route is almost never a straight line.

• How Habits Form and Break Psychology

  Your brain's reward chemical, dopamine, doesn't fire when something good happens. It fires when something is better than you expected, the size of the pleasant surprise. That 'surprise gap' is exactly the error signal that machine-learning programs use to teach themselves. So a habit forming is basically your nervous system fine-tuning itself toward whatever keeps pleasantly surprising it.

• Leadership and Decision-Making Business

  When a team is stuck in a rut, the smart move feels wrong: let them try some clearly worse ideas for a while and add a bit of chaos on purpose. Computers use the exact same trick, called simulated annealing, to escape a 'good enough' answer and find a much better one. You sometimes have to go downhill first to reach a higher hill.

• Recommendation Algorithms Media

  An app tunes your feed to get the most clicks, because clicks were supposed to mean 'people find this valuable.' But once you chase clicks above all, they stop meaning that. This is Goodhart's law: the moment a measurement becomes the target, it gets gamed and destroys the very thing it was standing in for.

• Does God Exist? Philosophy

  Faced with why a good God allows suffering, the thinker Leibniz claimed we live in 'the best of all possible worlds.' Notice the shape of that: it's an optimization claim, the highest total goodness you can reach under real constraints. Evil becomes the unavoidable price tag of that best-possible maximum, like a cost you can't fully remove.

• Social Media and Attention Media

  Your social feed is a math problem that's been solved, and the thing it's trying to maximize is your attention. Every post it ranks is the math creeping one tiny step closer to whatever keeps your thumb scrolling.

• Cancer Health

  The same body systems that stop cells from turning into cancer also help wear us out as we age. It's a tuned trade-off. Push too hard to block cancer and you tend to speed up aging, and vice versa.

• Addiction and the Brain Psychology

  Your brain uses a chemical called dopamine to signal 'that was better than expected', nudging you to do it again. That's exactly the math a learning computer uses to improve itself. Addiction is that system gone wild, endlessly chasing a reward signal that's been hijacked.

• Economic Systems Economics

  The old fight between free markets and government planning was secretly a math race: can a planner solve millions of equations about who needs what, faster than prices figure it out on their own? A whole branch of math for solving such puzzles was actually invented inside a Soviet plywood factory trying to do exactly that.

• Industrial Engineering Technology

  Industrial engineering turns objectives and resource limits into choices about schedules, layouts and capacity.

  Sources: U.S. Bureau of Labor Statistics — Industrial Engineers ↗ · ABET engineering program criteria 2025–2026 ↗