Optimization (traditional → modern)

You know the classical math core: formulate a model → minimize f(x) s.t. constraints → take the Lagrangian, multipliers μ/λ, KKT. This is the map of the whole field around that core, by era, with why each era exists.

The through-line (why it kept evolving)

Optimization grew every time the problem broke an assumption of the previous era:

classical — small, smooth, convex → solvable exactly (∇=0, KKT, duality).
numerical — too big for closed form → iterate (descend, Newton, split).
stochastic / ML — huge + non-convex + noisy gradients → SGD.
discrete / global — no gradient, combinatorial, many minima → search.
sequential / control — optimize over time / under uncertainty → DP, Bellman, control, online.
modern frontier — the objective itself is learned / adversarial / nested → bilevel, minimax, OT, learn-to-optimize.

The six clusters

classical-optimization — the math you know + convex optimization (the tractable core; convexity = the dividing line).
numerical-methods — how you actually solve: first/second-order, constrained solvers, line-search / trust regions, the gradient-flow ODE view.
stochastic-and-ml-optimization — Robbins–Monro → SGD/Adam; non-convex deep-learning landscapes.
discrete-and-global-optimization — integer/combinatorial (branch-and-bound, SAT/CDCL), metaheuristics (annealing, evolutionary), Bayesian / black-box.
sequential-and-control — DP / Bellman, optimal control (Pontryagin / HJB), RL, online learning (regret, bandits).
modern-frontier — bilevel / meta, minimax / games, optimal transport, learn-to-optimize, continuation / curriculum.

Why this connects to the cybernetics thread

This is the field our recursive-harness lives in:

optimizer = discretized ODE, convergence = contraction → dynamics-to-a-fixed-point;
Bellman = self-referential fixed point (HJB) → recursion-convergence-contraction;
stages-and-gates = continuation + splitting + trust-region for hard optimization → stages-gates-as-hard-optimization;
Ashby's requisite variety = the ceiling → ashby.