KKT Conditions in Finite Dimensions

Overview¶

In this lecture we show the core ideas needed to understand inequality constrained first-order and second-order optimality in finite dimensions.

We focus on the problem

with $f,\varphi_j\in C^1(\mathbb{R}^n)$.

The take-home message is simple:

1. at a constrained minimum there is no admissible descent direction;

2. this forces the objective gradient to balance active constraint normals;

3. this balance is exactly the KKT system;

4. second order is curvature of the Lagrangian along directions that remain tangent to active constraints.

5. with the convention $\mathcal L=f-\sum_j\lambda_j\varphi_j$ and $\varphi_j\ge 0$, active multipliers are nonnegative, and they vanish for inactive constraints (complementarity).

Problem Setting and Active Constraints¶

Define the feasible set

Let $\bar u\in\mathcal U_{\mathrm{ad}}$ be a local minimizer.

Not all constraints matter equally at $\bar u$. Only the constraints that are exactly at the boundary can influence first-order optimality. So we define the active set

Inactive constraints satisfy $\varphi_j(\bar u)>0$ and do not contribute at first order. For example: if we have a contact constraint between two bodies (that cannot penetrate), then only the constraints corresponding to points in contact are active, while the others are inactive.

Admissible First-Order Directions¶

Near $\bar u$, a direction $d\in\mathbb R^n$ is first-order admissible if it does not decrease active constraints. Linearizing active constraints gives

This condition is the local geometric ingredient we need.

Interpretation:

• if $\nabla\varphi_j(\bar u)\cdot d<0$, then moving along $d$ pushes outside feasibility for constraint $j$;

• if $\nabla\varphi_j(\bar u)\cdot d\ge 0$, then constraint $j$ is not violated at first order.

First-Order Necessary Condition¶

At a local minimizer, every admissible first-order direction must be non-descent:

Why this is unavoidable:

• if there existed one admissible $d$ with $\nabla f(\bar u)\cdot d<0$,

• then a small step $\bar u+t d$ (with $t>0$ small) would decrease the objective while remaining feasible at first order,

• contradicting local minimality.

This is the true origin of KKT.

From First-Order Geometry to KKT¶

For regular active constraints (the standard finite-dimensional nondegenerate case), there exist multipliers $\lambda_j\ge 0$ such that

Extending by $\lambda_j=0$ on inactive constraints yields the standard stationarity form

Together with primal feasibility and complementarity, we obtain

Complementarity meaning:

• if $\varphi_j(\bar u)>0$ (inactive), then $\lambda_j=0$;

• if $\lambda_j>0$, then necessarily $\varphi_j(\bar u)=0$ (active).

Box Constraints: The Prototype Case¶

Consider

Equivalent inequality form:

The componentwise first-order condition becomes

This is the exact finite-dimensional model behind bound-constrained control variables in PDE-constrained optimization.

Second-Order Conditions¶

Define the Lagrangian

After first-order conditions are satisfied, the next question is curvature along first-order feasible/tangent directions. A practical critical set is

Second-order necessary condition:

Second-order sufficient condition (strict local minimality):

Interpretation:

• first order kills linear descent along admissible directions;

• second order checks that curvature is positive on directions where first order is neutral.

Summary¶

At a constrained local minimizer:

1. no admissible first-order direction can decrease $f$;

2. this forces a balance between objective gradient and active constraint normals;

3. that balance is KKT with nonnegative multipliers and complementarity;

4. second order is positivity of Lagrangian curvature on critical directions.

This is the essential finite-dimensional picture we need before moving to PDE-constrained settings.