Unit 4: Multivariable Chain Rules, Directional Derivatives & Optimization

1. The Multivariable Chain Rule

Let $z = f(x, y)$ be a differentiable function of $x$ and $y$.

1. Case 1 (Single Independent Variable $t$):

If $x = g(t)$ and $y = h(t)$ are differentiable functions of $t$, then $z$ is a differentiable function of $t$, and:

2. Case 2 (Multiple Independent Variables $s, t$):

If $x = g(s, t)$ and $y = h(s, t)$ are differentiable functions of $s$ and $t$, then:

General Tree Diagram Rule: To evaluate the derivative of a dependent variable with respect to an independent variable, trace all paths from the dependent variable to the independent variable in the dependency tree. Form the product of partial derivatives along each branch, and sum across all paths.

2. Implicit Differentiation via Partial Derivatives

Let an equation $F(x, y) = 0$ define $y$ implicitly as a differentiable function of $x$. Differentiating both sides with respect to $x$ using the Chain Rule:

Assuming $F_y \neq 0$:

Similarly, if $F(x, y, z) = 0$ defines $z$ implicitly as a function of $x$ and $y$:

1. The Directional Derivative

Partial derivatives $f_x$ and $f_y$ measure rates of change along the directions of the coordinate axes $\hat{i}$ and $\hat{j}$. The directional derivative generalizes this to an arbitrary unit direction vector $\vec{u} = \langle a, b \rangle$ ($a^2 + b^2 = 1$).

Definition: The directional derivative of $f$ at $(x_0, y_0)$ in the direction of unit vector $\vec{u}$ is:

2. The Gradient Vector

Definition: If $f$ is a differentiable function of two variables, the gradient of $f$, denoted by $\nabla f$ (read "del $f$"), is the vector function:

For a function of three variables $w = f(x, y, z)$:

3. Computation Theorem for Directional Derivatives

Theorem: If $f$ is a differentiable function, then the directional derivative in the direction of unit vector $\vec{u}$ is given by the scalar dot product:

Proof:

Define $g(h) = f(x_0 + ha, y_0 + hb)$. By definition, $D_{\vec{u}} f(x_0, y_0) = g'(0)$. By the Multivariable Chain Rule:

Setting $h = 0$:

4. Fundamental Geometric Properties of the Gradient

Using the definition of the dot product:

where $\theta$ is the angle between $\nabla f$ and $\vec{u}$.

1. Direction of Maximum Increase (Steepest Ascent):

$\cos\theta = 1 \implies \theta = 0$. The maximum value of the directional derivative is $|\nabla f|$, and it occurs in the exact direction of the gradient vector $\nabla f$.

2. Direction of Maximum Decrease (Steepest Descent):

$\cos\theta = -1 \implies \theta = \pi$. The minimum value is $-|\nabla f|$, occurring in direction $-\nabla f$.

3. Orthogonality to Level Sets:

$\cos\theta = 0 \implies \theta = \pi/2$. The directional derivative is zero in directions orthogonal to $\nabla f$. Therefore, the gradient vector $\nabla f(x_0, y_0)$ is strictly perpendicular to the level curve $f(x, y) = c$ at $(x_0, y_0)$.

For a level surface $F(x, y, z) = c$, the gradient $\nabla F(x_0, y_0, z_0)$ is the normal vector to the tangent plane.

1. Critical Points and Extrema

A function $z = f(x, y)$ has a local maximum (or minimum) at $(a, b)$ if $f(x, y) \le f(a, b)$ (or $f(x, y) \ge f(a, b)$) for all $(x, y)$ in some open disk around $(a, b)$.

Fermat's Theorem for Multivariable Functions: If $f$ has a local extremum at $(a, b)$ and the first partial derivatives exist, then:

A point $(a, b)$ where $\nabla f(a, b) = \vec{0}$ or where either partial derivative fails to exist is called a critical point.

2. The Second Derivative Test (Hessian Discriminant)

Let $(a, b)$ be a critical point of $f(x, y)$, and assume the second partial derivatives are continuous on a disk containing $(a, b)$. Define the Hessian determinant:

1. If $D > 0$ and $f_{xx}(a, b) > 0$: $f(a, b)$ is a Local Minimum.

2. If $D > 0$ and $f_{xx}(a, b) < 0$: $f(a, b)$ is a Local Maximum.

3. If $D < 0$: $(a, b)$ is a Saddle Point (the surface curves upwards in one direction and downwards in another, resembling a mountain pass).

4. If $D = 0$: The test is inconclusive (higher-order terms must be analyzed).

3. Constrained Optimization via Lagrange Multipliers

To maximize or minimize an objective function $f(x, y, z)$ subject to a constraint $g(x, y, z) = k$ (where $\nabla g \neq \vec{0}$):

Geometric Principle: At an extremum on the constraint surface, the level surface of $f$ must be tangent to the constraint surface $g = k$. Therefore, their normal vectors must be parallel:

The scalar parameter $\lambda$ is called the Lagrange Multiplier.

Method of Lagrange Multipliers:

Solve the system of four simultaneous equations in four unknowns $(x, y, z, \lambda)$:

Dual Constraints:

To optimize $f(x, y, z)$ subject to two simultaneous constraints $g(x, y, z) = k$ and $h(x, y, z) = c$: