Taylor Approximation

17.1 INTRODUCTION
17.2 LECTURE
17.3 EXAMPLES
EXERCISES

17.1 INTRODUCTION

17.1.1 How Linearity Helped Feynman Conquer the Cube Root

According to legend¹, Richard Feynman got into the challenge to compute the cube root of against an Abacus computation. By using linear approximation and a bit of luck, he could get using paper and pencil. The actual cube root is . How did Feynman do it? The secret is in linear approximation. This means that we approximate a function like with a linear function. The same can be done with functions of several variables. The linear approximation if of the form .

**Figure 1.** The Abacus scene in the movie "Infinity".

17.1.2 Beyond Linear Approximations

One can also do higher order approximations. The function for example has the linear approximation at and the quadratic approximation at . To get the quadratic term, we just need to make sure that the first and second derivative at agree. This gives the formula Indeed, you can check that and have the same first derivatives and the same second derivatives at . A degree approximation is then the polynomial For the function for example, we have the ’th order approximation

17.1.3 Multivariable Approximations

The same can be done in higher dimensions. Everything is the same. We just have to use the derivative rather than the usual derivative . We look here only at linear and quadratic approximation of functions The linear approximation is then where is the Jacobian matrix, which ii a row vector. Now, since we can see the second derivative is a matrix . It is called the Hessian. It encodes all the second derivatives .

17.2 LECTURE

17.2.1 Unveiling the Multidimensional Taylor Formula

Given a function , its derivative is the Jacobian matrix. For every , we can use the matrix and a vector to get . For fixed , this defines a map , like the original . Because is a map on one calls it an operator. The Taylor formula holds in arbitrary dimensions:

Theorem 1.

Proof. It is the single variable Taylor on the line . The directional derivative is there the usual derivative as Technically, we need the sum to converge as well: like functions built from polynomials, , , . ◻

17.2.2 Tensors and the Multidimensional Taylor Series Representation

The Taylor formula can be written down using successive derivatives , , also, which are then called tensors. In the scalar case , the first derivative leads to the gradient , the second derivative to the Hessian matrix which is a bilinear form acting on pairs of vectors. The third derivative then acts on triples of vectors etc. One can still write as in one dimension

Theorem 2.

if we write . For a polynomial, this just means that we first write down the constant, then all linear terms then all quadratic terms, then all cubic terms etc.

17.2.3 The Local Approximation Through Linearization

Assume and stop the Taylor series after the first step. We get It is custom to write this with , as This function is called the linearization of . The kernel of is a linear manifold approximating the surface If , then the just said can be applied to every component of , with . One can not stress enough the importance of this linearization.²

17.2.4 Getting Even Closer: Quadratic Approximations with Hessians

If we stop the Taylor series after two steps, we get the function The matrix is called the Hessian matrix at the point . It is also here custom to eliminate by writing . is called the quadratic approximation of . The kernel of is the quadratic manifold where and . It approximates the surface even better than the linear one. If is of the order , then is of the order and is of the order . This follows from the exact Taylor with remainder formula.³

17.2.5 The Tangent Plane to a Surface

To get the tangent plane to a surface one can just look at the linear manifold . However, there is a better method:

The tangent plane to a surface at is , where and .

17.2.6 How Gradients Help Find Tangent Planes to Surfaces

This follows from the fundamental theorem of gradients:

Theorem 3. The gradient of is perpendicular to the surface at .

Proof. Let be a curve on with . The chain rule assures But because is constant, this is zero assuring being perpendicular to the gradient. As this works for any curve, we are done. ◻

17.3 EXAMPLES

Example 1. Let be given as . What is the quadratic approximation at ? We have and

The linearization is The quadratic approximation is This is the situation displayed to the left in Figure (17.2). For , the directional derivative The Taylor expansion given at the beginning is a finite series because was a polynomial:

Example 2. For , the gradient and Hessian are

The linearization is The quadratic approximation is the situation displayed to the right in Figure (17.2).

Example 3. What is the tangent plane to the surface for at the point ? The gradient is The tangent plane equation is , where the constant is obtained by plugging in the point. We end up with . The linearization is .

EXERCISES

Exercise 1. Let be a curve and be a coordinate change.

Compute at , then and and .
Compute first, then find . It should agree with a).

Exercise 2.

The surface is an ellipsoid. Compute at the point using the implicit differentiation rule. (Use the formula).
Apply the Newton step 3 times starting with to solve the equation .

Exercise 3. Evaluate without technology the cube root of using quadratic approximation. Especially look how close you are to the real value.

Exercise 4.

Find the tangent plane to the surface at .
Estimate using linear approximation (compute rather than .)

Exercise 5. Find the quadratic approximation of at the point by computing the gradient vector and the Hessian matrix . The vector is a matrix (row vector) and the Hessian matrix is a matrix.

Feynmans book "What do you care what other people think".↩︎
Again: the linearization idea is utmost important because it brings in linear algebra.↩︎
If , (prove this by induction!)↩︎

Table of Contents