Artificial Intelligence 🤖
Partial differentiation
Taylor Expansion

Taylor expansion

For a function of one variable, say g(x)g(x) we can write down a Taylor expansion about a point x0x_{0},

g(x)=g(x∗0)+dgdx(x∗0)h+12!d2gdx2(x∗0)h2+13!d3gdx3(x∗0)h3+⋯ , g(x)=g\left(x*{0}\right)+\frac{d g}{d x}\left(x*{0}\right) h+\frac{1}{2 !} \frac{d^{2} g}{d x^{2}}\left(x*{0}\right) h^{2}+\frac{1}{3 !} \frac{d^{3} g}{d x^{3}}\left(x*{0}\right) h^{3}+\cdots,

where h=x−x0h=x-x_{0}. In abbreviated notation, Eq. (1.14) is:

g(x)=g(x∗0)+g′(x∗0)h+12!g′′(x∗0)h2+13!g′′′(x∗0)h3+⋯ . g(x)=g\left(x*{0}\right)+g^{\prime}\left(x*{0}\right) h+\frac{1}{2 !} g^{\prime \prime}\left(x*{0}\right) h^{2}+\frac{1}{3 !} g^{\prime \prime \prime}\left(x*{0}\right) h^{3}+\cdots .

Recall that if we choose x0=0x_{0}=0, we have the Taylor series about x=0x=0, known as the Maclaurin series of g(x)g(x). Now, the approximation is valid if the function g(x)g(x) and any derivatives used are continuous in the interval [x,x+h][x, x+h] and the values of the function and the derivatives are known at the point x=x0x=x_{0}.

Under a similar set of conditions, we can use the same idea to expand a function of two variables. For the case of f=f(x,y)f=f(x, y), the Taylor expansion about the point (x0,y0)\left(x_{0}, y_{0}\right) is given by:

f(x,y)=f(x∗0,y∗0)+(∂f∂xh+∂f∂yk)(x∗0,y∗0)+12!(1∂2f∂x2h2+2∂2f∂x∂yhk+1∂2f∂y2k2)(x∗0,y∗0)+⋯ , \begin{aligned} f(x, y)=f\left(x*{0}, y*{0}\right) & +\left(\frac{\partial f}{\partial x} h+\frac{\partial f}{\partial y} k\right)\left(x*{0}, y*{0}\right) \\ & +\frac{1}{2 !}\left(\mathbf{1} \frac{\partial^{2} f}{\partial x^{2}} h^{2}+\mathbf{2} \frac{\partial^{2} f}{\partial x \partial y} h k+\mathbf{1} \frac{\partial^{2} f}{\partial y^{2}} k^{2}\right)\left(x*{0}, y*{0}\right)+\cdots, \end{aligned}

where h=x−x0h=x-x_{0} and k=y−y0k=y-y_{0}. Equation (1.16) is a second-order expansion. Notice the use of Clairaut's theorem and the equality of mixed partials, i.e. ∂2f∂x∂y=∂2f∂y∂x\frac{\partial^{2} f}{\partial x \partial y}=\frac{\partial^{2} f}{\partial y \partial x}. This results in the binomial coefficients (shown in bold) in the second-order term. It follows that including a third-order expansion in our approximation of f(x,y)f(x, y), and expressing it in abbreviated notation gives:

f(x,y)=f(x∗0,y∗0)+(f∗xh+f∗yk)(x∗0,y∗0)+12!(f∗xxh2+2f∗xyhk+f∗yyk2)(x∗0,y∗0)+13!(f∗xxxh3+3f∗xxyh2k+3f∗xyyhk2+f∗yyyk3)(x∗0,y_0)⋯ \begin{aligned} f(x, y) & =f\left(x*{0}, y*{0}\right)+\left(f*{x} h+f*{y} k\right)\left(x*{0}, y*{0}\right)+\frac{1}{2 !}\left(f*{x x} h^{2}+2 f*{x y} h k+f*{y y} k^{2}\right)\left(x*{0}, y*{0}\right) \\ & +\frac{1}{3 !}\left(f*{x x x} h^{3}+3 f*{x x y} h^{2} k+3 f*{x y y} h k^{2}+f*{y y y} k^{3}\right)\left(x*{0}, y\_{0}\right) \cdots \end{aligned}

Again, these ideas may be extended to functions of three or more variables. Finally, we make use of the notation introduced in Subsec. 1.2.2. namely ∇f\nabla f and Hf\mathcal{H} f, to express the expansion given by Eq. (1.16):

f(x,y)=f_0+hT⋅∇f+12!hT⋅Hf⋅h+⋯ , f(x, y)=f\_{0}+\mathbf{h}^{T} \cdot \nabla f+\frac{1}{2 !} \mathbf{h}^{T} \cdot \mathcal{H} f \cdot \mathbf{h}+\cdots,

where h=(hk)\mathbf{h}=\left(\begin{array}{c}h \\ k\end{array}\right) while ∇f\nabla f and Hf\mathcal{H} f are given by Eqs. (1.12) and (1.13), respectively.