Artificial Intelligence 🤖
Partial differentiation
Taylor Expansion

Taylor expansion

For a function of one variable, say g(x)g(x) we can write down a Taylor expansion about a point x0x_{0},

g(x)=g(x0)+dgdx(x0)h+12!d2gdx2(x0)h2+13!d3gdx3(x0)h3+, g(x)=g\left(x*{0}\right)+\frac{d g}{d x}\left(x*{0}\right) h+\frac{1}{2 !} \frac{d^{2} g}{d x^{2}}\left(x*{0}\right) h^{2}+\frac{1}{3 !} \frac{d^{3} g}{d x^{3}}\left(x*{0}\right) h^{3}+\cdots,

where h=xx0h=x-x_{0}. In abbreviated notation, Eq. (1.14) is:

g(x)=g(x0)+g(x0)h+12!g(x0)h2+13!g(x0)h3+. g(x)=g\left(x*{0}\right)+g^{\prime}\left(x*{0}\right) h+\frac{1}{2 !} g^{\prime \prime}\left(x*{0}\right) h^{2}+\frac{1}{3 !} g^{\prime \prime \prime}\left(x*{0}\right) h^{3}+\cdots .

Recall that if we choose x0=0x_{0}=0, we have the Taylor series about x=0x=0, known as the Maclaurin series of g(x)g(x). Now, the approximation is valid if the function g(x)g(x) and any derivatives used are continuous in the interval [x,x+h][x, x+h] and the values of the function and the derivatives are known at the point x=x0x=x_{0}.

Under a similar set of conditions, we can use the same idea to expand a function of two variables. For the case of f=f(x,y)f=f(x, y), the Taylor expansion about the point (x0,y0)\left(x_{0}, y_{0}\right) is given by:

f(x,y)=f(x0,y0)+(fxh+fyk)(x0,y0)+12!(12fx2h2+22fxyhk+12fy2k2)(x0,y0)+, \begin{aligned} f(x, y)=f\left(x*{0}, y*{0}\right) & +\left(\frac{\partial f}{\partial x} h+\frac{\partial f}{\partial y} k\right)\left(x*{0}, y*{0}\right) \\ & +\frac{1}{2 !}\left(\mathbf{1} \frac{\partial^{2} f}{\partial x^{2}} h^{2}+\mathbf{2} \frac{\partial^{2} f}{\partial x \partial y} h k+\mathbf{1} \frac{\partial^{2} f}{\partial y^{2}} k^{2}\right)\left(x*{0}, y*{0}\right)+\cdots, \end{aligned}

where h=xx0h=x-x_{0} and k=yy0k=y-y_{0}. Equation (1.16) is a second-order expansion. Notice the use of Clairaut's theorem and the equality of mixed partials, i.e. 2fxy=2fyx\frac{\partial^{2} f}{\partial x \partial y}=\frac{\partial^{2} f}{\partial y \partial x}. This results in the binomial coefficients (shown in bold) in the second-order term. It follows that including a third-order expansion in our approximation of f(x,y)f(x, y), and expressing it in abbreviated notation gives:

f(x,y)=f(x0,y0)+(fxh+fyk)(x0,y0)+12!(fxxh2+2fxyhk+fyyk2)(x0,y0)+13!(fxxxh3+3fxxyh2k+3fxyyhk2+fyyyk3)(x0,y_0) \begin{aligned} f(x, y) & =f\left(x*{0}, y*{0}\right)+\left(f*{x} h+f*{y} k\right)\left(x*{0}, y*{0}\right)+\frac{1}{2 !}\left(f*{x x} h^{2}+2 f*{x y} h k+f*{y y} k^{2}\right)\left(x*{0}, y*{0}\right) \\ & +\frac{1}{3 !}\left(f*{x x x} h^{3}+3 f*{x x y} h^{2} k+3 f*{x y y} h k^{2}+f*{y y y} k^{3}\right)\left(x*{0}, y\_{0}\right) \cdots \end{aligned}

Again, these ideas may be extended to functions of three or more variables. Finally, we make use of the notation introduced in Subsec. 1.2.2. namely f\nabla f and Hf\mathcal{H} f, to express the expansion given by Eq. (1.16):

f(x,y)=f_0+hTf+12!hTHfh+, f(x, y)=f\_{0}+\mathbf{h}^{T} \cdot \nabla f+\frac{1}{2 !} \mathbf{h}^{T} \cdot \mathcal{H} f \cdot \mathbf{h}+\cdots,

where h=(hk)\mathbf{h}=\left(\begin{array}{c}h \\ k\end{array}\right) while f\nabla f and Hf\mathcal{H} f are given by Eqs. (1.12) and (1.13), respectively.