Taylor/Maclaurin series

A power series centred at a point $x_{0}$ is defined to be an expression of the form,

\sum_{n=0}^{\infty} c_{n}\left(x-x_{0}\right)^{n}=c_{0}+c_{1}\left(x-x_{0}\right)+c_{2}\left(x-x_{0}\right)^{2} \cdots

where $x_{0}$ and $c_{i}(i=0,1, \cdots)$ are constants. A power series centred at the origin (i.e. when $x_{0}=0$ ), can be thought of as an 'infinite polynomial' and has the form,

\sum_{n=0}^{\infty} c_{n} x^{n}=c_{0}+c_{1} x+c_{2} x^{2}+\cdots

In what follows, we discuss a general method for writing a power series representation of a function. We first assume that a function $f(x)$ has a power series representation about $x=x_{0}$ ,

f(x)=\sum_{n=0}^{\infty} c_{n}\left(x-x_{0}\right)^{n}

where the first three terms are given by Eq. (6.1). The next assumption is that $f(x)$ has derivatives of every order and that we can find them. The latter allows us to determine the unknown coefficients $c_{i}$ . Evaluated at $x=x_{0}$ , we have $c_{0}=f\left(x_{0}\right)$ . Using differentiation, we can obtain formulae for the other coefficients. For instance, differentiating once wrt $x$ yields

f^{\prime}(x)=c_{1}+2 c_{2}\left(x-x_{0}\right)+3 c_{3}\left(x-x_{0}\right)^{2}+\cdots,

which, evaluated again at $x=x_{0}$ gives $c_{1}=f^{\prime}\left(x_{0}\right)$ . Continuing with this pattern, we can write down the following formula for the coefficients,

c_{n}=\frac{f^{(n)}\left(x_{0}\right)}{n !}

where $f^{(n)}$ represents the $n^{\text {th }}$ derivative. Note that the formula works for $n=0$ since $0 !=1$ and $f^{(0)}(x)=f(x)$ , by definition. Provided a power series representation for the function $f(x)$ about $x=x_{0}$ exists then, the Taylor series for $f(x)$ about $x=x_{0}$ is given by,

f(x)=\sum_{n=0}^{\infty} \frac{f^{(n)}\left(x_{0}\right)}{n !}\left(x-x_{0}\right)^{n} .

If we use $x_{0}=0$ , we have a Taylor series about $x=0$ sometimes referred to as the Maclaurin series for $f(x)$ given by the following expression,

f(x)=\sum_{n=0}^{\infty} \frac{f^{(n)}(0)}{n !} x^{n} .

Remainder

Now, at this point we have not yet discussed where (i.e. for what values of $x$ ) does the Taylor series for $f(x)$ converge. This is covered later in Section 6.2. What we are interested in here is another important question: in the case where the Taylor series for $f(x)$ does converge, does it converge to $f(x)$ ? To answer this, we refer to the remainder, a quantity which represents the difference between $f(x)$ and the Taylor polynomial of degree $n$ for $f(x)$ centred at $x_{0}$ . Define the $n^{\text {th }}$ degree Taylor polynomial of $f(x)$ as,

T_{n}(x)=\sum_{m=0}^{n} \frac{f^{(m)}\left(x_{0}\right)}{m !}\left(x-x_{0}\right)^{m} .

With the full Taylor series given by (6.5), the sum $T_{n}(x)$ is the $n^{\text {th }}$ partial sum of the series. The remainder is defined to be,

R_{n}(x)=f(x)-T_{n}(x)

which represents the error between the function $f(x)$ and the $n^{\text {th }}$ degree polynomial. Note that the remainder depends on $n$ . Rearranging (6.8) and using (6.7), we can write the function as,

f(x)=\sum_{m=0}^{n} \frac{f^{(m)}\left(x_{0}\right)}{m !}\left(x-x_{0}\right)^{m}+R_{n}(x) .

The sum in (6.7) can be used as an approximation to $f(x)$ if, at $x=x_{0}, T_{n}$ and its first $n$ derivatives agree with $f$ . This gives $(n+1)$ conditions:

\begin{aligned} T_{n}\left(x_{0}\right) & =f\left(x_{0}\right), \\ T_{n}^{(1)}\left(x_{0}\right) & =f^{(1)}\left(x_{0}\right), \\ T_{n}^{(2)}\left(x_{0}\right) & =f^{(2)}\left(x_{0}\right), \\ & \cdots, \\ T^{(n-1)}\left(x_{0}\right) & =f^{(n-1)}\left(x_{0}\right), \\ T^{(n)}\left(x_{0}\right) & =f^{(n)}\left(x_{0}\right) . \end{aligned}

We now state the following theorem.

Taylor's theorem

Let $f$ be an $(n+1)$ times differentiable function on an open interval containing the points $x_{0}$ and $x$ . Then,

f(x)=T_{n}(x)+R_{n}(x)

where

R_{n}(x)=\frac{f^{(n+1)}(c)}{(n+1) !}\left(x-x_{0}\right)^{n+1}

for some number $c$ between $x_{0}$ and $x$ . Equation (6.10) gives the Lagrange form of the remainder (see also Subsec. 6.1.3 for more details on remainders). Note that there are several formulae for the remainder like, for instance, the integral form given by

R_{n}(x)=\frac{1}{n !} \int_{a}^{x}(x-t)^{n} f^{(n+1)}(t) d t .

If we can show that, $R_{n}(x) \rightarrow 0$ as $n \rightarrow \infty$ then, we get a sequence of increasingly better approximations to $f$ leading to the Taylor series as given by Eq. (6.5). In general, the series will converge only for certain values of $x$ determined by the radius of convergence of the series (see Subsec. 6.2.3). Note that taking $x_{0}=0$ in Taylor's theorem gives a similar result for the Maclaurin series.

Example 6.1 Compute the Maclaurin series for $f(x)=\sin x$ with $n=4$ .

Solution The $4^{\text {th }}$ degree Maclaurin series is given by the sum,

S_{4}(x)=\sum_{m=0}^{4} \frac{f^{(m)}(0)}{m !} x^{m} .

We then proceed to evaluate the function $f$ and its derivatives at $0: \quad f(0)=0$ , $f^{\prime}(0)=1, f^{\prime \prime}(0)=0, f^{\prime \prime \prime}(0)=-1$ , and $f^{(4)}(0)=0$ . Back in (6.11), we have

S_{4}(x)=x-\frac{x^{3}}{6}

which approximates $\sin x$ near $x=0$ . Now, how good is this approximation? To show this, we need to determine that the remainder term [which gives the difference between $f(x)$ and its approximation, $\left.S_{4}(x)\right]$ is small. Taking $x=0$ in Eq. (6.9) and using Eq. (6.12), we write

\sin x=x-\frac{x^{3}}{6}+R_{4}(x)

where, using Eq. (6.10),

R_{4}(x)=\frac{f^{(5)}(c)}{5 !} x^{5}

for some $c$ between 0 and $x$ . Since $f^{(5)}(x)=\cos x$ , the remainder term is $R_{4}(x)=(\cos c) x^{5} / 5$ !. Further, $|\cos c| \leq 1$ which gives $\left|R_{4}(x)\right| \leq|x|^{5} / 5$ !. Finally, using Eq. (6.13),

\left|\sin x-\left(x-\frac{x^{3}}{6}\right)\right| \leq \frac{|x|^{5}}{5 !}

The $|x|^{5}$ term should decay rapidly in the vicinity of $x=0$ . Example 6.2 Compute the Maclaurin series for $f(x)=\sin x$ , as in Example 6.1 using arbitrary $n$ .

Solution The $\mathrm{n}^{\text {th }}$ degree Maclaurin series is given by the sum,

S_{n}(x)=\sum_{m=0}^{n} \frac{f^{(m)}(0)}{m !} x^{m} .

We evaluate the function $f$ and its derivatives at 0 : we find that $f^{(m)}(0)$ is 0 if $m$ is even and alternates between 1 and -1 if $m$ is odd, as seen above for Example 6.1. For arbitrary $n$ , we have

\sin x=x-\frac{x^{3}}{3 !}+\frac{x^{5}}{5 !}+\cdots+(-1)^{n-1} \frac{x^{2 n-1}}{(2 n-1) !}+R_{n}(x) .

The remainder term is given as

R_{n}(x)=\frac{\sin ^{(n+1)}(c)}{(n+1) !} x^{n+1},

for some $c$ between 0 and $x$ . Since the derivatives of $\sin x$ only take values between -1 and 1 , we have that

\left|R_{n}(x)\right| \leq \frac{|x|^{n+1}}{(n+1) !}

We need to show that the Maclaurin series converges to $\sin x$ for all $x$ which is equivalent to showing that for any fixed value of $x$ the limits of the remainders, $R_{n} \rightarrow 0$ as $n \rightarrow \infty$ . If $x>1,|x|^{n+1}$ grows fast as $n$ increases, however, for $n>>|x|$ , the denominator increases faster. See below for a rough sketch of the proof.

Limit of $\frac{|x|^{n}}{n !}$ as $n \rightarrow \infty$

We show that $|x|^{n} / n ! \rightarrow 0$ as $n \rightarrow \infty$ for a fixed value of $x$ by making use of the Sandwich theorem. This brief section aims to add to the intuition in Example 6.2 that $R_{n} \rightarrow 0$ as $n \rightarrow \infty$ for fixed $x$ . We assume the following inequality holds

a_{n}<b_{n}<c_{n}

The Sandwich theorem states that if

\lim _{n \rightarrow \infty} a_{n}=\lim _{n \rightarrow \infty} c_{n}=L

and there exists an integer $N$ for all $n>N$ , then

\lim _{n \rightarrow \infty} b_{n}=L .

Here, $a_{n}, b_{n}$ , and $c_{n}$ are assumed to be sequences. To simplify the discussion, we fix the value of $|x|$ to 3 and let $b_{n}=|3|^{n} / n$ ! where clearly $b_{n} \geq 0$ . We have

b_{n}=\frac{3 \cdot 3 \cdot 3 \cdots 3}{1 \cdot 2 \cdot 3 \cdots n}

which can be expressed as individuals fractions,

b_{n}=\frac{3}{1} \cdot \frac{3}{2} \cdot \frac{3}{3} \cdot \frac{3}{4} \cdots \frac{3}{n} .

We can already see that the limit approaches 0 as $n$ increases, since we are multiplying by smaller and smaller functions. Now, apart from the fractions $3 / 1,3 / 2,3 / 3$ , the rest of the $n-3$ fractions in the sequence above are at most equal to $3 / 4$ which means we can write

0 \leq b_{n} \leq \frac{9}{2}\left(\frac{3}{4}\right)^{n-3}

Finally, let $a_{n}=0$ and $c_{n}=(9 / 2)(3 / 4)^{n-3}$ ; the limits of both $a_{n}$ and $b_{n}$ are 0 as $n \rightarrow \infty$ and, since $a_{n} \leq b_{n} \leq c_{n}$ it follows, by the Sandwich theorem, $b_{n} \rightarrow 0$ .

Common power series

Using the definitions of series defined earlier, we can write down the Maclaurin series for $\cos x$ and $e^{x}$ , valid for any real $x$ :

\begin{gathered} \cos x=1-\frac{x^{2}}{2 !}+\frac{x^{4}}{4 !}-\cdots=\sum_{n=0}^{\infty} \frac{(-1)^{n}}{(2 n) !} x^{2 n} ; \\ e^{x}=1+x+\frac{x^{2}}{2 !}+\frac{x^{3}}{3 !}+\cdots=\sum_{n=0}^{\infty} \frac{x^{n}}{n !} . \end{gathered}

As a last example, consider the series of the binomial function given by $f_{p}(x)=(1+x)^{p}$ ,

(1+x)^{p}=1+p x+\frac{1}{2 !} p(p-1) x^{2}+\frac{1}{3 !} p(p-1)(p-2) x^{3}+\cdots .

The series is valid in $-1<x<1$ . If $p$ is not a positive integer, then the above series has infinitely many nonzero terms. If $p$ is a positive integer, the series terminates; the binomial function is a polynomial for positive $p$ with only the first $p+1$ terms are nonzero. For instance, with $p=2$ ,

f_{2}(x)=(1+x)^{2}

and its Maclaurin series, say $T$ , is given by

T(x)=1+2 x+x^{2},

i.e. $f_{2}(x)=T(x)$ , since all derivatives higher than or equal to the third, vanish.

Taylor's formula with integral remainder

Here, we derive Taylor's formula with integral remainder by reconstructing a function $f=f(x)$ through repeated integration (note that notation, definitions, and techniques pertaining to integration are covered in Chapter 8). Through the derivation in this subsection we arrive at the definitions of the remainder discussed in Subsec. 6.1.1. Starting with the identity,

\int_{x_{0}}^{x} f^{\prime}(x) d x=f(x)-f\left(x_{0}\right)

we have

f(x)=f\left(x_{0}\right)+\int_{x_{0}}^{x} f^{\prime}(x) d x .

Since $f(x)$ is arbitrary, (6.22) should also hold with $f(x)$ replaced by the function $f^{\prime}(x)$ , such that

f^{\prime}(x)=f^{\prime}\left(x_{0}\right)+\int_{x_{0}}^{x} f^{\prime \prime}(x) d x .

Using the above expression for $f^{\prime}(x)$ in the integral in (6.22), we obtain,

\begin{aligned} f(x) & =f\left(x_{0}\right)+\int_{x_{0}}^{x}\left\{f^{\prime}\left(x_{0}\right)+\int_{x_{0}}^{x} f^{\prime \prime}(x) d x\right\} d x \\ & =f\left(x_{0}\right)+f^{\prime}\left(x_{0}\right)\left(x-x_{0}\right)+\int_{x_{0}}^{x} \int_{x_{0}}^{x} f^{\prime \prime}(x) d x d x . \end{aligned}

Next, just as we obtained (6.23) from (6.22) by replacing $f(x)$ with $f^{\prime}(x)$ , we can replace $f^{\prime \prime}(x)$ in $(6.23)$ by $f^{\prime \prime \prime}(x)$ so that

f^{\prime \prime}(x)=f^{\prime \prime}\left(x_{0}\right)+\int_{x_{0}}^{x} f^{\prime \prime \prime}(x) d x

Using this result in (6.24) yields,

\begin{aligned} f(x) & =f\left(x_{0}\right)+f^{\prime}\left(x_{0}\right)\left(x-x_{0}\right)+\int_{x_{0}}^{x} \int_{x_{0}}^{x}\left\{f^{\prime \prime}\left(x_{0}\right)+\int_{x_{0}}^{x} f^{\prime \prime \prime}(x) d x\right\} d x d x \\ & =f\left(x_{0}\right)+f^{\prime}\left(x_{0}\right)\left(x-x_{0}\right)+\frac{f^{\prime \prime}\left(x_{0}\right)}{2 !}\left(x-x_{0}\right)^{2} \\ & +\int_{x_{0}}^{x} \int_{x_{0}}^{x} \int_{x_{0}}^{x} f^{\prime \prime \prime}(x) d x d x d x . \end{aligned}

Repeating this process, under the assumption that $f(x)$ is sufficiently differentiable of course, we find

f(x)=f\left(x_{0}\right)+f^{\prime}(x)\left(x-x_{0}\right)+\frac{f^{\prime \prime}(a)}{2 !}\left(x-x_{0}\right)^{2}+\cdots+\frac{f^{(n)}\left(x_{0}\right)}{n !}\left(x-x_{0}\right)^{n}+R_{n}(x)

where

R_{n}(x)=\int_{x_{0}}^{x} \cdots \int_{x_{0}}^{x} f^{(n+1)}(x)(d x)^{n+1}

and $(d x)^{n+1}$ means $d x \ldots d x, n+1$ times. Equation (6.27) is known as Taylor's formula with remainder, where the remainder is expressed in integral form by (6.28).

Suppose that $m \leq f^{(n+1)}(x) \leq M$ over $[a, x]$ , where $m$ and $M$ are constants. Then

\int_{x_{0}}^{x} \ldots \int_{x_{0}}^{x} m(d x)^{n+1} \leq R_{n} \leq \int_{x}^{x_{0}} \cdots \int_{x_{0}}^{x} M(d x)^{n+1} .

Integrating, we have

m \frac{\left(x-x_{0}\right)^{n+1}}{(n+1) !} \leq R_{n} \leq M \frac{\left(x-x_{0}\right)^{n+1}}{(n+1) !} .

It follows from (6.29) that we must be able to express $R_{n}$ as

R_{n}(x)=\frac{f^{(n+1)}(c)}{(n+1) !}\left(x-x_{0}\right)^{n+1}

where $c$ is some suitable point in $\left[x_{0}, x\right]$ ; this is the Lagrange form of $R_{n}$ which we saw in Eq. (6.10).

In fact, this result for the remainder can also be obtained with the mean value theorem of the integral calculus:

if $m \leq f(x) \leq M$ over $[a, b]$ then

m(b-a) \leq \int_{a}^{b} f(x) d x \leq M(b-a) .

If $m$ and $M$ are minimum and maximum values of $f$ over $[a, b]$ and $f$ is continuous, then there must be some point in $[a, b]$ , say $c$ , such that

\int_{a}^{b} f(x) d x=f(c)(b-a)

and this is known as the mean value theorem of the integral calculus. Using this theorem in (6.28), where we now have $n+1$ derivatives, yields (6.30).

Power series Convergence