Engineering at Alberta Courses

Introduction to Numerical Analysis: Taylor Series

Definitions

Let $f:\mathbb{R}\rightarrow \mathbb{R}$ be a smooth (differentiable) function, and let $a\in\mathbb{R}$ , then a Taylor series of the function $f(x)$ around the point $a$ is given by:

$f(x)=f(a)+f'(a) (x-a)+\frac{f''(a)}{2!}(x-a)^2+\frac{f'''(a)}{3!}(x-a)^3+\cdots+\frac{f^{(n)}(a)}{n!}(x-a)^n+\cdots$

In particular, if $a=0$ , then the expansion is known as the Maclaurin series and thus is given by:

$f(x)=f(0)+f'(0) x+\frac{f''(0)}{2!}x^2+\frac{f'''(0)}{3!}x^3+\cdots+\frac{f^{(n)}(0)}{n!}x^n+\cdots$

Taylor’s Theorem

Many of the numerical analysis methods rely on Taylor’s theorem. In this section, a few mathematical facts are presented (mostly without proof) which serve as the basis for Taylor’s theorem.

Extreme Values of Smooth Functions

Definition: Local Maximum and Local Minimum

Let $f:[a,b]\rightarrow \mathbb{R}$ . $f$ is said to have a local maximum at a point $c\in(a,b)$ if there exists an open interval $I\subset[a,b]$ such that $c\in I$ and $\forall x\in I:f(x)\leq f(c)$ . On the other hand, $f$ is said to have a local minimum at a point $c\in(a,b)$ if there exists an open interval $I\subset[a,b]$ such that $c\in I$ and $\forall x\in I:f(x)\geq f(c)$ . If $f$ has either a local maximum or a local minimum at $c$ , then $f$ is said to have a local extremum at $c$ .

Proposition 1

Let $f:[a,b]\rightarrow \mathbb{R}$ be smooth (differentiable). Assume that $f$ has a local extremum (maximum or minimum) at a point $c\in(a,b)$ , then $f'(c)=0$ .
This proposition simply means that if a smooth function attains a local maximum or minimum at a particular point, then the slope of the function is equal to zero at this point.
As an example, consider the function $f:[-3,3]\rightarrow \mathbb{R}$ with the relationship $f(x)=x^3-3x$ . In this case, $f(-1)=2$ is a local maximum value for $f$ attained at $x=-1$ and $f(1)=-2$ is a local minimum value of $f$ attained at $x=1$ . These local extrema values are associated with a zero slope for the function $f$ since

$f'(x)=\frac{\mathrm{d}f}{\mathrm{d}x}=3x^2-3$

$x=-1$ and $x=1$ are locations of local extrema and for both we have $f'(1)=f'(-1)=0$ . The red lines in the next figure show the slope of the function $f$ at the extremum values.
Theorem2
View Mathematica Code that Generated the Above Figure

Clear[x]
y = x^3 - 3 x;
Plot[y, {x, -3, 3}, Epilog -> {PointSize[0.04], Point[{-1, 2}], Point[{1, -2}], Red, Line[{{-3, 2}, {1.5, 2}}], Line[{{3, -2}, {-1.5, -2}}]}, Filling -> Axis, PlotRange -> All, Frame -> True, AxesLabel -> {"x", "y"}]

“Smoothness” or “Differentiability” is a very important requirement for the proposition to work. As an example, consider the function $f:[-1,1]\rightarrow \mathbb{R}$ defined as $f(x)=|x|$ . The function $f$ has a local minimum at $x=0$ , however, $f'(0)$ is not defined as the slope as $x\rightarrow 0$ from the right is different from the slope as $x\rightarrow 0$ from the left as shown in the next figure.
theorem3

View Mathematica Code that Generated the Above Figure

Clear[x]
y = Abs[x];
Plot[y, {x, -1, 1}, Epilog -> {PointSize[0.04], Point[{0, 0}]}, PlotRange -> All, Frame -> True, AxesLabel -> {"x", "y"}]

Proposition 2

Let $f:[a,b]\rightarrow\mathbb{R}$ be smooth (differentiable). Assume $\exists c\in(a,b)$ such that $f'(c)=0$ . Then,

If $f''(c)=0$ , then $f$ is constant around $c$ .
If $f''(c)>0$ , then $f(c)$ is a local minimum.
If $f''(c)<0$ , then $f(c)$ is a local maximum.

This proposition is very important for optimization problems when a local maximum or minimum is to be obtained for a particular function. In order to identify whether the solution corresponds to a local maximum or minimum, the second derivative of the function has to be evaluated. Considering the same example given above,

$f''(x)=\frac{\mathrm{d}^2f}{\mathrm{d}x^2}=6x$

We have already identified $x=1$ and $x=-1$ as locations of the local extremum values. To know whether they are local maxima or local minima, we can evaluate the second derivative at these points. $f''(1)=6>0$ . Therefore, $x=1$ is the location of a local minimum, while $f''(-1)=-6<0$ implying that $x=-1$ is the location of a local maximum.

Extreme Value Theorem

Statement: Let $f:[a,b]\rightarrow \mathbb{R}$ be continuous. Then, $f$ attains its maximum and its minimum value at some points $c_{max}$ and $c_{min}$ in the interval $[a,b]$ .

The theorem simply states that if we have a continuous function on a closed interval $[a,b]$ , then the image of $f$ contains a maximum value and a minimum value within the interval $[a,b]$ . The theorem is very intuitive. However, the proof is highly technical and relies on the definitions of real numbers and on continuous functions. For now, we will just illustrate the meaning of the theorem using an example. Consider the function $f:[-1.5,1.5]\rightarrow \mathbb{R}$ defined as:

$f(x)=x^3-3x$

The theorem states that $f$ has to attain a maximum value and a minimum value at a point within the interval. In this case, $f(-1)=2$ is the maximum value of $f$ attained at $x=-1\in[-1.5,1.5]$ and $f(1)=-2$ is the minimum value of $f$ attained at $x=1\in[-1.5,1.5]$ . Alternatively, if $f:[-3,3]\rightarrow \mathbb{R}$ with the same relationship as above, $f(-3)=-18$ is the minimum value of $f$ attained at $x=-3\in[-3,3]$ and $f(3)=18$ is the maximum value of $f$ attained at $x=3\in[-3,3]$
The following figure shows the graph of the function on the specified intervals.
theorem
View Mathematica Code that Generated the Above Figures

Clear[x]
y = x^3 - 3 x;
Plot[y, {x, -1.5, 1.5}, Epilog -> {PointSize[0.04], Point[{-1, 2}], Point[{1, -2}]}, Filling -> Axis, PlotRange -> All, Frame -> True, AxesLabel -> {"x", "y"}]
Plot[y, {x, -3, 3}, Epilog -> {PointSize[0.04], Point[{-3, y /. x -> -3}], Point[{3, y /. x -> 3}]}, Filling -> Axis, PlotRange -> All, Frame -> True, AxesLabel -> {"x", "y"}]

Rolle’s Theorem

Statement: Let $f:[a,b]\rightarrow \mathbb{R}$ be differentiable. Assume that $f(a)=f(b)$ , then there is at least one point $c\in(a,b)$ where $f'(c)=0$ .

The proof of Rolle’s theorem is straightforward from Proposition 1 and the Extreme Value Theorem above. The Extreme Value Theorem ensures that there is a local maximum or local minimum within the interval, while proposition 1 ensures that at this local extremum, the slope of the function is equal to zero. As an example, consider the function $f:[0.5,1.5]\rightarrow \mathbb{R}$ defined as $f(x)=12.5-5x-30x^2+20x^3$ . $f(0.5)=f(1.5)=5$ . This ensures that there is a point $c\in(0.5,1.5)$ with $f'(c)=0$ . Indeed, $f'\left(\frac{1}{2}+\frac{1}{\sqrt{3}}\right)=0$ and the point $\frac{1}{2}+\frac{1}{\sqrt{3}}\in (0.5,1.5)$ is the location of the local minimum. The following figure shows the graph of the function on the specified interval along with the point $c$ .

Theorem4

View Mathematica Code that Generated the Above Figure

Clear[x]
y = 20 (x - 1/2)^3 - 20 (x - 1/2) + 5;
Expand[y]
y /. x -> 1.5
y /. x -> 0.5
y /. x -> (1/2 + 1/Sqrt[3])
D[y, x] /. x -> (1/2 + 1/Sqrt[3])
Plot[y, {x, 0.5, 1.5}, Epilog -> {PointSize[0.04], Point[{1/2 + 1/Sqrt[3], y /. x -> 1/2 + 1/Sqrt[3]}], Red, Line[{{-3, y /. x -> 1/2 + 1/Sqrt[3]}, {1.5, y /. x -> 1/2 + 1/Sqrt[3]}}]}, Filling -> Axis, PlotRange -> All, Frame -> True, AxesLabel -> {"x", "y"}]

Mean Value Theorem

Statement: Let $f:[a,b]\rightarrow \mathbb{R}$ be differentiable. Then, there is at least one point $c\in(a,b)$ such that $f'(c)=\frac{f(b)-f(a)}{b-a}$ .

The proof of the mean value theorem is straightforward by applying Rolle’s theorem to the function $g:[a,b]\rightarrow \mathbb{R}$ defined as:

$g(x)=f(x)-f(a)- \frac{f(b)-f(a)}{b-a}(x-a)$

Clearly, $g$ satisfies the conditions of Rolle’s theorem (Differentiable and $g(a)=g(b)=0$ ). We also have:

$g'(x)=f'(x)-\frac{f(b)-f(a)}{b-a}$

Therefore, by Rolle’s theorem, there is a point $c\in(a,b)$ such that $g'(c)=0\Rightarrow f'(c)=\frac{f(b)-f(a)}{b-a}$ .

The mean value theorem states that there is a point $c$ inside the interval such that the slope of the function at $c$ is equal to the average slope along the interval. The following example will serve to illustrate the main concept of the mean value theorem. Consider the function $f:[-3,3]\rightarrow \mathbb{R}$ defined as:

$f(x)=x^3-3x$

The slope or first derivative of $f$ is given by:

$f'(x)=3x^2-3$

The average slope of $f$ on the interval is given by:

$\mbox{average slope}=\frac{f(b)-f(a)}{b-a}=\frac{f(3)-f(-3)}{3-(-3)}=\frac{2 \times 18}{6}=6$

The two points $x=\sqrt{3}$ and $x=-\sqrt{3}$ have a slope equal to the average slope:

$f'(\sqrt{3})=f'(-\sqrt{3})=3\times 3-3=6$

The figure below shows the function $f$ on the specified interval. The line representing the average slope is shown in black connecting the points $(a,f(a))$ and $(b,f(b))$ . The red lines show the slopes at the points $x=\sqrt{3}$ and $x=-\sqrt{3}$ .

Theorem5

View Mathematica Code that Generated the Above Figure

Clear[x]
y = x^3 - 3 x;
averageslope = ((y /. x -> 3) - (y /. x -> -3))/(3 + 3)
dydx = D[y, x];
a = Solve[D[y, x] == averageslope, x]
Point1 = {x /. a[[1, 1]], y /. a[[1, 1]]}
Point2 = {x /. a[[2, 1]], y /. a[[2, 1]]}
Plot[y, {x, -3, 3},  Epilog -> {PointSize[0.04], Point[{-3, y /. x -> -3}], Point[{3, y /. x -> 3}], Line[{{-3, y /. x -> -3}, {3, y /. x -> 3}}], Point[Point1],Point[Point2], Red, Line[{Point1 + {-1, -averageslope}, Point1, Point1 + {1, averageslope}}], Line[{Point2 + {-1, -averageslope}, Point2, Point2 + {1, averageslope}}]}, Filling -> Axis, PlotRange -> All, Frame -> True, AxesLabel -> {"x", "y"}]

Taylor’s Theorem

As an introduction to Taylor’s Theorem, let’s assume that we have a function $f:\mathbb{R}\rightarrow\mathbb{R}$ that can be represented as a polynomial function in the following form:

$f(x)=b_0 + b_1 (x-a)+b_2(x-a)^2+b_3(x-a)^3+b_4(x-a)^4+\cdots + b_n(x-a)^n+\cdots$

where $a$ is a fixed point and $\forall i\geq 0:b_i$ is a constant. The best way to find these constants is to find $f$ and its derivatives when $x=a$ . So, when $x=a$ we have:

$f(a)=b_0 + b_1 (a-a)+b_2(a-a)^2+b_3(a-a)^3+b_4(a-a)^4+\cdots + b_n(a-a)^n+\cdots=b_0$

Therefore, $b_0=f(a)$ . The derivatives of $f$ have the form:

$\begin{split} f'(x)&=b_1 +2b_2(x-a)+3b_3(x-a)^2+4b_4(x-a)^3+\cdots + nb_n(x-a)^{(n-1)}+\cdots\\ f''(x)&=2b_2+2\times 3b_3(x-a)+3\times 4b_4(x-a)^2+\cdots + (n-1)nb_n(x-a)^{(n-2)}+\cdots\\ \cdots\\ f^{(n)}(x)&=n!b_n+\cdots\\ \end{split}$

The derivatives of $f$ when $x=a$ have the form:

$\begin{split} f'(a)&=b_1 +2b_2(a-a)+3b_3(a-a)^2+4b_4(a-a)^3+\cdots + nb_n(a-a)^{(n-1)}+\cdots\\ f''(a)&=2b_2+2\times 3b_3(a-a)+3\times 4b_4(a-a)^2+\cdots + (n-1)nb_n(a-a)^{(n-2)}+\cdots\\ \cdots\\ f^{(n)}(a)&=n!b_n+\cdots \end{split}$

Therefore, $\forall n\geq 0$ :

$b_n=\frac{f^{(n)}(a)}{n!}$

The above does not really serve as a rigorous proof for Taylor’s Theorem but rather an illustration that if an infinitely differentiable function can be represented as the sum of an infinite number of polynomial terms, then, the Taylor series form of a function defined at the beginning of this section is obtained. The following is the exact statement of Taylor’s Theorem:

Statement of Taylor’s Theorem: Let $f:\mathbb{R}\rightarrow \mathbb{R}$ be $n+1$ times differentiable on an open interval $I$ . Let $a\in I$ . Then, $\forall x \in I:\exists \xi$ between $a$ and $x$ such that:

$f(x)=f(a)+f'(a) (x-a)+\frac{f''(a)}{2!}(x-a)^2+\frac{f'''(a)}{3!}(x-a)^3+\cdots+\frac{f^{(n)}(a)}{n!}(x-a)^n+\frac{f^{(n+1)}(\xi)}{(n+1)!}(x-a)^{n+1}$

Explanation and Importance: Taylor’s Theorem has numerous implications in analysis in engineering. In the following we will discuss the meaning of the theorem and some of its implications:

Simply put, Taylor’s Theorem states the following: if the function $f$ and its $n+1$ derivatives are known at a point $a$ , then, the function at a point $x$ away from $a$ can be approximated by the value of the Taylor’s approximation $P_n(x)$ :
$f(x)\approx P_n(x)=f(a)+f'(a) (x-a)+\frac{f''(a)}{2!}(x-a)^2+\frac{f'''(a)}{3!}(x-a)^3+\cdots+\frac{f^{(n)}(a)}{n!}(x-a)^n$

The error (difference between the approximation $P_n(x)$ and the exact $f(x)$ is given by:

$E=f(x)-P_n(x)=\frac{f^{(n+1)}(\xi)}{(n+1)!}(x-a)^{n+1}$

The term $f^{(n+1)}(\xi)$ is bounded since $f^{(n+1)}$ is a continuous function on the interval from $a$ to $x$ . Therefore, when $x>a$ , the upper bound of the error can be given as:

$|E|\leq \max_{\xi\in[a,x]}\frac{|f^{(n+1)}(\xi)|}{(n+1)!}(x-a)^{n+1}$

While, when $x<a$ , the upper bound of the error can be given as:

$|E|\leq \max_{\xi\in[x,a]}\frac{|f^{(n+1)}(\xi)|}{(n+1)!}(a-x)^{n+1}$

The above implies that the error is directly proportional to $(x-a)^{n+1}$ . This is traditionally written as follows:

$f(x)=P_n(x)+\mathcal{O} (h^{n+1})$

where $h=(x-a)$ . In other words, as $x-a$ gets smaller and smaller, the error gets smaller in proportion to $(x-a)^{n+1}$ . As an example, if we choose $x_1-a=0.1$ and then $x_2-a=0.05$ , then, $\frac{E_1}{E_2}\geq\left(\frac{0.1}{0.05}\right)^{n+1}=2^{n+1}$ . I.e., if the step size is halved, the error is divided by $2^{n+1}$ .
If the function $f:[c,d]\rightarrow \mathbb{R}$ is infinitely differentiable on an interval $I$ , and if $a\in I$ , then $\forall x\in I:f(x)$ is the limit of the sum of the Taylor series. The error which is the difference between the infinite sum and the approximation is called the truncation error as defined in the error section.
There are many rigorous proofs available for Taylor’s Theorem and the majority rely on the mean value theorem above. Notice that if we choose $n=0$ , then the mean value theorem is obtained. For a rigorous proof, you can check one of these links: link 1 or link 2. Note that these proofs rely on the mean value theorem. In particular, L’Hôpital’s rule was used in the Wikipedia proof which in turn relies on the mean value theorem.

The following code illustrates the difference between the function $f(x)=\sin{x}+0.01x^2$ and the Taylor’s polynomial $P(x)$ . You can download the code, change the function, the point $a$ , and the range of the plot to see how the Taylor series of other functions behave.

View Mathematica Code

Taylor[y_, x_, a_, n_] := (y /. x -> a) + 
  Sum[(D[y, {x, i}] /. x -> a)/i!*(x - a)^i, {i, 1, n}]
f = Sin[x] + 0.01 x^2;
Manipulate[
 s = Taylor[f, x, 1, nn];
 Grid[{{Plot[{f, s}, {x, -10, 10}, PlotLabel -> "f(x)=Sin[x]+0.01x^2",
      PlotLegends -> {"f(x)", "P(x)"}, 
     PlotRange -> {{-10, 10}, {-6, 30}}, 
     ImageSize -> Medium]}, {Expand[s]}}], {nn, 1, 30, 1}]

The Mathematica function Series can also be used to generate the Taylor expansion of any function:
View Mathematica Code

Series[Tan[x],{x,0,7}]
Series[1/(1+x^2),{x,0,10}]

The following tool shows how the Taylor series expansion around the point $a=1$ , termed $P(x)$ in the figure, can be used to provide an approximation of different orders to a cubic polynomial, termed $f(x)$ in the figure. Use the slider to change the order of the series expansion. The tool provides the error at $x=3$ , namely $E=f(3)-P(3)$ . What happens when the order reaches 3?

Applications and Examples

MacLaurin Series

The following are the MacLaurin series for some basic infinitely differentiable functions:

$\begin{split} e^x&=1+x+\frac{x^2}{2!}+\frac{x^3}{3!}+\cdots = \sum_{n=0}^\infty \frac{x^n}{n!}\\ \sin{x}&=x-\frac{x^3}{3!}+\frac{x^5}{5!}-\cdots = \sum_{n=0}^\infty \frac{(-1)^{n}x^{2n+1}}{(2n+1)!}\\ \cos{x}&=1-\frac{x^2}{2!}+\frac{x^4}{4!}-\cdots = \sum_{n=0}^\infty \frac{(-1)^{n}x^{2n}}{(2n)!} \end{split}$

Numerical Differentiation

Taylor’s Theorem provides a means for approximating the derivatives of a function. The first-order Taylor approximation of a function $f$ is given by:

$f(x)=f(a)+f'(a)h+\mathcal O (h^2)$

where $h=x-a$ is the step size. Then, :

$f'(a)=\frac{f(x)-f(a)}{h}+\frac{\mathcal O (h^2)}{h}=\frac{f(x)-f(a)}{h}+\mathcal O (h)$

Forward finite-difference

If we use the Taylor series approximation to estimate the value of the function $f$ at a point $x_{i+1}$ knowing the values at point $x_i<x_{i+1}$ then,we have:

(1) $\begin{equation*} f(x_{i+1})=f(x_{i})+f'(x_{i})h+\mathcal{O} (h^2) \end{equation*}$

where $h=x_{i+1}-x_i$ . In this case, the forward finite-difference can be used to approximate $f'(x_{i})$ as follows:

$f'(x_{i})=\frac{f(x_{i+1})-f(x_i)}{x_{i+1}-x_i}+\mathcal{O} (h)$

Backward finite-difference

If we use the Taylor series approximation to estimate the value of the function $f$ at a point $x_{i-1}$ knowing the values at point $x_i>x_{i-1}$ then, we have:

(2) $\begin{equation*} f(x_{i-1})=f(x_{i})-f'(x_{i})h+\mathcal O (h^2) \end{equation*}$

where $h=x_{i}-x_{i-1}$ . In this case, the backward finite-difference can be used to approximate $f'(x_{i})$ as follows:

$f'(x_{i})=\frac{f(x_{i-1})-f(x_i)}{x_{i-1}-x_i}+\mathcal O (h)$

Centred Finite Difference

The centred finite difference can provide a better estimate for the derivative of a function at a particular point. If the values of a function $f$ are known at the points $x_{i-1}<x_i<x_{i+1}$ and $x_{i+1}-x_i=x_i-x_{i-1}=h$ , then, we can use the Taylor series to find a good apprxoimation for the derivative as follows:

$\begin{split} f(x_{i+1})&=f(x_i)+f'(x_i) h+\frac{f''(x_i)}{2!}h^2+\mathcal{O}(h^3)\\ f(x_{i-1})&=f(x_i)+f'(x_i)(-h)+\frac{f''(x_i)}{2!}h^2+\mathcal{O}(h^3) \end{split}$

Subtracting the above two equations and dividing by $h$ gives the following:

$f'(x_i)=\frac{f(x_{i+1})-f(x_{i-1})}{2h}+\mathcal{O}(h^2)$

Building Differential Equations

See momentum balance and beam approximation for examples where the Taylor’s approximation is used to build a differential equation.

Approximating Continuous Functions

Taylor’s theorem essentially discusses approximating differentiable functions using polynomials. The approximation can be as close as needed by adding more polynomial terms and/or by ensuring that the step size $x-a$ is small enough. It is important to realize that polynomial approximations are valid for continuous functions that are not necessarily differentiable at every point. Let $f:[a,b]\rightarrow \mathbb{R}$ , then the Stone-Weierstrass Theorem states that for any small number $\varepsilon>0:\exists P(x)$ a polynomial function such that the biggest difference between $\max_{x\in[a,b]}|f(x)-P(x)|\leq \varepsilon$ . In other words, we can always find a polynomial function that approximates any continuous function on the interval $[a,b]$ within any degree of accuracy sought! This fact is extremely important in all engineering applications. Essentially, it allows modelling any continuous functions using polynomials. In other words, if a problem wishes to find the distribution of a variable (stress, strain, velocity, density, etc…) as a function of position, and if this variable is continuous, then, in the majority of applications we can assume differentiability and a Taylor approximation for the unknown function.

Examples

Example 1

Apply Taylor’s Theorem to the function $f:[0,\infty)\rightarrow\mathbb{R}$ defined as $f(x)=\sqrt{x}$ to estimate the value of $f(5)$ . Use $a=4$ . Estimate an upper bound for the error.

Solution

The Taylor approximation of the function $f(x)$ around the point $a=4$ is given as follows:

$f(x)=f(4)+f'(4) (x-4)+\frac{f''(4)}{2!}(x-4)^2+\frac{f'''(4)}{3!}(x-4)^3+\cdots+\frac{f^{(n)}(4)}{n!}(x-4)^n+\cdots$

If $n+1$ terms are used (including $f(a)$ ), then the upper bound for the error is:

$|E|\leq \max_{\xi\in[a,x]}\frac{|f^{(n+1)}(\xi)|}{(n+1)!}(x-a)^{n+1}$

With $a=4$ , an upper bound for the error can be calculated by assuming $\xi=4$ . That’s because the absolute values of the derivatives of $f$ attain their maximum value in the interval $[4,x]$ at 4. The derivatives of $f(x)$ have the following form:

$\begin{split} f'(x)&=\frac{1}{2\sqrt{x}}\\ f''(x)&=\frac{-1}{4x^{\frac{3}{2}}}=\frac{-1}{4x\sqrt{x}}\\ f'''(x)&=\frac{3}{8x^{\frac{5}{2}}}=\frac{3}{8x^2\sqrt{x}}\\ f''''(x)&=\frac{-15}{16x^{\frac{7}{2}}}=\frac{-15}{16x^3\sqrt{x}} \end{split}$

The derivatives of the function evaluated at the point $a=4$ can be calculated as:

$\begin{split} f'(4)&=\frac{1}{4}\\ f''(4)&=\frac{-1}{32}\\ f'''(4)&=\frac{3}{256}\\ f''''(4)&=\frac{-15}{2048} \end{split}$

If two terms are used, then:

$f(5)\approx f(4)+f'(4)(5-4)=2+\frac{1}{4}=2.25$

In this case, the upper bound for the error is:

$|E|\leq \frac{|f''(4)|}{2!}(5-4)^2=\frac{1}{32(2)}=\frac{1}{64}=0.015625$

Using Mathematica, the square root of $5$ approximated to 4 decimal places is $\sqrt{5}=2.2361$ . Therefore, the error when two terms are used is:

$E=2.2361-2.25=-0.0139$

which satisfies that the actual error is less the upper bound:

$|-0.0139|<0.015625$

If three terms are used, then:

$f(5)\approx f(4)+f'(4)(5-4)+\frac{f''(4)}{2!}(5-4)^2=2+\frac{1}{4}+\frac{-1}{64}=2.23438$

In this case, the upper bound for the error is:

$|E|\leq \frac{|f'''(4)|}{3!}(5-4)^3=\frac{3}{256\times 3\times 2}=0.001953$

The actual error in this case is indeed less than the upper bound:

$E=2.2361-2.23438=0.00172<0.001953$

The following code provides a user-defined function for the Taylor series having the following inputs: a function $f$ , the value of $x$ , the value of $a$ , and the number of terms $n$ including the constant term.
View Mathematica Code

Clear[x]
Taylor[f_, xi_, a_, n_] := Sum[(D[f, {x, i}] /. x -> a)/i!*(xi - a)^i, {i, 0, n - 1}]
f = (x)^(0.5);
a = Table[{i, Taylor[f, 5, 4, i]}, {i, 2, 21}];
a // MatrixForm

Example 2

Apply Taylor’s Theorem to the function $f:(-\infty,1]\rightarrow\mathbb{R}$ defined as $f(x)=\sqrt{1-x}$ to estimate the value of $f(0.1)$ and $f(-2)$ . Use $a=0$ . Estimate an upper bound for the error.

Solution

First, we will calculate the numerical solution for the $f(0.1)$ and $f(-2)$ :

$f(0.1)=\sqrt{1-0.1}=\sqrt{0.9}=0.948683 \qquad f(-2)=\sqrt{1+2}=\sqrt{3}=1.73205$

The Taylor approximation around $a=0$ is given as:

$f(x)=f(0)+f'(0) (x-0)+\frac{f''(0)}{2!}(x-0)^2+\frac{f'''(0)}{3!}(x-0)^3+\cdots+\frac{f^{(n)}(0)}{n!}(x-0)^n+\cdots$

If $n+1$ terms are used (including $f(0)$ ) and if $x>a$ , then the upper bound of the error is:

$|E|\leq \max_{\xi\in[a,x]}\frac{|f^{(n+1)}(\xi)|}{(n+1)!}(x-a)^{n+1}$

If $x<a$ , then, the upper bound of the error is:

$|E|\leq \max_{\xi\in[x,a]}\frac{|f^{(n+1)}(\xi)|}{(n+1)!}(a-x)^{n+1}$

The derivatives of the function are given by:

$\begin{split} f(x)&=(1-x)^{0.5}\\ f'(x)&=-\frac{0.5}{(1-x)^{0.5}}\\ f''(x)&=-\frac{0.25}{(1-x)^{1.5}}\\ f'''(x)&=-\frac{0.375}{(1-x)^{2.5}}\\ f''''(x)&=-\frac{0.9375}{(1-x)^{3.5}}\\ f^{(5)}(x)&=-\frac{3.28125}{(1-x)^{4.5}}\\ f^{(6)}(x)&=-\frac{14.7656}{(1-x)^{5.5}}\\ f^{(7)}(x)&=-\frac{81.2109}{(1-x)^{6.5}}\\ f^{(8)}(x)&=-\frac{527.871}{(1-x)^{7.5}} \end{split}$

when evaluated at $a=0$ these have the values:

$\begin{split} f(0)&=1\\ f'(0)&=-0.5\\ f''(0)&=-0.25\\ f'''(0)&=-0.375\\ f''''(0)&=-0.9375\\ f^{(5)}(0)&=-3.28125\\ f^{(6)}(0)&=-14.7656\\ f^{(7)}(0)&=-81.2109\\ f^{(8)}(0)&=-527.871 \end{split}$

For $x=0.1$ , and using two terms:

$f(0.1)\approx f(0)+f'(0)(0.1)=1-0.5\times 0.1 = 0.95$

Using three terms:

$f(0.1)\approx f(0)+f'(0)(0.1)+\frac{f''(0)}{2!}(0.1)^2=1-0.5\times 0.1-\frac{0.25}{2}0.1^2 = 0.94875$

Using four terms:

$\begin{split} f(0.1)&\approx f(0)+f'(0)(0.1)+\frac{f''(0)}{2!}(0.1)^2+\frac{f'''(0)}{3!}(0.1)^3\\ &=1-0.5\times 0.1-\frac{0.25}{2}0.1^2-\frac{0.375}{3\times 2}0.1^3 = 0.9486875 \end{split}$

The terms are getting closer to each other and four terms provide a good approximation. The error term in the theorem gives an upper bound for $|E|$ as follows:

$|E|\leq \max_{\xi\in[0,0.1]}\frac{|f^{(4)}(\xi)|}{4\times 3 \times 2\times 1}0.1^4=\frac{\frac{0.9375}{(1-\xi)^{3.5}}}{4\times 3 \times 2\times 1}0.1^4=\frac{3.9063(10)^{-6}}{(1-\xi)^{3.5}}$

The maximum value would be obtained for $\xi=0.1$ . Therefore:

$|E|\leq\frac{3.9063(10)^{-6}}{(1-0.1)^{3.5}}=5.64822(10)^{-6}$

Indeed, the actual value of the error is less than the upper bound:

$|E|=|0.948683-0.948688|=4.5(10)^{-6}<5.64822(10)^{-6}$

For $x=-2$ , the Taylor series for this function around $a=0$ doesn’t give a very good approximation as will be shown here but rather keeps oscillating. First, using two terms:

$f(-2)\approx f(0)+f'(0)(-2)=1-0.5\times (-2)=1+1=2$

Using three terms:

$f(-2)\approx f(0)+f'(0)(-2)+\frac{f''(0)}{2!}(-2)^2=1-0.5\times (-2)-\frac{0.25}{2}(-2)^2=1.5$

Using four terms:

$\begin{split} f(-2)&\approx f(0)+f'(0)(-2)+\frac{f''(0)}{2!}(-2)^2+\frac{f'''(0)}{3!}(-2)^3\\ &=1-0.5\times (-2)-\frac{0.25}{2}(-2)^2-\frac{0.375}{3\times 2}(-2)^3 \\ &= 2 \end{split}$

Using five terms:

$\begin{split} f(-2)&\approx f(0)+f'(0)(-2)+\frac{f''(0)}{2!}(-2)^2+\frac{f'''(0)}{3!}(-2)^3+\frac{f''''(0)}{4!}(-2)^4\\ &=1-0.5\times (-2)-\frac{0.25}{2}(-2)^2-\frac{0.375}{3\times 2}(-2)^3 -\frac{0.9375}{4\times 3\times 2}(-2)^4\\ &= 1.375 \end{split}$

Using six terms:

$\begin{split} f(-2)&\approx f(0)+f'(0)(-2)+\frac{f''(0)}{2!}(-2)^2+\frac{f'''(0)}{3!}(-2)^3+\frac{f''''(0)}{4!}(-2)^4+\frac{f'''''(0)}{5!}(-2)^5\\ &=1-0.5\times (-2)-\frac{0.25}{2}(-2)^2-\frac{0.375}{3\times 2}(-2)^3 -\frac{0.9375}{4\times 3\times 2}(-2)^4-\frac{-3.28125}{5\times 4\times 3\times 2}(-2)^5\\ &=2.25 \end{split}$

In fact, the following table gives the values up to 21 terms. It is clear that the Taylor series is diverging.
Taylor1

The error term in the theorem gives an upper bound for $|E|$ when six terms are used as follows:

$|E|\leq\max_{\xi\in[-2,0]}\frac{|f^{(6)}(\xi)|}{6\times 5\times 4\times 3 \times 2\times 1}(2)^6=\frac{\frac{14.7656}{(1-\xi)^{5.5}}}{6\times 5\times 4\times 3 \times 2\times 1}(2)^6=\frac{1.3125}{(1-\xi)^{5.5}}$

The maximum value will be obtained when $\xi=0$ :

$|E|\leq \frac{1.3125}{(1-\xi)^{5.5}}=1.3125$

This is a large upper bound and indicates that using six terms is not giving a good approximation. In general, the Taylor series works best if the distance between $x$ and $a$ is as small as possible. For some functions, like $\sin(x)$ , $\cos(x)$ , and $e^x$ , the Taylor series always converges. However, for functions with square roots, the Taylor series converges when $x$ is relatively close to $a$ . There are some analytical conditions that would indicate the radius of convergence (x-a) of a Taylor series; however, this is beyond the scope of this course!

Clear[x]
Taylor[f_, xi_, a_, n_] := Sum[(D[f, {x, i}] /. x -> a)/i!*(xi - a)^i, {i, 0, n-1}]
f = (1 - x)^(0.5);
a = Table[{i, Taylor[f, -2, 0, i]}, {i, 2, 21}];
a // MatrixForm
a = Table[{i, Taylor[f, 0.1, 0, i]}, {i, 2, 21}];
a // MatrixForm

Example 3

Use zero through fourth order Taylor’s series expansion to approximate the value of the function $f:\mathbb{R}\rightarrow\mathbb{R}$ defined as $f(x)=-0.1x^4-0.15x^3-0.5x^2-0.25x+1.2$ at $x=1$ . Use $a=0$ . Calculate the error associated with each expansion.

Solution

The true value of the function at $x=1$ is given by:

$f_{true}(1)=-0.1-0.15-0.5-0.25+1.2=0.2$

The zero order Taylor series expansion around $a=0$ has the following form:

$f(1)\approx f(0)=1.2$