The traditional "pencil and paper" derivations for numerical differentiation formulas for   and   are done independently as if there was no connection among the derivations.  This new approach gives a parallel development of the formulas.  It requires the solution of a "linear system" that includes symbolic quantities as coefficients and constants. The power of a computer algebra system such as Mathematica is used to elegantly solve the linear system for   and  .  The extension to proofs of higher order numerical differentiation formulas for the central, forward or backward differences is easy to accomplish.  The notation convention using square brackets "[ and ]" instead of parenthesis is used to make all the formulas look compatible with the Mathematica input and output that is used.  

The Three Point Central Difference Formulas

    Using three points , , and we give a parallel development of the numerical differentiation formulas for   and  .  Start with the Taylor series for expanded in powers of  .  Terms must be included so that the remainder term is included.  Using degree will suffice.    

The Mathematica notation for the remainder term is  .  The term is removed from the series with the command Normal, and a Taylor polynomial of degree is formed.  However, we must remember that the accuracy for the Taylor polynomial is  .   

Construct two "equations" by setting the series equal to the function at .    

We have two equations in the two unknown and  , and all the other quantities ,  , , , and   are considered as constants.  Now use the Mathematica command "Solve" to solve for   and  .    

The result is not too easy to read, so we can use the commands and to manipulate the formula.  In the print statements we add  ""  to remind us that we are using truncated infinite series.  The following commands will create a nicer looking printout for the above solution.   

Thus, we have derived the numerical differentiation formula for     and    and the first term in the series expansion for the remainder which involves, or  , respectively.  Since the "numerical differentiation formulae" are "truncated" infinite series, we know that the ellipses "" means that there are infinitely many more term which are not shown.   

    When we do not include the  "", we must evaluate the lowest order derivative in the series for the remainder at the value instead of , then we can "chop off" the infinite series at the term involving   or  , respectively.  The Mathematica command  ReplaceAll is used to accomplish this task.   

    We have obtained the desired numerical differentiation formulas and their remainder terms.  We should mention that new in version 4.1 of  Mathematica is symbolic recognition of order of approximation for divided difference formulas.  We can add the term to the numerical differentiation formula term, and Mathematica will conclude that it is the derivative plus .   The following commands illustrate this feature of Mathematica.   

Therefore, we have established the numerical differentiation formulas   

          
and
        

And the corresponding formulas with the big "O" notation   are   

          
and
        

Comparison with the Traditional Derivations

    The above derivation differs only slightly from the traditional derivations which also start with the two equations  

           
(1)
          

    When deriving the numerical differentiation formula for the equations (1) are subtracted and the terms involving   cancel and manipulations are used to solve for   and its truncation error term.  

     The traditional derivation of the numerical differentiation formula for starts with the same equations (1).  But this time the equations are added and the terms involving   cancel and manipulations are used to solve for   and its truncation error term.  

     There is no "big deal" made about the fact that the starting place is the same.  We shall see for the higher order formulas that using the same starting place will be the key to successful computer derivations of numerical differentiation formulas.  

The Five Point Central Difference Formulas

    Using five points , , ,, and we can give a parallel development of the numerical differentiation formulas for  ,  , and .  Start with the Taylor series for expanded in powers of  .  Terms must be included so that the remainder term is included.  Using degree will suffice.  Since the remainder term  , we will use Mathematica's Normal command to remove it from the series, and form the Taylor polynomial of degree .   

Use five points and set up the "equations" by setting   for  , which is automated by using Mathematica's Table command.  Although the value   , will produce   which is the boolean value True, it will do no harm when solving the set of equations and we will see that the generations of higher order formulas which require more complicated sets of equations is more easily automated with the use of a table.  

We can consider that these are four equations in the four unknowns , ,   and , and all the other quantities , , , , , , and   are constants.  Now use Mathematica to solve for , ,   and .  Since this requires typing of four derivatives to be solved, we will automate this process too, by using a table to construct the "variables."  

Amazingly, the algebra involved in solving the four equations will result in the cancellation of in the odd order derivatives and the term   will cancel in the even order derivatives.  This can be accomplished using Mathematica's Solve procedure.   

The output is not too clear to read, so we shall use the formula manipulation commands and to group the numerical differentiation portion, and then use the ReplaceAll command to change and   to and  , then a final use of Together will clean up the numerical differentiation formula part.  All this will be accomplished in the following three Mathematica commands.  If the reader is curious about exactly what is happening in each step, then the semi-colons can be deleted and the results of each operation will be shown.   

Notice that the formula for approximating and have truncation error terms involving   so they are numerical differentiation formulas of order .   But the formula for approximating and have truncation error terms involving   so they are numerical differentiation formulas of order .  This is one of the many surprises in the theory of numerical analysis.   

Again, we can uses Mathematica's 4.1 ability to symbolic recognize the order of approximation for divided difference formulas.  If we add the term   or to the numerical differentiation formula then we will be surprised to learn that Mathematica will tell us that it is the derivative plus   or .   The following commands illustrate this point.   

Therefore, we have established the numerical differentiation formulas   

            


          


           


           

These formulas can be written with the big "O" notation and if desired.

            


           


           


           

Appendix. Subroutines for Generating Numerical Differentiation Formulae

The central difference formulae use an odd number of points  ,  and an even number of   equations.  Since we want to include the remainder terms, we need to use series expansions of order  .  The remainder term in these formulas all involve even powers of  h  and even and odd derivatives depending on the situation.  Also, the subroutine requires a replacement of the point where the remainder term is evaluated to be instead of  .  

The Five Point Central Difference Formulae

    These are to difficult to do by hand, so we use the computer.  

Higher order formulas are easy to obtain with Mathematica.
Exploration

Subroutines for Generating the Forward and Backward Difference Formulae

Once we have the idea for the central difference formula, all we need to do is adjust the set of to be forward step sizes or backward step sizes and we can generate the forward and backward differentiation formulas.  

Explorations with the Higher Order Formulas

Higher order formulas are easy to obtain with Mathematica.
Exploration

Reference

John Mathews, Computer Derivations of Numerical Differentiation Formulae (Classroom Notes),  International Journal of Mathematics Education in Science and Technology, Volume 34 No 2 (March-April 2003), pp.280-287.  

Research Experience for Undergraduates

Numerical Differentiation  Numerical Differentiation  Internet hyperlinks to web sites and a bibliography of articles.  

Download this Mathematica Notebook Numerical Differentiation Formulae

(c) John H. Mathews 2004