-------------------------------------------
lines 5-126 of file: xrst/introduction.xrst
-------------------------------------------

{xrst_begin Introduction}
{xrst_spell
   andreas
   griewank
}

An Introduction by Example to Algorithmic Differentiation
#########################################################

Purpose
*******
This is an introduction by example
to Algorithmic Differentiation.
Its purpose is to aid in understand what AD calculates,
how the calculations are preformed,
and the amount of computation and memory required
for a forward or reverse sweep.

Preface
*******

Algorithmic Differentiation
===========================
Algorithmic Differentiation
(often referred to as Automatic Differentiation or just AD)
uses the software representation
of a function to obtain an efficient method for calculating its derivatives.
These derivatives can be of arbitrary order and are analytic in nature
(do not have any truncation error).

Forward Mode
============
A forward mode sweep computes
the partial derivative of all the dependent variables with respect
to one independent variable (or independent variable direction).

Reverse Mode
============
A reverse mode sweep computes
the derivative of one dependent variable
(or one dependent variable direction)
with respect to all the independent variables.

Operation Count
===============
The number of floating point operations for either a
forward or reverse mode sweep
is a small multiple of the number required to evaluate the original function.
Thus, using reverse mode,
you can evaluate the derivative of a scalar valued function
with respect to thousands of variables in a small multiple of the
work to evaluate the original function.

Efficiency
==========
AD automatically takes advantage of the
speed of your algorithmic representation of a function.
For example,
if you calculate a determinant using LU factorization,
AD will use the LU representation for
the derivative of the determinant
(which is faster than using the definition of the determinant).

Outline
*******

A. get_started
==============
See :ref:`get_started.cpp-name` for an example that uses
CppAD to calculate derivatives of a polynomial.

B. Example Algorithms
=====================
Present two algorithms that approximate the exponential function.
The first algorithm :ref:`exp_2.hpp-name` is simpler and does not
include any logical variables or loops.
The second algorithm :ref:`exp_eps.hpp-name` includes
logical operations and a ``while`` loop.

C. Example Steps
================
For each of the algorithms, exp_2 and exp_eps, do the following:

#. Define the mathematical function corresponding to the algorithm
   (:ref:`exp_2-name` and :ref:`exp_eps-name` ).
#. Write out the floating point operation sequence,
   and corresponding values,
   that correspond to executing the algorithm for a specific input
   (:ref:`exp_2_for0-name` and :ref:`exp_eps_for0-name` ).
#. Compute a forward sweep derivative of the operation sequence
   (:ref:`exp_2_for1-name` and :ref:`exp_eps_for1-name` ).
#. Compute a reverse sweep derivative of the operation sequence
   (:ref:`exp_2_rev1-name` and :ref:`exp_eps_rev1-name` ).
#. Use CppAD to compute both a forward and reverse sweep
   of the operation sequence
   (:ref:`exp_2_cppad-name` and :ref:`exp_eps_cppad-name` ).

D. Testing Examples
===================
The program :ref:`exp_apx.cpp-name` runs all of the test
routines that validate the calculations in the :ref:`exp_2-name`
and :ref:`exp_eps-name` presentation.

Reference
*********
An in-depth review of AD theory and methods can be found in
the book
*Evaluating Derivatives:*
*Principles and Techniques of Algorithmic Differentiation* ,
Andreas Griewank,
SIAM Frontiers in Applied Mathematics,
2000.

Contents
********
{xrst_toc_table
   introduction/exp_2.hpp
   introduction/exp_eps.hpp
   introduction/introduction.cpp
}

{xrst_end Introduction}