lines 325-647 of file: introduction/exp_eps.xrst

{xrst_begin exp_eps_rev1}
{xrst_spell
   forward forward
   preform
   rcll
}

exp_eps: First Order Reverse Sweep
##################################

Purpose
*******
First order reverse mode uses the
:ref:`exp_eps_for0@Operation Sequence` ,
and zero order forward sweep values,
to compute the first order derivative
of one dependent variable with respect to all the independent variables.
The computations are done in reverse
of the order of the computations in the original algorithm.

Mathematical Form
*****************
Suppose that we use the algorithm :ref:`exp_eps.hpp-name`
to compute ``exp_eps`` ( *x* , *epsilon* )
with *x* is equal to .5
and *epsilon* is equal to .2.
For this case, the mathematical function for the operation sequence
corresponding to ``exp_eps`` is

.. math::

   f ( x , \varepsilon ) =   1 + x + x^2 / 2

The corresponding partial derivatives,
and the value of the derivatives, are

.. math::
   :nowrap:

   \begin{eqnarray}
   \partial_x f ( x , \varepsilon ) & = &  1 + x  = 1.5
   \\
   \partial_\varepsilon f ( x , \varepsilon ) & = &  0
   \end{eqnarray}

epsilon
*******
Since :math:`\varepsilon` is an independent variable,
it could included as an argument to all of the
:math:`f_j` functions below.
The result would be that all the partials with respect to
:math:`\varepsilon` would be zero and hence we drop it to simplify
the presentation.

f_7
***
In reverse mode we choose one dependent variable and
compute its derivative with respect to all the independent variables.
For our example, we chose the value returned by :ref:`exp_eps.hpp-name`
which is :math:`v_7`.
We begin with the function :math:`f_7` where :math:`v_7`
is both an argument and the value of the function; i.e.,

.. math::
   :nowrap:

   \begin{eqnarray}
   f_7 ( v_1 , v_2 , v_3 , v_4 , v_5 , v_6 , v_7 ) & = & v_7
   \\
   \D{f_7}{v_7} & = & 1
   \end{eqnarray}

All the other partial derivatives of :math:`f_7` are zero.

Index 7: f_6
************
The last operation has index 7,

.. math::

   v_7 =   v_4 + v_6

We define the function
:math:`f_6 ( v_1 , v_2 , v_3 , v_4 , v_5 , v_6 )`
as equal to :math:`f_7`
except that :math:`v_7` is eliminated using
this operation; i.e.

.. math::

   f_6  =
   f_7 [ v_1 , v_2 , v_3 , v_4 , v_5 , v_6 , v_7 ( v_4 , v_6 ) ]

It follows that

.. math::

   \begin{array}{rcll}
   \D{f_6}{v_4}
   & = & \D{f_7}{v_4} +
      \D{f_7}{v_7} * \D{v_7}{v_4}
   & = 1
   \\
   \D{f_6}{v_6}
   & = & \D{f_7}{v_6} +
      \D{f_7}{v_7} * \D{v_7}{v_6}
   & = 1
   \end{array}

All the other partial derivatives of :math:`f_6` are zero.

Index 6: f_5
************
The previous operation has index 6,

.. math::

   v_6 = v_5 / 2

We define the function
:math:`f_5 (  v_1 , v_2 , v_3 , v_4 , v_5 )`
as equal to :math:`f_6`
except that :math:`v_6` is eliminated using this operation; i.e.,

.. math::

   f_5 =
   f_6 [  v_1 , v_2 , v_3 , v_4 , v_5 , v_6 ( v_5 ) ]

It follows that

.. math::

   \begin{array}{rcll}
   \D{f_5}{v_4}
   & = & \D{f_6}{v_4}
   & = 1
   \\
   \D{f_5}{v_5}
   & = & \D{f_6}{v_5} +
      \D{f_6}{v_6} * \D{v_6}{v_5}
   & = 0.5
   \end{array}

All the other partial derivatives of :math:`f_5` are zero.

Index 5: f_4
************
The previous operation has index 5,

.. math::

   v_5 = v_3 * v_1

We define the function
:math:`f_4 (  v_1 , v_2 , v_3 , v_4 )`
as equal to :math:`f_5`
except that :math:`v_5` is eliminated using this operation; i.e.,

.. math::

   f_4 =
   f_5 [  v_1 , v_2 , v_3 , v_4 , v_5 ( v_3 , v_1 )  ]

Given the information from the forward sweep, we have
:math:`v_3 =  0.5` and :math:`v_1 = 0.5`.
It follows that

.. math::

   \begin{array}{rcll}
   \D{f_4}{v_1}
   & = & \D{f_5}{v_1} +
      \D{f_5}{v_5} * \D{v_5}{v_1}
   & =  0.25
   \\
   \D{f_4}{v_2} & = & \D{f_5}{v_2}  & = 0
   \\
   \D{f_4}{v_3}
   & = & \D{f_5}{v_3} +
      \D{f_5}{v_5} * \D{v_5}{v_3}
   & =  0.25
   \\
   \D{f_4}{v_4}
   & = & \D{f_5}{v_4}
   & = 1
   \end{array}

Index 4: f_3
************
The previous operation has index 4,

.. math::

   v_4 = 1 + v_3

We define the function
:math:`f_3 (  v_1 , v_2 , v_3 )`
as equal to :math:`f_4`
except that :math:`v_4` is eliminated using this operation; i.e.,

.. math::

   f_3 =
   f_4 [  v_1 , v_2 , v_3 , v_4 ( v_3 )  ]

It follows that

.. math::

   \begin{array}{rcll}
   \D{f_3}{v_1}
   & = & \D{f_4}{v_1}
   & =  0.25
   \\
   \D{f_3}{v_2} & = & \D{f_4}{v_2}  & = 0
   \\
   \D{f_3}{v_3}
   & = & \D{f_4}{v_3} +
      \D{f_4}{v_4} * \D{v_4}{v_3}
   & =  1.25
   \end{array}

Index 3: f_2
************
The previous operation has index 3,

.. math::

   v_3 = v_2 / 1

We define the function
:math:`f_2 (  v_1 , v_2 )`
as equal to :math:`f_3`
except that :math:`v_3` is eliminated using this operation; i.e.,

.. math::

   f_2 =
   f_4 [  v_1 , v_2 , v_3 ( v_2 )  ]

It follows that

.. math::

   \begin{array}{rcll}
   \D{f_2}{v_1}
   & = & \D{f_3}{v_1}
   & =  0.25
   \\
   \D{f_2}{v_2} & = & \D{f_3}{v_2}  +
      \D{f_3}{v_3} * \D{v_3}{v_2}
   & = 1.25
   \end{array}

Index 2: f_1
************
The previous operation has index 1,

.. math::

   v_2 = 1 * v_1

We define the function
:math:`f_1 (  v_1 )`
as equal to :math:`f_2`
except that :math:`v_2` is eliminated using this operation; i.e.,

.. math::

   f_1 =
   f_2 [  v_1 , v_2 ( v_1 )  ]

It follows that

.. math::

   \begin{array}{rcll}
   \D{f_1}{v_1} & = & \D{f_2}{v_1}  +
      \D{f_2}{v_2} * \D{v_2}{v_1}
   & = 1.5
   \end{array}

Note that :math:`v_1` is equal to :math:`x`,
so the derivative of ``exp_eps`` ( *x* , *epsilon* )
at *x* equal to .5 and *epsilon* equal .2 is
1.5 in the *x* direction and zero in the *epsilon* direction.
We also note that
:ref:`forward<exp_eps_for1-name>` forward mode gave the
same result for the partial in the *x* direction.
{xrst_toc_hidden
   introduction/exp_eps_rev1.cpp
}

Verification
************
The file :ref:`exp_eps_rev1.cpp-name` contains a routine
that verifies the values computed above.
It only tests the partial derivatives of
:math:`f_j` that might not be equal to the corresponding
partials of :math:`f_{j+1}`; i.e., the
other partials of :math:`f_j` must be equal to the corresponding
partials of :math:`f_{j+1}`.

Exercises
*********

#. Consider the case where :math:`x = .1`
   and we first preform a zero order forward mode sweep
   for the operation sequence used above (in reverse order).
   What are the results of a
   first order reverse mode sweep; i.e.,
   what are the corresponding values for
   :math:`\D{f_j}{v_k}` for all :math:`j, k` such that
   :math:`\D{f_j}{v_k} \neq 0`.
#. Create a modified version of
   :ref:`exp_eps_rev1.cpp-name`
   that verifies the values you obtained for the previous exercise.
   Also create and run a main program that reports the result
   of calling the modified version of
   :ref:`exp_eps_rev1.cpp-name` .

{xrst_end exp_eps_rev1}