reverse_any

View page source

Any Order Reverse Mode

Syntax

dw = f . Reverse ( q , w )

Purpose

We use \(F : \B{R}^n \rightarrow \B{R}^m\) to denote the AD Function corresponding to f . Reverse mode computes the derivative of the Forward mode Taylor coefficients with respect to the domain variable \(x\). To be specific, it computes the derivative \(W^{(1)} (u)\) at \(u = x\) which is specified by the following notation:

Notation

u^(k)

For \(k = 0, \ldots , q-1\), the vector \(u^{(k)} \in \B{R}^n\) is defined as the value of x_k in the previous calls of the form

f . Forward ( k , x_k )

If there is no previous call with \(k = 0\), \(u^{(0)}\) is the value of the independent variables when the corresponding AD of Base operation sequence was recorded.

X(t, u)

The function \(X : \B{R} \times \B{R}^{n \times q} \rightarrow \B{R}^n\) is defined by

\[X ( t , u ) = u^{(0)} + u^{(1)} * t + \cdots + u^{(q-1)} * t^{q-1}\]

Note that for \(k = 0 , \ldots , q-1\), \(u^{(k)}\) is related to the k-th partial of \(X(t, u)\) with respect to \(t\) by

\[u^{(k)} = \frac{1}{k !} \Dpow{k}{t} X(0, u)\]

Y(t, u)

The function \(Y : \B{R} \times \B{R}^{n \times q} \rightarrow \B{R}^m\) is defined by

\[Y(t, u) = F [ X(t,u) ]\]

w^(k)

If the argument w has size m * q , for \(k = 0 , \ldots , q-1\) and \(i = 0, \ldots , m-1\),

\[w_i^{(k)} = w [ i * q + k ]\]

If the argument w has size m , for \(k = 0 , \ldots , q-1\) and \(i = 0, \ldots , m-1\),

\[\begin{split}w_i^{(k)} = \left\{ \begin{array}{ll} w [ i ] & {\rm if} \; k = q-1 \\ 0 & {\rm otherwise} \end{array} \right.\end{split}\]

W(u)

The function \(W : \B{R}^{n \times q} \rightarrow \B{R}\) is defined by

\[W(u) = \sum_{k=0}^{q-1} ( w^{(k)} )^\R{T} \frac{1}{k !} \Dpow{k}{t} Y(0, u)\]

f

The object f has prototype

const ADFun < Base > f

Before this call to Reverse , the value returned by

f . size_order ()

must be greater than or equal q (see size_order ).

q

The argument q has prototype

size_t q

and specifies the number of Taylor coefficient orders to be differentiated (for each variable).

w

The argument w has prototype

const Vector & w

(see Vector below) and its size must be equal to m or m * q , It specifies the weighting vector w in the definition of W(u) .

dw

The return value dw has prototype

Vector dw

(see Vector below). It is a vector with size \(n \times q\). For \(j = 0, \ldots, n-1\) and \(k = 0 , \ldots , q-1\) If the argument w has size m * q ,

\[dw[ j * q + k ] = W^{(1)} ( x )_{j,k}\]

where \(u = x\) is value of the Taylor coefficients where the derivative is evaluated.

If the argument w has size m ,

\[dw[ j * q + q - k - 1 ] = W^{(1)} ( x )_{j,k}\]

where \(u = x\) is value of the Taylor coefficients where the derivative is evaluated. Note the reverse order in which the order indices are stored. This is an unfortunate consequence of keeping Reverse backward compatible.

First Order

We consider the case where q = 1 and w . size () == m . In this case

\begin{eqnarray} W(u) & = & w_0 Y_0 (0, u) + \cdots + w_m Y_m (0, u) \\ W(u) & = & w_0 F_0 [ X(0, u) ] + \cdots + w_m F_m [ X(0, u) ] \\ W^{(1)} (x) & = & w_0 F_0^{(1)} ( x^{(0)} ) + \cdots + w_m F_m^{(1)} ( x^{(0)} ) \end{eqnarray}

This is the same as the result documented in reverse_one .

Second Order

We consider the case where q = 2 and w . size () == m . In this case

\begin{eqnarray} W(u) & = & w_0 \partial_t Y_0 (0, u) + \cdots + w_m \partial_t Y_m (0, u) \\ W(u) & = & w_0 \partial_t \{ F_0 [ X(t, u) ] \}_{t = 0} + \cdots + w_m \partial_t \{ F_m [ X(t, u) ] \}_{t = 0} \\ W(u) & = & w_0 F_0^{(1)} ( u^{(0)} ) u^{(1)} + \cdots + w_0 F_m^{(1)} ( u^{(0)} ) u^{(1)} \\ \partial_{u(0)} W(x) & = & w_0 ( x^{(1)} )^\R{T} F_0^{(2)} ( x^{(0)} ) + \cdots + w_m ( x^{(1)} )^\R{T} F_m^{(2)} ( x^{(0)} ) \\ \partial_{u(1)} W(x) & = & w_0 F_0^{(1)} ( x^{(0)} ) + \cdots + w_m F_m^{(1)} ( x^{(0)} ) \end{eqnarray}

where \(\partial{u(0)}\) denotes partial with respect to \(u^{(0)}\). These are the same as the result documented in reverse_two .

Vector

The type Vector must be a SimpleVector class with elements of type Base . The routine CheckSimpleVector will generate an error message if this is not the case.

Example

  1. The file reverse_three.cpp contains an example and test of using reverse mode to compute third order derivatives.

  2. The file rev_checkpoint.cpp contains an example and test of the general reverse mode case.