log_reverse

View page source

Logarithm Function Reverse Mode Theory

We use the reverse theory standard math function definition for the functions \(H\) and \(G\).

The zero order forward mode formula for the logarithm is

\[z^{(0)} = F( x^{(0)} )\]

and for \(j > 0\),

\[z^{(j)} = \frac{1}{ \bar{b} + x^{(0)} } \frac{1}{j} \left( j x^{(j)} - \sum_{k=1}^{j-1} k z^{(k)} x^{(j-k)} \right)\]

where

\[\begin{split}\bar{b} = \left\{ \begin{array}{ll} 0 & \R{if} \; F(x) = \R{log}(x) \\ 1 & \R{if} \; F(x) = \R{log1p}(x) \end{array} \right.\end{split}\]

We note that for \(j > 0\)

\begin{eqnarray} \D{ z^{(j)} } { x^{(0)} } & = & - \frac{1}{ \bar{b} + x^{(0)} } \frac{1}{ \bar{b} + x^{(0)} } \frac{1}{j} \left( j x^{(j)} - \sum_{k=1}^{j-1} k z^{(k)} x^{(j-k)} \right) \\ & = & - \frac{z^{(j)}}{ \bar{b} + x^{(0)} } \end{eqnarray}

Removing the zero order partials are given by

\begin{eqnarray} \D{H}{ x^{(0)} } & = & \D{G}{ x^{(0)} } + \D{G}{ z^{(0)} } \D{ z^{(0)} }{ x^{(0)} } \\ & = & \D{G}{ x^{(0)} } + \D{G}{ z^{(0)} } \frac{1}{ \bar{b} + x^{(0)} } \end{eqnarray}

For orders \(j > 0\) and for \(k = 1 , \ldots , j-1\)

\begin{eqnarray} \D{H}{ x^{(0)} } & = & \D{G}{ x^{(0)} } + \D{G}{ z^{(j)} } \D{ z^{(j)} }{ x^{(0)} } \\ & = & \D{G}{ x^{(0)} } - \D{G}{ z^{(j)} } \frac{ z^{(j)} }{ \bar{b} + x^{(0)} } \\ \D{H}{ x^{(j)} } & = & \D{G}{ x^{(j)} } + \D{G}{ z^{(j)} } \D{ z^{(j)} }{ x^{(j)} } \\ & = & \D{G}{ x^{(j)} } + \D{G}{ z^{(j)} } \frac{1}{ \bar{b} + x^{(0)} } \\ \D{H}{ x^{(j-k)} } & = & \D{G}{ x^{(j-k)} } - \D{G}{ z^{(j)} } \frac{1}{ \bar{b} + x^{(0)} } \frac{k}{j} z^{(k)} \\ \D{H}{ z^{(k)} } & = & \D{G}{ z^{(k)} } - \D{G}{ z^{(j)} } \frac{1}{ \bar{b} + x^{(0)} } \frac{k}{j} x^{(j-k)} \end{eqnarray}