\(\newcommand{\W}[1]{ \; #1 \; }\) \(\newcommand{\R}[1]{ {\rm #1} }\) \(\newcommand{\B}[1]{ {\bf #1} }\) \(\newcommand{\D}[2]{ \frac{\partial #1}{\partial #2} }\) \(\newcommand{\DD}[3]{ \frac{\partial^2 #1}{\partial #2 \partial #3} }\) \(\newcommand{\Dpow}[2]{ \frac{\partial^{#1}}{\partial {#2}^{#1}} }\) \(\newcommand{\dpow}[2]{ \frac{ {\rm d}^{#1}}{{\rm d}\, {#2}^{#1}} }\)

tan_reverse

View page source

Tangent and Hyperbolic Tangent Reverse Mode Theory

Notation

We use the reverse theory standard math function definition for the functions \(H\) and \(G\). In addition, we use the forward mode notation in tan_forward for \(X(t)\), \(Y(t)\) and \(Z(t)\).

Eliminating Y(t)

For \(j > 0\), the forward mode coefficients are given by

\[y^{(j-1)} = \sum_{k=0}^{j-1} z^{(k)} z^{(j-k-1)}\]

Fix \(j > 0\) and suppose that \(H\) is the same as \(G\) except that \(y^{(j-1)}\) is replaced as a function of the Taylor coefficients for \(Z(t)\). To be specific, for \(k = 0 , \ldots , j-1\),

\begin{eqnarray} \D{H}{ z^{(k)} } & = & \D{G}{ z^{(k)} } + \D{G}{ y^{(j-1)} } \D{ y^{(j-1)} }{ z^{(k)} } \\ & = & \D{G}{ z^{(k)} } + \D{G}{ y^{(j-1)} } 2 z^{(j-k-1)} \end{eqnarray}

Positive Orders Z(t)

For order \(j > 0\), suppose that \(H\) is the same as \(G\) except that \(z^{(j)}\) is expressed as a function of the coefficients for \(X(t)\), and the lower order Taylor coefficients for \(Y(t)\), \(Z(t)\).

\[z^{(j)} = x^{(j)} \pm \frac{1}{j} \sum_{k=1}^j k x^{(k)} y^{(j-k)}\]

For \(k = 1 , \ldots , j\), the partial of \(H\) with respect to \(x^{(k)}\) is given by

\begin{eqnarray} \D{H}{ x^{(k)} } & = & \D{G}{ x^{(k)} } + \D{G}{ z^{(j)} } \D{ z^{(j)} }{ x^{(k)} } \\ & = & \D{G}{ x^{(k)} } + \D{G}{ z^{(j)} } \left[ \delta ( j - k ) \pm \frac{k}{j} y^{(j-k)} \right] \end{eqnarray}

where \(\delta ( j - k )\) is one if \(j = k\) and zero otherwise. For \(k = 1 , \ldots , j\) The partial of \(H\) with respect to \(y^{j-k}\), is given by

\begin{eqnarray} \D{H}{ y^{(j-k)} } & = & \D{G}{ y^{(j-k)} } + \D{G}{ z^{(j)} } \D{ z^{(j)} }{ y^{(j-k)} } \\ & = & \D{G}{ y^{(j-k)} } \pm \D{G}{ z^{(j)} }\frac{k}{j} x^{k} \end{eqnarray}

Order Zero Z(t)

The order zero coefficients for the tangent and hyperbolic tangent are

\begin{eqnarray} z^{(0)} & = & \left\{ \begin{array}{c} \tan ( x^{(0)} ) \\ \tanh ( x^{(0)} ) \end{array} \right. \end{eqnarray}

Suppose that \(H\) is the same as \(G\) except that \(z^{(0)}\) is expressed as a function of the Taylor coefficients for \(X(t)\). In this case,

\begin{eqnarray} \D{H}{ x^{(0)} } & = & \D{G}{ x^{(0)} } + \D{G}{ z^{(0)} } \D{ z^{(0)} }{ x^{(0)} } \\ & = & \D{G}{ x^{(0)} } + \D{G}{ z^{(0)} } ( 1 \pm y^{(0)} ) \end{eqnarray}