\(\newcommand{\W}[1]{ \; #1 \; }\) \(\newcommand{\R}[1]{ {\rm #1} }\) \(\newcommand{\B}[1]{ {\bf #1} }\) \(\newcommand{\D}[2]{ \frac{\partial #1}{\partial #2} }\) \(\newcommand{\DD}[3]{ \frac{\partial^2 #1}{\partial #2 \partial #3} }\) \(\newcommand{\Dpow}[2]{ \frac{\partial^{#1}}{\partial {#2}^{#1}} }\) \(\newcommand{\dpow}[2]{ \frac{ {\rm d}^{#1}}{{\rm d}\, {#2}^{#1}} }\)
reverse_theory¶
View page sourceThe Theory of Reverse Mode¶
Taylor Notation¶
In Taylor notation, each variable corresponds to a function of a single argument which we denote by t (see Section 10.2 of Evaluating Derivatives ). Here and below \(X(t)\), \(Y(t)\), and Z ( t ) are scalar valued functions and the corresponding p-th order Taylor coefficients row vectors are \(x\), \(y\) and \(z\); i.e.,
For the purposes of this discussion, we are given the p-th order Taylor coefficient row vectors \(x\), \(y\), and \(z\). In addition, we are given the partial derivatives of a scalar valued function
We need to compute the partial derivatives of the scalar valued function
where \(z^{(j)}\) is expressed as a function of the j-1-th order Taylor coefficient row vector for \(Z\) and the vectors \(x\), \(y\); i.e., \(z^{(j)}\) above is a shorthand for
If we do not provide a formula for a partial derivative of \(H\), then that partial derivative has the same value as for the function \(G\).
Binary Operators¶
Addition¶
The forward mode formula for Addition is
If follows that for \(k = 0 , \ldots , j\) and \(l = 0 , \ldots , j-1\)
Subtraction¶
The forward mode formula for Subtraction is
If follows that for \(k = 0 , \ldots , j\)
Multiplication¶
The forward mode formula for Multiplication is
If follows that for \(k = 0 , \ldots , j\) and \(l = 0 , \ldots , j-1\)
Division¶
The forward mode formula for Division is
If follows that for \(k = 1 , \ldots , j\)
Standard Math Functions¶
The standard math functions have only one argument. Hence we are given the partial derivatives of a scalar valued function
We need to compute the partial derivatives of the scalar valued function
where \(z^{(j)}\) is expressed as a function of the j-1-th order Taylor coefficient row vector for \(Z\) and the vector \(x\); i.e., \(z^{(j)}\) above is a shorthand for
Contents¶
Name |
Title |
---|---|
exp_reverse |
|
log_reverse |
|
sqrt_reverse |
|
sin_cos_reverse |
|
atan_reverse |
|
asin_reverse |
|
acos_reverse |
|
tan_reverse |
|
erf_reverse |
|
pow_reverse |