\(\newcommand{\W}[1]{ \; #1 \; }\) \(\newcommand{\R}[1]{ {\rm #1} }\) \(\newcommand{\B}[1]{ {\bf #1} }\) \(\newcommand{\D}[2]{ \frac{\partial #1}{\partial #2} }\) \(\newcommand{\DD}[3]{ \frac{\partial^2 #1}{\partial #2 \partial #3} }\) \(\newcommand{\Dpow}[2]{ \frac{\partial^{#1}}{\partial {#2}^{#1}} }\) \(\newcommand{\dpow}[2]{ \frac{ {\rm d}^{#1}}{{\rm d}\, {#2}^{#1}} }\)

ForSparseHes

View page source

Hessian Sparsity Pattern: Forward Mode

Syntax

h = f . ForSparseHes ( r , s )

Purpose

We use \(F : \B{R}^n \rightarrow \B{R}^m\) to denote the AD Function corresponding to f . we define

\begin{eqnarray} H(x) & = & \partial_x \left[ \partial_u S \cdot F[ x + R \cdot u ] \right]_{u=0} \\ & = & R^\R{T} \cdot (S \cdot F)^{(2)} ( x ) \cdot R \end{eqnarray}

Where \(R \in \B{R}^{n \times n}\) is a diagonal matrix and \(S \in \B{R}^{1 \times m}\) is a row vector. Given a Sparsity Pattern for the diagonal of \(R\) and the vector \(S\), ForSparseHes returns a sparsity pattern for the \(H(x)\).

f

The object f has prototype

const ADFun < Base > f

x

If the operation sequence in f is Independent of the independent variables in \(x \in \B{R}^n\), the sparsity pattern is valid for all values of (even if it has CondExp or VecAD operations).

r

The argument r has prototype

const SetVector & r

(see SetVector below) If it has elements of type bool , its size is \(n\). If it has elements of type std::set<size_t> , its size is one and all the elements of s [0] are between zero and \(n - 1\). It specifies a Sparsity Pattern for the diagonal of \(R\). The fewer non-zero elements in this sparsity pattern, the faster the calculation should be and the more sparse \(H(x)\) should be.

s

The argument s has prototype

const SetVector & s

(see SetVector below) If it has elements of type bool , its size is \(m\). If it has elements of type std::set<size_t> , its size is one and all the elements of s [0] are between zero and \(m - 1\). It specifies a Sparsity Pattern for the vector S . The fewer non-zero elements in this sparsity pattern, the faster the calculation should be and the more sparse \(H(x)\) should be.

h

The result h has prototype

SetVector & h

(see SetVector below). If h has elements of type bool , its size is \(n * n\). If it has elements of type std::set<size_t> , its size is \(n\) and all the set elements are between zero and n -1 inclusive. It specifies a Sparsity Pattern for the matrix \(H(x)\).

SetVector

The type SetVector must be a SimpleVector class with elements of type bool or std::set<size_t> ; see Sparsity Pattern for a discussion of the difference. The type of the elements of SetVector must be the same as the type of the elements of r .

Algorithm

See Algorithm II in Computing sparse Hessians with automatic differentiation by Andrea Walther. Note that s provides the information so that ‘dead ends’ are not included in the sparsity pattern.

Example

The file for_sparse_hes.cpp contains an example and test of this operation.