\(\newcommand{\W}[1]{ \; #1 \; }\) \(\newcommand{\R}[1]{ {\rm #1} }\) \(\newcommand{\B}[1]{ {\bf #1} }\) \(\newcommand{\D}[2]{ \frac{\partial #1}{\partial #2} }\) \(\newcommand{\DD}[3]{ \frac{\partial^2 #1}{\partial #2 \partial #3} }\) \(\newcommand{\Dpow}[2]{ \frac{\partial^{#1}}{\partial {#2}^{#1}} }\) \(\newcommand{\dpow}[2]{ \frac{ {\rm d}^{#1}}{{\rm d}\, {#2}^{#1}} }\)

sparse_hessian

View page source

Sparse Hessian

Syntax

hes = f . SparseHessian ( x , w )

hes = f . SparseHessian ( x , w , p )

n_sweep = f . SparseHessian ( x , w , p , row , col , hes , work )

Purpose

We use \(n\) for the Domain size, and \(m\) for the Range size of f . We use \(F : \B{R}^n \rightarrow \B{R}^m\) do denote the AD Function corresponding to f . The syntax above sets hes to the Hessian

\[H(x) = \dpow{2}{x} \sum_{i=1}^m w_i F_i (x)\]

This routine takes advantage of the sparsity of the Hessian in order to reduce the amount of computation necessary. If row and col are present, it also takes advantage of the reduced set of elements of the Hessian that need to be computed. One can use speed tests (e.g. speed_test ) to verify that results are computed faster than when using the routine Hessian .

f

The object f has prototype

ADFun < Base > f

Note that the ADFun object f is not const (see Uses Forward below).

x

The argument x has prototype

const BaseVector & x

(see BaseVector below) and its size must be equal to n , the dimension of the Domain space for f . It specifies that point at which to evaluate the Hessian.

w

The argument w has prototype

const BaseVector & w

and size \(m\). It specifies the value of \(w_i\) in the expression for hes . The more components of \(w\) that are identically zero, the more sparse the resulting Hessian may be (and hence the more efficient the calculation of hes may be).

p

The argument p is optional and has prototype

const SetVector & p

(see SetVector below) If it has elements of type bool , its size is \(n * n\). If it has elements of type std::set<size_t> , its size is \(n\) and all its set elements are between zero and \(n - 1\). It specifies a Sparsity Pattern for the Hessian \(H(x)\).

Purpose

If this sparsity pattern does not change between calls to SparseHessian , it should be faster to calculate p once and pass this argument to SparseHessian . If you specify p , CppAD will use the same type of sparsity representation (vectors of bool or vectors of std::set<size_t> ) for its internal calculations. Otherwise, the representation for the internal calculations is unspecified.

work

If you specify work in the calling sequence, it is not necessary to keep the sparsity pattern; see the heading p under the work description.

Column Subset

If the arguments row and col are present, and color_method is cppad.general or cppad.symmetric , it is not necessary to compute the entire sparsity pattern. Only the following subset of column values will matter:

{ col [ k ] : k = 0 , … , K -1 }

.

row, col

The arguments row and col are optional and have prototype

      const SizeVector & row

      const SizeVector & col

(see SizeVector below). They specify which rows and columns of \(H (x)\) are returned and in what order. We use \(K\) to denote the value hes . size () which must also equal the size of row and col . Furthermore, for \(k = 0 , \ldots , K-1\), it must hold that \(row[k] < n\) and \(col[k] < n\). In addition, all of the \((row[k], col[k])\) pairs must correspond to a true value in the sparsity pattern p .

hes

The result hes has prototype

BaseVector hes

In the case where row and col are not present, the size of hes is \(n * n\) and its size is \(n * n\). In this case, for \(i = 0 , \ldots , n - 1\) and \(ell = 0 , \ldots , n - 1\)

\[hes [ j * n + \ell ] = \DD{ w^{\rm T} F }{ x_j }{ x_\ell } ( x )\]

In the case where the arguments row and col are present, we use \(K\) to denote the size of hes . The input value of its elements does not matter. Upon return, for \(k = 0 , \ldots , K - 1\),

\[hes [ k ] = \DD{ w^{\rm T} F }{ x_j }{ x_\ell } (x) \; , \; \; {\rm where} \; j = row[k] \; {\rm and } \; \ell = col[k]\]

work

If this argument is present, it has prototype

sparse_hessian_work& work

This object can only be used with the routines SparseHessian . During its the first use, information is stored in work . This is used to reduce the work done by future calls to SparseHessian with the same f , p , row , and col . If a future call is made where any of these values have changed, you must first call work . clear () to inform CppAD that this information needs to be recomputed.

color_method

The coloring algorithm determines which rows and columns can be computed during the same sweep. This field has prototype

std::string work . color_method

This value only matters on the first call to sparse_hessian that follows the work constructor or a call to work . clear () .

"cppad.symmetric"

This is the default coloring method (after a constructor or clear() ). It takes advantage of the fact that the Hessian matrix is symmetric to find a coloring that requires fewer sweeps .

"cppad.general"

This is the same as the "cppad" method for the sparse_jacobian calculation.

"colpack.symmetric"

This method requires that colpack_prefix was specified on the CMake Command line. It also takes advantage of the fact that the Hessian matrix is symmetric.

"colpack.general"

This is the same as the "colpack" method for the sparse_jacobian calculation.

colpack.star Deprecated 2017-06-01

The colpack.star method is deprecated. It is the same as the colpack.symmetric which should be used instead.

p

If work is present, and it is not the first call after its construction or a clear, the sparsity pattern p is not used. This enables one to free the sparsity pattern and still compute corresponding sparse Hessians.

n_sweep

The return value n_sweep has prototype

size_t n_sweep

It is the number of first order forward sweeps used to compute the requested Hessian values. Each first forward sweep is followed by a second order reverse sweep so it is also the number of reverse sweeps. This is proportional to the total work that SparseHessian does, not counting the zero order forward sweep, or the work to combine multiple columns into a single forward-reverse sweep pair.

BaseVector

The type BaseVector must be a SimpleVector class with elements of type Base . The routine CheckSimpleVector will generate an error message if this is not the case.

SetVector

The type SetVector must be a SimpleVector class with elements of type bool or std::set<size_t> ; see Sparsity Pattern for a discussion of the difference. The routine CheckSimpleVector will generate an error message if this is not the case.

Restrictions

If SetVector has elements of std::set<size_t> , then p [ i ] must return a reference (not a copy) to the corresponding set. According to section 26.3.2.3 of the 1998 C++ standard, std::valarray< std::set<size_t> > does not satisfy this condition.

SizeVector

The type SizeVector must be a SimpleVector class with elements of type size_t . The routine CheckSimpleVector will generate an error message if this is not the case.

Uses Forward

After each call to Forward , the object f contains the corresponding Taylor coefficients . After a call to any of the sparse Hessian routines, the zero order Taylor coefficients correspond to f . Forward (0, x ) and the other coefficients are unspecified.

Example

The routine sparse_hessian.cpp is examples and tests of sparse_hessian . It return true , if it succeeds and false otherwise.

Subset Hessian

The routine sub_sparse_hes.cpp is an example and test that compute a sparse Hessian for a subset of the variables. It returns true , for success, and false otherwise.