Inexact Newton-CG Optimization

optimization

This module provides a modern wrapper to Hippylib's Newton-CG solver.

Parametrization is done via data classes, all input and output vectors are numpy arrays.

The input Hippliyb inference model used for optimization is assumed to provide a cost functional in form of a negative log-posterior. Thus, the optimal parameter value found by the optimizer approximizes the maximum a-posteriori (MAP) estimate. In SPIN, we consider Gaussian prior and noise models, resulting in an optimization problem of the form

\[ \mathbf{m}_{\text{MAP}} = \underset{\mathbf{m}}{\text{argmin}}\ \frac{1}{2}||\mathbf{F}(m)-\mathbf{d}_{obs}||_{R_{\text{noise}}}^2 +\frac{1}{2}||\mathbf{m}-\mathbf{m}_{pr}||_{R_{\text{prior}}}^2. \]

Classes:

Name	Description
SolverSettings	Configuration of the Newton-CG solver.
SolverResult	Data class for storage of solver results
NewtonCGSolver	Wrapper class for the inexact Newton-CG solver implementation in Hippylib.

spin.hippylib.optimization.SolverSettings dataclass

Configuration of the Newton-CG solver.

All attributes have default values.

Attributes:

Name	Type	Description
relative_tolerance	Real	Relative tolerance for the gradient norm (compared to initial guess).
absolute_tolerance	Real	Absolute tolerance for the gradient norm.
gradient_projection_tolerance	Real	Tolerance for the inner product (g,dm), where g is the current gradient and dm the search direction.
max_num_newton_iterations	int	Maximum number of Newton iterations.
num_gauss_newton_iterations	int	Number of Gauss-Newton iterations performed initially, before switching to full Newton.
coarsest_tolerance_cg	Real	Termination tolerance for the conjugate gradient solver.
max_num_cg_iterations	int	Maximum number of conjugate gradient iterations.
armijo_line_search_constant	Real	Constant for the Armijo line search.
max_num_line_search_iterations	int	Maximum number of line search iterations.
verbose	bool	Whether to print the solver output.

spin.hippylib.optimization.SolverResult dataclass

Data class for storage of solver results.

Attributes:

Name	Type	Description
optimal_parameter	npt.NDArray[np.floating]	Optimal parameter, found by the optimizer.
forward_solution	npt.NDArray[np.floating]	Solution of the PDE problem for the optimal parameter.
adjoint_solution	npt.NDArray[np.floating]	Solution of the adjoint problem for the optimal parameter.
converged	bool	Whether the solver has converged.
num_iterations	int	Number of Newton iterations.
termination_reason	str	Reason for termination.
final_gradient_norm	Real	Final gradient norm.

spin.hippylib.optimization.NewtonCGSolver

Wrapper class for the inexact Newton-CG solver implementation in Hippylib.

This class mainly exists to provide a more modern, convenient, and consistent interface to the underlying Hippylib functionality. The incomplite Newton-CG method constitutes a combination of algorithms that efficienly solve large-scale optimization problems, with good scalability in terms of the number of degrees of freedom. The outer Newton iterations are known to converge independently of the problem size for a wide range of applications. The linear system at each Newton step is solved inexactly using the conjugate gradient (CG) method. The CG solver is terminated early according to Steihaug and Eisenstat-Walker stopping criteria. This ensures termination independently of the problem size as well. For globalization, armijo line search is utilized.

For more information on the implementation of the Solver in hippylib, check out the NewtonCG documentation.

Methods:

Name	Description
run	Start the solver with an initial guess.

__init__

__init__(solver_settings: SolverSettings, inference_model: hl.Model) -> None

Constructor, initializing solver according to settings.

Parameters:

Name	Type	Description	Default
solver_settings	SolverSettings	Solver configuration.	required
inference_model	hl.Model	Hippylib inference model defining the optimization problem.	required

solve

solve(initial_guess: npt.NDArray[np.floating]) -> SolverResult

Run the solver, given an initial guess.

Parameters:

Name	Type	Description	Default
initial_guess	npt.NDArray[np.floating]	Initial guess for the optimization problem.	required

Raises:

Type	Description
ValueError	Checks if the initial guess has the correct size.

Returns:

Name	Type	Description
SolverResult	SolverResult	Optimal solution and metadata.