Nonlinear conjugate gradient method#

\(\rule{125mm}{0.7pt}\)

Algorithm Flowchart

\(\rule{125mm}{0.7pt}\)

Nonlinear conjugate gradient methodNonlinear conjugate gradient method

\(\rule{125mm}{0.7pt}\)

nonlinear_cgm(function, x0, epsilon=1e-05, max_iter=500, verbose=False, keep_history=False)[source]#

Returns a tensor n x 1 with optimal point and history. Algorithm works when the function is approximately quadratic near the minimum, which is the case when the function is twice differentiable at the minimum and the second derivative is non-singular there [1]

References

Parameters:

  • function (Callable[[Tensor], Tensor]) – callable that depends on the first positional argument

  • x0 (Tensor) – Torch tensor which is initial approximation

  • epsilon (float) – optimization accuracy

  • max_iter (int) – maximum number of iterations

  • verbose (bool) – flag of printing iteration logs

  • keep_history (bool) – flag of return history

Returns:

tuple with point and history.

Return type:

Tuple[Tensor, HistoryGD]

Examples

Example for \(f(x, y) = (x + 0.5)^2 + (y - 0.5)^2, \quad x = 1\)

>>> def func(x): return 10 * x[0] ** 2 + x[1] ** 2 / 5
>>> x_0 = torch.tensor([1, 2])
>>> solution = nonlinear_cgm(func, x_0)
>>> print(solution[0])
tensor([6.9846e+25, 4.2454e+26], dtype=torch.float64)