BFGS (Broyden–Fletcher–Goldfarb–Shanno) algorithm#

bfgs(function, x0, tolerance=1e-08, max_iter=500, verbose=False, keep_history=False)[source]#

Returns a tensor n x 1 with optimal point and history using the BFGS method [1]

Broyden–Fletcher–Goldfarb–Shanno algorithm The algorithm does not use Wolfe conditions. Instead of wolfe, alg uses the optimal step.

Note

The algorithm only works for a flat x0, and the functions should depend on a flat array

References

\(\rule{125mm}{0.2pt} \\\)

Parameters:

function (Callable[[Tensor], Tensor]) – callable that depends on the first positional argument. Other arguments are passed through kwargs
x0 (Tensor) – start minimization point
tolerance (float) – criterion of stop os l2 norm(grad f) < tolerance
max_iter (int) – maximum number of iterations
verbose (bool) – flag of printing iteration logs
keep_history (bool) – flag of return history

Returns:

tuple with point and history.

Return type:

Tuple[Tensor, HistoryGD]

Examples

>>> def func(x): return 10 * x[0] ** 2 + x[1] ** 2 / 5
>>> x_0 = torch.tensor([1, 2])
>>> solution = bfgs(func, x_0)
>>> print(solution[0])
tensor([3.4372e-14, 1.8208e-14], dtype=torch.float64)