calculus

gradient(function, x0, delta_x=0.0001)

Returns the gradient of the function at a specific point \(x_0\). A two-point finite difference formula that approximates the derivative

(1)\[\frac{\partial f}{\partial x_i} \approx {\frac {f(x_1, \dots, x_i+h, \dots, x_n)- f(x_1, \dots, x_i-h, \dots, x_n)}{2h}}\]

Gradient

(2)\[\displaystyle \nabla f = \left[\frac{\partial f}{\partial x_1} \enspace \frac{\partial f}{\partial x_2} \enspace \dots \enspace \frac{\partial f}{\partial x_n}\right]^\top\]

Parameters:

• function (Callable[[Tensor], Tensor]) – function which depends on n variables from x

• x0 (Tensor) – n x 1 - dimensional array \(\in \mathbb{R}^{n}\). dtype is torch.double (float64)

• delta_x (float) – precision of two-point formula above (delta_x = h)

Returns:

vector of partial derivatives

Return type:

Tensor

Note

If we make delta_x \(\leq\) 1e-4 gradient will return values with large error rate

Examples

>>> # f(x, y)  = x ** 2 + y ** 2
>>> gradient(lambda x: (x ** 2).sum(), torch.tensor([1., 2.]))
    tensor([2., 4.], dtype=torch.float64)

hessian(function, x0, delta_x=0.0001)

Returns a hessian of function at point \(x_0\)

(3)\[\ H(f) = \begin{bmatrix} \displaystyle \frac{\partial^2 f}{\partial x_1^2} & \displaystyle \frac{\partial^2 f}{\partial x_1\,\partial x_2} & \cdots & \displaystyle \frac{\partial^2 f}{\partial x_1\,\partial x_n} \\ \ \displaystyle \frac{\partial^2 f}{\partial x_2\,\partial x_1} & \displaystyle \frac{\partial^2 f} {\partial x_2^2} & \cdots & \displaystyle \frac{\partial^2 f}{\partial x_2\,\partial x_n} \\ \ \vdots & \vdots & \ddots & \vdots \\ \ \displaystyle \frac{\partial^2 f}{\partial x_n\,\partial x_1} & \displaystyle \frac{\partial^2 f} {\partial x_n\,\partial x_2} & \cdots & \displaystyle \frac{\partial^2 f}{\partial x_n^2} \end{bmatrix}\\ \]

Parameters:

• function (Callable[[Tensor], Tensor]) – function which depends on n variables from x

• x0 (Tensor) – n - dimensional array

• delta_x (float) – precision of two-point formula above (delta_x = h)

Returns:

the hessian of function

Return type:

Tensor

Note

If we make delta_x \(\leq\) 1e-4 hessian returns matrix with large error rate

Examples

>>> def paraboloid(x): return x[0] ** 2 + 2 * x[1] ** 2
>>> print(hessian(paraboloid, torch.tensor([1, 1])).round())
[[2. 0.]
 [0. 4.]]

jacobian(f_vector, x0, delta_x=0.0001)

Returns the Jacobian matrix of a sequence of m functions from f_vector by n variables from x.

(4)\[{\displaystyle J ={\begin{bmatrix}{\dfrac {\partial f_{1}}{\partial x_{1}}}&\cdots &{\dfrac {\partial f_{1}}{\partial x_{n}}}\\\vdots &\ddots &\vdots \\{\dfrac {\partial f_{m}} {\partial x_{1}}}&\cdots &{\dfrac {\partial f_{m}}{\partial x_{n}}}\end{bmatrix}}}_{m \times n}\]

Parameters:

• f_vector (Sequence[Callable[[Tensor], Tensor]]) – a flat sequence, list or tuple or other containing m functions

• x0 (Tensor) – an n-dimensional array. The specific point at which we will calculate the Jacobian

• delta_x (float) – precision of gradient

Returns:

the Jacobian matrix according to the above formula. Matrix n x m

Return type:

Tensor

Examples

>>> func_3 = [lambda x: x[0] ** 2 + x[1], lambda x: 2 * x[0] + 5 * x[1], lambda x: x[0] * x[1]]
>>> print(jacobian(func_3, torch.tensor([-1, 2])).round())
tensor([[-2.,  1.],
        [ 2.,  5.],
        [ 2., -1.]], dtype=torch.float64)