ml.classification#
- class LogisticRegression[source]#
Bases:
BaseClassificationBinary classification model
Let \(x \in \mathbb{R}^{n \times m}, \ w \in \mathbb{R}^{m \times 1}, \ I = [1]_{n \times 1}\), \(x_i\) – is a row and \(x_i \in \mathbb{R}^{1 \times m}\)
Model:
(1)#\[\mathbb{P}(y_i = 1 | w) = \frac{1}{1 + \exp (x_i \cdot w + b)}\]- __init__(kernel='linear')[source]#
- Parameters:
kernel (Literal['linear', 'perceptron']) – ‘linear’ or ‘perceptron’. linear - basic logistic regression, perceptron - nn with 2 hidden layer with dim1 = 1024, dim2 = 512
- fit(x, y, epochs=1000, l1_lambda=0, show_epoch=0, print_function=<built-in function print>)[source]#
Returns trained model Logistic Regression
- Parameters:
x (Tensor) – training set
y (Tensor) – target value
epochs – max number of sgd implements
l1_lambda (float) – l1 regularization weight
show_epoch (int) – amount of showing epochs
print_function (Callable) – print or streamlit.write
- Returns:
trained model
- Return type:
Module
- class LogisticRegressionRBF[source]#
Bases:
BaseClassificationThis is a logistic regression, but before we make a basic linear prediction and apply the sigmoid, we transfer x to another space using radial basis functions. The dimension of this space depends on the basis matrix x (x_basis) [1]
Radial basis functions
gaussian \(\displaystyle \varphi (x, x_b)=e^{-\Vert x - x_b \Vert^2}\)
linear \(\varphi (x, x_b) = \Vert x - x_b \Vert\)
multiquadratic \(\displaystyle \varphi (x, x_b) = \sqrt{1 + \Vert x - x_b \Vert^2}\)
References
- __init__(x_basis, rbf='gaussian')[source]#
- Parameters:
x_basis (Tensor) – centers of basis functions
rbf (Literal['linear', 'gaussian', 'multiquadratic']) – type of rbf function. Available: [‘linear’, ‘gaussian’]
- fit(x, y, epochs=100, l1_lambda=0, show_epoch=0, print_function=<built-in function print>)[source]#
Returns trained model Logistic Regression with RBF
- Parameters:
x (Tensor) – training set
y (Tensor) – target value
epochs – max number of sgd implements
l1_lambda (float) – l1 regularization weight
show_epoch (int) – amount of showing epochs
print_function (Callable) – e.g. print or streamlit.write
- Returns:
trained model
- Return type:
Module
- forward(x=None, phi_matrix=None)[source]#
Returns a “probability” (confidence) of class 1
- Parameters:
x (Optional[Tensor]) – 2D array
phi_matrix (Optional[Tensor]) – 2D array
- Returns:
1D array
- Return type:
Tensor
- make_phi_matrix(x)[source]#
Returns n x k array with calculated phi(x_i, x_basis_j). n is number of observation from x (x.shape[0]) k is number of basis from initialization.
(2)#\[\begin{bmatrix} \varphi(x_1, x^\text{basis}_1) & \varphi(x_1, x^\text{basis}_2) & \dots & \varphi(x_1, x^\text{basis}_k) \\ \varphi(x_2, x^\text{basis}_1) & \varphi(x_2, x^\text{basis}_2) & \dots & \varphi(x_2, x^\text{basis}_k) \\ \vdots & \vdots & \ddots & \vdots \\ \varphi(x_n, x^\text{basis}_1) & \varphi(x_n, x^\text{basis}_2) & \dots & \varphi(x_n, x^\text{basis}_k) \ \end{bmatrix}\]- Parameters:
x (Tensor) – Array k x m dimensional. k different x_i and m features
- Return type:
Tensor
- class SVM[source]#
Bases:
BaseClassificationBinary classification model. Method predict: SVM.predict(x) –> original names
Mathematical model:
(3)#\[\hat y = \operatorname{sign}(x \cdot w - b \cdot I)\]\(x \in \mathbb{R}^{n \times m}, \ w \in \mathbb{R}^{m \times 1}, \ I = [1]_{n \times 1}\)
And search of best \(w, b\) calculates by minimization of Hinge loss
(4)#\[{\displaystyle \lambda \lVert \mathbf {w} \rVert ^{2}+\left[{\frac {1}{n}}\sum _{i=1}^{n}\max \left(0,1-y_{i}(x_i \cdot w - b)\right)\right] \longrightarrow \min }\]or PEGASOS algorithm
- Variables:
scale – for the best training and prediction, the model will standard normalize the input x data. The first time you call model, std and mean will be saved and in the future use the parameters for scaling. x = (x is the average value) / std
weights – parameters of model. Initialize after first calling
- _fit_pegasos(x, y, epochs=20, lambda_reg=0.95, show_epoch=0, print_function=<built-in function print>)[source]#
Returns trained model SVM [2]
- Parameters:
x (Tensor) – training set
y (Tensor) – target value
epochs – max number of sgd implements
lambda_reg (float) – regularization parameter
show_epoch (int) – amount of showing epochs
print_function (Callable) – print or streamlit.write
- Returns:
trained model
- Return type:
Module
References
- _fit_sgd(x, y, epochs=500, l2_lambda=0, show_epoch=0, print_function=<built-in function print>)[source]#
Returns trained model SVM
- Parameters:
x (Tensor) – training set
y (Tensor) – target value
epochs – max number of sgd implements
l2_lambda (float) – l2 regularization weight
show_epoch (int) – amount of showing epochs
print_function (Callable) – print or streamlit.write
- Returns:
trained model
- fit(x, y, method='sgd', epochs=100, lambda_reg=0.1, show_epoch=0, print_function=<built-in function print>)[source]#
Returns trained model SVM
- Parameters:
x (Tensor) – training set
y (Tensor) – target value. binary classes
method (Literal['pegasos', 'sgd']) – optimization method. Available PEGASOS or sgd
epochs – max number of sgd and pegasos steps
lambda_reg (float) – l2 regularization weight
show_epoch (int) – amount of showing epochs
print_function (Callable) – print or streamlit.write
- Returns:
trained model
- forward(x)[source]#
Returns x @ w + b
(5)#\[f(x) = w_0 + w_1 \cdot x_1 + w_2 \cdot x_2 + \dots + w_m \cdot x_m \]- Parameters:
x (Tensor) – input observations, tensor n x m (n is the number of observations that have m parameters)
- Returns:
regression value (yes, no classification, for binary classes call predict)
- Return type:
Tensor