# Documentation for Basis Functions¶

Basis Function for NARMAX models

## Fourier¶

Build Fourier basis function. Generate a new feature matrix consisting of all Fourier features with respect to the number of harmonics.

Parameters:

Name Type Description Default
degree int(max_degree)

The maximum degree of the polynomial features.

2

#### Notes¶

Be aware that the number of features in the output array scales significantly as the number of inputs, the max lag of the input and output.

Source code in sysidentpy\basis_function\_basis_function.py
  96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 class Fourier: """Build Fourier basis function. Generate a new feature matrix consisting of all Fourier features with respect to the number of harmonics. Parameters ---------- degree : int (max_degree), default=2 The maximum degree of the polynomial features. Notes ----- Be aware that the number of features in the output array scales significantly as the number of inputs, the max lag of the input and output. """ def __init__(self, n=1, p=2 * np.pi, degree=1, ensemble=True): self.n = n self.p = p self.degree = degree self.ensemble = ensemble self.repetition = None def _fourier_expansion(self, data, n): base = np.column_stack( [ np.cos(2 * np.pi * data * n / self.p), np.sin(2 * np.pi * data * n / self.p), ] ) return base def fit( self, data: np.ndarray, max_lag: int = 1, predefined_regressors: Union[np.ndarray, None] = None, ): """Build the Polynomial information matrix. Each columns of the information matrix represents a candidate regressor. The set of candidate regressors are based on xlag, ylag, and degree defined by the user. Parameters ---------- data : ndarray of floats The lagged matrix built with respect to each lag and column. max_lag : int Target data used on training phase. predefined_regressors : ndarray of int The index of the selected regressors by the Model Structure Selection algorithm. Returns ------- psi = ndarray of floats The lagged matrix built in respect with each lag and column. """ # remove intercept (because the data always have the intercept) if self.degree > 1: data = Polynomial().fit(data, max_lag, predefined_regressors=None) data = data[:, 1:] else: data = data[max_lag:, 1:] columns = list(range(data.shape[1])) harmonics = list(range(1, self.n + 1)) psi = np.zeros([len(data), 1]) for col in columns: base_col = np.column_stack( [self._fourier_expansion(data[:, col], h) for h in harmonics] ) psi = np.column_stack([psi, base_col]) self.repetition = self.n * 2 if self.ensemble: psi = psi[:, 1:] psi = np.column_stack([data, psi]) else: psi = psi[:, 1:] if predefined_regressors is None: return psi, self.ensemble return psi[:, predefined_regressors], self.ensemble def transform( self, data: np.ndarray, max_lag: int = 1, predefined_regressors: Union[np.ndarray, None] = None, ): return self.fit(data, max_lag, predefined_regressors) 

### fit(data, max_lag=1, predefined_regressors=None)¶

Build the Polynomial information matrix.

Each columns of the information matrix represents a candidate regressor. The set of candidate regressors are based on xlag, ylag, and degree defined by the user.

Parameters:

Name Type Description Default
data ndarray of floats

The lagged matrix built with respect to each lag and column.

required
max_lag int

Target data used on training phase.

1
predefined_regressors ndarray of int

The index of the selected regressors by the Model Structure Selection algorithm.

None

Returns:

Type Description
psi = ndarray of floats

The lagged matrix built in respect with each lag and column.

Source code in sysidentpy\basis_function\_basis_function.py
 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 def fit( self, data: np.ndarray, max_lag: int = 1, predefined_regressors: Union[np.ndarray, None] = None, ): """Build the Polynomial information matrix. Each columns of the information matrix represents a candidate regressor. The set of candidate regressors are based on xlag, ylag, and degree defined by the user. Parameters ---------- data : ndarray of floats The lagged matrix built with respect to each lag and column. max_lag : int Target data used on training phase. predefined_regressors : ndarray of int The index of the selected regressors by the Model Structure Selection algorithm. Returns ------- psi = ndarray of floats The lagged matrix built in respect with each lag and column. """ # remove intercept (because the data always have the intercept) if self.degree > 1: data = Polynomial().fit(data, max_lag, predefined_regressors=None) data = data[:, 1:] else: data = data[max_lag:, 1:] columns = list(range(data.shape[1])) harmonics = list(range(1, self.n + 1)) psi = np.zeros([len(data), 1]) for col in columns: base_col = np.column_stack( [self._fourier_expansion(data[:, col], h) for h in harmonics] ) psi = np.column_stack([psi, base_col]) self.repetition = self.n * 2 if self.ensemble: psi = psi[:, 1:] psi = np.column_stack([data, psi]) else: psi = psi[:, 1:] if predefined_regressors is None: return psi, self.ensemble return psi[:, predefined_regressors], self.ensemble 

## Polynomial¶

Bases: BaseBasisFunction

Build polynomial basis function. Generate a new feature matrix consisting of all polynomial combinations of the features with degree less than or equal to the specified degree.

$y_k = \sum_{i=1}^{p}\Theta_i \times \prod_{j=0}^{n_x}u_{k-j}^{b_i, j} \prod_{l=1}^{n_e}e_{k-l}^{d_i, l}\prod_{m=1}^{n_y}y_{k-m}^{a_i, m}$

where $$p$$ is the number of regressors, $$\Theta_i$$ are the model parameters, and $$a_i, m, b_i, j$$ and $$d_i, l \in \mathbb{N}$$ are the exponents of the output, input and noise terms, respectively.

Parameters:

Name Type Description Default
degree int(max_degree)

The maximum degree of the polynomial features.

2

#### Notes¶

Be aware that the number of features in the output array scales significantly as the number of inputs, the max lag of the input and output, and degree increases. High degrees can cause overfitting.

Source code in sysidentpy\basis_function\_basis_function.py
 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 class Polynomial(BaseBasisFunction): r"""Build polynomial basis function. Generate a new feature matrix consisting of all polynomial combinations of the features with degree less than or equal to the specified degree. $$y_k = \sum_{i=1}^{p}\Theta_i \times \prod_{j=0}^{n_x}u_{k-j}^{b_i, j} \prod_{l=1}^{n_e}e_{k-l}^{d_i, l}\prod_{m=1}^{n_y}y_{k-m}^{a_i, m}$$ where $p$ is the number of regressors, $\Theta_i$ are the model parameters, and $a_i, m, b_i, j$ and $d_i, l \in \mathbb{N}$ are the exponents of the output, input and noise terms, respectively. Parameters ---------- degree : int (max_degree), default=2 The maximum degree of the polynomial features. Notes ----- Be aware that the number of features in the output array scales significantly as the number of inputs, the max lag of the input and output, and degree increases. High degrees can cause overfitting. """ def __init__( self, degree=2, ): self.degree = degree def fit( self, data: np.ndarray, max_lag: int = 1, predefined_regressors: Union[np.ndarray, None] = None, ): """Build the Polynomial information matrix. Each columns of the information matrix represents a candidate regressor. The set of candidate regressors are based on xlag, ylag, and degree defined by the user. Parameters ---------- data : ndarray of floats The lagged matrix built with respect to each lag and column. max_lag : int Target data used on training phase. predefined_regressors : ndarray of int The index of the selected regressors by the Model Structure Selection algorithm. Returns ------- psi = ndarray of floats The lagged matrix built in respect with each lag and column. """ # Create combinations of all columns based on its index iterable_list = range(data.shape[1]) combinations = list(combinations_with_replacement(iterable_list, self.degree)) if predefined_regressors is not None: combinations = [combinations[index] for index in predefined_regressors] psi = np.column_stack( [ np.prod(data[:, combinations[i]], axis=1) for i in range(len(combinations)) ] ) psi = psi[max_lag:, :] return psi def transform( self, data: np.ndarray, max_lag: int = 1, predefined_regressors: Union[np.ndarray, None] = None, ): return self.fit(data, max_lag, predefined_regressors) 

### fit(data, max_lag=1, predefined_regressors=None)¶

Build the Polynomial information matrix.

Each columns of the information matrix represents a candidate regressor. The set of candidate regressors are based on xlag, ylag, and degree defined by the user.

Parameters:

Name Type Description Default
data ndarray of floats

The lagged matrix built with respect to each lag and column.

required
max_lag int

Target data used on training phase.

1
predefined_regressors ndarray of int

The index of the selected regressors by the Model Structure Selection algorithm.

None

Returns:

Type Description
psi = ndarray of floats

The lagged matrix built in respect with each lag and column.

Source code in sysidentpy\basis_function\_basis_function.py
 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 def fit( self, data: np.ndarray, max_lag: int = 1, predefined_regressors: Union[np.ndarray, None] = None, ): """Build the Polynomial information matrix. Each columns of the information matrix represents a candidate regressor. The set of candidate regressors are based on xlag, ylag, and degree defined by the user. Parameters ---------- data : ndarray of floats The lagged matrix built with respect to each lag and column. max_lag : int Target data used on training phase. predefined_regressors : ndarray of int The index of the selected regressors by the Model Structure Selection algorithm. Returns ------- psi = ndarray of floats The lagged matrix built in respect with each lag and column. """ # Create combinations of all columns based on its index iterable_list = range(data.shape[1]) combinations = list(combinations_with_replacement(iterable_list, self.degree)) if predefined_regressors is not None: combinations = [combinations[index] for index in predefined_regressors] psi = np.column_stack( [ np.prod(data[:, combinations[i]], axis=1) for i in range(len(combinations)) ] ) psi = psi[max_lag:, :] return psi