Consider now the case of regression
according to functional (247)
with an adaptive template .
The system of stationarity equations for
the regression function (corresponding to )
and becomes
(448) |
(453) |
For comparison with Eqs.(449,450,451)
we give the stationarity equations
for parameters
for a parameterized regression functional
including an additional prior term
with hyperparameters
(455) |
Let us now compare the various regression functionals we have met up to now. The non-parameterized and regularized regression functional (247) implements prior information explicitly by a regularization term.
A parameterized and regularized functional of the form (353) corresponds to a functional of the form (454) for fixed. It imposes restrictions on the regression function in two ways, by choosing a specific parameterization and by including an explicit prior term. If the number of data is large enough, compared to the flexibility of the parameterization, the data term of alone can have a unique minimum. Then, at least technically, no additional prior term would be required. This corresponds to the classical error minimization methods used typically for parametric approaches. Nevertheless, also in such situations the explicit prior term can be useful if it implements useful prior knowledge over .
The regularized functional with prior- or hyperparameters (432) implements, compared to , effectively weaker prior restrictions. The prior term corresponds to a soft restriction of to the space spanned by the parameterized . In the limit where the parameterization of is rich enough to allow = at the stationary point the prior term vanishes completely.
The parameterized and regularized functional (454), including prior parameters , implements prior information explicitly by a regularization term and implicitly by the parameterization of . The explicit prior term vanishes if = at the stationary point. The functional combines a hard restriction of with respect to the space spanned by the parameterization and a soft restriction of with respect to the space spanned by the parameterized . Finally, the parameterized and non-regularized functional (452) implements prior information only implicitly by parameterizing . In contrast to the functionals and it implements only a hard restriction for . The following table summarizes the discussion:
Functional | Eq. | prior implemented |
(247) | explicitly | |
(353) | explicitly and implicitly | |
(432) | explicitly | |
no prior for | ||
(454) | explicitly and implicitly | |
no expl. prior for | ||
(452) | implicitly |