Having defined the fundamentals of prior concepts in a bit more detail we now give a short and rather compact outline of combinations of concepts. A more detailed paper is in preparation. We also include some comments on learning algorithms.
Commonly one wants to approximate training data AND an additional regularization (e.g. smoothness) constraint. Interpreting the functional up to a constant as posterior log-probability, an AND of independent events is realized by a sum. A sum of quadratic terms = is again quadratic and one finds easily
Proposition 1 (probabilistic AND)
AND-ing quadratic concepts
by adding their square distances
results in
= +
with square distance
=
from template average
where
,
and -independent minimal component energy
=
,
which has, up to a factor , the structure of a variance.
The linear stationarity equation for a functional
reads
.
The inverse exists for positive definite giving the solution . This case includes for example the classical regularization approaches of Radial Basis Functions (RBF) and various spline methods [3,4,5,6].
As example for OR-like combinations. consider an old, only partly conserved image. Parts of it might be trees OR persons. Having available a collection of example images of trees and persons one may wish to use this information to improve the reconstruction of the image. One has to be careful not to reconstruct a tree as person. Also we do not want to simply replace objects by their prototypes, but to reconstruct the original image as good as possible.
A probabilistic OR for disjunct events appears for probabilities as sum, i.e. as mixture model, = = . Assuming to be easier to specify than and using the notations , , , and we obtain
Proposition 2 (probabilistic OR)
Modeling a probabilistic OR for concepts by
a mixture (or finite temperature) model
Here we introduced a convexity parameter (inverse temperature, Lagrange multiplier to determine the average error) controlling the number of local minima of and an analogous for mixture probabilities . (See for example [7] and, similarly, for clustering [8].)
Instead of a mixture of Gaussian (``free'') models one may specify non-concave (``interacting'') log-probabilities by fuzzy logical combinations. Choosing the product as a simple realization of a fuzzy logical OR for distances we can consider polynomial interactions:
Proposition 3 (``a fuzzy OR'')
A product implementation of OR gives
the polynomial model
The error surface becomes convex in the limit . This parallelizes the phenomenological Landau-Ginzburg treatment of phase transitions in statistical physics.