Next: A numerical example
Up: Prior mixtures
Previous: High and low temperature
  Contents
Especially interesting
are -independent
=
with -independent determinants so
or
, respectively,
do not have to be calculated.
Notice that this still allows
completely arbitrary parameterisations
of .
Thus, the template function can for example be
a parameterised model,
e.g., a neural network or decision tree,
and maximising the posterior with respect to corresponds
to training that model.
In such cases the prior term
forces the maximum posterior solution to be similar
(as defined by )
to this trained parameterised reference model.
The condition of invariant
does not exclude
adaption of covariances.
For example, transformations for real, symmetric positive definite
leaving
determinant and eigenvalues (but not eigenvectors) invariant
are of the form
with real, orthogonal
= .
This allows for example to adapt the sensible
directions of multidimensional Gaussians.
A second kind of transformations
changing eigenvalues but not eigenvectors and determinant
is of the form
if the product of eigenvalues
of the real, diagonal
and
are equal.
Eqs.(29,35)
show that the high temperature solution becomes
a linear combination
of the (potential) low temperature solutions
|
(41) |
Similarly,
Eq.(21) simplifies to
|
(42) |
and
Eq.(23) to
|
(43) |
introducing
vector with components ,
matrices
defined in (39).
Eq.(42) is still a nonlinear equation
for , it shows however that the solutions
must be convex combinations of the -independent
(see Fig. 2).
Thus, it is sufficient to solve
Eq.(43) for mixture coefficients
instead of Eq.(21) for the function .
Figure 2:
Left: Example of a solution space for = 3.
Shown are three low temperature solutions ,
high temperature solution , and a possible
solution at finite .
Right: Exact vs. (dominant) (dashed)
for = , = 2,
= 0.405,
= 0.605.
|
Figure 3:
Shown are
the plots of
and
within the inverse temperature range
(for ,
= ).
Notice the appearance of a second stable
solution at low temperatures.
|
For two prior components, i.e., ,
Eq.(42) becomes
|
(44) |
with
|
(45) |
because
the matrices are in this case zero except
.
For
uniform in
we have
=
so that = .
The stationarity Eq.(43),
being analogous to
the celebrated mean field equation of a ferromagnet,
can be solved graphically
(see Fig.3 and Fig.2
for a comparison with ),
the solution is given by the point
where
|
(46) |
Next: A numerical example
Up: Prior mixtures
Previous: High and low temperature
  Contents
Joerg_Lemm
1999-12-21