High and low temperature limits

Next: Equal covariances Up: Prior mixtures Previous: Analytical solution Contents

High and low temperature limits

Low and high temperature limits are extremely useful because in both cases the stationarity Eq.(21) becomes linear, corresponding thus to classical quadratic regularisation approaches.

In the high temperature limit $\beta \rightarrow 0$ the exponential factors become -independent

$\begin{displaymath} a_j \stackrel{\beta\rightarrow0}{\longrightarrow} a^0_j = \... ...\sum_k^me^{-E_{\theta,\beta,k}+\frac{1}{2}\ln\det{\bf K}_k}} , \end{displaymath}$

(34)

(for $b_j^{*}\rightarrow b_j^{0,*}$ replace ${\bf K}_j$ by $\widetilde{\bf K}_j$ ). The solution $h = \bar t$ is a (generalised) `complete template average'

$\begin{displaymath} \bar t = {\bf K}_{a^0}^{-1} \left( {\bf K_T} t_T + \sum_l^m a^0_j \,{\bf K}_j t_j \right) , \end{displaymath}$

(35)

with

$\begin{displaymath} {\bf K}_{a^0} = {\bf K}_T+\sum_j a_j^0 \,{\bf K}_j . \end{displaymath}$

(36)

This high temperature solution corresponds to the minimum of the quadratic functional $E_{T} +\sum_j^m a_j^0 E_{h,j}$ ,

In the low temperature limit $\beta\rightarrow \infty$ only the maximal component contributes, i.e.,

$\begin{displaymath} a_j \stackrel{\beta\rightarrow\infty}{\longrightarrow} a^\in... ...,j} \\ 0 & j \ne {\rm argmin}_j E_{h,j} \end{array} \right. , \end{displaymath}$

(37)

(for

replace $E_{h,j}$ by $\widetilde E_j$ ) assuming $E_{\beta,\theta,j}$ = $E_{\beta}$ + $E_{\theta,j}$ or $E_{\beta,\theta,j}$ = $E_{\beta}$ + $E_{j}$ + $\beta E_{\theta}$ . Hence, low temperature solutions $h = \bar t_{j}$ , are all (generalised) `component averages' $\bar t_j$ provided they fulfil the stability condition

$\begin{displaymath} E_{h,j} (h=\bar t_j) <E_{h,j^\prime} (h=\bar t_j) , \quad \forall j^\prime \ne j , \end{displaymath}$

(38)

or, after performing a (generalised) `bias-variance' decomposition, $2 V_{j}<{B}_{j^\prime} (j,j) + 2 V_{j^\prime}$ , with $m\times m$ matrices

$\begin{displaymath} {B}_j (k,l) = \Big(\bar t_k-\bar t_j,\,\left( {\bf K}_D + {\bf K}_j\right) \,(\bar t_l-\bar t_j)\Big) \end{displaymath}$

(39)

and (generalised) `template variances'

$\displaystyle V_j \!\!$	$\textstyle =$	$\displaystyle \frac{1}{2} \Bigg( \Big(t_T,\,{\bf K}_T\,t_T\Big) +\Big(t_j,\,{\bf K}_j\,t_j\Big) \quad$
	$\textstyle -$	$\displaystyle \Big(\bar t_j,({\bf K}_T+{\bf K}_j) \,\bar t_j\Big) \Bigg) = \widetilde E_{0,j}.$	(40)

That means single component averages $\bar t_j$ (which minimise $E_{h,j}$ and thus $-\beta E_j+c_j$ ) become solutions at zero temperature $1/\beta$ in case their (generalised) variance

measuring the discrepancy between data and prior term is small enough.

Next: Equal covariances Up: Prior mixtures Previous: Analytical solution Contents

Joerg_Lemm 1999-12-21