Estimation using Gaussian Mixtures

The GM representation of the density has the a remarkable property that $ p({\bf x}\vert{\bf y})$ can be computed in closed form. This is especially useful in visualization of information. For example, it is useful to show a human operator the distribution of likely $ {\bf x}$ after $ {\bf y}$ is measured. If desired, the MMSE can be computed in closed form as well. The MAP estimate can also be computed, but that requires a search over $ {\bf x}$.

Let the GM approximation to the distribution be given by

$\displaystyle p({\bf x},{\bf y})={ \displaystyle \sum}_{i}\alpha_{i}p_{i}({\bf x},{\bf y}).$ (13.4)

By Bayes rule,

$\displaystyle p({\bf x}\vert{\bf y})=\frac{p({\bf x},{\bf y})}{p({\bf y})} =
\frac{1}{p({\bf y})} { \displaystyle \sum}_{i}\alpha_{i}p_{i}({\bf x},{\bf y})

where $ p({\bf y})$ is the marginal distribution of $ {\bf y}$. We now define $ p_{i}({\bf y})$ as the marginal distributions of $ {\bf y}$ given that $ {\bf y}$ is a member of mode $ i$. These are, of course, Gaussian with means and covariances taken from the $ {\bf y}$-partitions of the mode $ i$ mean and covariance $ \mu$$ _{i},{\bf\Sigma}_{i}$.

   $\displaystyle \mbox{\boldmath$\mu$}$$\displaystyle _{i} = \left[ \begin{array}{l}
\mbox{\boldmath$\mu$}_{x,i}  \m...
...}_{xy,i} \\
{\bf\Sigma}_{yx,i} & {\bf\Sigma}_{yy,i} \\
\end{array} \right]


\begin{displaymath}\begin{array}{rcl} p({\bf x}\vert{\bf y}) & = &{\displaystyle...
...alpha_{i} p_{i}({\bf y})p_{i}({\bf x}\vert{\bf y})} \end{array}\end{displaymath} (13.5)

where $ p_{i}({\bf x}\vert{\bf y})$ is the conditional density for $ {\bf x}$ given $ {\bf y}$ assuming that $ {\bf x}$ and $ {\bf y}$ are from that certain Gaussian sub-class $ i$. Fortunately, there is a closed-form equation for $ p_{i}({\bf x}\vert{\bf y})$ [55]. $ p_{i}({\bf x}\vert{\bf y})$ is Gaussian with mean

$\displaystyle {\bf E}_{i}({\bf x}\vert{\bf y}) =$   $\displaystyle \mbox{\boldmath$\mu$}$$\displaystyle _{x,i}+ {\bf\Sigma}_{xy,i} {\bf\Sigma}_{yy,i}^{-1} ({\bf y}-$   $\displaystyle \mbox{\boldmath$\mu$}$$\displaystyle _{y,i}).$ (13.6)

and covariance

$\displaystyle {\rm cov}_{i}({\bf x}\vert{\bf y}) = {\bf\Sigma}_{xx,i} - {\bf\Sigma}_{xy,i} {\bf\Sigma}_{yy,i}^{-1} {\bf\Sigma}_{yx,i}.$ (13.7)

Note that the conditional distribution is a Gaussian Mixture in its own right, with mode weights modified by $ p_{i}({\bf y})$ which tends to ``select" the modes closest to $ {\bf y}$. To reduce the number of modes in the conditioning process, one could easily remove those modes with a low value of $ p_{i}({\bf y})$ (suggested by R. L. Streit).

This conditional distribution can be used for data visualization or, to easily calculate the conditional mean estimate, which is a by-product of equations (13.5),(13.6),(13.7):

\begin{displaymath}\begin{array}{rcl} {\bf E}({\bf x}\vert{\bf y}) & = & {\displ...
...{i} p_{i}({\bf y}){\bf E}_{i}({\bf x}\vert{\bf y})} \end{array}\end{displaymath} (13.8)

Baggenstoss 2017-05-19