PDF projection is illustrated in Figure 2.1. In the figure, we see a feature transformation $ T({\bf x})$ that transforms the high-dimensional input data $ {\bf x}$ into a lower-dimensional feature $ {\bf z}$, illustrated as a 1-dimentional density. On the right side of the figure is an illustration of a feature distribution $ g({\bf z})$, which is given (it can be estimated from training data). On the left, we see two potential input distributions, denoted by $ G_1({\bf x}; H_a, T, g)$ and $ G_2({\bf x}; H_b, T, g)$ for reasons that will become clear. Both distributions are consistent with $ g({\bf z})$. This means that if samples are drawn from either distribution, and passed through $ T({\bf x})$, the feature distribution will be precisely $ g({\bf z})$ in either case. We call them projected PDFs since the feature PDF $ g({\bf z})$ has been projected onto the high-dimensional input data space. It can be shown that each projected PDF that generates $ g({\bf z})$ can be defined by a reference hypothesis $ H_0$ under which the distribution of $ {\bf x}$ is written $ p({\bf x}\vert H_0)$. The two projected PDFs $ G_1({\bf x}; H_a, T, g)$ and $ G_2({\bf x}; H_b, T, g)$ shown in the figure are based on reference distributions $ p({\bf x}\vert H_a)$ and $ p({\bf x}\vert H_b)$ respectively. The shape of the projected PDFs depend on these reference distributions and on $ g({\bf z})$. Intuitively, the reference distributions define the characteristics that cannot be observed through $ T({\bf x})$ and therefore cannot be determined by $ g({\bf z})$. Figure 2.1 is notional only, because generally speaking, all distributions have support almost everywhere in the input data space, so are overlapping. The same is true of the reference distributions. In Chapter 3, we will discuss maximum entropy PDF projection in which we choose $ H_0$ to produce the distribution that has maximum entropy among all possible PDFs consistent with $ g({\bf z})$. But for now, assume that $ H_0$ is user-selected.
Figure 2.1: Illustration of PDF projection.

Baggenstoss 2017-05-19