PDF projection is illustrated in Figure
In the figure, we see a feature transformation
that transforms the high-dimensional
input data into a lower-dimensional feature ,
illustrated as a 1-dimentional density.
On the right side of the figure is an illustration of
a feature distribution
, which is given
(it can be estimated from training data).
On the left, we see two potential input distributions,
for reasons that will become clear. Both
distributions are consistent with
This means that if samples are drawn from either distribution,
and passed through
, the feature distribution
will be precisely
in either case.
We call them projected PDFs since
the feature PDF
has been projected
onto the high-dimensional input data space.
It can be shown that each projected PDF that generates
can be defined by a reference hypothesis
under which the distribution of is written
The two projected PDFs
shown in the figure are based on reference distributions
The shape of the projected PDFs depend on these reference distributions
. Intuitively, the reference distributions
define the characteristics that cannot be observed through
and therefore cannot be determined by
Figure 2.1 is notional only, because generally
speaking, all distributions have support almost everywhere in the
input data space, so are overlapping.
The same is true of the reference distributions.
In Chapter 3, we will discuss maximum entropy PDF projection
in which we choose to produce the distribution that has maximum entropy
among all possible PDFs consistent with
. But for now, assume that is user-selected.
Illustration of PDF projection.