PDF projection is illustrated in Figure
2.1.
In the figure, we see a feature transformation
that transforms the highdimensional
input data into a lowerdimensional feature ,
illustrated as a 1dimentional density.
On the right side of the figure is an illustration of
a feature distribution
, which is given
(it can be estimated from training data).
On the left, we see two potential input distributions,
denoted by
and
for reasons that will become clear. Both
distributions are consistent with
.
This means that if samples are drawn from either distribution,
and passed through
, the feature distribution
will be precisely
in either case.
We call them projected PDFs since
the feature PDF
has been projected
onto the highdimensional input data space.
It can be shown that each projected PDF that generates
can be defined by a reference hypothesis
under which the distribution of is written
.
The two projected PDFs
and
shown in the figure are based on reference distributions
and
respectively.
The shape of the projected PDFs depend on these reference distributions
and on
. Intuitively, the reference distributions
define the characteristics that cannot be observed through
and therefore cannot be determined by
.
Figure 2.1 is notional only, because generally
speaking, all distributions have support almost everywhere in the
input data space, so are overlapping.
The same is true of the reference distributions.
In Chapter 3, we will discuss maximum entropy PDF projection
in which we choose to produce the distribution that has maximum entropy
among all possible PDFs consistent with
. But for now, assume that is userselected.
Figure 2.1:
Illustration of PDF projection.

Baggenstoss
20170519