Discriminative Alignment of PBN
It was shown in [1] that a generative classifier
(a PBN) can compete with state of the art discriminative classifiers.
This seems to contradict the widely-held belief that the
generative task is much harder, and unnecessary for classifying [93].
However, generative models are useful in their own right, and
can be applied in some tasks where discriminative classifiers
cannot [94]. A generative classifier than
can perform as well as a discriminative classifier,
is highly desireable.
Sand box analogy. Discriminative alignment can be understood in terms of the
conceptual image of a sand box enclosed by a wooden frame. The frame can be
thought of as the discriminative task, separating the probability
mass (the sand) from the other classes at the decision boundaries.
The generative model can be thought of as heaping the sand
at the places where data is more likely.
The two tasks can be achieved simultaneously using
discriminative alignment of a PBN.