Signal Processing for Generative Model design

Clearly, designing or estimating high-dimensional generative models for competely unknown natural processes or man-made signals in natural noise is daunting. But high-dimensions does not deter anyone from designing signal processing algorithms. We propose that signal processing and generative model design is one and the same. The signal processing chain is designed to remove nuisance information or concentrate signal energy based on a theoretical or intuitive understanding. This process of concentrating information and removing noise and nuisance information is in general class-dependent. In MaxEnt PDF projection, which we will describe shortly, the signal processing chain, seen as a dimension-reducing transformation $ {\bf z}=T({\bf x})$, together with the specified or estimated feature PDF $ g({\bf z})$, uniquely defines a complete generative model for which we can compute the likelihood function and generate unlimited random samples. The signal processing can be comprised of many stages and can even be iterative if posed as a maximum likelihood estimator.

Baggenstoss 2017-05-19