Simple Example

We now validate the proposition above, that solving for the maximum entropy multivariate truncated exponential (MaxEnt-MTE) distribution approximates the asymptotic distribution of MCMC-UMS for doubly-bounded data. In this experiment, we used a data size of $ N=128$. The matrix $ {\bf A}$ in (5.1) computed the first $ D=6$ Coefficients of the length-$ N$ DCT of $ {\bf x}$. We created a data sample $ {\bf x}$ using a raised sine-wave plus Gaussian noise, clipped it to the interval [0,1], then computed the feature $ {\bf z}$. The original $ {\bf x}$ was then discarded. We then got a starting point for $ \lambda$ using a linear programming solver as previously explained, with the lower and upper bounds of 0 and $ 1$. We then maximized (6.6) over $ \lambda$ subject to (5.12) using the method above. Figure 6.2 shows the results. See software/test_lin01.m.
Figure: The Maximum entropy asymptotic mean (dark line) and the sample mean (circles). One random MCMC-UMS sample is shown (light jagged line). The pseudo-inverse solution (dotted lin) is seen to have values outside $ (0,\; 1)$.
The sample-mean of MCMC-UMS after 10000 samples matches the Maximum entropy mean solution as close as could be determined.

Baggenstoss 2017-05-19