Simple Example

We now validate the proposition above, that solving for the maximum entropy multivariate truncated exponential (MaxEnt-MTE) distribution approximates the asymptotic distribution of MCMC-UMS for doubly-bounded data. In this experiment, we used a data size of $N=128$. The matrix ${\bf A}$ in (5.1) computed the first $D=6$ Coefficients of the length-$N$ DCT of ${\bf x}$. We created a data sample ${\bf x}$ using a raised sine-wave plus Gaussian noise, clipped it to the interval [0,1], then computed the feature ${\bf z}$. The original ${\bf x}$ was then discarded. We then got a starting point for $\lambda$ using a linear programming solver as previously explained, with the lower and upper bounds of 0 and $1$. We then maximized (7.6) over $\lambda$ subject to (5.12) using the method above. Figure 7.2 shows the results. See software/test_lin01.m.

Figure: The Maximum entropy asymptotic mean (dark line) and the sample mean (circles). One random MCMC-UMS sample is shown (light jagged line). The pseudo-inverse solution (dotted lin) is seen to have values outside $(0,\; 1)$.

The sample-mean of MCMC-UMS after 10000 samples matches the Maximum entropy mean solution as close as could be determined.