## Statement of MaxEnt theorem

Maximum entropy PDF projection [5] is a means of finding the unique member of with highest entropy, more precisely: to find the that produces with highest entropy. The entropy of is given by

It can be shown (See [5], equation 8) that this can be expanded as follows:

 (3.1)

where the entropy of is and the manifold entropy is

 (3.2)

where comes from (2.5). Since is fixed, to absolutely maximize (i.e. for each ), must be maximized for each . This is achieved if is the uniform distribution, which is the MaxEnt distribution on regions of compact support. But, in (2.5), is shaped by . Therefore, we have two requirements, (a) must be of constant value on any manifold (2.4), and (b) all manifolds must be compact. There are two ways to achieve these requirements depending on .

When is itself a compact set, such as the unit hypercube for , we can make the uniform distribution. Then, so long as the manifold is compact for all 3.1, then will be a proper uniform distribution for all , which has maximum entropy. Alternatively, when is infinite in extent, the manifold can be forced to be compact by the inclusion of an energy statistic in (first proposed in [5]). The solution for compact and the solution for unbounded are formalized by the following two theorems.

Theorem 2   Maximum Entropy PDF Projection - Compact . Starting with the same assumptions as Theorem 1, we further assume that is a compact set and Furthermore, we assume that is a compact set for all . Then, the PDF

 (3.3)

where is the distribution of under the uniform assumption , is the member of with highest entropy.

The second case considers when is not compact.

Theorem 3   Maximum Entropy PDF Projection - Unbounded . Starting with the same assumptions as Theorem 1, we further assume that there exists a function such that

 (3.4)

for some norm valid in . We further assume that for all finite , is a compact set. Then, if the reference distribution can be written in the form

 (3.5)

for some function , then the projected PDF (2.2) is the member of with highest entropy.

The highest-entropy distribution, which is found by maximizing the entropy of over , is denoted by . Intuitively, has the MaxEnt property because all samples generated by for a given are equally likely (the uniform distribution, which has maximum entropy on a compact set). This is a result of the fact that for a given , all samples on the manifold have constant value on , making also constant on the manifold, implying that the manifold distribution is in fact the uniform distribution. The reader is referred to a previous article for additional details of the proof [5].

Baggenstoss 2017-05-19