PDF Projection validation: the Acid test

In constructing a classifier, it is of utmost importance that the J-function is accurate. This will insure that the resulting projected PDF is, in fact, a valid PDF. If the J-function is inaccurate, classification errors may occur. To verify the “J" function, we have developed an end-to-end test that we call the “Acid Test" because of its foolproof nature. To use the method, it is first necessary to define a fixed synthetic data hypothesis, denoted by $H_s$, for which we can compute the PDF $p({\bf x}\vert H_s)$ readily, and for which we can create synthetic raw data. Note that $H_s$ is not a reference hypothesis. The synthetic data is converted into features and the PDF $\hat{p}({\bf z}\vert H_s)$ is estimated from the synthetic features (using a Gaussian Mixture PDF, HMM, or any appropriate parametric or non-parametric statistical model). Next, the theoretical PDF $p({\bf x}\vert H_s)$ is compared with the projected PDF

$$\hat{p}({\bf x}\vert H_s) = J({\bf x}; H_0,T) \; \hat{p}({\bf z}\vert H_s)$$

for each sample of synthetic data. The log-PDF values are plotted on each axis and the results should fall on the X=Y line. Since the acid test checks the equality of two entirely different paths, it should find any systematic error in PDF estimation or in the J-function calculation. To see an example of the Acid test, jump to Section 5.2.1, example 8.