Test Results

Standard acid and re-synthesis tests are implemented by software/module_sinar_test.m. It also implements a "crlb" test that compares ML estimation error with the CRLB. The following thorough test is implemented by software/test_sinar.m.

1. We first generated data with two sinewaves and background data from a second-order AR spectrum. In Figure 9.3, on top, we see the the raw spectrum (cyan), and the median filtered spectrum (magenta), the initial AR spectrum (red), and the detected sinusoids (asterisk).
2. Next, we estimated the SINAR model parameters as described ( software/module_sinar.m). On the bottom of Figure 9.3, we see again the the raw spectrum (cyan), the final AR spectrum (magenta), and the spectrum of the final sinusoidal component ${\bf y}$ (green).

   Figure 9.3: Results of SINAR model test.

   

3. Next, we compressed the parameters (features) by converting the real/imaginary amplitude pairs to just log-amplitude using the two modules:

     [z,jout]=module_c2r(z,jout,nsin+[1:2*nsin]);
     [z,jout]=module_log(z,jout,[ nsin+[1:nsin]  2*nsin+P+1]);
   

4. We then re-synthesized the expanded parameters using the inverse of the above chain:

     zs=module_exp(z,0,[ nsin+[1:nsin]  2*nsin+P+1]);
     zs=module_c2r_synth(zs,nsin+[1:nsin]);
   

5. Using the SINAR parameters from the above, we re-synthesized the raw data using ( software/module_sinar_synth.m).

6. We then passed this raw data back into the chain software/module_sinar.m, software/module_c2r.m, software/module_log.mand compared the features with the original set:

      0.335671813882368   0.335671815103722  -0.000000001221354
      0.349466557731445   0.349466557694834   0.000000000036611
      3.702689056085881   3.702689045510666   0.000000010575215
      2.296026775397141   2.296026769375969   0.000000006021172
     -1.248813788132684  -1.248813788102529  -0.000000000030155
      0.748129007066103   0.748129007111796  -0.000000000045693
      3.868379046063297   3.868379046072616  -0.000000000009318
   

   Note that the features, including frequencies, amplitides, and AR coefficients, agree to many decimal places.

7. Next, we conducted an experiment to verify the Cramér Rao lower bound (CRLB). We generated data with a fixed parameters set, and measured the estimation covariance of the ML parameter estimates. We then compared this with the CRLB bound. The CRLB was obtained by averaging the Fisher's Information matrices, then inverting. For frequency estimation, the result was:

   Actual and estimated freq:
   ans =
      0.332898504064205   0.332893584758298
      0.362250453516612   0.362245648564146
   ans =
    Freq CRLB and actual (stdev):
   ans =
      1.0e-03 *
      0.296121883718455   0.306671006314352
      0.433985843610495   0.461426272795449
   

   For amplitude estimation, the result was:

   ans =
    Ampl CRLB and actual (stdev):
   ans =
      0.493213972877173   0.481101790137884
      0.433333457180508   0.428876675859441
      0.484875224240585   0.481079868355394
      0.450742290250430   0.428879298178772
   

   And, for AR parameters, the result was:

   AR CRLB, actual:
   ans =
      0.001824998585736  -0.001277523563861   0.001916486591848  -0.001355315073806
     -0.001277523563861   0.001824998585736  -0.001355315073806   0.001853675830711
   CR bound for variance: 4.240074, actual stdev: 4.361447
   

   All results agree to within about 10%.

8. Next, we conducted an acid test (See Section 2.3.8). As a “theoretical PDF", we generated data according to a circularly-stationary process (Sections 10.1.4, 10.1.2) with a very peaked spectrum. Note that in theory, a sinusoid in noise is not a circularly-stationary process. But, for a very sharply-peaked spectrum, it is an approximation. We did this because we could generate data and compute the exact PDF of the circularly-stationary process. In Figure 9.4, we show the spectral shape we used. The spectal peak falls directly on an FFT bin, so depending on how sharp the peak is, the adjacent bins can have little or no spectral energy above the background. If the data is generated in the frequencyy-domain as independent exponentially-distributed bins ^9.2, the PDF computed using (10.7) is the exact PDF of the generated data. This is a perfect synthetic data / “theoretical" PDF pair for the acid test of the SINAR model.

   Figure 9.4: Data spectrum used for acid test.

   
   In Figure 9.5, we show the results of the acid test. On the top is the SINAR model acid test results. On the bottom is the result for the software/module_ar_mlx.m module. Clearly the AR features alone produce a poor PDF estimate because the features are insufficient.

   Figure 9.5: Results of SINAR model acid test. Left: acid test using data with spectrum shown in Figure 9.4. Right: with just background AR spectrum. Top: SINAR model, bottom: AR model.