Contents

• Introduction
  • Overview
    • Is This a Book?
    • Purpose
    • What is the Class-Specific Method (CSM)?
    • What are the advantages of CSM?
    • What is a class-specific module?
    • What are the different topologies of a CSM implementation?
    • What kinds of features can be used by CSM?
    • What software tools are available for CSM?
    • Book organization
  
  
• PDF Projection
  • Illustration of PDF Projection
  • Mathematical Definition of PDF Projection
    • Problem Setup
    • Generation of Samples from
    • Class-Specific Feature Classifier
    • Chain Rule
  • Implementation
    • The Fixed Reference Hypothesis and the PDF tail problem
    • The Saddle Point Approximation
    • Floating Reference Hypothesis
    • Data Normalization
    • Maximum likelihood and PDF Projection
    • J-function Types
    • Interpretation of the J-function
    • PDF Projection validation: the Acid test
  • PDF Projection Software Module Design
    • The module function
    • Computing the projected PDF
    • Multiple dimensions and segments or samples
  
  
• Maximum Entropy PDF Projection
  • Introduction
    • Why Maximum entropy?
  • Mathematical description of MaxEnt PDF projection
    • Statement of MaxEnt PDF Projection theorem
    • Energy Statistic (ES)
    • Discussion on the choice of
    • MaxEnt and the Chain Rule
  • Data Synthesis
  • Information Maximization
  
  
• Simple Feature Transformations
  • General 1:1 transformations
    • Log
    • Square root of positive data
    • DCT
    • The Exp Function (module_exp.m)
    • Log Bilinear (module_bilinear.m)
  • Squaring of Gaussian data
  • Magnitude Squared DFT of Gaussian data
  • Linear transformation of Gaussian data (ES approach)
    • Feature Transformation and Reference hypothesis
    • J-Function
    • UMS
    • Asymptotic (large ) behavior
    • Example
  • Linear transformation of Gaussian data (fixed variance approach)
  • Gaussian Model with Non-Linear Parameter Dependence
    • Data model
    • ML estimation
    • PDF Projection
    • UMS
  
  
• Linear transformation of positive-valued exponentially-distributed data
  • Feature transformation and Energy Statistic (ES approach)
  • J-function Implementations
    • General solution using SPA (module_A_chisq.m)
    • Application of SPA to Circular Auto-Correlation Function Analysis
    • Application of SPA to MEL Bank analysis
    • General solution using CLT (module_A_chisq_clt.m)
    • Application of CLT to Auto-Correlation Function Analysis
    • Application of CLT to MEL Band Analysis
    • General solution using ML
    • ML Solution for ACF/AR
    • ML Solution for MEL band analysis
  • Data re-synthesis of positive data from linear features: UMS
    • MCMC-UMS
    • Predicting the Mean (centroid) of MCMC-UMS.
    • Example: Circular AR/ACF Spectrum
    • Accuracy of Predicted Mean and Mixing Rate
    • Whitened MCMC-UMS
    • Dimension effects
    • Example: MFCC
    • Software for UMS
  • Fixed mean assumption
  
  
• Linear transformation of positive-valued truncated Gaussian data
• Linear transformation of doubly-bounded data
  • Feature transformation and J-function
  • MCMC-UMS for doubly-bounded
  • The Truncated Exponential Distribution (TED)
  • Solving for asymptotic mean of MCMC-UMS
    • Method 1 (requiring starting point)
    • Method 2 (not requiring starting point)
  • Simple Example
  • Image Example
    • Cameraman image
  
  
• Order Statistics
  • Arbitrary order statistics of a set of iid RV, and sum of remainder energy.
    • Features
    • Computing : Integral Solution
    • Largest RV
    • Class-Specific Module module_bestkfft.m.
    • Largest values of a set of independent RVs with different distributions, and sum of remainder, and indexes (exp_ord_spect.m).
  • SPA Solution for Order Statistics of Exponential RVs (module_exp_ord.m)
    • Comparison
  
  
• Special Models
  • M Quadratic and Linear Forms of Correlated Random Variables
    • Form of the statistics
    • Saddlepoint Approximation
    • Autocorrelation Function.
    • Implementation: Circular and Non-Circular Autocorrelation
    • Linear Filtering
  • Sinewave plus AR noise (SINAR)
    • Statistical Model
    • MATLAB module
    • Data Re-synthesis
    • Test Results
    • Amplitude Compression
    • Feature PDF (Spectral Histogram)
  
  
• Timeseries Analysis using AR, MA, and ARMA models
  • Stationary and Circularly Stationary processes and the ACF
    • Stationary Processes and the Power Spectrum
    • Circularly-stationary process
    • Data PDF of stationary process
    • Data PDF of circularly stationary process
    • When to use Circulary-Stationary Processes
    • Rational Transfer Function Models
  • ARMA Modeling
    • Computing the ACF of an ARMA process
    • Exact PDF of an ARMA process - Efficient method using Levinson algorithm
    • Exact PDF of an ARMA process - Very efficient method using filtering
    • Circular ARMA process
    • ARMA parameter estimation
    • CR Bound analysis of ARMA PDF - exact forms.
    • CR bound analysis of circular ARMA parameters pdf_arma_circ.m.
    • CR Bound analysis of the exact PDF of an ARMA processes - filtering approach.
    • ARMA Modules
  • MA Modeling
    • Exact PDF for MA process
    • MA parameter estimation
    • CR bound analysis of MA parameters
    • MA Modules
  • AR modeling
    • AR Data PDF - exact
    • AR data PDF, Large (circular process)
    • Estimating the AR coefficients - deterministic approach (Levinson Algorithm)
    • CR bound analysis of AR parameters - frequency domain approach
    • CR Bound analysis of AR PDF - exact forms.
    • Reflection coefficients
    • Polynomial Roots (module_poly2root.m).
    • LAR coefficients (module_bilinear.m)
    • Block Diagram of AR Software Modules
    • Chains for ACF/AR features
  
  
• Cepstral Analysis
  • Comparison of MFCC approaches
    • Experimental Approach
    • PDF based on Circular Power Spectrum
    • Feature Chains
    • Experimental Setup
    • Acid Tests
    • Classification experiments
  • MFCC: Recommended chain
  • Rectangular Bands
  
  
• Classifier Topologies
  • Segmentation and Window Functions
    • Unsegmented Data
    • Block Segmentation
    • Hanning-3 Segmentation
    • Hanning-3 Example Experiment
    • On-the-fly Segmentation
    • Multi-Resolution HMM
  • Likelihood Comparison and Combination methods
    • One Class/One Model: Class-Specific Features (CSF)
    • Class-Specific Feature Mixture (CSFM)
    • Class-Specific Hidden Markov Models (CSHMM)
  • PDF Sculpting
  
  
• Feature PDF Estimation
  • PDF Modeling Introduction and Notation
  • PDF Estimation using Gaussian Mixtures
    • Gaussian Mixtures
    • Gaussian Mixtures and the E-M Algorithm
    • Implementation Overview
    • Implementation of the E-M algorithm : gmix_step.m
    • Training
    • Conditional PDFs and Conditional Mean using Gaussian Mixtures
    • An Example Script for Gaussian Mixtures
  • PDF Estimation using HMMs
    • Introduction to HMM's
    • The standard HMM
    • MATLAB toolbox for HMM
  
  
• Multi-resolution HMM
  • Introduction
  • Multi-Resolution HMM
    • MR-HMM Definition
    • MR-HMM Likelihood function
    • Proxy HMM
    • Computation of likelihood function using Proxy HMM
    • Computation of segment Likelihood functions using PPT
    • Algorithm Summary
    • MR-HMM illustrative example
    • MR-HMM initialization
  • MR-HMM data generation
    • MR-HMM Terminology
  • Software
    • CMEX function
    • MR-HMM initialization
    • MRHMM Training and Evaluation
    • MRHMM Synthesis
  • Illustrative Example 1 : Two pulses
    • Data Generation
    • Segment Set Definition
    • Data Truncation
    • Feature Definition and Computation
    • Running the Script
  • Illustrative Example 2: Using Manual Segmentation: Analysis of speech
  • Illustrative Example 3 : Auto Segmentation
  
  
• Derivative-augmented Features
  • Introduction
    • Background and Motivation
  • Mathematical Preliminaries
    • DAF
    • DAF-HMM
  • The DAF integral.
  • Experiments
    • Data sets
    • Features
    • PDF estimation
    • Experimental Procedure
    • Results
  • Conclusions
  
  
• The Projected Belief Network (PBN)
  • Projected Belief Network (PBN) Fundamentals
    • Motivation: Advantages of PBN
    • Discriminative Alignment of PBN
  • The Deterministic Projected Belief Network (PBN)
    • The D-PBN as optimal reconstructor
  
  
• Appendix
  • Univariate PDFs
    • Gaussian
    • Chi-Square
    • Chi
  • Multivariate PDFs
    • Gaussian
    • Multivariate Gaussian in Frequency Domain
    • Non-Gaussian
  • Least Squares
  • Proof of the PDF Projection Theorem
  • Maximum Likelihood Parameter Estimation and the CR bound
  • Saddlepoint Approximation for Linear Function of Real Positive Data
    • The Saddlepoint.
    • Arbitrary Scaling
    • MATLAB implementation
  • Computing the PDF of Linear Function of Uniformly-Distributed Data
    • Saddlepoint Approximation for Linear Function of Uniformly-Distributed Data
    • ML approach
    • Numerical validation
  
  
• References
• Bibliography

PDF Projection and Related Methods in Statistical Signal Processing, and Hypothesis Testing, and Machine Learning

Dr. Paul M. Baggenstoss
Fraunhofer FKIE
Fraunhoferstrasse 20
Wachtberg, Germany
http://class-specific.com/csf/index.html
p.m.baggenstoss@ieee.org

M-files The HTML version of this document is automatically translated from LaTeX using LaTeX2HTML. For more information about LaTeX, the reader is referred to the LaTeX project homepage .