Next:
Contents
Contents
Contents
Introduction
Overview
Is This a Book?
Purpose
What is the Class-Specific Method (CSM)?
What are the advantages of CSM?
What is a class-specific module?
What are the different topologies of a CSM implementation?
What kinds of features can be used by CSM?
What software tools are available for CSM?
Book organization
PDF Projection
Illustration of PDF Projection
Mathematical Definition of PDF Projection
Problem Setup
Feature Transformation
Mathematical Notation
Reference Hypothesis
Feature PDF
The Projected PDF
Statement of the PDF Projection Theorem
Generation of Samples from
Optimality Conditions of the Theorem
Class-Specific Feature Classifier
Chain Rule
Implementation
The Fixed Reference Hypothesis and the PDF tail problem
The Saddle Point Approximation
Floating Reference Hypothesis
Data Normalization
Maximum likelihood and PDF Projection
J-function Types
One-to-one (invertible) transformations
Fixed reference hypothesis
Floating reference hypothesis modules
Maximum Likelihood Modules
Interpretation of the J-function
PDF Projection validation: the Acid test
PDF Projection Software Module Design
The module function
Computing the projected PDF
Multiple dimensions and segments or samples
Maximum Entropy PDF Projection
Introduction
Why Maximum entropy?
Mathematical description of MaxEnt PDF projection
Statement of MaxEnt PDF Projection theorem
Energy Statistic (ES)
Discussion on the choice of
MaxEnt and the Chain Rule
Data Synthesis
Information Maximization
Simple Feature Transformations
General 1:1 transformations
Log
Square root of positive data
DCT
The Exp Function (
module_exp.m
)
Log Bilinear (
module_bilinear.m
)
Squaring of Gaussian data
Magnitude Squared DFT of Gaussian data
Linear transformation of Gaussian data (ES approach)
Feature Transformation and Reference hypothesis
J-Function
UMS
Asymptotic (large
) behavior
Example
Linear transformation of Gaussian data (fixed variance approach)
Gaussian Model with Non-Linear Parameter Dependence
Data model
ML estimation
First derivatives
Fisher's information
Iteration
PDF Projection
UMS
Linear transformation of positive-valued exponentially-distributed data
Feature transformation and Energy Statistic (ES approach)
J-function Implementations
General solution using SPA (
module_A_chisq.m
)
Calculating
Reference Hypothesis
Module
Application of SPA to Circular Auto-Correlation Function Analysis
Application of SPA to MEL Bank analysis
General solution using CLT (
module_A_chisq_clt.m
)
Application of CLT to Auto-Correlation Function Analysis
Application of CLT to MEL Band Analysis
General solution using ML
ML Solution for ACF/AR
Parameters
PDF form
CR Bound (FIM)
ML Solution for MEL band analysis
Parameters and PDF form
CR Bound (FIM)
Software Module
Comparison with SPA and CLT
Data re-synthesis of positive data from linear features: UMS
MCMC-UMS
Algorithm description
Starting point
Illustration
Predicting the Mean (centroid) of MCMC-UMS.
Method 1 (requiring starting point)
Method 2 (not requiring starting point)
Example: Circular AR/ACF Spectrum
Experimental approach
Results
Accuracy of Predicted Mean and Mixing Rate
Whitened MCMC-UMS
Dimension effects
Example: MFCC
Software for UMS
Fixed mean assumption
Linear transformation of positive-valued truncated Gaussian data
Linear transformation of doubly-bounded data
Feature transformation and J-function
MCMC-UMS for doubly-bounded
The Truncated Exponential Distribution (TED)
Solving for asymptotic mean of MCMC-UMS
Method 1 (requiring starting point)
Method 2 (not requiring starting point)
Simple Example
Image Example
Cameraman image
Order Statistics
Arbitrary order statistics of a set of
iid RV, and sum of remainder energy.
Features
Computing
: Integral Solution
Probability Density Function (
pdf_ord_rem_chisq.m
)
Exponential Distribution.
Chi Distribution
MATLAB functions
Largest
RV
Class-Specific Module
module_bestkfft.m
.
Largest
values of a set of
independent RVs with different distributions, and sum of remainder, and indexes (
exp_ord_spect.m
).
SPA Solution for Order Statistics of Exponential RVs (
module_exp_ord.m
)
Joint MGF of
.
Joint MGF of
.
Partial Derivatives of the CGF.
MATLAB Functions.
Comparison
Special Models
M Quadratic and Linear Forms of Correlated Random Variables
Form of the statistics
Saddlepoint Approximation
Autocorrelation Function.
General Autocorrelation Function.
Cyclic Autocorrelation Function.
Cross-correlations
Numerical Validation
Implementation: Circular and Non-Circular Autocorrelation
Linear Filtering
Sinewave plus AR noise (SINAR)
Statistical Model
Initialization
Estimation of Sinusoidal Parameters
Estimation of AR Parameters
Amplitude Normalization
MATLAB module
Data Re-synthesis
Test Results
Amplitude Compression
Feature PDF (Spectral Histogram)
Timeseries Analysis using AR, MA, and ARMA models
Stationary and Circularly Stationary processes and the ACF
Stationary Processes and the Power Spectrum
Circularly-stationary process
Data PDF of stationary process
Data PDF of circularly stationary process
When to use Circulary-Stationary Processes
Rational Transfer Function Models
ARMA Modeling
Computing the ACF of an ARMA process
Exact PDF of an ARMA process - Efficient method using Levinson algorithm
Exact PDF of an ARMA process - Very efficient method using filtering
Circular ARMA process
Data PDF of Circular ARMA process
ARMA parameter estimation
CR Bound analysis of ARMA PDF - exact forms.
CR bound analysis of circular ARMA parameters
pdf_arma_circ.m
.
CR Bound analysis of the exact PDF of an ARMA processes - filtering approach.
ARMA Modules
MA Modeling
Exact PDF for MA process
MA parameter estimation
CR bound analysis of MA parameters
Alternative Parameterization
MA Modules
AR modeling
AR Data PDF - exact
AR data PDF, Large
(circular process)
Estimating the AR coefficients - deterministic approach (Levinson Algorithm)
CR bound analysis of AR parameters - frequency domain approach
CR Bound analysis of AR PDF - exact forms.
Reflection coefficients
Polynomial Roots (
module_poly2root.m
).
LAR coefficients (
module_bilinear.m
)
Block Diagram of AR Software Modules
Chains for ACF/AR features
Chain Comparisons
Prediction error vs. Power
Recommended AR Processing Chain
Cepstral Analysis
Comparison of MFCC approaches
Experimental Approach
PDF based on Circular Power Spectrum
AR circular power spectral model
MFCC circular power spectral model
Feature Chains
Experimental Setup
Acid Tests
Comparing various MFCC features
Comparing MFCC with AR features
Classification experiments
MFCC: Recommended chain
Rectangular Bands
Classifier Topologies
Segmentation and Window Functions
Unsegmented Data
Block Segmentation
Hanning-3 Segmentation
Hanning-3 Example Experiment
On-the-fly Segmentation
Multi-Resolution HMM
Likelihood Comparison and Combination methods
One Class/One Model: Class-Specific Features (CSF)
Class-Specific Feature Mixture (CSFM)
Class-Specific Hidden Markov Models (CSHMM)
PDF Sculpting
Feature PDF Estimation
PDF Modeling Introduction and Notation
PDF Estimation using Gaussian Mixtures
Gaussian Mixtures
Gaussian Mixtures and the E-M Algorithm
Derivation of the EM Algorithm for GM
Implementation Overview
Implementation of the E-M algorithm :
gmix_step.m
Working in the log domain.
Using the Cholesky Decomposition of
.
Choosing the covariance constraints
Conditioning the Covariances
Training
Determining the number of modes.
E-M algorithm (
gmix_step.m
)
Pruning (
gmix_deflate.m
)
Merging Modes (
gmix_merge.m
)
Splitting modes (
gmix_kurt.m
)
Convergence
Training script (
gmix_trainscript.m
)
Training on Huge data sets
Conditional PDFs and Conditional Mean using Gaussian Mixtures
Conditional Estimation in general
Estimation using Gaussian Mixtures
MATLAB implementation
Example of Estimation: Beam Interpolation
CR Bound analysis
An Example Script for Gaussian Mixtures
PDF Estimation using HMMs
Introduction to HMM's
How HMM's are used.
The role of HMM's in class-specific classifiers
The standard HMM
Using Gaussian Mixtures for
.
Forward/Backward Procedure
Reestimation of HMM Parameters
Reestimation of Observation PDF's
Reestimation of Gaussian Mixture Parameters
Structured State Transition Matrices
Multiple Records
MATLAB toolbox for HMM
An HMM example
Creating feature data for training.
Initializing HMM parameters
Training using the Baum-Welch algorithm
Viewing the state PDF's
Annealing
Creating Synthetic Observations
Estimating the states: the Viterbi algorithm.
Classifying using the trained HMM parameters
Multi-resolution HMM
Introduction
Multi-Resolution HMM
MR-HMM Definition
MR-HMM Likelihood function
Proxy HMM
Computation of likelihood function using Proxy HMM
Computation of segment Likelihood functions using PPT
Algorithm Summary
MR-HMM illustrative example
MR-HMM initialization
Initial segmentation
Feature PDF initialization
Discrete probabilities initialization
MR-HMM data generation
MR-HMM Terminology
Software
CMEX function
MR-HMM initialization
Data storage andSegment Set Definition
Segment Set Definition
Defining and Computing Features
Data Labeling
Signal class Feature Map
States vs Subclasses
PDF type
MRHMM Initialization function
MRHMM Training and Evaluation
MRHMM Synthesis
Illustrative Example 1 : Two pulses
Data Generation
Segment Set Definition
Data Truncation
Feature Definition and Computation
Running the Script
Manual Labeling
Auto Labeling
Viterbi
Data Synthesis
Illustrative Example 2: Using Manual Segmentation: Analysis of speech
Illustrative Example 3 : Auto Segmentation
Derivative-augmented Features
Introduction
Background and Motivation
Mathematical Preliminaries
DAF
DAF-HMM
The DAF integral.
Experiments
Data sets
Features
PDF estimation
Experimental Procedure
Results
Conclusions
The Projected Belief Network (PBN)
Projected Belief Network (PBN) Fundamentals
Motivation: Advantages of PBN
Discriminative Alignment of PBN
The Deterministic Projected Belief Network (PBN)
The D-PBN as optimal reconstructor
Appendix
Univariate PDFs
Gaussian
Chi-Square
Chi
Multivariate PDFs
Gaussian
Multivariate Gaussian in Frequency Domain
Parameter Estimation
Non-Gaussian
Least Squares
Proof of the PDF Projection Theorem
Maximum Likelihood Parameter Estimation and the CR bound
Saddlepoint Approximation for Linear Function of Real Positive Data
The Saddlepoint.
Arbitrary Scaling
MATLAB implementation
Computing the PDF of Linear Function of Uniformly-Distributed Data
Saddlepoint Approximation for Linear Function of Uniformly-Distributed Data
ML approach
Numerical validation
References
Bibliography
About this document ...