# Maximum Likelihood Parameter Estimation and the CR bound

Consider a general log PDF that depends on parameters: :

The Fisher's information between any two parameters and is defined by

 (16.13)

Collecting all these values into the matrix , we have the Fisher's information matrix. The Cramer-Rao lower bound states that the covariance matrix of any joint unbiased estimator for the parameters is such that

. This effectively means that is the lower bound for the covariance of any unbiased estimator.

The inverse of the Fisher's information matrix is a good estimate of the parameter estimation error covariance and is useful for iterative optimization. Given a parameter estimate , the new estimate is obtained as

 (16.14)

where

is the gradient vector formed from the first partial derivatives

It is possible to optimize only subsets of the features as well. A feature pair is updated according to

 (16.15)

Baggenstoss 2017-05-19