ML estimation

Note that in these problems, $\hat{\mbox{\boldmath $\theta$}}$ can be found independently of $\hat{{\bf a}}, \; \hat{\sigma^2}$ by maximizing

$$Q={\bf x}^\prime {\bf H}_\theta \left({\bf H}_{\hat\theta}^\prime {\bf H}_{\hat\theta}\right)^{-1} {\bf H}_{\hat\theta}^\prime{\bf x}.$$

The other parameters $\hat{{\bf a}}$ and $\hat{\sigma}^2$ can be obtained in closed form for a fixed $\hat{\mbox{\boldmath $\theta$}}$,

$$\hat{{\bf a}}=\left({\bf H}_{\hat\theta}^\prime {\bf H}_{\hat\theta}\right)^{-1}{\bf H}_{\hat\theta}^\prime{\bf x},$$ (4.2)

$$\hat{\sigma^2}=\frac{1}{N}({\bf x}-{\bf H}_{\hat\theta}\hat{{\bf a}})^\prime ({\bf x}-{\bf H}_{\hat\theta}\hat{{\bf a}}).$$ (4.3)

Alternatively, the Newton-Raphson algorithm may be used to find $\hat{\mbox{\boldmath $\theta$}}$ as described in Section 17.5. For this we need the first derivatives and Fisher's information (See Section 17.5).