Least Squares

Suppose we would like to approximate the vector ${\bf x}$ as a linear combination of the columns of some matrix ${\bf H}$, or

$${\bf x}\simeq {\bf H} \; {\bf a},$$

where ${\bf a}$ is a vector of coefficients. To find the least-squares approximation, we let

$${\bf a} = ({\bf H}^\prime {\bf H})^{-1} {\bf H}^\prime {\bf x}.$$ (17.12)