To solve (5.15), define
as the element-wise inverse of
Then, (5.15) can be written
which means that
must be fully in the column space of , or that
for some vector .
Therefore, to find the mean of the surrogate density,
we solve for the free variable such that
We can solve this by driving the square error to zero:
We can easily find derivative of
is the diagonal matrix with diagonal elements
We have found that if we use the negative-definite
the resulting Newton-Raphson algorithm has excellent convergence properties
when starting with
be the value of
at the solution to (5.18).
The algorithm to find
can be summarized as follows.
The algorithm is implemnted by software/lam_solve.m with dbound=0.
Although this method corresponds to classical
methods, it is based on a novel sampling argument
and can be extend to other manifolds as we will see.
- Set iteration counter .
- To initialize, let
where is the vector of ones.
will be the
vector of ones.
- Compute derivative and Hessian according to (5.21),(5.22) and (5.23), then
- Increment and go to step 3.