To solve (5.15), define
as the elementwise inverse of
.
Then, (5.15) can be written
,
which means that
must be fully in the column space of , or that

(5.17) 
for some vector .
Therefore, to find the mean of the surrogate density,
we solve for the free variable such that
where
We can solve this by driving the square error to zero:
We can easily find derivative of
with respect
to :
where
is the diagonal matrix with diagonal elements

(5.22) 
We have found that if we use the negativedefinite
Hessian approximation

(5.23) 
the resulting NewtonRaphson algorithm has excellent convergence properties
when starting with
.
Let
be the value of
at the solution to (5.18).
The algorithm to find
can be summarized as follows.
 Set iteration counter .
 To initialize, let
where is the vector of ones.

. Initially,
will be the
vector of ones.

 Compute derivative and Hessian according to (5.21),(5.22) and (5.23), then
update :
 Increment and go to step 3.
The algorithm is implemnted by software/lam_solve.m with dbound=0.
Although this method corresponds to classical
methods, it is based on a novel sampling argument
and can be extend to other manifolds as we will see.
Baggenstoss
20170519