Gradient-based techniques

 

The gradient of L(Theta) is obtained by computing, for each theta_ij:

pdL(Theta)/pdtheta_ij = dlogp(x_q = a, e)/dtheta_ij - dlogp(e)/dtheta_ij.

This expression (derivation can be found in [10]) is:

  dL(Theta)/dtheta_ij = p(z'_i = j|x_q = a, e)/theta_ij - p(z'_i = j|e)/theta_ij,

which can be obtained through standard Bayesian network algorithms using local computations. A conjugate gradient descent can be constructed by selecting an initial value for Theta and, at each step, normalizing the values of Theta to ensure they represent proper distributions [31].