Expectation Maximization Algorithm

• use Jensen's Inequality to lower bound a concave function

Definition

E-Step

• goal is to maximize the lower bound with respect to the posterior probabilities of the latent variables
  Use lagrange multipliers decoupled for each point:

$\lambda$ is introduced to keep $q$ normalized.
Deriving the first order optimality condition:

M-Step

Maximize lower bound w.r.t. the parameters of $p$ with the current guess of $q$