Maximum Likelihood Estimation

• uses likelihood function as an objective
• probability distribution thought of as a function of parameters while keeping observed outcomes fixed
• more convenient to work with log likelihood
• suggests to chose the model parameters which maximize the probability of the observed data.

Definition

Log likelihood function:

Where $p(x_t;{\theta }_z)$ is the probability distribution which we chose for our clusters.
Example for Gaussian:

Optimization problem find the maximum likelihood estimate (MLE):

Computation

Use expectation-maximization-algorightm