#machine-learning, computer-science/machine-learning/nlp, eth/cil/theory

Probabilistic Latent Semantic Analysis (pLSA)

derive a low-dimensional representation of the observed variables in terms of their affinity to certain hidden variables
assign each word occurence a topic
document $d$ , word $w$ , topic $z$

Definition

Define a distribution over the latent (i.e. topic) variables
- document is characterized by a distribution over topics: $p (z | d)$
Define a conditional distribution of a word given a topic
- topics are represented as distributions over words: $p (w | z)$
  Use two stage sampling process:
Sample a topic from $p (z | d)$
Sample a token given the topic from $p (w | z)$

Log-Likelihood

l (θ; N) = \log p (N; θ) = \sum_{i, j} N_{i j} \log p (w_{j} | d_{i}), p (w_{j} | d_{i}) = \sum_{t = 1}^{k} p (w_{j} | z_{t}) p (z_{t} | d_{i})