Deep Learning for Language Modeling

Language modeling

• Language model

  • Estimated the probability of word sequence: $P(w_1,w_2,w_3,...,w_n)$

• Application: speech recognition, sentence generation

N-gram

NN-based LM

RNN-based LM

Matrix Factorization

Matrix Factorization (NN-based)

RNN-based LM