[Review]Neural Language Models

Foreword

This a literature review-ish article to allow me to summarise my understanding on the domain of NLM.

Papers

1. Factored Neural Language Models

Profile:

Authors : Andrei Alexandrescu, Kartin Krinchhoff
Link: http://melodi.ee.washington.edu/people/katrin/Papers/alexand-kirchhoff-hlt06.pdf
Published Year: 2006

Issue: ?? they have just investigated the further improvement in NLM.

Approach:

They viewed a word as a vector of word features or “factors” : $w = $<$f_1, f_2, . . . , f_k$> and predict a word jointly from previous words and their factors.
Based on the original neural language model introduced by Bengio et al. in 2000, they have combined the factorisation and NLM, then this technique called Factored Neural Language Model.

*Bengio's NLM

*Factored NLM