Literature

The entire Deep Learning Book by Goodfellow, Bengio, and Courville is excellent and worth reading. In addition to the theory of various neural network architectures, it also has practical tips and pointers to a lot of the applied literature (including to NLP).

Here are some papers about Deep Learning in NLP that we like, and that might serve you as inspiration for your own projects.

Applications in machine translation:

• Sutskever et al. (2014), Sequence to Sequence Learning with Neural Networks. Proc. NIPS.
• Bahdanau et al. (2015), Neural machine translation by jointly learning to align and translate. Proc. ICLR.

Applications in semantics and question answering:

• Bowman et al. (2015), Recursive neural networks can learn logical semantics. Proc. ACL.
• Sukhbaatar et al. (2015), End-to-end Memory Networks. Proc. NIPS.
• Hermann et al. (2015), Teaching machines to read and comprehend. Proc. NIPS.
• Rocktäschel et al. (2016), Reasoning about entailment with neural attention. Proc. ICLR.
• Lots and lots of papers about word embeddings.

Applications in parsing:

• Durrett & Klein (2015), Neural CRF parsing. Proc. ACL.
• Xu et al. (2015), CCG supertagging with a recurrent neural networks. Proc. ACL.
• Vinyals et al. (2015), Grammar as a foreign language. Proc. NIPS.
• Lewis et al. (2016), LSTM CCG Parsing. Proc. NAACL.

Others in NLP:

• Collobert et al. (2011), Natural Language Processing (almost) from scratch. JMLR.
• Socher et al. (2013), Recursive deep models for semantic compositionality over a sentiment treebank. Proc. EMNLP.
• Rush et al. (2015), A neural attention model for abstractive summarization. Proc. EMNLP.
• Vinyals & Le (2015), A neural conversational model. Proc. ICML.

Some cool non-NLP applications:

• Mnih et al. (2013), Playing Atari with deep reinforcement learning. Proc. NIPS Deep Learning Workshop.
• Zaremba & Sutskever (2015), Learning to execute. Proc. ICLR.
• Silver et al. (2016), Mastering the game of Go with deep neural networks and tree search. Nature.