Deep Learning for NLP

PM 1: Deep Learning for Natural Language Processing
Summer Semester 2016
Prof. Dr. Alexander Koller and Dr. Sebastian Stober
Fri 12-14; Golm, Haus 14, Raum 0.09

Begins: Friday, April 15

Deep Learning is one of the most exciting developments in Artificial Intelligence right now. “Deep” neural networks, which have one or more hidden layers, and recurrent neural networks, which can read a sequence of inputs and produce a sequence of outputs, have long been recognized as powerful models of machine learning. But it is only through recent advances in computing power (especially on graphics cards) and training methods that they have become useable in practice. Now they have been breaking records in many areas of AI, including image recognition, speech recognition, and, most recently, game playing.

Within the past five years, deep neural networks have also led to breakthroughs in computational linguistics. Word embeddings based on neural networks have all but replaced traditional methods in distributional semantics, and remarkably simple networks have achieved or improved the state of the art in problems as diverse as part-of-speech tagging, parsing, and machine translation.

In this PM, we will look at neural methods for natural language processing in detail. We start by an introduction to neural networks and some of their coolest uses in NLP. These will take the form of short lectures, based on recent research papers you should read in advance so we can discuss them in class. At the same time, you will do weekly assignments in order to try out some of these techniques yourselves, and to collect hands-on experience.

After about one third of the semester, you will then team up with other students to carry out your own project. Each team will define its own project and work on it for the rest of the semester. We will meet weekly to discuss your progress and help where we can. At the end of the semester, you will present your final project to the rest of the course. After the semester, you will write a term paper that documents your project. The grade is a weighted mean of a grade for the project (assigned to the entire team) and grades for presentations and documentation (assigned to individual students).

This will be a rather intensive course, both during the preparatory phase (weekly assignments) and during the project phase (completing an ambitious research and programming project). It is worth 12 credit points, which corresponds to 40% of your work time each week, so plan your semester accordingly. However, we believe that you’ll be able to build very cool systems within this PM, and we look forward to seeing what you’ll come up with!

Please use the Piazza Forum for the course to discuss questions with us and with each other!