Deep Learning Natural Language Processing (DNLP)

(The course design comes from Stanford NLP with deep learning course with some modification)

Gary Geunbae Lee, Eng 2-211, gblee@postech.ac.kr, 279-2254

1.     Course objectives

This course will cover a cutting-edge research knowledge in deep learning natural language processing. Through lectures, students will learn the necessary skills to design, implement, and understand their own neural network models for various NLP problems such as word embedding/contextual word embedding, text classification, syntactic parsing, recurrent language modeling, machine translation, question answering, natural language generation, dialog systems, multi-task deep learning models, etc

2.     Course prerequisites

no required pre-requisite

3.     Grading

midterm 35%

final 35%

homework 30%

4  texts or references

Dan Jurafsky and James H. Martin. Speech and Language Processing (3rd ed. draft)

Jacob Eisenstein. Natural Language Processing

Yoav Goldberg. A Primer on Neural Network Models for Natural Language Processing

Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning

Michael A. Nielsen. Neural Networks and Deep Learning

Eugene Charniak. Introduction to Deep Learning

4.     Others

instruction language: English

2 homeworks: solve deep learning NLP application problems including python programming/coding

5.     Course schedule

1. DNLP overview

2. Word embedding

3. Text classification/BP

4. Neural dependency parsing

5. Neural language modeling (RNN)

6. Neural machine translation (seq-to-seq)

7. Some NLP Project Data

8. Question answering

9. ConvNet NLP

10. Sub-word modeling

11. Contextual word embedding (BERT/Transformer)

    11.1 LLM Prompting RLHF

12. NLG (dialog, summarization)

13. Co-reference resolution

14. Multi-task DNLP

    14.1 Multimodal NLP

15. Tree RNN

16. Future deep learning NLP