Natural Language Processing

Courses

These courses will help you understand the basics of Natural Language Processing along with enabling you to read and implement papers.

1. Sequence Models This course, like all the other courses by Andrew Ng (on Coursera) is a simple overview or montage of NLP. It will prolly give you some confidence but since the assignments have a lot of readymade code, which once you see, you cannot unsee. Also, if you try reading a paper on NLP, you will not be able to do so (if all your knowledge comes from this course). So all in all, it is a simple, basic and gentle scratch on the surface.

2. CS224n: Natural Language Processing with Deep Learning This is one of the most rigorous courses on NLP and it starts from the very basics and scales up almost all the way to SOTA (since it is a relatively new course). It has a good amount of depth of concept and reasonable math. The best part is that they provide you with concepts and let you figure out how to implement them. This is very very helpful in the long run and once you are done with like 12-13 lectures of this course, you can actually read, understand and implement papers right from scratch in PyTorch. **Note:** This course may seem to be boring in the beginning but the concepts (which may seem too rudimentary) taught in the lectures really go a long way.

3. Natural Language Processing by National Research University, Russia This course aims to teach you Natural Language Processing from the ground up, starting from dated statistical methods, and covering everything upto the latest Deep Learning based techniques. The quizzes test mathematical and theoritical knowledge, and the programming assignments make you build stuff. These result in a good blend of theory and practice, which can help if you’re the kind that gets bored of just studying and not doing. The final project is enticing and will require you to deploy a telegram chatbot on an AWS machine, which gives you real world experience of how these systems run in production.

More Courses

1. Natural Language Processing Specialization by Coursera
2. Fast.ai - NLP
3. Hugging Face Course

Tutorials/Implementations

1. IvLabs’ Natural Language Processing Repository
2. NLP From Scratch: Generating names with a character-level RNN
3. Sequence-to-Sequence Modeling with nn.Transformer and TorchText
4. NLP From Scratch: Translation with a Sequence to Sequence Network and Attention
5. PyTorch Seq-2-Seq

Blog Posts

1. Introduction to Word Embedding and Word2Vec
2. Illustrated Word2Vec
3. Illustrated: Self-Attention
4. The Illustrated Transformer

Research Papers

Notes for some of the below-mentioned papers can be found here.

1. Sequence to Sequence Learning with Neural Networks
2. Neural Machine Translation by Jointly Learning to Align and Translate
3. Convolutional Sequence to Sequence Learning
4. Attention Is All You Need
5. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
6. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks