Deep Learning Course, 2025

This course provides a comprehensive exploration of modern deep learning techniques, from foundational concepts to advanced topics.

• Introduction to Neural Networks: MLP, Backpropagation, Initialization, Optimization, Regularization, CNN
• Natural Language Processing: Word Embeddings, RNN, LSTM, Attention, Transformer
• Computer Vision: Classification, Object detection, Segmentation
• Reinforcement Learning: Multi-armed Bandits, Monte Carlo Methods, Policy Improvement
• Generative Models: Autoregression, VAE, GAN, Diffusion, Flow Matching
• Advanced NLP: LLMs, Fine-tuning, RAG, Agents, Multi-modal Models
• Acceleration: Quantization, Pruning, Distillation, KV-Cache, Flash Attention

Course Staff

• Eduard Vladimirov @Edyarich
• Daniil Dorin @DorinDaniil
• Nikita Kiselev @kisnikser
• Sergey Firsov @Schaft-s
• Vadim Kasiuk @KasiukVadim

Materials

Week #	Date	Topic	Lecture	Seminar	Recording
1	September, 9	MLP, Backpropagation	slides, slides with notes	ipynb	record
2	September, 16	Optimization, Regularization	slides	ipynb	record
3	September, 23	Initialization, Normalization, CNN	slides	ipynb, notes	lecture record, seminar record
4	September, 30	Intro to NLP, Word Embeddings	slides	ipynb	record
5	October, 7	RNN, LSTM, Attention, Transformer	slides	ipynb	record
6	October, 14	Classification, Object Detection	slides	ipynb	lecture, seminar
7	October, 21	Segmentation	slides	ipynb_1, ipynb_2	lecture, seminar
8	October, 28	Multi-armed Bandits, Bellman Equations, Monte Carlo Methods, TD Learning, Q-Learning	-	-	record
9	November, 11	-	-	-	-
10	November, 18	Autoregression, VAE, GAN	slides	ipynb	record
11	November, 25	Diffusion Models, Flow Matching	slides	ipynb	record
12	December, 2	-	-	-	-
13	December, 9	Multimodality, CLIP, BLIP, LLaVA	slides	ipynb	lecture, seminar
14	December, 16	Quantization, Pruning, Distillation, KV-Cache, Flash Attention	slides	-	record

Homeworks

Homework #	Date	Deadline	Description	Link
1	September, 8	September, 29	Autograd implementation	google form
2	September, 8	October, 13	Alexnet implementation on PyTorch	google form
3	September, 8	October, 28	Image captioning with attention	google form
4	November, 5	November, 21	Satellite images segmentation	google form
5	November, 5	December, 12	Multi-armed bandits & CartPole	google form

Game Rules

• 5 Homeworks = 70 points
• Oral Exam = 30 points
• Maximum Points: 70 + 30 = 100 points

Final Grade: min(round(#points/10), 10)

Prerequisities

• Probability Theory + Statistics
• Machine Learning
• Python

Previous Episodes

• 2024
• 2023
• 2022