Reinforcement learning 2022 2023

Lecturers and Seminarists

Lecturer	Alexey Naumov	[anaumov@hse.ru]	T924
Seminarist	Sergey Samsonov	[svsamsonov@hse.ru]	T926

About the course

This page contains materials for Mathematical Foundations of Reinforcement learning course in 2021/2022 year, optional one for 2nd year Master students of the Math of Machine Learning program (HSE and Skoltech).

Grading

The final grade consists of 2 components (each is non-negative real number from 0 to 10, without any intermediate rounding) :

• O_HW for the hometasks
• O_Project for the course project

The formula for the final grade is

• O_Final = 0.5*O_HW + 0.5*O_Project

with the usual (arithmetical) rounding rule.

Table with grades

Lectures

• Lecture 09.11
• Lecture 16.11

Seminars

• Seminar 09.11, Seminar 09.11, Video, Seminar 09.11, Notebook
• Seminar 16.11, Video,
• Seminar 23.11, Video,
• Seminar 07.12, Video,

Recommended literature

Lecture and seminar 09.11

• Sebastien Bubek, Nicolo Cesa-Bianchi. Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. Chapter 2. http://sbubeck.com/SurveyBCB12.pdf
• Richard S. Sutton, Andrew G. Barto. Reinforcement Learning: An Introduction. Chapter 2. http://incompleteideas.net/book/the-book-2nd.html;
• Botao Hao et al. Bootstrapping Upper Confidence Bound. https://arxiv.org/abs/1906.05247

Lecture and seminar 16.11

• Seminar 09.11, Seminar 09.11, Video,

Homeworks

• Homework №1, deadline 19.12.2021, 23:59, Environment for task №2,
• Homework №2, deadline 19.12.2021, 23:59.