Statistical learning theory 2026

General Information

First lecture on Monday Sept 7 as described below. Afterwards, lectures will probably on Mondays 9h30, see this sheet.

Lectures: on Monday 13h -- 14h20 in Pokrovkaya Room TBA and in zoom by Bruno Bauwens

Seminars: on Monday 14h40 -- 16h online in Zoom by Nikita Lukianenko.

For questions about materials and last minute practical issues, join the telegram group. The course is similar to last year.

Course materials

Video	Old Video	Summary	Slides	Lecture notes	Problem list	Solutions
		Part 1. Online learning
07 Sept	oldrec 01	Philosophy. The online mistake bound model. The halving and weighted majority algorithms.	sl01	ch00 ch01	prob01	sol01
14 Sept	oldrec 02	The standard optimal algorithm. The perceptron algorithm.		ch02 ch03	prob02	sol02
21 Sept	oldrec 03	Prediction with expert advice. Recap probability theory (seminar).		ch04 ch05	prob03 Upd 7 Oct	sol03
		Part 2. Distribution independent risk bounds
28 Sept	oldrec 04	Necessity of a hypothesis class. Sample complexity in the realizable setting, examples: threshold functions and finite classes.		ch06	prob04	sol04
05 Oct	oldrec 05	Growth functions, VC-dimension and the characterization of sample comlexity with VC-dimensions		ch07 ch08	prob05	sol05
12 Oct	oldrec 06	Risk decomposition and the fundamental theorem of statistical learning theory (previous recording covers more)		ch09	prob06	sol06
19 Oct	oldrec 07	Bounded differences inequality, Rademacher complexity, symmetrization, contraction lemma.	sl07	ch10 ch11	prob07	sol07
		Part 3. Margin risk bounds with applications
02 Nov	oldrec 08	Simple regression, support vector machines, margin risk bounds, and dropout in neural nets (switch to old recording for SVM stuff).	sl08	ch12 ch13	prob08	sol08
09 Nov	oldrec 09	Kernels: RKHS, representer theorem, risk bounds	sl09	ch14	prob09	sol09
16 Nov	oldrec 10	AdaBoost and the margin hypothesis	sl10	ch15	prob10	sol10
		Part 4. Neural nets
23 Nov	oldrec 11	Exponential (and cross entropy loss) find maximal margin solutions. Losses of neural nets are not locally convex.		ch16	See next
30 Nov	oldrec 12	Lazy training and the neural tangent kernel in overparameterized nets.		ch17	prob11	sol11
07 Dec	oldrec 13	Finnish previous lecture. Optional: a label dependent risk bound for overparameterized nets.		ch18	Consult 15.12
		Colloquium Rules and questions.

The lectures in October and November are based on the book: Foundations of machine learning 2nd ed, Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalker, 2018.

A gentle introduction to the materials of the first 3 lectures and an overview of probability theory, can be found in chapters 1-6 and 11-12 of the following book: Sanjeev Kulkarni and Gilbert Harman: An Elementary Introduction to Statistical Learning Theory, 2012.

Grading formula

Final grade = 0.25 * ([score colloquium 1] + [score of exam 1] + [score on colloquium 2] + [score on exam 2]) + bonus from quizzes.

There is a colloquium + exam during the session at the end of October. There is also a colloquium+exam during the session in December.

At the end of the lectures there is a short quiz in which you may earn 0.1 bonus points on the final non-rounded grade.

There is no rounding except for transforming the final grade to the official grade. Arithmetic rounding is used. There are no autogrades.

Problems exam

Date: Saturday 20.12, 13h-17h, room D203
-- You may use handwritten notes, lecture materials from this wiki (either printed or through your PC, it is a computer room), Mohri's book
-- You may not search on the internet or interact with other humans (e.g. by phone, forums, etc)

About questions
-- 4 or 5 questions of the difficulty of the homework. (Many homework questions were from former exams.)
-- I always ask to calculate VC dimension and to give/prove some risk bound with Rademacher complexity. -- Example of an exam (a bit easier, during COVID).

If you have a passing grade without attending the exam, you may skip the exam and I will mark you as present by default.

Office hours

Bruno Bauwens: Tuesday 15h-21h Better send me an email in advance.

Nikita Lukianenko: Write in Telegram, the time is flexible