Neurobayesian models 2020 - История изменений

Alexgri в 11:26, 6 марта 2020

2020-03-06T11:26:53Z

← Предыдущая версия		Версия от 11:26, 6 марта 2020
Строка 96:		Строка 96:

	[https://drive.google.com/file/d/1cp3Ek9XJAgxrXC6tfTOEcSx1FyxBT_iH/view?usp=sharing Discrete Latent Variables and Variance Reduction] by Artem Sobolev		[https://drive.google.com/file/d/1cp3Ek9XJAgxrXC6tfTOEcSx1FyxBT_iH/view?usp=sharing Discrete Latent Variables and Variance Reduction] by Artem Sobolev

			[https://docs.google.com/document/d/1yaXWGx4TeGfNPgotIDc1zbJ1Dv6NKV26aB1QWpxwvek/edit?usp=sharing SIVI lecture papers] by Dmitry Molchanov

	====Reading List ====		====Reading List ====

Alexgri в 10:15, 24 февраля 2020

2020-02-24T10:15:38Z

← Предыдущая версия		Версия от 10:15, 24 февраля 2020
Строка 93:		Строка 93:
	[https://docs.google.com/document/d/1Mqzo89OeX5gteQ9qYF5rhz6Ap00JQEyoiq2WKo54vA8/edit?usp=sharing List of materials from previous year (relevant papers, blogposts, etc.)]		[https://docs.google.com/document/d/1Mqzo89OeX5gteQ9qYF5rhz6Ap00JQEyoiq2WKo54vA8/edit?usp=sharing List of materials from previous year (relevant papers, blogposts, etc.)]

	[https://drive.google.com/~~open?id=~~1yu_77loRT3Th3-YkYc6sbnf9QDcYIO9U Natural gradient, Fisher Information matrix, VI] by Artem Sobolev		[https://drive.google.com/file/d/1yu_77loRT3Th3-YkYc6sbnf9QDcYIO9U/view?usp=sharing Natural gradient, Fisher Information matrix, VI] by Artem Sobolev

	[https://drive.google.com/~~open~~?id=~~1cp3Ek9XJAgxrXC6tfTOEcSx1FyxBT_iH~~ Discrete Latent Variables and Variance Reduction] by Artem Sobolev		[https://drive.google.com/file/d/1cp3Ek9XJAgxrXC6tfTOEcSx1FyxBT_iH/view?usp=sharing Discrete Latent Variables and Variance Reduction] by Artem Sobolev

	====Reading List ====		====Reading List ====

Alexgri в 10:48, 23 февраля 2020

2020-02-23T10:48:38Z

← Предыдущая версия		Версия от 10:48, 23 февраля 2020
Строка 92:		Строка 92:
	=== Course Materials ===		=== Course Materials ===
	[https://docs.google.com/document/d/1Mqzo89OeX5gteQ9qYF5rhz6Ap00JQEyoiq2WKo54vA8/edit?usp=sharing List of materials from previous year (relevant papers, blogposts, etc.)]		[https://docs.google.com/document/d/1Mqzo89OeX5gteQ9qYF5rhz6Ap00JQEyoiq2WKo54vA8/edit?usp=sharing List of materials from previous year (relevant papers, blogposts, etc.)]

	[https://drive.google.com/open?id=1yu_77loRT3Th3-YkYc6sbnf9QDcYIO9U Natural gradient, Fisher Information matrix, VI] by Artem Sobolev		[https://drive.google.com/open?id=1yu_77loRT3Th3-YkYc6sbnf9QDcYIO9U Natural gradient, Fisher Information matrix, VI] by Artem Sobolev

	[https://drive.google.com/open?id=1cp3Ek9XJAgxrXC6tfTOEcSx1FyxBT_iH Discrete Latent Variables and Variance Reduction] by Artem Sobolev		[https://drive.google.com/open?id=1cp3Ek9XJAgxrXC6tfTOEcSx1FyxBT_iH Discrete Latent Variables and Variance Reduction] by Artem Sobolev

Alexgri в 10:48, 23 февраля 2020

2020-02-23T10:48:25Z

← Предыдущая версия		Версия от 10:48, 23 февраля 2020
Строка 59:		Строка 59:
	\| rowspan="2" \| 1 \|\| 30 January \|\| Lecture: Stochastic Variational Inference		\| rowspan="2" \| 1 \|\| 30 January \|\| Lecture: Stochastic Variational Inference
	\|-		\|-
	\|30 January\|\|Seminar: Application of SVI to Latent Dirichlet Allocation model		\|30 January\|\|Seminar: Application of SVI to Latent Dirichlet Allocation model
	\|-		\|-
	\| rowspan="2" \| 2 \|\| 30 January \|\| Lecture: Doubly Stochastic Variational Inference		\| rowspan="2" \| 2 \|\| 30 January \|\| Lecture: Doubly Stochastic Variational Inference
Строка 92:		Строка 92:
	=== Course Materials ===		=== Course Materials ===
	[https://docs.google.com/document/d/1Mqzo89OeX5gteQ9qYF5rhz6Ap00JQEyoiq2WKo54vA8/edit?usp=sharing List of materials from previous year (relevant papers, blogposts, etc.)]		[https://docs.google.com/document/d/1Mqzo89OeX5gteQ9qYF5rhz6Ap00JQEyoiq2WKo54vA8/edit?usp=sharing List of materials from previous year (relevant papers, blogposts, etc.)]
			[https://drive.google.com/open?id=1yu_77loRT3Th3-YkYc6sbnf9QDcYIO9U Natural gradient, Fisher Information matrix, VI] by Artem Sobolev
			[https://drive.google.com/open?id=1cp3Ek9XJAgxrXC6tfTOEcSx1FyxBT_iH Discrete Latent Variables and Variance Reduction] by Artem Sobolev

	====Reading List ====		====Reading List ====

Alexgri в 13:37, 13 февраля 2020

2020-02-13T13:37:32Z

← Предыдущая версия		Версия от 13:37, 13 февраля 2020
Строка 12:		Строка 12:
	This course is devoted to Bayesian reasoning in application to deep learning models. Attendees would learn how to use probabilistic modeling to construct neural generative and discriminative models, how to use the paradigm of generative adversarial networks to perform approximate Bayesian inference and how to model the uncertainty about the weights of neural networks. Selected open problems in the field of deep learning would also be discussed. The practical assignments will cover implementation of several modern Bayesian deep learning models.		This course is devoted to Bayesian reasoning in application to deep learning models. Attendees would learn how to use probabilistic modeling to construct neural generative and discriminative models, how to use the paradigm of generative adversarial networks to perform approximate Bayesian inference and how to model the uncertainty about the weights of neural networks. Selected open problems in the field of deep learning would also be discussed. The practical assignments will cover implementation of several modern Bayesian deep learning models.

	Course syllabus ~~TBA~~		[https://docs.google.com/document/d/1k6xMnyWzpKHO6SmA7GEFjZVCw85paSCJAgMXTwu_zsE/edit?usp=sharing Course syllabus]

	=== News ===		=== News ===

			The first assignment has been uploaded to anytask. Deadline: 15 February, 21:00.

	===Grading System===		===Grading System===
Строка 30:		Строка 32:
	All grades are in ten-point grading scale. If О_cumulative or О_final has a fractional part greater or equal than 0.5 then it is rounded up.		All grades are in ten-point grading scale. If О_cumulative or О_final has a fractional part greater or equal than 0.5 then it is rounded up.

	~~(*) There are several extra conditions which will~~ be ~~described soon~~		Assignment is considered completed if the grade is equal or greater than 5 out of 10.
			Necessary condition for O_final to be greater or equal to 4/6/8 is to complete one/two/three assignment and pass the exam.

	===Assignments ===		===Assignments ===
Строка 38:		Строка 41:
	* Assignments are scored up to 10 points. Each assignment has a deadline, a penalty is charged in the amount of 0.3 points for each day of delay, but in total not more than 6 points. You will have 1 to 2 weeks to solve an assignment. Some assignments may contain bonus parts.		* Assignments are scored up to 10 points. Each assignment has a deadline, a penalty is charged in the amount of 0.3 points for each day of delay, but in total not more than 6 points. You will have 1 to 2 weeks to solve an assignment. Some assignments may contain bonus parts.

	Approximate dates of assignments' upload: 6 February, 13 February, ~~27 February~~, 12 March		Approximate dates of assignments' upload: 8 February, 15 February, 1 March, 15 March

	===Exam ===		===Exam ===
Строка 64:		Строка 67:
	\| rowspan="2" \| 3 \|\| 6 February \|\| Lecture: Bayesian Neural Networks		\| rowspan="2" \| 3 \|\| 6 February \|\| Lecture: Bayesian Neural Networks
	\|-		\|-
	\| 6 February \|\| Seminar: ~~TBA~~		\| 6 February \|\| Seminar: Local Reparametrization Trick
	\|-		\|-
	\| rowspan="2" \| 4 \|\| 13 February \|\| Lecture: Variational autoencoders (VAE) and normalizing flows (NF)		\| rowspan="2" \| 4 \|\| 13 February \|\| Lecture: Variational autoencoders (VAE) and normalizing flows (NF)

Alexgri в 10:37, 25 января 2020

2020-01-25T10:37:52Z

← Предыдущая версия		Версия от 10:37, 25 января 2020
Строка 18:		Строка 18:
	===Grading System===		===Grading System===

	The assessment consists of 4 practical assignments and a final oral exam. Practical assignments consist of programming some models/methods from the course in Python and analysing their behavior: Sparse Variational Dropout, Normalizing flows, VAE, and ~~(4)~~ Discrete Latent Variables. At the final exam students have to demonstrate knowledge of the material covered during the entire course.		The assessment consists of 4 practical assignments and a final oral exam. Practical assignments consist of programming some models/methods from the course in Python and analysing their behavior: Sparse Variational Dropout, Normalizing flows, VAE, and Discrete Latent Variables. At the final exam students have to demonstrate knowledge of the material covered during the entire course.

	Final course grade is obtained (*) from the following formula:		Final course grade is obtained (*) from the following formula:

Alexgri в 10:33, 25 января 2020

2020-01-25T10:33:55Z

← Предыдущая версия		Версия от 10:33, 25 января 2020
Строка 1:		Строка 1:
	'''Lector:''' [https://www.hse.ru/en/staff/dvetrov Dmitry Vetrov]		'''Lector:''' [https://www.hse.ru/en/staff/dvetrov Dmitry Vetrov]

	'''Tutors:''' ~~[https://www.hse.ru/en/org/persons/190884100 Alexander Grishin],~~ [https://www.hse.ru/en/org/persons/165140955 Kirill Struminsky], [https://www.hse.ru/en/org/persons/205487138 Dmitry Molchanov], [https://www.hse.ru/en/org/persons/191263008 Kirill Neklyudov], [http://artem.sobolev.name/ Artem Sobolev], [https://ars-ashuha.ru/ Arsenii Ashukha], [https://bayesgroup.ru/people/oleg-ivanov/ Oleg Ivanov], [https://www.hse.ru/en/org/persons/~~131072080 Ekaterina Lobacheva~~], [https://www.hse.ru~~/en~~/org/persons/~~137291612 Artyom Gadetsky~~], [https://www.hse.ru/org/persons/~~318659918 Aibek Alanov~~].		'''Tutors:''' [https://www.hse.ru/en/org/persons/165140955 Kirill Struminsky], [https://www.hse.ru/en/org/persons/205487138 Dmitry Molchanov], [https://www.hse.ru/en/org/persons/191263008 Kirill Neklyudov], [http://artem.sobolev.name/ Artem Sobolev], [https://ars-ashuha.ru/ Arsenii Ashukha], [https://bayesgroup.ru/people/oleg-ivanov/ Oleg Ivanov], [https://www.hse.ru/en/org/persons/137291612 Artyom Gadetsky], [https://www.hse.ru/org/persons/318659918 Aibek Alanov].

			'''Manager:''' [https://www.hse.ru/en/org/persons/190884100 Alexander Grishin]

	'''Contacts:''' All the questions should be addressed to '''bayesml@gmail.com'''. Subject line of any letter must contain the following tag: '''[HSE NBM20]'''. Letters without the tag will be most probably lost in the inbox.		'''Contacts:''' All the questions should be addressed to '''bayesml@gmail.com'''. Subject line of any letter must contain the following tag: '''[HSE NBM20]'''. Letters without the tag will be most probably lost in the inbox.

Alexgri в 10:26, 25 января 2020

2020-01-25T10:26:25Z

← Предыдущая версия		Версия от 10:26, 25 января 2020
Строка 36:		Строка 36:
	* Assignments are scored up to 10 points. Each assignment has a deadline, a penalty is charged in the amount of 0.3 points for each day of delay, but in total not more than 6 points. You will have 1 to 2 weeks to solve an assignment. Some assignments may contain bonus parts.		* Assignments are scored up to 10 points. Each assignment has a deadline, a penalty is charged in the amount of 0.3 points for each day of delay, but in total not more than 6 points. You will have 1 to 2 weeks to solve an assignment. Some assignments may contain bonus parts.

	Approximate dates of assignments' upload: 28 February, ~~6 March~~, 20 March~~, 3 April~~		Approximate dates of assignments' upload: 6 February, 13 February, 27 February, 12 March

	===Exam ===		===Exam ===

Alexgri в 10:22, 25 января 2020

2020-01-25T10:22:17Z

← Предыдущая версия		Версия от 10:22, 25 января 2020
Строка 5:		Строка 5:
	'''Contacts:''' All the questions should be addressed to '''bayesml@gmail.com'''. Subject line of any letter must contain the following tag: '''[HSE NBM20]'''. Letters without the tag will be most probably lost in the inbox.		'''Contacts:''' All the questions should be addressed to '''bayesml@gmail.com'''. Subject line of any letter must contain the following tag: '''[HSE NBM20]'''. Letters without the tag will be most probably lost in the inbox.

	We also have a chat in Telegram ~~(invite TBA)~~. Its main language is English. All important news will be announced in the chat.		We also have a chat in Telegram. Its main language is English. All important news will be announced in the chat.

	===Course description===		===Course description===
Строка 16:		Строка 16:
	===Grading System===		===Grading System===

	~~TBA~~		The assessment consists of 4 practical assignments and a final oral exam. Practical assignments consist of programming some models/methods from the course in Python and analysing their behavior: Sparse Variational Dropout, Normalizing flows, VAE, and (4) Discrete Latent Variables. At the final exam students have to demonstrate knowledge of the material covered during the entire course.

			Final course grade is obtained (*) from the following formula:

			О_final = 0,7 * О_cumulative + 0,3 * О_exam,

			where О_cumulative is a weighted average grade for the practical assignments:

			О_cumulative = 1/6 * O(SVDO) + 1/3 * O(NF) + 1/3 * O(VAE) + 1/6 * O(DLV)

			All grades are in ten-point grading scale. If О_cumulative or О_final has a fractional part greater or equal than 0.5 then it is rounded up.

			(*) There are several extra conditions which will be described soon

	===Assignments ===		===Assignments ===
	* The course contains four practical assignments. Solutions should be submitted to anytask ~~(url TBA)~~. To get the invite, please write to the course e-mail. The site has an interface only in Russian, so non-Russian speaking students may submit their solutions to the course e-mail. In this case, the subject line of the letter in addition to the tag should contain your name, surname and assignment number.		* The course contains four practical assignments. Solutions should be submitted to [https://anytask.org/course/602 anytask]. To get the invite, please write to the course e-mail. The site has an interface only in Russian, so non-Russian speaking students may submit their solutions to the course e-mail. In this case, the subject line of the letter in addition to the tag should contain your name, surname and assignment number.
	* All assignments should be coded in Python 3 using PyTorch.		* All assignments should be coded in Python 3 using PyTorch.
	* Students have to complete all assignments by themselves. Using code of your colleagues or code from open implementations is prohibited and will be considered as plagiarism. All involved students (including those who shared their solutions) will be severely punished.		* Students have to complete all assignments by themselves. Using code of your colleagues or code from open implementations is prohibited and will be considered as plagiarism. All involved students (including those who shared their solutions) will be severely punished.
	* Assignments are scored up to ~~TBA~~. Each assignment has a deadline, a penalty is charged in the amount of 0.3 points for each day of delay, but in total not more than 6 points. ~~Usually you~~ will have 2 weeks to solve an assignment. Some assignments may contain bonus parts.		* Assignments are scored up to 10 points. Each assignment has a deadline, a penalty is charged in the amount of 0.3 points for each day of delay, but in total not more than 6 points. You will have 1 to 2 weeks to solve an assignment. Some assignments may contain bonus parts.

	Approximate dates of assignments' upload: ~~TBA~~		Approximate dates of assignments' upload: 28 February, 6 March, 20 March, 3 April

	~~A hard deadline for all assignments: TBA~~		===Exam ===

	~~===~~Exam ~~===~~		Exam questions will be published here.
	~~TBA~~
			At the beginning of the exam, we will give you a random question from the main part of the exam questions list. You will have one hour to prepare your answer. During this hour you may use any materials (including the ones on your laptop).

			Then you discuss your answer with an examiner, answer questions from the theoretical minimum, answer additional questions on the course and solve problems. At this part of the exam, you CAN NOT use any materials. Please pay attention to the theoretical minimum - inability to answer any questions from it automatically entails an unsatisfactory mark for the exam.

	=== Course Plan ===		=== Course Plan ===

	TBA		{\| class="wikitable"
			\|-
			! '''№''' !! '''Date''' !! '''Theme'''
			\|-
			\| rowspan="2" \| 1 \|\| 30 January \|\| Lecture: Stochastic Variational Inference
			\|-
			\|30 January\|\|Seminar: Application of SVI to Latent Dirichlet Allocation model
			\|-
			\| rowspan="2" \| 2 \|\| 30 January \|\| Lecture: Doubly Stochastic Variational Inference
			\|-
			\| 31 January \|\| Seminar: Reparametrization Trick and Variance Reduction
			\|-
			\| rowspan="2" \| 3 \|\| 6 February \|\| Lecture: Bayesian Neural Networks
			\|-
			\| 6 February \|\| Seminar: TBA
			\|-
			\| rowspan="2" \| 4 \|\| 13 February \|\| Lecture: Variational autoencoders (VAE) and normalizing flows (NF)
			\|-
			\| 13 February \|\| Seminar: Importance Weighted Autoencoders + more complex NF
			\|-
			\| rowspan="2" \| 5 \|\| 20 February \|\| Lecture: Discrete Latent Variables and Variance Reduction
			\|-
			\| 20 February \|\| Seminar: Discrete Latent Variables and Variance Reduction
			\|-
			\| rowspan="2" \| 6 \|\| 27 February \|\| Lecture: Implicit Variational Inference using Adversarial Training
			\|-
			\| 27 February \|\| Seminar: f-GAN
			\|-
			\| rowspan="2" \| 7 \|\| 5 March \|\| Lecture: TBA
			\|-
			\| 5 March \|\| Seminar: TBA
			\|-
			\| rowspan="2" \| 8 \|\| 12 March \|\| Lecture: Deep Markov chain Monte Carlo (MCMC)
			\|-
			\| 12 March \|\| Seminar: Deep Markov chain Monte Carlo (MCMC)
			\|}

	=== Course Materials ===		=== Course Materials ===

Alexgri в 11:29, 20 января 2020

2020-01-20T11:29:49Z

← Предыдущая версия		Версия от 11:29, 20 января 2020
Строка 16:		Строка 16:
	===Grading System===		===Grading System===

	The assessment consists of 4 practical assignments and a final oral exam. Practical assignments consist of programming some models/methods from the course in Python and analysing their behavior: VAE, Normalizing flows, Sparse Variational Dropout and Discrete Latent Variables. At the final exam students have to demonstrate knowledge of the material covered during the entire course.		TBA

	~~Final course grade is obtained from the following formula:~~

	~~О_final = 0,7 * О_cumulative + 0,3 * О_exam,~~

	~~where О_cumulative is an average grade for the practical assignments.~~

	~~Grades for the practical assignments are~~ TBA~~. If О_cumulative or О_final has a fractional part greater or equal than 0.5 then it is rounded up.~~

	===Assignments ===		===Assignments ===