Wiki - Факультет компьютерных наук - Вклад [ru]

Data analysis (Software Engineering) 2017

2017-12-09T16:44:24Z

Apogentus:

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Exam questions ==

[https://yadi.sk/i/2w5_Ts2p3K7jQb Exam questions]

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

Baseline loss: 0.89

[https://yadi.sk/d/vfVkVE__3JSbMT Baseline solution.]

[https://docs.google.com/spreadsheets/d/1gfmWjrigiVbIjBv9G40q3n_OHHaeRg6gf0o2vqFxGzk/edit?usp=sharing Evaluation results]

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/IT2V7Dfe3J4Qxh Download]

'''Lecture 15. Neural networks.'''
[https://yadi.sk/i/ZOPEDMW03JJAaj Download]

'''Lecture 16. Clustering.'''
[https://yadi.sk/i/Cb3IlxK83JXGZq Download]

'''Lecture 17. Mixture density models.'''
[https://yadi.sk/i/1h3fvfmn3JXH5R Download]

'''Lecture 18. Clustering evaluation.'''
[https://yadi.sk/i/HPt4xzPP3JXGo6 Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

'''Seminar 11. Ensembles, bagging'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZVjk1c2dIWmVSUkk/view?usp=sharing Practical task 11], Deadline: May 18

'''Seminar 12. Ensembles, boosting'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZTFJGZ2VmcURVZDg/view?usp=sharing Theoretical task 12], Deadline: May 18

'''Seminar 13. Neural networks'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTnJTd2d0QldneDA/view?usp=sharing Practical task 13], [https://drive.google.com/open?id=0B7TWwiIrcJstMEU3bXkxY1hwZ00 Data], [https://drive.google.com/open?id=0B7TWwiIrcJstcjB4eFJaUkxDdEE Short training set], Deadline: June 6

Additional materials: [https://drive.google.com/open?id=0B7TWwiIrcJstbHFYbFcwTkNpNWc Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/open?id=0B7TWwiIrcJstOTZidU1BREtSRjA PyBrain example from the seminar]

''By default, you should use the whole training set from Data. But if you have MemoryError then use Short training set. ''

'''Seminar 14. Clustering'''

[https://drive.google.com/file/d/0B7TWwiIrcJstSjRBMlNLSW00YlU/view?usp=sharing Theoretical task 14], Deadline: June 1

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-09-10T12:25:50Z

Apogentus: /* Kaggle competition */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Exam questions ==

[https://yadi.sk/i/2w5_Ts2p3K7jQb Exam questions]

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

Baseline loss: 0.89

[https://yadi.sk/d/vfVkVE__3JSbMT Baseline solution.]

[https://docs.google.com/spreadsheets/d/1gfmWjrigiVbIjBv9G40q3n_OHHaeRg6gf0o2vqFxGzk/edit?usp=sharing Evaluation results]

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/IT2V7Dfe3J4Qxh Download]

'''Lecture 15. Neural networks.'''
[https://yadi.sk/i/ZOPEDMW03JJAaj Download]

'''Lecture 16. Clustering.'''
[https://yadi.sk/i/Cb3IlxK83JXGZq Download]

'''Lecture 17. Mixture density models.'''
[https://yadi.sk/i/1h3fvfmn3JXH5R Download]

'''Lecture 18. Clustering evaluation.'''
[https://yadi.sk/i/HPt4xzPP3JXGo6 Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

'''Seminar 11. Ensembles, bagging'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZVjk1c2dIWmVSUkk/view?usp=sharing Practical task 11], Deadline: May 18

'''Seminar 12. Ensembles, boosting'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZTFJGZ2VmcURVZDg/view?usp=sharing Theoretical task 12], Deadline: May 18

'''Seminar 13. Neural networks'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTnJTd2d0QldneDA/view?usp=sharing Practical task 13], [https://drive.google.com/open?id=0B7TWwiIrcJstMEU3bXkxY1hwZ00 Data], [https://drive.google.com/open?id=0B7TWwiIrcJstcjB4eFJaUkxDdEE Short training set], Deadline: June 6

Additional materials: [https://drive.google.com/open?id=0B7TWwiIrcJstbHFYbFcwTkNpNWc Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/open?id=0B7TWwiIrcJstOTZidU1BREtSRjA PyBrain example from the seminar]

''By default, you should use the whole training set from Data. But if you have MemoryError then use Short training set. ''

'''Seminar 14. Clustering'''

[https://drive.google.com/file/d/0B7TWwiIrcJstSjRBMlNLSW00YlU/view?usp=sharing Theoretical task 14], Deadline: June 1

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-08-23T12:29:12Z

Apogentus: /* Preliminary exam questions */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Exam questions ==

[https://yadi.sk/i/2w5_Ts2p3K7jQb Exam questions]

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

Baseline loss: 0.89

[https://yadi.sk/d/vfVkVE__3JSbMT Baseline solution.]

[https://docs.google.com/spreadsheets/d/1gfmWjrigiVbIjBv9G40q3n_OHHaeRg6gf0o2vqFxGzk/edit?usp=sharing Evaluation results]

===Winner's solutions===
[https://yadi.sk/i/1RNo2e_B3JpxCf Лиза Поваляева - N1]

[https://yadi.sk/i/F7JQrvTw3Jpx5A Исмаил Хамитов - N2]

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/IT2V7Dfe3J4Qxh Download]

'''Lecture 15. Neural networks.'''
[https://yadi.sk/i/ZOPEDMW03JJAaj Download]

'''Lecture 16. Clustering.'''
[https://yadi.sk/i/Cb3IlxK83JXGZq Download]

'''Lecture 17. Mixture density models.'''
[https://yadi.sk/i/1h3fvfmn3JXH5R Download]

'''Lecture 18. Clustering evaluation.'''
[https://yadi.sk/i/HPt4xzPP3JXGo6 Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

'''Seminar 11. Ensembles, bagging'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZVjk1c2dIWmVSUkk/view?usp=sharing Practical task 11], Deadline: May 18

'''Seminar 12. Ensembles, boosting'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZTFJGZ2VmcURVZDg/view?usp=sharing Theoretical task 12], Deadline: May 18

'''Seminar 13. Neural networks'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTnJTd2d0QldneDA/view?usp=sharing Practical task 13], [https://drive.google.com/open?id=0B7TWwiIrcJstMEU3bXkxY1hwZ00 Data], [https://drive.google.com/open?id=0B7TWwiIrcJstcjB4eFJaUkxDdEE Short training set], Deadline: June 6

Additional materials: [https://drive.google.com/open?id=0B7TWwiIrcJstbHFYbFcwTkNpNWc Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/open?id=0B7TWwiIrcJstOTZidU1BREtSRjA PyBrain example from the seminar]

''By default, you should use the whole training set from Data. But if you have MemoryError then use Short training set. ''

'''Seminar 14. Clustering'''

[https://drive.google.com/file/d/0B7TWwiIrcJstSjRBMlNLSW00YlU/view?usp=sharing Theoretical task 14], Deadline: June 1

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-06-14T12:27:35Z

Apogentus:

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Preliminary exam questions ==

[https://yadi.sk/i/2w5_Ts2p3K7jQb Preliminary exam questions]

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

Baseline loss: 0.89

[https://yadi.sk/d/vfVkVE__3JSbMT Baseline solution.]

[https://docs.google.com/spreadsheets/d/1gfmWjrigiVbIjBv9G40q3n_OHHaeRg6gf0o2vqFxGzk/edit?usp=sharing Evaluation results]

===Winner's solutions===
[https://yadi.sk/i/1RNo2e_B3JpxCf Лиза Поваляева - N1]

[https://yadi.sk/i/F7JQrvTw3Jpx5A Исмаил Хамитов - N2]

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/IT2V7Dfe3J4Qxh Download]

'''Lecture 15. Neural networks.'''
[https://yadi.sk/i/ZOPEDMW03JJAaj Download]

'''Lecture 16. Clustering.'''
[https://yadi.sk/i/Cb3IlxK83JXGZq Download]

'''Lecture 17. Mixture density models.'''
[https://yadi.sk/i/1h3fvfmn3JXH5R Download]

'''Lecture 18. Clustering evaluation.'''
[https://yadi.sk/i/HPt4xzPP3JXGo6 Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

'''Seminar 11. Ensembles, bagging'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZVjk1c2dIWmVSUkk/view?usp=sharing Practical task 11], Deadline: May 18

'''Seminar 12. Ensembles, boosting'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZTFJGZ2VmcURVZDg/view?usp=sharing Theoretical task 12], Deadline: May 18

'''Seminar 13. Neural networks'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTnJTd2d0QldneDA/view?usp=sharing Practical task 13], [https://drive.google.com/open?id=0B7TWwiIrcJstMEU3bXkxY1hwZ00 Data], [https://drive.google.com/open?id=0B7TWwiIrcJstcjB4eFJaUkxDdEE Short training set], Deadline: June 6

Additional materials: [https://drive.google.com/open?id=0B7TWwiIrcJstbHFYbFcwTkNpNWc Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/open?id=0B7TWwiIrcJstOTZidU1BREtSRjA PyBrain example from the seminar]

''By default, you should use the whole training set from Data. But if you have MemoryError then use Short training set. ''

'''Seminar 14. Clustering'''

[https://drive.google.com/file/d/0B7TWwiIrcJstSjRBMlNLSW00YlU/view?usp=sharing Theoretical task 14], Deadline: June 1

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-06-05T08:26:15Z

Apogentus: /* Решения победителей */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

Baseline loss: 0.89

[https://yadi.sk/d/vfVkVE__3JSbMT Baseline solution.]

[https://docs.google.com/spreadsheets/d/1gfmWjrigiVbIjBv9G40q3n_OHHaeRg6gf0o2vqFxGzk/edit?usp=sharing Evaluation results]

===Winner's solutions===
[https://yadi.sk/i/1RNo2e_B3JpxCf Лиза Поваляева - N1]

[https://yadi.sk/i/F7JQrvTw3Jpx5A Исмаил Хамитов - N2]

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/IT2V7Dfe3J4Qxh Download]

'''Lecture 15. Neural networks.'''
[https://yadi.sk/i/ZOPEDMW03JJAaj Download]

'''Lecture 16. Clustering.'''
[https://yadi.sk/i/Cb3IlxK83JXGZq Download]

'''Lecture 17. Mixture density models.'''
[https://yadi.sk/i/1h3fvfmn3JXH5R Download]

'''Lecture 18. Clustering evaluation.'''
[https://yadi.sk/i/HPt4xzPP3JXGo6 Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

'''Seminar 11. Ensembles, bagging'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZVjk1c2dIWmVSUkk/view?usp=sharing Practical task 11], Deadline: May 18

'''Seminar 12. Ensembles, boosting'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZTFJGZ2VmcURVZDg/view?usp=sharing Theoretical task 12], Deadline: May 18

'''Seminar 13. Neural networks'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTnJTd2d0QldneDA/view?usp=sharing Practical task 13], [https://drive.google.com/open?id=0B7TWwiIrcJstMEU3bXkxY1hwZ00 Data], [https://drive.google.com/open?id=0B7TWwiIrcJstcjB4eFJaUkxDdEE Short training set], Deadline: June 5

Additional materials: [https://drive.google.com/open?id=0B7TWwiIrcJstbHFYbFcwTkNpNWc Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/open?id=0B7TWwiIrcJstOTZidU1BREtSRjA PyBrain example from the seminar]

''By default, you should use the whole training set from Data. But if you have MemoryError then use Short training set. ''

'''Seminar 14. Clustering'''

[https://drive.google.com/file/d/0B7TWwiIrcJstSjRBMlNLSW00YlU/view?usp=sharing Theoretical task 14], Deadline: June 1

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-06-05T08:24:21Z

Apogentus: /* Решения победителей */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

Baseline loss: 0.89

[https://yadi.sk/d/vfVkVE__3JSbMT Baseline solution.]

[https://docs.google.com/spreadsheets/d/1gfmWjrigiVbIjBv9G40q3n_OHHaeRg6gf0o2vqFxGzk/edit?usp=sharing Evaluation results]

===Решения победителей===
[https://yadi.sk/i/1RNo2e_B3JpxCf Лиза Поваляева - N1]

[https://yadi.sk/i/F7JQrvTw3Jpx5A Исмаил Хамитов - N2]

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/IT2V7Dfe3J4Qxh Download]

'''Lecture 15. Neural networks.'''
[https://yadi.sk/i/ZOPEDMW03JJAaj Download]

'''Lecture 16. Clustering.'''
[https://yadi.sk/i/Cb3IlxK83JXGZq Download]

'''Lecture 17. Mixture density models.'''
[https://yadi.sk/i/1h3fvfmn3JXH5R Download]

'''Lecture 18. Clustering evaluation.'''
[https://yadi.sk/i/HPt4xzPP3JXGo6 Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

'''Seminar 11. Ensembles, bagging'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZVjk1c2dIWmVSUkk/view?usp=sharing Practical task 11], Deadline: May 18

'''Seminar 12. Ensembles, boosting'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZTFJGZ2VmcURVZDg/view?usp=sharing Theoretical task 12], Deadline: May 18

'''Seminar 13. Neural networks'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTnJTd2d0QldneDA/view?usp=sharing Practical task 13], [https://drive.google.com/open?id=0B7TWwiIrcJstMEU3bXkxY1hwZ00 Data], [https://drive.google.com/open?id=0B7TWwiIrcJstcjB4eFJaUkxDdEE Short training set], Deadline: June 5

Additional materials: [https://drive.google.com/open?id=0B7TWwiIrcJstbHFYbFcwTkNpNWc Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/open?id=0B7TWwiIrcJstOTZidU1BREtSRjA PyBrain example from the seminar]

''By default, you should use the whole training set from Data. But if you have MemoryError then use Short training set. ''

'''Seminar 14. Clustering'''

[https://drive.google.com/file/d/0B7TWwiIrcJstSjRBMlNLSW00YlU/view?usp=sharing Theoretical task 14], Deadline: June 1

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-06-05T08:22:33Z

Apogentus: /* Решения победителей */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

Baseline loss: 0.89

[https://yadi.sk/d/vfVkVE__3JSbMT Baseline solution.]

[https://docs.google.com/spreadsheets/d/1gfmWjrigiVbIjBv9G40q3n_OHHaeRg6gf0o2vqFxGzk/edit?usp=sharing Evaluation results]

===Решения победителей===
[https://yadi.sk/i/1RNo2e_B3JpxCf Лиза Поваляева - N1]

[https://yadi.sk/i/F7JQrvTw3Jpx5A Исмаил Хамитов - N2]

Вывод: в обоих решениях - в 1м явно, а во 2м неявно, через рекомендательную систему, использовались характеристики пользователя - как конкретный пользователь в среднем ставит оценки. Тем самым, разделялись пользователи, которые, в целом, склонны высоко либо низко оценивать большинство фильмов.

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/IT2V7Dfe3J4Qxh Download]

'''Lecture 15. Neural networks.'''
[https://yadi.sk/i/ZOPEDMW03JJAaj Download]

'''Lecture 16. Clustering.'''
[https://yadi.sk/i/Cb3IlxK83JXGZq Download]

'''Lecture 17. Mixture density models.'''
[https://yadi.sk/i/1h3fvfmn3JXH5R Download]

'''Lecture 18. Clustering evaluation.'''
[https://yadi.sk/i/HPt4xzPP3JXGo6 Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

'''Seminar 11. Ensembles, bagging'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZVjk1c2dIWmVSUkk/view?usp=sharing Practical task 11], Deadline: May 18

'''Seminar 12. Ensembles, boosting'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZTFJGZ2VmcURVZDg/view?usp=sharing Theoretical task 12], Deadline: May 18

'''Seminar 13. Neural networks'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTnJTd2d0QldneDA/view?usp=sharing Practical task 13], [https://drive.google.com/open?id=0B7TWwiIrcJstMEU3bXkxY1hwZ00 Data], [https://drive.google.com/open?id=0B7TWwiIrcJstcjB4eFJaUkxDdEE Short training set], Deadline: June 5

Additional materials: [https://drive.google.com/open?id=0B7TWwiIrcJstbHFYbFcwTkNpNWc Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/open?id=0B7TWwiIrcJstOTZidU1BREtSRjA PyBrain example from the seminar]

''By default, you should use the whole training set from Data. But if you have MemoryError then use Short training set. ''

'''Seminar 14. Clustering'''

[https://drive.google.com/file/d/0B7TWwiIrcJstSjRBMlNLSW00YlU/view?usp=sharing Theoretical task 14], Deadline: June 1

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-06-05T08:16:31Z

Apogentus: /* Решения победителей */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

Baseline loss: 0.89

[https://yadi.sk/d/vfVkVE__3JSbMT Baseline solution.]

[https://docs.google.com/spreadsheets/d/1gfmWjrigiVbIjBv9G40q3n_OHHaeRg6gf0o2vqFxGzk/edit?usp=sharing Evaluation results]

===Решения победителей===
[https://yadi.sk/i/1RNo2e_B3JpxCf Лиза Поваляева - N1]

[https://yadi.sk/i/F7JQrvTw3Jpx5A Исмаил Хамитов - N2]

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/IT2V7Dfe3J4Qxh Download]

'''Lecture 15. Neural networks.'''
[https://yadi.sk/i/ZOPEDMW03JJAaj Download]

'''Lecture 16. Clustering.'''
[https://yadi.sk/i/Cb3IlxK83JXGZq Download]

'''Lecture 17. Mixture density models.'''
[https://yadi.sk/i/1h3fvfmn3JXH5R Download]

'''Lecture 18. Clustering evaluation.'''
[https://yadi.sk/i/HPt4xzPP3JXGo6 Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

'''Seminar 11. Ensembles, bagging'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZVjk1c2dIWmVSUkk/view?usp=sharing Practical task 11], Deadline: May 18

'''Seminar 12. Ensembles, boosting'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZTFJGZ2VmcURVZDg/view?usp=sharing Theoretical task 12], Deadline: May 18

'''Seminar 13. Neural networks'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTnJTd2d0QldneDA/view?usp=sharing Practical task 13], [https://drive.google.com/open?id=0B7TWwiIrcJstMEU3bXkxY1hwZ00 Data], [https://drive.google.com/open?id=0B7TWwiIrcJstcjB4eFJaUkxDdEE Short training set], Deadline: June 5

Additional materials: [https://drive.google.com/open?id=0B7TWwiIrcJstbHFYbFcwTkNpNWc Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/open?id=0B7TWwiIrcJstOTZidU1BREtSRjA PyBrain example from the seminar]

''By default, you should use the whole training set from Data. But if you have MemoryError then use Short training set. ''

'''Seminar 14. Clustering'''

[https://drive.google.com/file/d/0B7TWwiIrcJstSjRBMlNLSW00YlU/view?usp=sharing Theoretical task 14], Deadline: June 1

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-06-05T08:08:59Z

Apogentus: /* Решения победителей */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

Baseline loss: 0.89

[https://yadi.sk/d/vfVkVE__3JSbMT Baseline solution.]

[https://docs.google.com/spreadsheets/d/1gfmWjrigiVbIjBv9G40q3n_OHHaeRg6gf0o2vqFxGzk/edit?usp=sharing Evaluation results]

===Решения победителей===
[https://yadi.sk/i/F7JQrvTw3Jpx5A Лиза Поваляева - N1]

[https://yadi.sk/i/1RNo2e_B3JpxCf Исмаил Хамитов - N2]

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/IT2V7Dfe3J4Qxh Download]

'''Lecture 15. Neural networks.'''
[https://yadi.sk/i/ZOPEDMW03JJAaj Download]

'''Lecture 16. Clustering.'''
[https://yadi.sk/i/Cb3IlxK83JXGZq Download]

'''Lecture 17. Mixture density models.'''
[https://yadi.sk/i/1h3fvfmn3JXH5R Download]

'''Lecture 18. Clustering evaluation.'''
[https://yadi.sk/i/HPt4xzPP3JXGo6 Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

'''Seminar 11. Ensembles, bagging'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZVjk1c2dIWmVSUkk/view?usp=sharing Practical task 11], Deadline: May 18

'''Seminar 12. Ensembles, boosting'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZTFJGZ2VmcURVZDg/view?usp=sharing Theoretical task 12], Deadline: May 18

'''Seminar 13. Neural networks'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTnJTd2d0QldneDA/view?usp=sharing Practical task 13], [https://drive.google.com/open?id=0B7TWwiIrcJstMEU3bXkxY1hwZ00 Data], [https://drive.google.com/open?id=0B7TWwiIrcJstcjB4eFJaUkxDdEE Short training set], Deadline: June 5

Additional materials: [https://drive.google.com/open?id=0B7TWwiIrcJstbHFYbFcwTkNpNWc Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/open?id=0B7TWwiIrcJstOTZidU1BREtSRjA PyBrain example from the seminar]

''By default, you should use the whole training set from Data. But if you have MemoryError then use Short training set. ''

'''Seminar 14. Clustering'''

[https://drive.google.com/file/d/0B7TWwiIrcJstSjRBMlNLSW00YlU/view?usp=sharing Theoretical task 14], Deadline: June 1

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-06-05T08:08:46Z

Apogentus: /* Kaggle competition */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

Baseline loss: 0.89

[https://yadi.sk/d/vfVkVE__3JSbMT Baseline solution.]

[https://docs.google.com/spreadsheets/d/1gfmWjrigiVbIjBv9G40q3n_OHHaeRg6gf0o2vqFxGzk/edit?usp=sharing Evaluation results]

===Решения победителей===
[https://yadi.sk/i/F7JQrvTw3Jpx5A Лиза Поваляева - N1]
[https://yadi.sk/i/1RNo2e_B3JpxCf Исмаил Хамитов - N2]

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/IT2V7Dfe3J4Qxh Download]

'''Lecture 15. Neural networks.'''
[https://yadi.sk/i/ZOPEDMW03JJAaj Download]

'''Lecture 16. Clustering.'''
[https://yadi.sk/i/Cb3IlxK83JXGZq Download]

'''Lecture 17. Mixture density models.'''
[https://yadi.sk/i/1h3fvfmn3JXH5R Download]

'''Lecture 18. Clustering evaluation.'''
[https://yadi.sk/i/HPt4xzPP3JXGo6 Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

'''Seminar 11. Ensembles, bagging'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZVjk1c2dIWmVSUkk/view?usp=sharing Practical task 11], Deadline: May 18

'''Seminar 12. Ensembles, boosting'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZTFJGZ2VmcURVZDg/view?usp=sharing Theoretical task 12], Deadline: May 18

'''Seminar 13. Neural networks'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTnJTd2d0QldneDA/view?usp=sharing Practical task 13], [https://drive.google.com/open?id=0B7TWwiIrcJstMEU3bXkxY1hwZ00 Data], [https://drive.google.com/open?id=0B7TWwiIrcJstcjB4eFJaUkxDdEE Short training set], Deadline: June 5

Additional materials: [https://drive.google.com/open?id=0B7TWwiIrcJstbHFYbFcwTkNpNWc Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/open?id=0B7TWwiIrcJstOTZidU1BREtSRjA PyBrain example from the seminar]

''By default, you should use the whole training set from Data. But if you have MemoryError then use Short training set. ''

'''Seminar 14. Clustering'''

[https://drive.google.com/file/d/0B7TWwiIrcJstSjRBMlNLSW00YlU/view?usp=sharing Theoretical task 14], Deadline: June 1

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-06-05T08:07:53Z

Apogentus: /* Kaggle competition */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

Baseline loss: 0.89

[https://yadi.sk/d/vfVkVE__3JSbMT Baseline solution.]

[https://docs.google.com/spreadsheets/d/1gfmWjrigiVbIjBv9G40q3n_OHHaeRg6gf0o2vqFxGzk/edit?usp=sharing Evaluation results]

[https://yadi.sk/i/F7JQrvTw3Jpx5A Top1 решение]
[https://yadi.sk/i/1RNo2e_B3JpxCf Top2 решение]

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/IT2V7Dfe3J4Qxh Download]

'''Lecture 15. Neural networks.'''
[https://yadi.sk/i/ZOPEDMW03JJAaj Download]

'''Lecture 16. Clustering.'''
[https://yadi.sk/i/Cb3IlxK83JXGZq Download]

'''Lecture 17. Mixture density models.'''
[https://yadi.sk/i/1h3fvfmn3JXH5R Download]

'''Lecture 18. Clustering evaluation.'''
[https://yadi.sk/i/HPt4xzPP3JXGo6 Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

'''Seminar 11. Ensembles, bagging'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZVjk1c2dIWmVSUkk/view?usp=sharing Practical task 11], Deadline: May 18

'''Seminar 12. Ensembles, boosting'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZTFJGZ2VmcURVZDg/view?usp=sharing Theoretical task 12], Deadline: May 18

'''Seminar 13. Neural networks'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTnJTd2d0QldneDA/view?usp=sharing Practical task 13], [https://drive.google.com/open?id=0B7TWwiIrcJstMEU3bXkxY1hwZ00 Data], [https://drive.google.com/open?id=0B7TWwiIrcJstcjB4eFJaUkxDdEE Short training set], Deadline: June 5

Additional materials: [https://drive.google.com/open?id=0B7TWwiIrcJstbHFYbFcwTkNpNWc Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/open?id=0B7TWwiIrcJstOTZidU1BREtSRjA PyBrain example from the seminar]

''By default, you should use the whole training set from Data. But if you have MemoryError then use Short training set. ''

'''Seminar 14. Clustering'''

[https://drive.google.com/file/d/0B7TWwiIrcJstSjRBMlNLSW00YlU/view?usp=sharing Theoretical task 14], Deadline: June 1

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-06-01T21:48:59Z

Apogentus: /* Kaggle competition */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

Baseline loss: 0.89

[https://yadi.sk/d/vfVkVE__3JSbMT Baseline solution.]

[https://docs.google.com/spreadsheets/d/1gfmWjrigiVbIjBv9G40q3n_OHHaeRg6gf0o2vqFxGzk/edit?usp=sharing Evaluation results]

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/IT2V7Dfe3J4Qxh Download]

'''Lecture 15. Neural networks.'''
[https://yadi.sk/i/ZOPEDMW03JJAaj Download]

'''Lecture 16. Clustering.'''
[https://yadi.sk/i/Cb3IlxK83JXGZq Download]

'''Lecture 17. Mixture density models.'''
[https://yadi.sk/i/1h3fvfmn3JXH5R Download]

'''Lecture 18. Clustering evaluation.'''
[https://yadi.sk/i/HPt4xzPP3JXGo6 Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

'''Seminar 11. Ensembles, bagging'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZVjk1c2dIWmVSUkk/view?usp=sharing Practical task 11], Deadline: May 18

'''Seminar 12. Ensembles, boosting'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZTFJGZ2VmcURVZDg/view?usp=sharing Theoretical task 12], Deadline: May 18

'''Seminar 13. Neural networks'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTnJTd2d0QldneDA/view?usp=sharing Practical task 13], [https://drive.google.com/open?id=0B7TWwiIrcJstMEU3bXkxY1hwZ00 Data], [https://drive.google.com/open?id=0B7TWwiIrcJstcjB4eFJaUkxDdEE Short training set], Deadline: June 5

Additional materials: [https://drive.google.com/open?id=0B7TWwiIrcJstbHFYbFcwTkNpNWc Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/open?id=0B7TWwiIrcJstOTZidU1BREtSRjA PyBrain example from the seminar]

''By default, you should use the whole training set from Data. But if you have MemoryError then use Short training set. ''

'''Seminar 14. Clustering'''

[https://drive.google.com/file/d/0B7TWwiIrcJstSjRBMlNLSW00YlU/view?usp=sharing Theoretical task 14], Deadline: June 1

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-05-25T16:51:21Z

Apogentus: /* Lecture materials */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

Baseline loss: 0.89

[https://yadi.sk/d/vfVkVE__3JSbMT Baseline solution.]

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/IT2V7Dfe3J4Qxh Download]

'''Lecture 15. Neural networks.'''
[https://yadi.sk/i/ZOPEDMW03JJAaj Download]

'''Lecture 16. Clustering.'''
[https://yadi.sk/i/Cb3IlxK83JXGZq Download]

'''Lecture 17. Mixture density models.'''
[https://yadi.sk/i/1h3fvfmn3JXH5R Download]

'''Lecture 18. Clustering evaluation.'''
[https://yadi.sk/i/HPt4xzPP3JXGo6 Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

'''Seminar 11. Ensembles, bagging'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZVjk1c2dIWmVSUkk/view?usp=sharing Practical task 11], Deadline: May 18

'''Seminar 12. Ensembles, boosting'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZTFJGZ2VmcURVZDg/view?usp=sharing Theoretical task 12], Deadline: May 18

'''Seminar 13. Neural networks'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTnJTd2d0QldneDA/view?usp=sharing Practical task 13], [https://drive.google.com/open?id=0B7TWwiIrcJstMEU3bXkxY1hwZ00 Data], [https://drive.google.com/open?id=0B7TWwiIrcJstcjB4eFJaUkxDdEE Short training set], Deadline: June 1

Additional materials: [https://drive.google.com/open?id=0B7TWwiIrcJstbHFYbFcwTkNpNWc Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/open?id=0B7TWwiIrcJstOTZidU1BREtSRjA PyBrain example from the seminar]

''By default, you should use the whole training set from Data. But if you have MemoryError then use Short training set. ''

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-05-25T16:47:39Z

Apogentus: /* Lecture materials */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

Baseline loss: 0.89

[https://yadi.sk/d/vfVkVE__3JSbMT Baseline solution.]

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/IT2V7Dfe3J4Qxh Download]

'''Lecture 15. Neural networks.'''
[https://yadi.sk/i/ZOPEDMW03JJAaj Download]

'''Lecture 16. Clustering.'''
[https://yadi.sk/i/Cb3IlxK83JXGZq Download]

'''Lecture 17. Mixtures, EM.'''
[https://yadi.sk/i/1h3fvfmn3JXH5R Download]

'''Lecture 18. Clustering evaluation.'''
[https://yadi.sk/i/HPt4xzPP3JXGo6 Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

'''Seminar 11. Ensembles, bagging'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZVjk1c2dIWmVSUkk/view?usp=sharing Practical task 11], Deadline: May 18

'''Seminar 12. Ensembles, boosting'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZTFJGZ2VmcURVZDg/view?usp=sharing Theoretical task 12], Deadline: May 18

'''Seminar 13. Neural networks'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTnJTd2d0QldneDA/view?usp=sharing Practical task 13], [https://drive.google.com/open?id=0B7TWwiIrcJstMEU3bXkxY1hwZ00 Data], [https://drive.google.com/open?id=0B7TWwiIrcJstcjB4eFJaUkxDdEE Short training set], Deadline: June 1

Additional materials: [https://drive.google.com/open?id=0B7TWwiIrcJstbHFYbFcwTkNpNWc Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/open?id=0B7TWwiIrcJstOTZidU1BREtSRjA PyBrain example from the seminar]

''By default, you should use the whole training set from Data. But if you have MemoryError then use Short training set. ''

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-05-25T16:44:59Z

Apogentus: /* Lecture materials */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

Baseline loss: 0.89

[https://yadi.sk/d/vfVkVE__3JSbMT Baseline solution.]

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/IT2V7Dfe3J4Qxh Download]

'''Lecture 15. Neural networks.'''
[https://yadi.sk/i/ZOPEDMW03JJAaj Download]

'''Lecture 16. Clustering.'''
[https://yadi.sk/i/Cb3IlxK83JXGZq Download]

'''Lecture 17. Mixtures, EM.'''
[https://yadi.sk/d/fubKLlKi3JXGmq Download]

'''Lecture 18. Clustering evaluation.'''
[https://yadi.sk/i/HPt4xzPP3JXGo6 Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

'''Seminar 11. Ensembles, bagging'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZVjk1c2dIWmVSUkk/view?usp=sharing Practical task 11], Deadline: May 18

'''Seminar 12. Ensembles, boosting'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZTFJGZ2VmcURVZDg/view?usp=sharing Theoretical task 12], Deadline: May 18

'''Seminar 13. Neural networks'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTnJTd2d0QldneDA/view?usp=sharing Practical task 13], [https://drive.google.com/open?id=0B7TWwiIrcJstMEU3bXkxY1hwZ00 Data], [https://drive.google.com/open?id=0B7TWwiIrcJstcjB4eFJaUkxDdEE Short training set], Deadline: June 1

Additional materials: [https://drive.google.com/open?id=0B7TWwiIrcJstbHFYbFcwTkNpNWc Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/open?id=0B7TWwiIrcJstOTZidU1BREtSRjA PyBrain example from the seminar]

''By default, you should use the whole training set from Data. But if you have MemoryError then use Short training set. ''

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-05-23T12:42:44Z

Apogentus: /* Kaggle competition */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

Baseline loss: 0.89

[https://yadi.sk/d/vfVkVE__3JSbMT Baseline solution.]

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/IT2V7Dfe3J4Qxh Download]

'''Lecture 15. Neural networks.'''
[https://yadi.sk/i/ZOPEDMW03JJAaj Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

'''Seminar 11. Ensembles, bagging'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZVjk1c2dIWmVSUkk/view?usp=sharing Practical task 11], Deadline: May 18

'''Seminar 12. Ensembles, boosting'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZTFJGZ2VmcURVZDg/view?usp=sharing Theoretical task 12], Deadline: May 18

'''Seminar 13. Neural networks'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTnJTd2d0QldneDA/view?usp=sharing Practical task 13], [https://drive.google.com/open?id=0B7TWwiIrcJstMEU3bXkxY1hwZ00 Data], [https://drive.google.com/open?id=0B7TWwiIrcJstcjB4eFJaUkxDdEE Short training set], Deadline: June 1

Additional materials: [https://drive.google.com/open?id=0B7TWwiIrcJstbHFYbFcwTkNpNWc Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/open?id=0B7TWwiIrcJstOTZidU1BREtSRjA PyBrain example from the seminar]

''By default, you should use the whole training set from Data. But if you have MemoryError then use Short training set. ''

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-05-18T14:30:51Z

Apogentus: /* Lecture materials */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/IT2V7Dfe3J4Qxh Download]

'''Lecture 15. Neural networks.'''
[https://yadi.sk/i/ZOPEDMW03JJAaj Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

'''Seminar 11. Ensembles, bagging'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZVjk1c2dIWmVSUkk/view?usp=sharing Practical task 11], Deadline: May 18

'''Seminar 12. Ensembles, boosting'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZTFJGZ2VmcURVZDg/view?usp=sharing Theoretical task 12], Deadline: May 18

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-05-14T08:45:45Z

Apogentus: /* Lecture materials */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/IT2V7Dfe3J4Qxh Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

'''Seminar 11. Ensembles, bagging'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZVjk1c2dIWmVSUkk/view?usp=sharing Practical task 11], Deadline: May 18

'''Seminar 12. Ensembles, boosting'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZTFJGZ2VmcURVZDg/view?usp=sharing Theoretical task 12], Deadline: May 18

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-05-11T19:57:59Z

Apogentus: /* Lecture materials */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/IT2V7Dfe3J4Qxh Download]

'''Lecture 15. xgBoost.'''
[https://yadi.sk/i/PfnWMyJU3J4R7K Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

'''Seminar 11. Ensembles'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZVjk1c2dIWmVSUkk/view?usp=sharing Practical task 11], Deadline: May 18

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-04-27T18:39:39Z

Apogentus: /* Lecture materials */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

'''Lecture 13. Ensemble methods.'''
[https://yadi.sk/i/5K8fg85Q3HRbEF Download]

'''Lecture 14. Boosting.'''
[https://yadi.sk/i/-RwlAWj53HRbFi Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-04-27T09:52:47Z

Apogentus: /* Kaggle competition */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Follow this link to [https://kaggle.com/join/HSE_Competition participate].

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

'''Seminar 10. Feature selection + text mining'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZdHk1MmFxUlVwNm8/view?usp=sharing Theoretical task 10], Deadline: April 27

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-04-20T21:13:17Z

Apogentus: /* Lecture materials */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Full data and rules will be availbale on kaggle platform in several days.
Link to [https://yadi.sk/d/kJass7GR3HBXct train set].

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

'''Lecture 11. Feature selection.'''
[https://yadi.sk/i/lyx3mLCc3HBwML Download]

'''Lecture 12. Working with text.'''
[https://yadi.sk/i/NmoN6-I43HBwN5 Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-04-20T17:02:19Z

Apogentus: /* Data prediction competition */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Kaggle competition ==
Full data and rules will be availbale on kaggle platform in several days.
Link to [https://yadi.sk/d/kJass7GR3HBXct train set].

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-04-20T17:01:38Z

Apogentus:

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Data prediction competition ==
Full data and rules will be availbale on kaggle platform in several days.
Link to [https://yadi.sk/d/kJass7GR3HBXct train set].

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

'''Seminar 9. PCA'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZT1U0aUhjQmphZDA/view?usp=sharing Theoretical task 9], Deadline: April 20

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-04-13T19:35:19Z

Apogentus: /* Lecture materials */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

'''Lecture 9. Principal component analysis.'''
[https://yadi.sk/i/dfImvp3O3Gwoaw Download]

'''Lecture 10. Singular value decomposition.'''
[https://yadi.sk/i/znlevSwu3Gwobb Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-03-30T12:42:45Z

Apogentus: /* Colloquium */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except single A4 prepared before the exam and handwritten personally by you (from two sides). You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-03-30T12:42:03Z

Apogentus: /* Colloquium */

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Colloquium ==
Colloquium will be held on April 7th during lecture & seminars time slot.

You may not use any materials during colloquium except A4 prepared before the exam and handwritten personally by you. You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-03-24T06:44:12Z

Apogentus:

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Colloquium ==
You may not use any materials during colloquium except A4 prepared before the exam and handwritten personally by you. You will have 2 questions from the [https://yadi.sk/i/myp4_88l3GJDwe questions list] with 25 minutes for preparation and may receive additional questions or tasks.

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

'''Lecture 3. Decision trees.'''
[https://yadi.sk/i/sUJamzb83Ao4xL Download]

'''Lecture 4. Regression methods.'''
[https://yadi.sk/i/mvozzv_s3CReQQ Download]

'''Lecture 5. Properties of convex functions.'''
[https://yadi.sk/i/zCcyHHCz3DhLTx Download]

'''Lecture 6. Linear methods of classification.'''
[https://yadi.sk/i/CsWN0nwW3DhDJp Download]

'''Lecture 7. Classifier evaluation.'''
[https://yadi.sk/i/r3tYJwZ73FNNWk Download]

'''Lecture 8. SVM and kernel trick.'''
[https://yadi.sk/i/f35uPeOe3FzvKg Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

'''Seminar 2. Metric Classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVmJ5NnNBY3YwV2c/view?usp=sharing Theoretical task 2], Deadline: January 26

[https://drive.google.com/file/d/0B7TWwiIrcJstUTg0czdlVkpMaWc/view?usp=sharing Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstSEZOZzBoQUo3bk0 data]. Deadline: February 2

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstelVRUVBXNktKenc/view?usp=sharing Theoretical task 3], Deadline: February 2

'''Seminar 4. Regression methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJsta29CeXdfcXFwWFU/view?usp=sharing Theoretical task 4], Deadline: February 9

'''Seminar 5. Linear classification: loss functions '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUzQxWndJaTB1cGM/view?usp=sharing Theoretical task 5], Deadline: February 16

'''Seminar 6. Linear classification: optimization '''

[https://drive.google.com/file/d/0B7TWwiIrcJstaHFVRFV2Z0F0ZWs/view?usp=sharing Theoretical task 6], Deadline: March 2

[https://drive.google.com/file/d/0B7TWwiIrcJstRjNSbXhxWDZfRzA/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstbGZZTDNjazg5Mmc/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstZ1M3NmhXU0EwSlE/view?usp=sharing diabetes dataset]. Deadline: March 16

'''Seminar 7. Classifier evaluation'''

[https://drive.google.com/file/d/0B4DmUfeAdxyZUF9FMlVWYUVmQUE/view?usp=sharing Theoretical task 7], Deadline: March 16

'''Seminar 8. SVM and kernel trick'''

[https://drive.google.com/file/d/0B7TWwiIrcJstazFWSXVSREJzSGc/view?usp=sharing Theoretical task 8], Deadline: March 23

[https://drive.google.com/file/d/0B7TWwiIrcJstOVotNEhnYUJYdGs/view?usp=sharing Practical task 8], [https://drive.google.com/file/d/0B7TWwiIrcJstU0gxeDhYX1hHN1E/view?usp=sharing data]. Deadline: March 30

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

If you solve the theoretical problem in class you obtain 1.5 points (if you solve it at home you obtain 1 point).
Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [https://github.com/esokolov/ml-course-hse Machine learning course from Evgeny Sokolov on Github]
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-03-16T17:55:59Z

Apogentus: /* Lecture materials */

Data analysis (Software Engineering) 2017

2017-03-09T16:11:36Z

Apogentus: /* Lecture materials */

Data analysis (Software Engineering) 2017

2017-02-09T17:01:14Z

Apogentus: /* Lecture materials */

Data analysis (Software Engineering) 2017

2017-02-09T17:01:00Z

Apogentus: /* Lecture materials */

Data analysis (Software Engineering) 2017

2017-02-09T16:43:35Z

Apogentus: /* Lecture materials */

Data analysis (Software Engineering) 2017

2017-02-02T09:19:25Z

Apogentus: /* Lecture materials */

Data analysis (Software Engineering) 2017

2017-02-02T09:14:37Z

Apogentus: /* Lecture materials */

Data analysis (Software Engineering) 2017

2017-01-25T09:18:13Z

Apogentus: /* Lecture materials */

Data analysis (Software Engineering) 2017

2017-01-24T18:42:16Z

Apogentus:

Data analysis (Software Engineering) 2017

2017-01-19T10:40:42Z

Apogentus: /* Lecture materials */

'''This page isn't finished yet!!!'''

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
[https://yadi.sk/i/TKVKjQMi38fUFm Download]

'''Lecture 2. Metric methods of classification & regression.'''
[https://yadi.sk/i/D4YvGVx739oAZ8 Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering) 2017

2017-01-19T10:39:44Z

Apogentus: /* Lecture materials */

'''This page isn't finished yet!!!'''

'''Class email:''' cshse.ml@gmail.com 
'''Anonymous feedback form:''' [https://docs.google.com/forms/d/e/1FAIpQLSdxP-U47SLedjTvF0CyxHSIYy8eTUnzcDOc9DIl4gFSD2-ixA/viewform here] 
'''Scores: [https://docs.google.com/spreadsheets/d/124qVkoSpftuRhYCEcY7KC-ALPiLn47_guyqAQIZJUYk/edit?usp=sharing here]''' 

 

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''
https://yadi.sk/i/TKVKjQMi38fUFm

'''Lecture 2. Metric methods of classification & regression.'''
https://yadi.sk/i/D4YvGVx739oAZ8

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

Additional materials: [https://github.com/esokolov/ml-course-hse/blob/master/2016-fall/seminars/sem01-tools.ipynb 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

[https://drive.google.com/file/d/0B7TWwiIrcJstaVo4b0FQYTZpejg/view?usp=sharing Practical task 1], [https://drive.google.com/file/d/0B7TWwiIrcJstazJRLVFRZ3dHM1k/view?usp=sharing data]. Deadline: January 19.

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

Standard period for working on a homework assignment is 2 weeks for practical assignments and 1 week for theoretical ones. The first practical assignment is an exception. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

'''Deadline time:''' 23:59 of the day before seminar (Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group"

''Example'': Practice 1 - Ivanov Ivan - 141

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in jupyter notebook format, theoretical ones in pdf. Practical assignments must use '''Python 3'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===

[https://www.continuum.io/downloads anaconda]

Data analysis (Software Engineering)

2017-01-12T19:32:20Z

Apogentus: /* Lecture materials */

'''Scores and deadlines: []'''

'''Class email:''' cshse.ml@gmail.com

'''Anonymous feedback form:''' [https://docs.google.com/forms/d/12E9Bsd8UpY9KpEdhOsKdPOawcTY1g-7mpefcSh7Qm7Y here]
 

== Announcements ==

== Course description ==
In this class we consider the main problems of machine learning and data analysis: classification, regression, dimensionality reduction, clustering, collaborative filtering. We will also study mathematical methods and concepts which machine learning is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

==Prerequisites==
Firm knowledge of linear algebra, mathematical analysis and probability theory is required.

== Syllabus ==

# Introduction. Core concepts of machine learning.
# K-nearest neighbours method.
# Decision tree methods.
# Regression methods. Regularization.
# Convex functions. Classification with linear methods. Loss functions.
# Classification with linear methods. Gradient descent and stochastic gradient descent. Regularization.
# Model evaluation. Logistic regression.
# Support vector machines.
# Generalization with kernels.
# Linear dimensionality reduction - PCA, SVD.
# Feature selection.
# Ensemble methods.
# Boosting. xgBoost.
# Neural networks - architecture.
# Neural networks - optimization.
# Clustering. EM algorithm for Gaussian mixtures.
# Collaborative filtering
# Bayes decision theory. Naive Bayes assumption. Kernel density estimation.
# Semi-supervised learning
# Active learnning

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''

[https://yadi.sk/i/TKVKjQMi38fUFm Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

[https://drive.google.com/open?id=0B7TWwiIrcJstdkJyam9rNHpEcDg Practical task 1], [https://drive.google.com/open?id=0B7TWwiIrcJstQldxcThZRnF3ZVk data]

Additional materials: [http://nbviewer.ipython.org/gist/anonymous/fba8bf7f1ad379df9d63 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

'''Seminar 2. kNN '''

[https://drive.google.com/open?id=0B7TWwiIrcJstbGRxREhGeDBNd3M Theoretical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstNXVXYTcydUMzUUk Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstTXUyYUstMmJNckk data]

Additional materials: [http://ogrisel.github.io/scikit-learn.org/sklearn-tutorial/auto_examples/tutorial/plot_knn_iris.html Visualization tutorial]

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstbmFmMmE5bWh2Y28/view?usp=sharing Theoretical task 3]

'''Seminar 4. Linear classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstc1J4SnRWTnlxZlE/view?usp=sharing Theoretical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstckZCN1pYcEo2NW8/view?usp=sharing Practical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstM3VSQTVrYWhqYk0/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstN0VOUV9PcDc1ZlE/view?usp=sharing diabetes dataset]

UPD: At all parts of practical task 4 you should use GD and SGD functions that you program at the fisrt part!

''Deadline for this practical task has been changed for some groups! Check it in the table!''

'''Seminar 5. Model evaluation '''

[https://drive.google.com/file/d/0B7TWwiIrcJstdTFST0Z4UkRoaEk/view?usp=sharing Theoretical task 5]

'''Seminar 6. Bayesian decision rule '''

[https://drive.google.com/file/d/0B7TWwiIrcJstWF9zcllDU01ZY2M/view?usp=sharing Theoretical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstSHRKcFlVcy1xd3M/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstVlhBdGZYMm94SHc/view?usp=sharing data]

Practical task 6 was completed: the last part was described in more details + there are two small corrections in the first part (they are in bold font). Read it carefully!

'''Deadline for this practical task has been changed for all groups! Check it in the table!'''

'''Seminar 7. SVM and kernel trick '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUGhEX2QxV1gycDA/view?usp=sharing Theoretical task 7]

Additional materials: [http://www.ccas.ru/voron/download/SVM.pdf Лекция К.В. Воронцова по SVM]

'''Seminar 8. Regression '''

[https://drive.google.com/file/d/0B7TWwiIrcJstNkE0dVJscGFSOEE/view?usp=sharing Theoretical task 8]

'''Seminar 9. Boosting '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVkx2Z0tGbi1yNFE/view?usp=sharing Practical task 9],
[https://drive.google.com/file/d/0B7TWwiIrcJstelRydjBUcUlleUk/view?usp=sharing data]

'''Seminar 10. Ensemble methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJstR28zVlN3OUZJTEU/view?usp=sharing Theoretical task 10]

''Problem 2: a small typo was corrected in the loss function formula.''

'''Seminar 11. Summary'''

'''Seminar 12. How to solve practical problems'''

[https://inclass.kaggle.com/c/cmc-msu-machine-learning-spring-2015-2016-dota-competition Dota Competition from the seminar], [https://drive.google.com/file/d/0B7TWwiIrcJstVnF0RVkzc1c1TVE/view?usp=sharing ipython notebook]

'''Seminar 13. Feature selection'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTUU4OGFnQzBPa3c/view?usp=sharing Theoretical task 13], [https://drive.google.com/file/d/0B7TWwiIrcJstM3JkWnhtQ2YzZms/view?usp=sharing Practical task 13]

''Practical task is completed.''

'''Seminar 14. Feature extraction'''

You can read about computing PCA through SVD at the end of [https://drive.google.com/file/d/0B7TWwiIrcJstWFFSOUI5aTRBM00/view?usp=sharing this paper].

'''Seminar 15. Neural networks'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxbk4yeXZyVi10dVE/view?usp=sharing Practical task 15],
[https://www.dropbox.com/s/r0u3xh5ybtstw9c/mnist.zip?dl=0 Data],
[https://drive.google.com/file/d/0B7TWwiIrcJstX1FzMWhQdmdGekk/view?usp=sharing Data in csv format],
[https://drive.google.com/file/d/0B0QWEJMlsxfRS2lWSkR5LVc2MzA/view?usp=sharing Censored training set],
[https://drive.google.com/file/d/0B0QWEJMlsxfRZmlHVHdlYWdQSTQ/view?usp=sharing Theoretical task 15]

Additional materials: [https://drive.google.com/file/d/0B7TWwiIrcJstd3pOWjcwUUNOaUk/view?usp=sharing Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/file/d/0B7TWwiIrcJstSGp0SzNTa1RJeTQ/view?usp=sharing PyBrain example from the seminar]

''New data files have been uploaded (there were some problems with reading old ones). Therefore deadline has been changed for some groups! Check it in the table!''

If you have MemoryError then read only part of training data from csv files (for example, 30000 objects). You can download '''censored training set''' (find link above) or use the following code:

mnist_train = np.loadtxt('mnist_train.csv', delimiter=',') 
train_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_train)): 
:train_data.appendLinked(mnist_train[i, 1:] / 255., int(mnist_train[i, 0])) 
train_data._convertToOneOfMany() 

mnist_test = np.loadtxt('mnist_test.csv', delimiter=',') 
test_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_test)): 
:test_data.appendLinked(mnist_test[i, 1:] / 255., int(mnist_test[i, 0])) 
test_data._convertToOneOfMany()

'''Seminar 16. Clustering'''

[https://drive.google.com/file/d/0B0QWEJMlsxfRRy1fd1RNVVhpMm8/view?usp=sharing Theoretical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstM1l5M3kwNDFZQlU/view?usp=sharing Practical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstZ2xIRU00dTB0OHc/view?usp=sharing parrots.jpg], [https://drive.google.com/file/d/0B7TWwiIrcJstb0RDc0RiQ1M3OXc/view?usp=sharing grass.jpg]

'''Seminar 17. Clustering, EM-algorithm'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxLWJjUEtCVVNFa00/view?usp=sharing Theoretical task 17]

'''Seminar 18. Recommender systems'''

[https://drive.google.com/file/d/0B7TWwiIrcJstWkt3Qk5Cb0FWTUk/view?usp=sharing Theoretical task 18],[https://drive.google.com/file/d/0B7TWwiIrcJstbW5Sd3hBMjNoNkE/view?usp=sharing Practical task 18], [https://www.dropbox.com/s/5as2k79dhajtw7n/data.zip?dl=0, data]

Additional materials: [https://www.semanticscholar.org/paper/Factorization-Machines-Rendle/2ef7d506b25731d0f3ec0c8f90b718b6e5bbd069/pdf Factorization Machines]

Columns in the data: 0 - user, 1 - item, 2 - rating, 3 - time (you don't need this one).

In the practical task you should train models on the train data (base) and evaluate on the test data.

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

All the deadlines can be found in the second tab [https://drive.google.com/open?id=1TQ97B8rqC7sUxTnCMKXoskgRPXO8rAyWoBWBezY58h4 here].

We have two deadlines for each assignments: normal and late. An assignment sent prior to normal deadline is scored with no penalty. The maximum score is penalized by 50% for assignments sent in between of the normal and the late deadline. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

Standard period for working on a homework assignment is 2 and 4 weeks (normal and late deadlines correspondingly) for practical assignments and 1 and 2 weeks for theoretical ones. The first practical assignment is an exception.

'''Deadline time:''' 23:59 of the day before seminar (Sunday for students attending Monday seminars and Wednesday for students that have seminars on Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group(subgroup)"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group(subgroup)"

''Example'': Practice 1 - Ivanov Ivan - 131(1)

If you want to address a particular teacher, mention his name in the subject.

''Example'': Question - Ivanov Ivan - 131(1) - Ekaterina

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in ipython notebook format, theoretical ones in pdf. Practical assignments must use '''Python 2.7'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Windows|Windows]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Mac_OS|Mac OS]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Linux | Linux]]

Data analysis (Software Engineering)

2017-01-12T19:27:11Z

Apogentus:

'''Scores and deadlines: []'''

'''Class email:''' cshse.ml@gmail.com

'''Anonymous feedback form:''' [https://docs.google.com/forms/d/12E9Bsd8UpY9KpEdhOsKdPOawcTY1g-7mpefcSh7Qm7Y here]
 

== Announcements ==

== Course description ==
In this class we consider the main problems of machine learning and data analysis: classification, regression, dimensionality reduction, clustering, collaborative filtering. We will also study mathematical methods and concepts which machine learning is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

==Prerequisites==
Firm knowledge of linear algebra, mathematical analysis and probability theory is required.

== Syllabus ==

# Introduction. Core concepts of machine learning.
# K-nearest neighbours method.
# Decision tree methods.
# Regression methods. Regularization.
# Convex functions. Classification with linear methods. Loss functions.
# Classification with linear methods. Gradient descent and stochastic gradient descent. Regularization.
# Model evaluation. Logistic regression.
# Support vector machines.
# Generalization with kernels.
# Linear dimensionality reduction - PCA, SVD.
# Feature selection.
# Ensemble methods.
# Boosting. xgBoost.
# Neural networks - architecture.
# Neural networks - optimization.
# Clustering. EM algorithm for Gaussian mixtures.
# Collaborative filtering
# Bayes decision theory. Naive Bayes assumption. Kernel density estimation.
# Semi-supervised learning
# Active learnning

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''

[https://yadi.sk/i/RajIebEkmqgzw Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

[https://drive.google.com/open?id=0B7TWwiIrcJstdkJyam9rNHpEcDg Practical task 1], [https://drive.google.com/open?id=0B7TWwiIrcJstQldxcThZRnF3ZVk data]

Additional materials: [http://nbviewer.ipython.org/gist/anonymous/fba8bf7f1ad379df9d63 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

'''Seminar 2. kNN '''

[https://drive.google.com/open?id=0B7TWwiIrcJstbGRxREhGeDBNd3M Theoretical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstNXVXYTcydUMzUUk Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstTXUyYUstMmJNckk data]

Additional materials: [http://ogrisel.github.io/scikit-learn.org/sklearn-tutorial/auto_examples/tutorial/plot_knn_iris.html Visualization tutorial]

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstbmFmMmE5bWh2Y28/view?usp=sharing Theoretical task 3]

'''Seminar 4. Linear classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstc1J4SnRWTnlxZlE/view?usp=sharing Theoretical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstckZCN1pYcEo2NW8/view?usp=sharing Practical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstM3VSQTVrYWhqYk0/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstN0VOUV9PcDc1ZlE/view?usp=sharing diabetes dataset]

UPD: At all parts of practical task 4 you should use GD and SGD functions that you program at the fisrt part!

''Deadline for this practical task has been changed for some groups! Check it in the table!''

'''Seminar 5. Model evaluation '''

[https://drive.google.com/file/d/0B7TWwiIrcJstdTFST0Z4UkRoaEk/view?usp=sharing Theoretical task 5]

'''Seminar 6. Bayesian decision rule '''

[https://drive.google.com/file/d/0B7TWwiIrcJstWF9zcllDU01ZY2M/view?usp=sharing Theoretical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstSHRKcFlVcy1xd3M/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstVlhBdGZYMm94SHc/view?usp=sharing data]

Practical task 6 was completed: the last part was described in more details + there are two small corrections in the first part (they are in bold font). Read it carefully!

'''Deadline for this practical task has been changed for all groups! Check it in the table!'''

'''Seminar 7. SVM and kernel trick '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUGhEX2QxV1gycDA/view?usp=sharing Theoretical task 7]

Additional materials: [http://www.ccas.ru/voron/download/SVM.pdf Лекция К.В. Воронцова по SVM]

'''Seminar 8. Regression '''

[https://drive.google.com/file/d/0B7TWwiIrcJstNkE0dVJscGFSOEE/view?usp=sharing Theoretical task 8]

'''Seminar 9. Boosting '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVkx2Z0tGbi1yNFE/view?usp=sharing Practical task 9],
[https://drive.google.com/file/d/0B7TWwiIrcJstelRydjBUcUlleUk/view?usp=sharing data]

'''Seminar 10. Ensemble methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJstR28zVlN3OUZJTEU/view?usp=sharing Theoretical task 10]

''Problem 2: a small typo was corrected in the loss function formula.''

'''Seminar 11. Summary'''

'''Seminar 12. How to solve practical problems'''

[https://inclass.kaggle.com/c/cmc-msu-machine-learning-spring-2015-2016-dota-competition Dota Competition from the seminar], [https://drive.google.com/file/d/0B7TWwiIrcJstVnF0RVkzc1c1TVE/view?usp=sharing ipython notebook]

'''Seminar 13. Feature selection'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTUU4OGFnQzBPa3c/view?usp=sharing Theoretical task 13], [https://drive.google.com/file/d/0B7TWwiIrcJstM3JkWnhtQ2YzZms/view?usp=sharing Practical task 13]

''Practical task is completed.''

'''Seminar 14. Feature extraction'''

You can read about computing PCA through SVD at the end of [https://drive.google.com/file/d/0B7TWwiIrcJstWFFSOUI5aTRBM00/view?usp=sharing this paper].

'''Seminar 15. Neural networks'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxbk4yeXZyVi10dVE/view?usp=sharing Practical task 15],
[https://www.dropbox.com/s/r0u3xh5ybtstw9c/mnist.zip?dl=0 Data],
[https://drive.google.com/file/d/0B7TWwiIrcJstX1FzMWhQdmdGekk/view?usp=sharing Data in csv format],
[https://drive.google.com/file/d/0B0QWEJMlsxfRS2lWSkR5LVc2MzA/view?usp=sharing Censored training set],
[https://drive.google.com/file/d/0B0QWEJMlsxfRZmlHVHdlYWdQSTQ/view?usp=sharing Theoretical task 15]

Additional materials: [https://drive.google.com/file/d/0B7TWwiIrcJstd3pOWjcwUUNOaUk/view?usp=sharing Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/file/d/0B7TWwiIrcJstSGp0SzNTa1RJeTQ/view?usp=sharing PyBrain example from the seminar]

''New data files have been uploaded (there were some problems with reading old ones). Therefore deadline has been changed for some groups! Check it in the table!''

If you have MemoryError then read only part of training data from csv files (for example, 30000 objects). You can download '''censored training set''' (find link above) or use the following code:

mnist_train = np.loadtxt('mnist_train.csv', delimiter=',') 
train_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_train)): 
:train_data.appendLinked(mnist_train[i, 1:] / 255., int(mnist_train[i, 0])) 
train_data._convertToOneOfMany() 

mnist_test = np.loadtxt('mnist_test.csv', delimiter=',') 
test_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_test)): 
:test_data.appendLinked(mnist_test[i, 1:] / 255., int(mnist_test[i, 0])) 
test_data._convertToOneOfMany()

'''Seminar 16. Clustering'''

[https://drive.google.com/file/d/0B0QWEJMlsxfRRy1fd1RNVVhpMm8/view?usp=sharing Theoretical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstM1l5M3kwNDFZQlU/view?usp=sharing Practical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstZ2xIRU00dTB0OHc/view?usp=sharing parrots.jpg], [https://drive.google.com/file/d/0B7TWwiIrcJstb0RDc0RiQ1M3OXc/view?usp=sharing grass.jpg]

'''Seminar 17. Clustering, EM-algorithm'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxLWJjUEtCVVNFa00/view?usp=sharing Theoretical task 17]

'''Seminar 18. Recommender systems'''

[https://drive.google.com/file/d/0B7TWwiIrcJstWkt3Qk5Cb0FWTUk/view?usp=sharing Theoretical task 18],[https://drive.google.com/file/d/0B7TWwiIrcJstbW5Sd3hBMjNoNkE/view?usp=sharing Practical task 18], [https://www.dropbox.com/s/5as2k79dhajtw7n/data.zip?dl=0, data]

Additional materials: [https://www.semanticscholar.org/paper/Factorization-Machines-Rendle/2ef7d506b25731d0f3ec0c8f90b718b6e5bbd069/pdf Factorization Machines]

Columns in the data: 0 - user, 1 - item, 2 - rating, 3 - time (you don't need this one).

In the practical task you should train models on the train data (base) and evaluate on the test data.

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

All the deadlines can be found in the second tab [https://drive.google.com/open?id=1TQ97B8rqC7sUxTnCMKXoskgRPXO8rAyWoBWBezY58h4 here].

We have two deadlines for each assignments: normal and late. An assignment sent prior to normal deadline is scored with no penalty. The maximum score is penalized by 50% for assignments sent in between of the normal and the late deadline. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

Standard period for working on a homework assignment is 2 and 4 weeks (normal and late deadlines correspondingly) for practical assignments and 1 and 2 weeks for theoretical ones. The first practical assignment is an exception.

'''Deadline time:''' 23:59 of the day before seminar (Sunday for students attending Monday seminars and Wednesday for students that have seminars on Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group(subgroup)"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group(subgroup)"

''Example'': Practice 1 - Ivanov Ivan - 131(1)

If you want to address a particular teacher, mention his name in the subject.

''Example'': Question - Ivanov Ivan - 131(1) - Ekaterina

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in ipython notebook format, theoretical ones in pdf. Practical assignments must use '''Python 2.7'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Windows|Windows]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Mac_OS|Mac OS]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Linux | Linux]]

Data analysis (Software Engineering)

2017-01-12T17:54:28Z

Apogentus:

'''Scores and deadlines: [--- here]'''

'''Class email:''' cshse.ml@gmail.com

'''Anonymous feedback form:''' [https://docs.google.com/forms/d/12E9Bsd8UpY9KpEdhOsKdPOawcTY1g-7mpefcSh7Qm7Y here]
 

== Announcements ==

== Course description ==
In this class we consider the main problems of machine learning and data analysis: classification, regression, dimensionality reduction, clustering, collaborative filtering. We will also study mathematical methods and concepts which machine learning is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

==Prerequisites==
Firm knowledge of linear algebra, mathematical analysis and probability theory is required.

== Syllabus ==

# Introduction. Core concepts of machine learning.
# K-nearest neighbours method.
# Decision tree methods.
# Regression methods. Regularization.
# Convex functions. Classification with linear methods. Loss functions.
# Classification with linear methods. Gradient descent and stochastic gradient descent. Regularization.
# Model evaluation. Logistic regression.
# Support vector machines.
# Generalization with kernels.
# Linear dimensionality reduction - PCA, SVD.
# Feature selection.
# Ensemble methods.
# Boosting. xgBoost.
# Neural networks - architecture.
# Neural networks - optimization.
# Clustering. EM algorithm for Gaussian mixtures.
# Collaborative filtering
# Bayes decision theory. Naive Bayes assumption. Kernel density estimation.
# Semi-supervised learning
# Active learnning

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''

[https://yadi.sk/i/RajIebEkmqgzw Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

[https://drive.google.com/open?id=0B7TWwiIrcJstdkJyam9rNHpEcDg Practical task 1], [https://drive.google.com/open?id=0B7TWwiIrcJstQldxcThZRnF3ZVk data]

Additional materials: [http://nbviewer.ipython.org/gist/anonymous/fba8bf7f1ad379df9d63 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

'''Seminar 2. kNN '''

[https://drive.google.com/open?id=0B7TWwiIrcJstbGRxREhGeDBNd3M Theoretical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstNXVXYTcydUMzUUk Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstTXUyYUstMmJNckk data]

Additional materials: [http://ogrisel.github.io/scikit-learn.org/sklearn-tutorial/auto_examples/tutorial/plot_knn_iris.html Visualization tutorial]

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstbmFmMmE5bWh2Y28/view?usp=sharing Theoretical task 3]

'''Seminar 4. Linear classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstc1J4SnRWTnlxZlE/view?usp=sharing Theoretical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstckZCN1pYcEo2NW8/view?usp=sharing Practical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstM3VSQTVrYWhqYk0/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstN0VOUV9PcDc1ZlE/view?usp=sharing diabetes dataset]

UPD: At all parts of practical task 4 you should use GD and SGD functions that you program at the fisrt part!

''Deadline for this practical task has been changed for some groups! Check it in the table!''

'''Seminar 5. Model evaluation '''

[https://drive.google.com/file/d/0B7TWwiIrcJstdTFST0Z4UkRoaEk/view?usp=sharing Theoretical task 5]

'''Seminar 6. Bayesian decision rule '''

[https://drive.google.com/file/d/0B7TWwiIrcJstWF9zcllDU01ZY2M/view?usp=sharing Theoretical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstSHRKcFlVcy1xd3M/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstVlhBdGZYMm94SHc/view?usp=sharing data]

Practical task 6 was completed: the last part was described in more details + there are two small corrections in the first part (they are in bold font). Read it carefully!

'''Deadline for this practical task has been changed for all groups! Check it in the table!'''

'''Seminar 7. SVM and kernel trick '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUGhEX2QxV1gycDA/view?usp=sharing Theoretical task 7]

Additional materials: [http://www.ccas.ru/voron/download/SVM.pdf Лекция К.В. Воронцова по SVM]

'''Seminar 8. Regression '''

[https://drive.google.com/file/d/0B7TWwiIrcJstNkE0dVJscGFSOEE/view?usp=sharing Theoretical task 8]

'''Seminar 9. Boosting '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVkx2Z0tGbi1yNFE/view?usp=sharing Practical task 9],
[https://drive.google.com/file/d/0B7TWwiIrcJstelRydjBUcUlleUk/view?usp=sharing data]

'''Seminar 10. Ensemble methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJstR28zVlN3OUZJTEU/view?usp=sharing Theoretical task 10]

''Problem 2: a small typo was corrected in the loss function formula.''

'''Seminar 11. Summary'''

'''Seminar 12. How to solve practical problems'''

[https://inclass.kaggle.com/c/cmc-msu-machine-learning-spring-2015-2016-dota-competition Dota Competition from the seminar], [https://drive.google.com/file/d/0B7TWwiIrcJstVnF0RVkzc1c1TVE/view?usp=sharing ipython notebook]

'''Seminar 13. Feature selection'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTUU4OGFnQzBPa3c/view?usp=sharing Theoretical task 13], [https://drive.google.com/file/d/0B7TWwiIrcJstM3JkWnhtQ2YzZms/view?usp=sharing Practical task 13]

''Practical task is completed.''

'''Seminar 14. Feature extraction'''

You can read about computing PCA through SVD at the end of [https://drive.google.com/file/d/0B7TWwiIrcJstWFFSOUI5aTRBM00/view?usp=sharing this paper].

'''Seminar 15. Neural networks'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxbk4yeXZyVi10dVE/view?usp=sharing Practical task 15],
[https://www.dropbox.com/s/r0u3xh5ybtstw9c/mnist.zip?dl=0 Data],
[https://drive.google.com/file/d/0B7TWwiIrcJstX1FzMWhQdmdGekk/view?usp=sharing Data in csv format],
[https://drive.google.com/file/d/0B0QWEJMlsxfRS2lWSkR5LVc2MzA/view?usp=sharing Censored training set],
[https://drive.google.com/file/d/0B0QWEJMlsxfRZmlHVHdlYWdQSTQ/view?usp=sharing Theoretical task 15]

Additional materials: [https://drive.google.com/file/d/0B7TWwiIrcJstd3pOWjcwUUNOaUk/view?usp=sharing Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/file/d/0B7TWwiIrcJstSGp0SzNTa1RJeTQ/view?usp=sharing PyBrain example from the seminar]

''New data files have been uploaded (there were some problems with reading old ones). Therefore deadline has been changed for some groups! Check it in the table!''

If you have MemoryError then read only part of training data from csv files (for example, 30000 objects). You can download '''censored training set''' (find link above) or use the following code:

mnist_train = np.loadtxt('mnist_train.csv', delimiter=',') 
train_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_train)): 
:train_data.appendLinked(mnist_train[i, 1:] / 255., int(mnist_train[i, 0])) 
train_data._convertToOneOfMany() 

mnist_test = np.loadtxt('mnist_test.csv', delimiter=',') 
test_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_test)): 
:test_data.appendLinked(mnist_test[i, 1:] / 255., int(mnist_test[i, 0])) 
test_data._convertToOneOfMany()

'''Seminar 16. Clustering'''

[https://drive.google.com/file/d/0B0QWEJMlsxfRRy1fd1RNVVhpMm8/view?usp=sharing Theoretical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstM1l5M3kwNDFZQlU/view?usp=sharing Practical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstZ2xIRU00dTB0OHc/view?usp=sharing parrots.jpg], [https://drive.google.com/file/d/0B7TWwiIrcJstb0RDc0RiQ1M3OXc/view?usp=sharing grass.jpg]

'''Seminar 17. Clustering, EM-algorithm'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxLWJjUEtCVVNFa00/view?usp=sharing Theoretical task 17]

'''Seminar 18. Recommender systems'''

[https://drive.google.com/file/d/0B7TWwiIrcJstWkt3Qk5Cb0FWTUk/view?usp=sharing Theoretical task 18],[https://drive.google.com/file/d/0B7TWwiIrcJstbW5Sd3hBMjNoNkE/view?usp=sharing Practical task 18], [https://www.dropbox.com/s/5as2k79dhajtw7n/data.zip?dl=0, data]

Additional materials: [https://www.semanticscholar.org/paper/Factorization-Machines-Rendle/2ef7d506b25731d0f3ec0c8f90b718b6e5bbd069/pdf Factorization Machines]

Columns in the data: 0 - user, 1 - item, 2 - rating, 3 - time (you don't need this one).

In the practical task you should train models on the train data (base) and evaluate on the test data.

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

All the deadlines can be found in the second tab [https://drive.google.com/open?id=1TQ97B8rqC7sUxTnCMKXoskgRPXO8rAyWoBWBezY58h4 here].

We have two deadlines for each assignments: normal and late. An assignment sent prior to normal deadline is scored with no penalty. The maximum score is penalized by 50% for assignments sent in between of the normal and the late deadline. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

Standard period for working on a homework assignment is 2 and 4 weeks (normal and late deadlines correspondingly) for practical assignments and 1 and 2 weeks for theoretical ones. The first practical assignment is an exception.

'''Deadline time:''' 23:59 of the day before seminar (Sunday for students attending Monday seminars and Wednesday for students that have seminars on Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group(subgroup)"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group(subgroup)"

''Example'': Practice 1 - Ivanov Ivan - 131(1)

If you want to address a particular teacher, mention his name in the subject.

''Example'': Question - Ivanov Ivan - 131(1) - Ekaterina

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in ipython notebook format, theoretical ones in pdf. Practical assignments must use '''Python 2.7'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Windows|Windows]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Mac_OS|Mac OS]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Linux | Linux]]

Data analysis (Software Engineering)

2017-01-12T17:51:14Z

Apogentus: /* Lecture materials */

'''Scores and deadlines: [--- here]'''

'''Class email:''' cshse.ml@gmail.com

'''Anonymous feedback form:''' [https://docs.google.com/forms/d/12E9Bsd8UpY9KpEdhOsKdPOawcTY1g-7mpefcSh7Qm7Y here]
 

== Announcements ==

== Course description ==
In this class we consider the main problems of machine learning and data analysis: classification, regression, dimensionality reduction, clustering, collaborative filtering. We will also study mathematical methods and concepts which machine learning is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

==Prerequisites==
Firm knowledge of linear algebra, mathematical analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Syllabus ==

# Introduction. Core concepts of machine learning.
# K-nearest neighbours method.
# Decision tree methods.
# Regression methods. Regularization.
# Convex functions. Classification with linear methods. Loss functions.
# Classification with linear methods. Gradient descent and stochastic gradient descent. Regularization.
# Model evaluation. Logistic regression.
# Support vector machines.
# Generalization with kernels.
# Linear dimensionality reduction - PCA, SVD.
# Feature selection.
# Ensemble methods.
# Boosting. xgBoost.
# Neural networks - architecture.
# Neural networks - optimization.
# Clustering. EM algorithm for Gaussian mixtures.
# Collaborative filtering
# Bayes decision theory. Naive Bayes assumption. Kernel density estimation.
# Semi-supervised learning
# Active learnning

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''

[https://yadi.sk/i/RajIebEkmqgzw Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

[https://drive.google.com/open?id=0B7TWwiIrcJstdkJyam9rNHpEcDg Practical task 1], [https://drive.google.com/open?id=0B7TWwiIrcJstQldxcThZRnF3ZVk data]

Additional materials: [http://nbviewer.ipython.org/gist/anonymous/fba8bf7f1ad379df9d63 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

'''Seminar 2. kNN '''

[https://drive.google.com/open?id=0B7TWwiIrcJstbGRxREhGeDBNd3M Theoretical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstNXVXYTcydUMzUUk Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstTXUyYUstMmJNckk data]

Additional materials: [http://ogrisel.github.io/scikit-learn.org/sklearn-tutorial/auto_examples/tutorial/plot_knn_iris.html Visualization tutorial]

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstbmFmMmE5bWh2Y28/view?usp=sharing Theoretical task 3]

'''Seminar 4. Linear classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstc1J4SnRWTnlxZlE/view?usp=sharing Theoretical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstckZCN1pYcEo2NW8/view?usp=sharing Practical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstM3VSQTVrYWhqYk0/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstN0VOUV9PcDc1ZlE/view?usp=sharing diabetes dataset]

UPD: At all parts of practical task 4 you should use GD and SGD functions that you program at the fisrt part!

''Deadline for this practical task has been changed for some groups! Check it in the table!''

'''Seminar 5. Model evaluation '''

[https://drive.google.com/file/d/0B7TWwiIrcJstdTFST0Z4UkRoaEk/view?usp=sharing Theoretical task 5]

'''Seminar 6. Bayesian decision rule '''

[https://drive.google.com/file/d/0B7TWwiIrcJstWF9zcllDU01ZY2M/view?usp=sharing Theoretical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstSHRKcFlVcy1xd3M/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstVlhBdGZYMm94SHc/view?usp=sharing data]

Practical task 6 was completed: the last part was described in more details + there are two small corrections in the first part (they are in bold font). Read it carefully!

'''Deadline for this practical task has been changed for all groups! Check it in the table!'''

'''Seminar 7. SVM and kernel trick '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUGhEX2QxV1gycDA/view?usp=sharing Theoretical task 7]

Additional materials: [http://www.ccas.ru/voron/download/SVM.pdf Лекция К.В. Воронцова по SVM]

'''Seminar 8. Regression '''

[https://drive.google.com/file/d/0B7TWwiIrcJstNkE0dVJscGFSOEE/view?usp=sharing Theoretical task 8]

'''Seminar 9. Boosting '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVkx2Z0tGbi1yNFE/view?usp=sharing Practical task 9],
[https://drive.google.com/file/d/0B7TWwiIrcJstelRydjBUcUlleUk/view?usp=sharing data]

'''Seminar 10. Ensemble methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJstR28zVlN3OUZJTEU/view?usp=sharing Theoretical task 10]

''Problem 2: a small typo was corrected in the loss function formula.''

'''Seminar 11. Summary'''

'''Seminar 12. How to solve practical problems'''

[https://inclass.kaggle.com/c/cmc-msu-machine-learning-spring-2015-2016-dota-competition Dota Competition from the seminar], [https://drive.google.com/file/d/0B7TWwiIrcJstVnF0RVkzc1c1TVE/view?usp=sharing ipython notebook]

'''Seminar 13. Feature selection'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTUU4OGFnQzBPa3c/view?usp=sharing Theoretical task 13], [https://drive.google.com/file/d/0B7TWwiIrcJstM3JkWnhtQ2YzZms/view?usp=sharing Practical task 13]

''Practical task is completed.''

'''Seminar 14. Feature extraction'''

You can read about computing PCA through SVD at the end of [https://drive.google.com/file/d/0B7TWwiIrcJstWFFSOUI5aTRBM00/view?usp=sharing this paper].

'''Seminar 15. Neural networks'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxbk4yeXZyVi10dVE/view?usp=sharing Practical task 15],
[https://www.dropbox.com/s/r0u3xh5ybtstw9c/mnist.zip?dl=0 Data],
[https://drive.google.com/file/d/0B7TWwiIrcJstX1FzMWhQdmdGekk/view?usp=sharing Data in csv format],
[https://drive.google.com/file/d/0B0QWEJMlsxfRS2lWSkR5LVc2MzA/view?usp=sharing Censored training set],
[https://drive.google.com/file/d/0B0QWEJMlsxfRZmlHVHdlYWdQSTQ/view?usp=sharing Theoretical task 15]

Additional materials: [https://drive.google.com/file/d/0B7TWwiIrcJstd3pOWjcwUUNOaUk/view?usp=sharing Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/file/d/0B7TWwiIrcJstSGp0SzNTa1RJeTQ/view?usp=sharing PyBrain example from the seminar]

''New data files have been uploaded (there were some problems with reading old ones). Therefore deadline has been changed for some groups! Check it in the table!''

If you have MemoryError then read only part of training data from csv files (for example, 30000 objects). You can download '''censored training set''' (find link above) or use the following code:

mnist_train = np.loadtxt('mnist_train.csv', delimiter=',') 
train_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_train)): 
:train_data.appendLinked(mnist_train[i, 1:] / 255., int(mnist_train[i, 0])) 
train_data._convertToOneOfMany() 

mnist_test = np.loadtxt('mnist_test.csv', delimiter=',') 
test_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_test)): 
:test_data.appendLinked(mnist_test[i, 1:] / 255., int(mnist_test[i, 0])) 
test_data._convertToOneOfMany()

'''Seminar 16. Clustering'''

[https://drive.google.com/file/d/0B0QWEJMlsxfRRy1fd1RNVVhpMm8/view?usp=sharing Theoretical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstM1l5M3kwNDFZQlU/view?usp=sharing Practical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstZ2xIRU00dTB0OHc/view?usp=sharing parrots.jpg], [https://drive.google.com/file/d/0B7TWwiIrcJstb0RDc0RiQ1M3OXc/view?usp=sharing grass.jpg]

'''Seminar 17. Clustering, EM-algorithm'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxLWJjUEtCVVNFa00/view?usp=sharing Theoretical task 17]

'''Seminar 18. Recommender systems'''

[https://drive.google.com/file/d/0B7TWwiIrcJstWkt3Qk5Cb0FWTUk/view?usp=sharing Theoretical task 18],[https://drive.google.com/file/d/0B7TWwiIrcJstbW5Sd3hBMjNoNkE/view?usp=sharing Practical task 18], [https://www.dropbox.com/s/5as2k79dhajtw7n/data.zip?dl=0, data]

Additional materials: [https://www.semanticscholar.org/paper/Factorization-Machines-Rendle/2ef7d506b25731d0f3ec0c8f90b718b6e5bbd069/pdf Factorization Machines]

Columns in the data: 0 - user, 1 - item, 2 - rating, 3 - time (you don't need this one).

In the practical task you should train models on the train data (base) and evaluate on the test data.

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

All the deadlines can be found in the second tab [https://drive.google.com/open?id=1TQ97B8rqC7sUxTnCMKXoskgRPXO8rAyWoBWBezY58h4 here].

We have two deadlines for each assignments: normal and late. An assignment sent prior to normal deadline is scored with no penalty. The maximum score is penalized by 50% for assignments sent in between of the normal and the late deadline. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

Standard period for working on a homework assignment is 2 and 4 weeks (normal and late deadlines correspondingly) for practical assignments and 1 and 2 weeks for theoretical ones. The first practical assignment is an exception.

'''Deadline time:''' 23:59 of the day before seminar (Sunday for students attending Monday seminars and Wednesday for students that have seminars on Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group(subgroup)"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group(subgroup)"

''Example'': Practice 1 - Ivanov Ivan - 131(1)

If you want to address a particular teacher, mention his name in the subject.

''Example'': Question - Ivanov Ivan - 131(1) - Ekaterina

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in ipython notebook format, theoretical ones in pdf. Practical assignments must use '''Python 2.7'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Windows|Windows]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Mac_OS|Mac OS]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Linux | Linux]]

Data analysis (Software Engineering)

2017-01-12T17:50:38Z

Apogentus: /* Syllabus */

'''Scores and deadlines: [--- here]'''

'''Class email:''' cshse.ml@gmail.com

'''Anonymous feedback form:''' [https://docs.google.com/forms/d/12E9Bsd8UpY9KpEdhOsKdPOawcTY1g-7mpefcSh7Qm7Y here]
 

== Announcements ==

== Course description ==
In this class we consider the main problems of machine learning and data analysis: classification, regression, dimensionality reduction, clustering, collaborative filtering. We will also study mathematical methods and concepts which machine learning is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

==Prerequisites==
Firm knowledge of linear algebra, mathematical analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Syllabus ==

# Introduction. Core concepts of machine learning.
# K-nearest neighbours method.
# Decision tree methods.
# Regression methods. Regularization.
# Convex functions. Classification with linear methods. Loss functions.
# Classification with linear methods. Gradient descent and stochastic gradient descent. Regularization.
# Model evaluation. Logistic regression.
# Support vector machines.
# Generalization with kernels.
# Linear dimensionality reduction - PCA, SVD.
# Feature selection.
# Ensemble methods.
# Boosting. xgBoost.
# Neural networks - architecture.
# Neural networks - optimization.
# Clustering. EM algorithm for Gaussian mixtures.
# Collaborative filtering
# Bayes decision theory. Naive Bayes assumption. Kernel density estimation.
# Semi-supervised learning
# Active learnning

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''

[https://yadi.sk/i/RajIebEkmqgzw Download]

Additional materials: [https://yadi.sk/i/x2lrKdbVmr2bf The Field Guide to Data Science], [http://www.machinelearning.ru/wiki/images/f/fc/Voron-ML-Intro-slides.pdf Лекция К.В.Воронцова]

'''Lecture 2. K nearest neighbours method. '''

[https://yadi.sk/i/Od8HM9h-nUWob Download]

Additional materials: [http://www.machinelearning.ru/wiki/images/c/c3/Voron-ML-Metric-slides.pdf Лекция К.В.Воронцова], [http://arxiv.org/pdf/1306.6709v4.pdf Metric learning survey]

'''Lecture 3. Decision trees. '''

[https://yadi.sk/i/-vPl2vaBqXrt5 Download]

Additional materials: ''Webb, Copsey "Statistical Pattern Recognition", chapter 7.2.''

'''Lecture 4a. Model evaluation. '''

Binary quality measures. ROC curve, AUC.

[https://yadi.sk/i/7V7U_1QtnfYZQ Download]

Additional materials: ''Webb, Copsey "Statistical Pattern Recognition", chapter 9.''

'''Lecture 4b. Bayes minimum cost classification. '''

Case of general losses, common within-class losses and 0,1 losses. Gaussian classifier.

[https://yadi.sk/i/6jIvLiYMosuz5 Download]

'''Lecture 5. Linear classifiers. '''

Discriminant function. Invariance to monotonous transformations for them. Definition for multi-class and binary class cases.

[https://yadi.sk/i/0IQ6P3LDoqpRk Download]

Additional materials: [http://www.machinelearning.ru/wiki/images/5/53/Voron-ML-Lin-SG.pdf Лекции К.В.Воронцова по линейным методам классификации]

'''Lecture 6. Support vector machines. '''

Linear separable and linearly non-separable case. Equivalent definition with loss function. Support vectors and non-informative vectors.

[https://yadi.sk/i/baP7gbWXoqpTE Download]

'''Lecture 7. Kernel trick. '''

Application of kernel trick to SVM. Gaussian, polynomial kernels.

[https://yadi.sk/i/3W6A9FmZoqpU9 Download]

'''Lecture 8. Regression. '''

Linear regression and extensions: weighted regression, robust regression, different loss-functions, regression with non-linear features, locally-constant (Nadaraya-Watson) regression.

[https://yadi.sk/i/HSp51pmepjQBq Download]

'''Lecture 9. Boosting. '''

Forward stagewise additive modelling. AdaBoost. Gradient boosting.

[https://yadi.sk/i/3yTKLDcCpjLhG Download]

Additional materials:

[http://statweb.stanford.edu/~tibs/ElemStatLearn/ Friedman, Hastie, Tibshirani "The Elements of Statistical Learning"] - section 10: Boosting and additive trees.,

[http://www.recognition.mccme.ru/pub/RecognitionLab.html/slbook.pdf Мерков "Введение в методы статистического обучения"] - секция 4: Линейные комбинации распознавателей.

'''Lecture 10. Ensemble methods. '''

Motivation. Bias-variance tradeoff. Bagging, RandomForest, ExtraRandomTrees. Stacking.

[https://yadi.sk/i/8E-wZxIpq9Sgf Download]

'''Lectures 11, 12. Summary. '''

'''Lecture 13. Feature selection. '''

[https://yadi.sk/i/2ZLC3J6dr3iAc Download]

'''Lecture 14. Principal components analysis. '''

[https://yadi.sk/i/6DxoScrKrN3LK Download]

'''Lecture 14. Singular values decomposition. '''

[https://yadi.sk/i/zwJftdkUrN6Td Download] - updated pages 17,18,19.

'''Lecture 15. Working with text. '''

[https://yadi.sk/i/YqdRr-0erEXba Download]

'''Lecture 16. Neural networks. '''

[https://yadi.sk/i/9yMM7mkrrEXbc Download]

'''Lecture 17. Parametric distributions. '''

[https://yadi.sk/i/D33hdVXTrkxDN Download]

'''Lecture 18. Clustering. '''

[https://yadi.sk/i/3DJY4Oo7rkxEW Download]

'''Lecture 19. Mixture densities, EM-algorithm. ''' - updated.

[https://yadi.sk/i/22wLOL2krkxGL Download]

'''Lecture 20. Recommender systems. '''

[https://yadi.sk/i/mMhaZlqLrvjph Download]

'''Lecture 21. Kernel density estimation. '''

[https://yadi.sk/i/zF7ZPyTCsAjFH Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

[https://drive.google.com/open?id=0B7TWwiIrcJstdkJyam9rNHpEcDg Practical task 1], [https://drive.google.com/open?id=0B7TWwiIrcJstQldxcThZRnF3ZVk data]

Additional materials: [http://nbviewer.ipython.org/gist/anonymous/fba8bf7f1ad379df9d63 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

'''Seminar 2. kNN '''

[https://drive.google.com/open?id=0B7TWwiIrcJstbGRxREhGeDBNd3M Theoretical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstNXVXYTcydUMzUUk Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstTXUyYUstMmJNckk data]

Additional materials: [http://ogrisel.github.io/scikit-learn.org/sklearn-tutorial/auto_examples/tutorial/plot_knn_iris.html Visualization tutorial]

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstbmFmMmE5bWh2Y28/view?usp=sharing Theoretical task 3]

'''Seminar 4. Linear classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstc1J4SnRWTnlxZlE/view?usp=sharing Theoretical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstckZCN1pYcEo2NW8/view?usp=sharing Practical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstM3VSQTVrYWhqYk0/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstN0VOUV9PcDc1ZlE/view?usp=sharing diabetes dataset]

UPD: At all parts of practical task 4 you should use GD and SGD functions that you program at the fisrt part!

''Deadline for this practical task has been changed for some groups! Check it in the table!''

'''Seminar 5. Model evaluation '''

[https://drive.google.com/file/d/0B7TWwiIrcJstdTFST0Z4UkRoaEk/view?usp=sharing Theoretical task 5]

'''Seminar 6. Bayesian decision rule '''

[https://drive.google.com/file/d/0B7TWwiIrcJstWF9zcllDU01ZY2M/view?usp=sharing Theoretical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstSHRKcFlVcy1xd3M/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstVlhBdGZYMm94SHc/view?usp=sharing data]

Practical task 6 was completed: the last part was described in more details + there are two small corrections in the first part (they are in bold font). Read it carefully!

'''Deadline for this practical task has been changed for all groups! Check it in the table!'''

'''Seminar 7. SVM and kernel trick '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUGhEX2QxV1gycDA/view?usp=sharing Theoretical task 7]

Additional materials: [http://www.ccas.ru/voron/download/SVM.pdf Лекция К.В. Воронцова по SVM]

'''Seminar 8. Regression '''

[https://drive.google.com/file/d/0B7TWwiIrcJstNkE0dVJscGFSOEE/view?usp=sharing Theoretical task 8]

'''Seminar 9. Boosting '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVkx2Z0tGbi1yNFE/view?usp=sharing Practical task 9],
[https://drive.google.com/file/d/0B7TWwiIrcJstelRydjBUcUlleUk/view?usp=sharing data]

'''Seminar 10. Ensemble methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJstR28zVlN3OUZJTEU/view?usp=sharing Theoretical task 10]

''Problem 2: a small typo was corrected in the loss function formula.''

'''Seminar 11. Summary'''

'''Seminar 12. How to solve practical problems'''

[https://inclass.kaggle.com/c/cmc-msu-machine-learning-spring-2015-2016-dota-competition Dota Competition from the seminar], [https://drive.google.com/file/d/0B7TWwiIrcJstVnF0RVkzc1c1TVE/view?usp=sharing ipython notebook]

'''Seminar 13. Feature selection'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTUU4OGFnQzBPa3c/view?usp=sharing Theoretical task 13], [https://drive.google.com/file/d/0B7TWwiIrcJstM3JkWnhtQ2YzZms/view?usp=sharing Practical task 13]

''Practical task is completed.''

'''Seminar 14. Feature extraction'''

You can read about computing PCA through SVD at the end of [https://drive.google.com/file/d/0B7TWwiIrcJstWFFSOUI5aTRBM00/view?usp=sharing this paper].

'''Seminar 15. Neural networks'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxbk4yeXZyVi10dVE/view?usp=sharing Practical task 15],
[https://www.dropbox.com/s/r0u3xh5ybtstw9c/mnist.zip?dl=0 Data],
[https://drive.google.com/file/d/0B7TWwiIrcJstX1FzMWhQdmdGekk/view?usp=sharing Data in csv format],
[https://drive.google.com/file/d/0B0QWEJMlsxfRS2lWSkR5LVc2MzA/view?usp=sharing Censored training set],
[https://drive.google.com/file/d/0B0QWEJMlsxfRZmlHVHdlYWdQSTQ/view?usp=sharing Theoretical task 15]

Additional materials: [https://drive.google.com/file/d/0B7TWwiIrcJstd3pOWjcwUUNOaUk/view?usp=sharing Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/file/d/0B7TWwiIrcJstSGp0SzNTa1RJeTQ/view?usp=sharing PyBrain example from the seminar]

''New data files have been uploaded (there were some problems with reading old ones). Therefore deadline has been changed for some groups! Check it in the table!''

If you have MemoryError then read only part of training data from csv files (for example, 30000 objects). You can download '''censored training set''' (find link above) or use the following code:

mnist_train = np.loadtxt('mnist_train.csv', delimiter=',') 
train_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_train)): 
:train_data.appendLinked(mnist_train[i, 1:] / 255., int(mnist_train[i, 0])) 
train_data._convertToOneOfMany() 

mnist_test = np.loadtxt('mnist_test.csv', delimiter=',') 
test_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_test)): 
:test_data.appendLinked(mnist_test[i, 1:] / 255., int(mnist_test[i, 0])) 
test_data._convertToOneOfMany()

'''Seminar 16. Clustering'''

[https://drive.google.com/file/d/0B0QWEJMlsxfRRy1fd1RNVVhpMm8/view?usp=sharing Theoretical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstM1l5M3kwNDFZQlU/view?usp=sharing Practical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstZ2xIRU00dTB0OHc/view?usp=sharing parrots.jpg], [https://drive.google.com/file/d/0B7TWwiIrcJstb0RDc0RiQ1M3OXc/view?usp=sharing grass.jpg]

'''Seminar 17. Clustering, EM-algorithm'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxLWJjUEtCVVNFa00/view?usp=sharing Theoretical task 17]

'''Seminar 18. Recommender systems'''

[https://drive.google.com/file/d/0B7TWwiIrcJstWkt3Qk5Cb0FWTUk/view?usp=sharing Theoretical task 18],[https://drive.google.com/file/d/0B7TWwiIrcJstbW5Sd3hBMjNoNkE/view?usp=sharing Practical task 18], [https://www.dropbox.com/s/5as2k79dhajtw7n/data.zip?dl=0, data]

Additional materials: [https://www.semanticscholar.org/paper/Factorization-Machines-Rendle/2ef7d506b25731d0f3ec0c8f90b718b6e5bbd069/pdf Factorization Machines]

Columns in the data: 0 - user, 1 - item, 2 - rating, 3 - time (you don't need this one).

In the practical task you should train models on the train data (base) and evaluate on the test data.

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

All the deadlines can be found in the second tab [https://drive.google.com/open?id=1TQ97B8rqC7sUxTnCMKXoskgRPXO8rAyWoBWBezY58h4 here].

We have two deadlines for each assignments: normal and late. An assignment sent prior to normal deadline is scored with no penalty. The maximum score is penalized by 50% for assignments sent in between of the normal and the late deadline. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

Standard period for working on a homework assignment is 2 and 4 weeks (normal and late deadlines correspondingly) for practical assignments and 1 and 2 weeks for theoretical ones. The first practical assignment is an exception.

'''Deadline time:''' 23:59 of the day before seminar (Sunday for students attending Monday seminars and Wednesday for students that have seminars on Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group(subgroup)"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group(subgroup)"

''Example'': Practice 1 - Ivanov Ivan - 131(1)

If you want to address a particular teacher, mention his name in the subject.

''Example'': Question - Ivanov Ivan - 131(1) - Ekaterina

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in ipython notebook format, theoretical ones in pdf. Practical assignments must use '''Python 2.7'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Windows|Windows]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Mac_OS|Mac OS]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Linux | Linux]]

Data analysis (Software Engineering)

2017-01-12T16:52:51Z

Apogentus:

'''Scores and deadlines: [--- here]'''

'''Class email:''' cshse.ml@gmail.com

'''Anonymous feedback form:''' [https://docs.google.com/forms/d/12E9Bsd8UpY9KpEdhOsKdPOawcTY1g-7mpefcSh7Qm7Y here]
 

== Announcements ==

== Course description ==
In this class we consider the main problems of machine learning and data analysis: classification, regression, dimensionality reduction, clustering, collaborative filtering. We will also study mathematical methods and concepts which machine learning is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

==Prerequisites==
Firm knowledge of linear algebra, mathematical analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''

[https://yadi.sk/i/RajIebEkmqgzw Download]

Additional materials: [https://yadi.sk/i/x2lrKdbVmr2bf The Field Guide to Data Science], [http://www.machinelearning.ru/wiki/images/f/fc/Voron-ML-Intro-slides.pdf Лекция К.В.Воронцова]

'''Lecture 2. K nearest neighbours method. '''

[https://yadi.sk/i/Od8HM9h-nUWob Download]

Additional materials: [http://www.machinelearning.ru/wiki/images/c/c3/Voron-ML-Metric-slides.pdf Лекция К.В.Воронцова], [http://arxiv.org/pdf/1306.6709v4.pdf Metric learning survey]

'''Lecture 3. Decision trees. '''

[https://yadi.sk/i/-vPl2vaBqXrt5 Download]

Additional materials: ''Webb, Copsey "Statistical Pattern Recognition", chapter 7.2.''

'''Lecture 4a. Model evaluation. '''

Binary quality measures. ROC curve, AUC.

[https://yadi.sk/i/7V7U_1QtnfYZQ Download]

Additional materials: ''Webb, Copsey "Statistical Pattern Recognition", chapter 9.''

'''Lecture 4b. Bayes minimum cost classification. '''

Case of general losses, common within-class losses and 0,1 losses. Gaussian classifier.

[https://yadi.sk/i/6jIvLiYMosuz5 Download]

'''Lecture 5. Linear classifiers. '''

Discriminant function. Invariance to monotonous transformations for them. Definition for multi-class and binary class cases.

[https://yadi.sk/i/0IQ6P3LDoqpRk Download]

Additional materials: [http://www.machinelearning.ru/wiki/images/5/53/Voron-ML-Lin-SG.pdf Лекции К.В.Воронцова по линейным методам классификации]

'''Lecture 6. Support vector machines. '''

Linear separable and linearly non-separable case. Equivalent definition with loss function. Support vectors and non-informative vectors.

[https://yadi.sk/i/baP7gbWXoqpTE Download]

'''Lecture 7. Kernel trick. '''

Application of kernel trick to SVM. Gaussian, polynomial kernels.

[https://yadi.sk/i/3W6A9FmZoqpU9 Download]

'''Lecture 8. Regression. '''

Linear regression and extensions: weighted regression, robust regression, different loss-functions, regression with non-linear features, locally-constant (Nadaraya-Watson) regression.

[https://yadi.sk/i/HSp51pmepjQBq Download]

'''Lecture 9. Boosting. '''

Forward stagewise additive modelling. AdaBoost. Gradient boosting.

[https://yadi.sk/i/3yTKLDcCpjLhG Download]

Additional materials:

[http://statweb.stanford.edu/~tibs/ElemStatLearn/ Friedman, Hastie, Tibshirani "The Elements of Statistical Learning"] - section 10: Boosting and additive trees.,

[http://www.recognition.mccme.ru/pub/RecognitionLab.html/slbook.pdf Мерков "Введение в методы статистического обучения"] - секция 4: Линейные комбинации распознавателей.

'''Lecture 10. Ensemble methods. '''

Motivation. Bias-variance tradeoff. Bagging, RandomForest, ExtraRandomTrees. Stacking.

[https://yadi.sk/i/8E-wZxIpq9Sgf Download]

'''Lectures 11, 12. Summary. '''

'''Lecture 13. Feature selection. '''

[https://yadi.sk/i/2ZLC3J6dr3iAc Download]

'''Lecture 14. Principal components analysis. '''

[https://yadi.sk/i/6DxoScrKrN3LK Download]

'''Lecture 14. Singular values decomposition. '''

[https://yadi.sk/i/zwJftdkUrN6Td Download] - updated pages 17,18,19.

'''Lecture 15. Working with text. '''

[https://yadi.sk/i/YqdRr-0erEXba Download]

'''Lecture 16. Neural networks. '''

[https://yadi.sk/i/9yMM7mkrrEXbc Download]

'''Lecture 17. Parametric distributions. '''

[https://yadi.sk/i/D33hdVXTrkxDN Download]

'''Lecture 18. Clustering. '''

[https://yadi.sk/i/3DJY4Oo7rkxEW Download]

'''Lecture 19. Mixture densities, EM-algorithm. ''' - updated.

[https://yadi.sk/i/22wLOL2krkxGL Download]

'''Lecture 20. Recommender systems. '''

[https://yadi.sk/i/mMhaZlqLrvjph Download]

'''Lecture 21. Kernel density estimation. '''

[https://yadi.sk/i/zF7ZPyTCsAjFH Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

[https://drive.google.com/open?id=0B7TWwiIrcJstdkJyam9rNHpEcDg Practical task 1], [https://drive.google.com/open?id=0B7TWwiIrcJstQldxcThZRnF3ZVk data]

Additional materials: [http://nbviewer.ipython.org/gist/anonymous/fba8bf7f1ad379df9d63 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

'''Seminar 2. kNN '''

[https://drive.google.com/open?id=0B7TWwiIrcJstbGRxREhGeDBNd3M Theoretical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstNXVXYTcydUMzUUk Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstTXUyYUstMmJNckk data]

Additional materials: [http://ogrisel.github.io/scikit-learn.org/sklearn-tutorial/auto_examples/tutorial/plot_knn_iris.html Visualization tutorial]

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstbmFmMmE5bWh2Y28/view?usp=sharing Theoretical task 3]

'''Seminar 4. Linear classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstc1J4SnRWTnlxZlE/view?usp=sharing Theoretical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstckZCN1pYcEo2NW8/view?usp=sharing Practical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstM3VSQTVrYWhqYk0/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstN0VOUV9PcDc1ZlE/view?usp=sharing diabetes dataset]

UPD: At all parts of practical task 4 you should use GD and SGD functions that you program at the fisrt part!

''Deadline for this practical task has been changed for some groups! Check it in the table!''

'''Seminar 5. Model evaluation '''

[https://drive.google.com/file/d/0B7TWwiIrcJstdTFST0Z4UkRoaEk/view?usp=sharing Theoretical task 5]

'''Seminar 6. Bayesian decision rule '''

[https://drive.google.com/file/d/0B7TWwiIrcJstWF9zcllDU01ZY2M/view?usp=sharing Theoretical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstSHRKcFlVcy1xd3M/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstVlhBdGZYMm94SHc/view?usp=sharing data]

Practical task 6 was completed: the last part was described in more details + there are two small corrections in the first part (they are in bold font). Read it carefully!

'''Deadline for this practical task has been changed for all groups! Check it in the table!'''

'''Seminar 7. SVM and kernel trick '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUGhEX2QxV1gycDA/view?usp=sharing Theoretical task 7]

Additional materials: [http://www.ccas.ru/voron/download/SVM.pdf Лекция К.В. Воронцова по SVM]

'''Seminar 8. Regression '''

[https://drive.google.com/file/d/0B7TWwiIrcJstNkE0dVJscGFSOEE/view?usp=sharing Theoretical task 8]

'''Seminar 9. Boosting '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVkx2Z0tGbi1yNFE/view?usp=sharing Practical task 9],
[https://drive.google.com/file/d/0B7TWwiIrcJstelRydjBUcUlleUk/view?usp=sharing data]

'''Seminar 10. Ensemble methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJstR28zVlN3OUZJTEU/view?usp=sharing Theoretical task 10]

''Problem 2: a small typo was corrected in the loss function formula.''

'''Seminar 11. Summary'''

'''Seminar 12. How to solve practical problems'''

[https://inclass.kaggle.com/c/cmc-msu-machine-learning-spring-2015-2016-dota-competition Dota Competition from the seminar], [https://drive.google.com/file/d/0B7TWwiIrcJstVnF0RVkzc1c1TVE/view?usp=sharing ipython notebook]

'''Seminar 13. Feature selection'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTUU4OGFnQzBPa3c/view?usp=sharing Theoretical task 13], [https://drive.google.com/file/d/0B7TWwiIrcJstM3JkWnhtQ2YzZms/view?usp=sharing Practical task 13]

''Practical task is completed.''

'''Seminar 14. Feature extraction'''

You can read about computing PCA through SVD at the end of [https://drive.google.com/file/d/0B7TWwiIrcJstWFFSOUI5aTRBM00/view?usp=sharing this paper].

'''Seminar 15. Neural networks'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxbk4yeXZyVi10dVE/view?usp=sharing Practical task 15],
[https://www.dropbox.com/s/r0u3xh5ybtstw9c/mnist.zip?dl=0 Data],
[https://drive.google.com/file/d/0B7TWwiIrcJstX1FzMWhQdmdGekk/view?usp=sharing Data in csv format],
[https://drive.google.com/file/d/0B0QWEJMlsxfRS2lWSkR5LVc2MzA/view?usp=sharing Censored training set],
[https://drive.google.com/file/d/0B0QWEJMlsxfRZmlHVHdlYWdQSTQ/view?usp=sharing Theoretical task 15]

Additional materials: [https://drive.google.com/file/d/0B7TWwiIrcJstd3pOWjcwUUNOaUk/view?usp=sharing Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/file/d/0B7TWwiIrcJstSGp0SzNTa1RJeTQ/view?usp=sharing PyBrain example from the seminar]

''New data files have been uploaded (there were some problems with reading old ones). Therefore deadline has been changed for some groups! Check it in the table!''

If you have MemoryError then read only part of training data from csv files (for example, 30000 objects). You can download '''censored training set''' (find link above) or use the following code:

mnist_train = np.loadtxt('mnist_train.csv', delimiter=',') 
train_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_train)): 
:train_data.appendLinked(mnist_train[i, 1:] / 255., int(mnist_train[i, 0])) 
train_data._convertToOneOfMany() 

mnist_test = np.loadtxt('mnist_test.csv', delimiter=',') 
test_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_test)): 
:test_data.appendLinked(mnist_test[i, 1:] / 255., int(mnist_test[i, 0])) 
test_data._convertToOneOfMany()

'''Seminar 16. Clustering'''

[https://drive.google.com/file/d/0B0QWEJMlsxfRRy1fd1RNVVhpMm8/view?usp=sharing Theoretical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstM1l5M3kwNDFZQlU/view?usp=sharing Practical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstZ2xIRU00dTB0OHc/view?usp=sharing parrots.jpg], [https://drive.google.com/file/d/0B7TWwiIrcJstb0RDc0RiQ1M3OXc/view?usp=sharing grass.jpg]

'''Seminar 17. Clustering, EM-algorithm'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxLWJjUEtCVVNFa00/view?usp=sharing Theoretical task 17]

'''Seminar 18. Recommender systems'''

[https://drive.google.com/file/d/0B7TWwiIrcJstWkt3Qk5Cb0FWTUk/view?usp=sharing Theoretical task 18],[https://drive.google.com/file/d/0B7TWwiIrcJstbW5Sd3hBMjNoNkE/view?usp=sharing Practical task 18], [https://www.dropbox.com/s/5as2k79dhajtw7n/data.zip?dl=0, data]

Additional materials: [https://www.semanticscholar.org/paper/Factorization-Machines-Rendle/2ef7d506b25731d0f3ec0c8f90b718b6e5bbd069/pdf Factorization Machines]

Columns in the data: 0 - user, 1 - item, 2 - rating, 3 - time (you don't need this one).

In the practical task you should train models on the train data (base) and evaluate on the test data.

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

All the deadlines can be found in the second tab [https://drive.google.com/open?id=1TQ97B8rqC7sUxTnCMKXoskgRPXO8rAyWoBWBezY58h4 here].

We have two deadlines for each assignments: normal and late. An assignment sent prior to normal deadline is scored with no penalty. The maximum score is penalized by 50% for assignments sent in between of the normal and the late deadline. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

Standard period for working on a homework assignment is 2 and 4 weeks (normal and late deadlines correspondingly) for practical assignments and 1 and 2 weeks for theoretical ones. The first practical assignment is an exception.

'''Deadline time:''' 23:59 of the day before seminar (Sunday for students attending Monday seminars and Wednesday for students that have seminars on Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group(subgroup)"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group(subgroup)"

''Example'': Practice 1 - Ivanov Ivan - 131(1)

If you want to address a particular teacher, mention his name in the subject.

''Example'': Question - Ivanov Ivan - 131(1) - Ekaterina

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in ipython notebook format, theoretical ones in pdf. Practical assignments must use '''Python 2.7'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Windows|Windows]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Mac_OS|Mac OS]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Linux | Linux]]

Data analysis (Software Engineering)

2016-09-13T21:34:23Z

Apogentus:

'''Scores and deadlines: [https://drive.google.com/open?id=1TQ97B8rqC7sUxTnCMKXoskgRPXO8rAyWoBWBezY58h4 here]'''

'''Class email:''' cshse.ml@gmail.com

'''[https://docs.google.com/forms/d/100_gMWQwp41zpHgKuf3fl3SpFSwNj6ggL13DtnxWEEw/viewform Anonymous overall course evaluation form] '''

'''Anonymous feedback form:''' [http://goo.gl/forms/CT3h4QaMeB here]
 

== Announcements ==

===Kaggle evaluation===
Kaggle evaluation [https://docs.google.com/spreadsheets/d/1h90zLpQ2Q8QD_xQLGJLY1xljnLGF175Re4fTMChHFgA/edit?usp=sharing is available here]. Please check that your work is in the list. Presentations were evaluated using [https://inclass.kaggle.com/c/hse-spring2016-stack-overflow/rules the rules of the competition]. In particular - I was expecting to see:
* that you tried different methods
* table with accuracy results of each method
* description how you tuned the parameters of your model (over which grid, with graphs showing accuracy change)
* data analysis and insights described with illustrative visualizations.
* in feature selection: quantitive results how each feature could be helpful/not helpful.

===Exam questions===
Exam questions are published and available [https://yadi.sk/i/Gi_T_R4AsD2aU here].

===Kaggle presentation requirements===
You should send presentations before June 3 (Friday) 23-59. Presentations should be sent to v.v.kitov@yandex.ru. The title should be "HSE kaggle presentation <team name>". On the title page of the presentation you should list all team participants. Presentation should be in pdf or ppt format and have all components listed in [https://inclass.kaggle.com/c/hse-spring2016-stack-overflow/rules competition rules]. Code in py or ipynb format should also be attached to the letter (it may consist of several files).

===Early exam===
On June 6th, 13-40 - 16-30 there will two lessons. They will cover:
1) a consultation before exam. Please read through all the material and come with your questions.
2) Presentations of top-3 kaggle teams with their solutions (15 minutes each). Teams with over 60 submissions are welcome to tell their findings in the data - what worked and what not (10 minutes each). Everybody else is also welcome (not obliged) to participate with short presentations (5-10 minutes) and tell interesting findings in the data and non-standard approaches that you tried (not necessarily successful).

For your convenience there will be a possibility to take exam in data analysis earlier - on June 6th at 16-40. To take exam earlier you need to request participation to e-mail v.v.kitov@yandex.ru. The number of participants is limited. Note that earlier exam will be the same as official exam and they mutually exclude each other, so you need to select in which exam to participate. Exam program will be available soon. Earlier exam schedule is proposed for your convenience - to give you the possibility to fully concentrate on preparation to data analysis exam.

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

== Events outside the course ==

[https://it.mail.ru/announcements/36/?utm_campaign=newsletter&utm_medium=email&utm_source=newsletter_2742016df Universal recomendation system of mail.ru]

[https://www.youtube.com/channel/UCeq6ZIlvC9SVsfhfKnSvM9w Description of solutions to different competitions on Kaggle]

[http://ria.ru/science/20160514/1432666353.html Neural networks adapt videos to the painting style of famous artists.]

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''

[https://yadi.sk/i/RajIebEkmqgzw Download]

Additional materials: [https://yadi.sk/i/x2lrKdbVmr2bf The Field Guide to Data Science], [http://www.machinelearning.ru/wiki/images/f/fc/Voron-ML-Intro-slides.pdf Лекция К.В.Воронцова]

'''Lecture 2. K nearest neighbours method. '''

[https://yadi.sk/i/Od8HM9h-nUWob Download]

Additional materials: [http://www.machinelearning.ru/wiki/images/c/c3/Voron-ML-Metric-slides.pdf Лекция К.В.Воронцова], [http://arxiv.org/pdf/1306.6709v4.pdf Metric learning survey]

'''Lecture 3. Decision trees. '''

[https://yadi.sk/i/-vPl2vaBqXrt5 Download]

Additional materials: ''Webb, Copsey "Statistical Pattern Recognition", chapter 7.2.''

'''Lecture 4a. Model evaluation. '''

Binary quality measures. ROC curve, AUC.

[https://yadi.sk/i/7V7U_1QtnfYZQ Download]

Additional materials: ''Webb, Copsey "Statistical Pattern Recognition", chapter 9.''

'''Lecture 4b. Bayes minimum cost classification. '''

Case of general losses, common within-class losses and 0,1 losses. Gaussian classifier.

[https://yadi.sk/i/6jIvLiYMosuz5 Download]

'''Lecture 5. Linear classifiers. '''

Discriminant function. Invariance to monotonous transformations for them. Definition for multi-class and binary class cases.

[https://yadi.sk/i/0IQ6P3LDoqpRk Download]

Additional materials: [http://www.machinelearning.ru/wiki/images/5/53/Voron-ML-Lin-SG.pdf Лекции К.В.Воронцова по линейным методам классификации]

'''Lecture 6. Support vector machines. '''

Linear separable and linearly non-separable case. Equivalent definition with loss function. Support vectors and non-informative vectors.

[https://yadi.sk/i/baP7gbWXoqpTE Download]

'''Lecture 7. Kernel trick. '''

Application of kernel trick to SVM. Gaussian, polynomial kernels.

[https://yadi.sk/i/3W6A9FmZoqpU9 Download]

'''Lecture 8. Regression. '''

Linear regression and extensions: weighted regression, robust regression, different loss-functions, regression with non-linear features, locally-constant (Nadaraya-Watson) regression.

[https://yadi.sk/i/HSp51pmepjQBq Download]

'''Lecture 9. Boosting. '''

Forward stagewise additive modelling. AdaBoost. Gradient boosting.

[https://yadi.sk/i/3yTKLDcCpjLhG Download]

Additional materials:

[http://statweb.stanford.edu/~tibs/ElemStatLearn/ Friedman, Hastie, Tibshirani "The Elements of Statistical Learning"] - section 10: Boosting and additive trees.,

[http://www.recognition.mccme.ru/pub/RecognitionLab.html/slbook.pdf Мерков "Введение в методы статистического обучения"] - секция 4: Линейные комбинации распознавателей.

'''Lecture 10. Ensemble methods. '''

Motivation. Bias-variance tradeoff. Bagging, RandomForest, ExtraRandomTrees. Stacking.

[https://yadi.sk/i/8E-wZxIpq9Sgf Download]

'''Lectures 11, 12. Summary. '''

'''Lecture 13. Feature selection. '''

[https://yadi.sk/i/2ZLC3J6dr3iAc Download]

'''Lecture 14. Principal components analysis. '''

[https://yadi.sk/i/6DxoScrKrN3LK Download]

'''Lecture 14. Singular values decomposition. '''

[https://yadi.sk/i/zwJftdkUrN6Td Download] - updated pages 17,18,19.

'''Lecture 15. Working with text. '''

[https://yadi.sk/i/YqdRr-0erEXba Download]

'''Lecture 16. Neural networks. '''

[https://yadi.sk/i/9yMM7mkrrEXbc Download]

'''Lecture 17. Parametric distributions. '''

[https://yadi.sk/i/D33hdVXTrkxDN Download]

'''Lecture 18. Clustering. '''

[https://yadi.sk/i/3DJY4Oo7rkxEW Download]

'''Lecture 19. Mixture densities, EM-algorithm. ''' - updated.

[https://yadi.sk/i/22wLOL2krkxGL Download]

'''Lecture 20. Recommender systems. '''

[https://yadi.sk/i/mMhaZlqLrvjph Download]

'''Lecture 21. Kernel density estimation. '''

[https://yadi.sk/i/zF7ZPyTCsAjFH Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

[https://drive.google.com/open?id=0B7TWwiIrcJstdkJyam9rNHpEcDg Practical task 1], [https://drive.google.com/open?id=0B7TWwiIrcJstQldxcThZRnF3ZVk data]

Additional materials: [http://nbviewer.ipython.org/gist/anonymous/fba8bf7f1ad379df9d63 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

'''Seminar 2. kNN '''

[https://drive.google.com/open?id=0B7TWwiIrcJstbGRxREhGeDBNd3M Theoretical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstNXVXYTcydUMzUUk Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstTXUyYUstMmJNckk data]

Additional materials: [http://ogrisel.github.io/scikit-learn.org/sklearn-tutorial/auto_examples/tutorial/plot_knn_iris.html Visualization tutorial]

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstbmFmMmE5bWh2Y28/view?usp=sharing Theoretical task 3]

'''Seminar 4. Linear classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstc1J4SnRWTnlxZlE/view?usp=sharing Theoretical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstckZCN1pYcEo2NW8/view?usp=sharing Practical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstM3VSQTVrYWhqYk0/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstN0VOUV9PcDc1ZlE/view?usp=sharing diabetes dataset]

UPD: At all parts of practical task 4 you should use GD and SGD functions that you program at the fisrt part!

''Deadline for this practical task has been changed for some groups! Check it in the table!''

'''Seminar 5. Model evaluation '''

[https://drive.google.com/file/d/0B7TWwiIrcJstdTFST0Z4UkRoaEk/view?usp=sharing Theoretical task 5]

'''Seminar 6. Bayesian decision rule '''

[https://drive.google.com/file/d/0B7TWwiIrcJstWF9zcllDU01ZY2M/view?usp=sharing Theoretical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstSHRKcFlVcy1xd3M/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstVlhBdGZYMm94SHc/view?usp=sharing data]

Practical task 6 was completed: the last part was described in more details + there are two small corrections in the first part (they are in bold font). Read it carefully!

'''Deadline for this practical task has been changed for all groups! Check it in the table!'''

'''Seminar 7. SVM and kernel trick '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUGhEX2QxV1gycDA/view?usp=sharing Theoretical task 7]

Additional materials: [http://www.ccas.ru/voron/download/SVM.pdf Лекция К.В. Воронцова по SVM]

'''Seminar 8. Regression '''

[https://drive.google.com/file/d/0B7TWwiIrcJstNkE0dVJscGFSOEE/view?usp=sharing Theoretical task 8]

'''Seminar 9. Boosting '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVkx2Z0tGbi1yNFE/view?usp=sharing Practical task 9],
[https://drive.google.com/file/d/0B7TWwiIrcJstelRydjBUcUlleUk/view?usp=sharing data]

'''Seminar 10. Ensemble methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJstR28zVlN3OUZJTEU/view?usp=sharing Theoretical task 10]

''Problem 2: a small typo was corrected in the loss function formula.''

'''Seminar 11. Summary'''

'''Seminar 12. How to solve practical problems'''

[https://inclass.kaggle.com/c/cmc-msu-machine-learning-spring-2015-2016-dota-competition Dota Competition from the seminar], [https://drive.google.com/file/d/0B7TWwiIrcJstVnF0RVkzc1c1TVE/view?usp=sharing ipython notebook]

'''Seminar 13. Feature selection'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTUU4OGFnQzBPa3c/view?usp=sharing Theoretical task 13], [https://drive.google.com/file/d/0B7TWwiIrcJstM3JkWnhtQ2YzZms/view?usp=sharing Practical task 13]

''Practical task is completed.''

'''Seminar 14. Feature extraction'''

You can read about computing PCA through SVD at the end of [https://drive.google.com/file/d/0B7TWwiIrcJstWFFSOUI5aTRBM00/view?usp=sharing this paper].

'''Seminar 15. Neural networks'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxbk4yeXZyVi10dVE/view?usp=sharing Practical task 15],
[https://www.dropbox.com/s/r0u3xh5ybtstw9c/mnist.zip?dl=0 Data],
[https://drive.google.com/file/d/0B7TWwiIrcJstX1FzMWhQdmdGekk/view?usp=sharing Data in csv format],
[https://drive.google.com/file/d/0B0QWEJMlsxfRS2lWSkR5LVc2MzA/view?usp=sharing Censored training set],
[https://drive.google.com/file/d/0B0QWEJMlsxfRZmlHVHdlYWdQSTQ/view?usp=sharing Theoretical task 15]

Additional materials: [https://drive.google.com/file/d/0B7TWwiIrcJstd3pOWjcwUUNOaUk/view?usp=sharing Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/file/d/0B7TWwiIrcJstSGp0SzNTa1RJeTQ/view?usp=sharing PyBrain example from the seminar]

''New data files have been uploaded (there were some problems with reading old ones). Therefore deadline has been changed for some groups! Check it in the table!''

If you have MemoryError then read only part of training data from csv files (for example, 30000 objects). You can download '''censored training set''' (find link above) or use the following code:

mnist_train = np.loadtxt('mnist_train.csv', delimiter=',') 
train_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_train)): 
:train_data.appendLinked(mnist_train[i, 1:] / 255., int(mnist_train[i, 0])) 
train_data._convertToOneOfMany() 

mnist_test = np.loadtxt('mnist_test.csv', delimiter=',') 
test_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_test)): 
:test_data.appendLinked(mnist_test[i, 1:] / 255., int(mnist_test[i, 0])) 
test_data._convertToOneOfMany()

'''Seminar 16. Clustering'''

[https://drive.google.com/file/d/0B0QWEJMlsxfRRy1fd1RNVVhpMm8/view?usp=sharing Theoretical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstM1l5M3kwNDFZQlU/view?usp=sharing Practical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstZ2xIRU00dTB0OHc/view?usp=sharing parrots.jpg], [https://drive.google.com/file/d/0B7TWwiIrcJstb0RDc0RiQ1M3OXc/view?usp=sharing grass.jpg]

'''Seminar 17. Clustering, EM-algorithm'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxLWJjUEtCVVNFa00/view?usp=sharing Theoretical task 17]

'''Seminar 18. Recommender systems'''

[https://drive.google.com/file/d/0B7TWwiIrcJstWkt3Qk5Cb0FWTUk/view?usp=sharing Theoretical task 18],[https://drive.google.com/file/d/0B7TWwiIrcJstbW5Sd3hBMjNoNkE/view?usp=sharing Practical task 18], [https://www.dropbox.com/s/5as2k79dhajtw7n/data.zip?dl=0, data]

Additional materials: [https://www.semanticscholar.org/paper/Factorization-Machines-Rendle/2ef7d506b25731d0f3ec0c8f90b718b6e5bbd069/pdf Factorization Machines]

Columns in the data: 0 - user, 1 - item, 2 - rating, 3 - time (you don't need this one).

In the practical task you should train models on the train data (base) and evaluate on the test data.

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

All the deadlines can be found in the second tab [https://drive.google.com/open?id=1TQ97B8rqC7sUxTnCMKXoskgRPXO8rAyWoBWBezY58h4 here].

We have two deadlines for each assignments: normal and late. An assignment sent prior to normal deadline is scored with no penalty. The maximum score is penalized by 50% for assignments sent in between of the normal and the late deadline. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

Standard period for working on a homework assignment is 2 and 4 weeks (normal and late deadlines correspondingly) for practical assignments and 1 and 2 weeks for theoretical ones. The first practical assignment is an exception.

'''Deadline time:''' 23:59 of the day before seminar (Sunday for students attending Monday seminars and Wednesday for students that have seminars on Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group(subgroup)"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group(subgroup)"

''Example'': Practice 1 - Ivanov Ivan - 131(1)

If you want to address a particular teacher, mention his name in the subject.

''Example'': Question - Ivanov Ivan - 131(1) - Ekaterina

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in ipython notebook format, theoretical ones in pdf. Practical assignments must use '''Python 2.7'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Windows|Windows]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Mac_OS|Mac OS]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Linux | Linux]]

Data analysis (Software Engineering)

2016-06-21T07:03:36Z

Apogentus: /* Lecture materials */

'''Scores and deadlines: [https://drive.google.com/open?id=1TQ97B8rqC7sUxTnCMKXoskgRPXO8rAyWoBWBezY58h4 here]'''

'''Class email:''' cshse.ml@gmail.com

'''[https://docs.google.com/forms/d/100_gMWQwp41zpHgKuf3fl3SpFSwNj6ggL13DtnxWEEw/viewform Anonymous overall course evaluation form] '''

'''Anonymous feedback form:''' [http://goo.gl/forms/CT3h4QaMeB here]
 

== Announcements ==

===Kaggle evaluation===
Kaggle evaluation [https://docs.google.com/spreadsheets/d/1h90zLpQ2Q8QD_xQLGJLY1xljnLGF175Re4fTMChHFgA/edit?usp=sharing is available here]. Please check that your work is in the list. Presentations were evaluated using [https://inclass.kaggle.com/c/hse-spring2016-stack-overflow/rules the rules of the competition]. In particular - I was expecting to see:
* that you tried different methods
* table with accuracy results of each method
* description how you tuned the parameters of your model (over which grid, with graphs showing accuracy change)
* data analysis and insights described with illustrative visualizations.
* in feature selection: quantitive results how each feature could be helpful/not helpful.

===Exam questions===
Exam questions are published and available [https://yadi.sk/i/Gi_T_R4AsD2aU here].

===Kaggle presentation requirements===
You should send presentations before June 3 (Friday) 23-59. Presentations should be sent to v.v.kitov@yandex.ru. The title should be "HSE kaggle presentation <team name>". On the title page of the presentation you should list all team participants. Presentation should be in pdf or ppt format and have all components listed in [https://inclass.kaggle.com/c/hse-spring2016-stack-overflow/rules competition rules]. Code in py or ipynb format should also be attached to the letter (it may consist of several files).

===Early exam===
On June 6th, 13-40 - 16-30 there will two lessons. They will cover:
1) a consultation before exam. Please read through all the material and come with your questions.
2) Presentations of top-3 kaggle teams with their solutions (15 minutes each). Teams with over 60 submissions are welcome to tell their findings in the data - what worked and what not (10 minutes each). Everybody else is also welcome (not obliged) to participate with short presentations (5-10 minutes) and tell interesting findings in the data and non-standard approaches that you tried (not necessarily successful).

For your convenience there will be a possibility to take exam in data analysis earlier - on June 6th at 16-40. To take exam earlier you need to request participation to e-mail v.v.kitov@yandex.ru. The number of participants is limited. Note that earlier exam will be the same as official exam and they mutually exclude each other, so you need to select in which exam to participate. Exam program will be available soon. Earlier exam schedule is proposed for your convenience - to give you the possibility to fully concentrate on preparation to data analysis exam.

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

==Kaggle competition==

[https://kaggle.com/join/hse_fcs Participate]

== Events outside the course ==

[https://it.mail.ru/announcements/36/?utm_campaign=newsletter&utm_medium=email&utm_source=newsletter_2742016df Universal recomendation system of mail.ru]

[https://www.youtube.com/channel/UCeq6ZIlvC9SVsfhfKnSvM9w Description of solutions to different competitions on Kaggle]

[http://ria.ru/science/20160514/1432666353.html Neural networks adapt videos to the painting style of famous artists.]

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''

[https://yadi.sk/i/RajIebEkmqgzw Download]

Additional materials: [https://yadi.sk/i/x2lrKdbVmr2bf The Field Guide to Data Science], [http://www.machinelearning.ru/wiki/images/f/fc/Voron-ML-Intro-slides.pdf Лекция К.В.Воронцова]

'''Lecture 2. K nearest neighbours method. '''

[https://yadi.sk/i/Od8HM9h-nUWob Download]

Additional materials: [http://www.machinelearning.ru/wiki/images/c/c3/Voron-ML-Metric-slides.pdf Лекция К.В.Воронцова], [http://arxiv.org/pdf/1306.6709v4.pdf Metric learning survey]

'''Lecture 3. Decision trees. '''

[https://yadi.sk/i/-vPl2vaBqXrt5 Download]

Additional materials: ''Webb, Copsey "Statistical Pattern Recognition", chapter 7.2.''

'''Lecture 4a. Model evaluation. '''

Binary quality measures. ROC curve, AUC.

[https://yadi.sk/i/7V7U_1QtnfYZQ Download]

Additional materials: ''Webb, Copsey "Statistical Pattern Recognition", chapter 9.''

'''Lecture 4b. Bayes minimum cost classification. '''

Case of general losses, common within-class losses and 0,1 losses. Gaussian classifier.

[https://yadi.sk/i/6jIvLiYMosuz5 Download]

'''Lecture 5. Linear classifiers. '''

Discriminant function. Invariance to monotonous transformations for them. Definition for multi-class and binary class cases.

[https://yadi.sk/i/0IQ6P3LDoqpRk Download]

Additional materials: [http://www.machinelearning.ru/wiki/images/5/53/Voron-ML-Lin-SG.pdf Лекции К.В.Воронцова по линейным методам классификации]

'''Lecture 6. Support vector machines. '''

Linear separable and linearly non-separable case. Equivalent definition with loss function. Support vectors and non-informative vectors.

[https://yadi.sk/i/baP7gbWXoqpTE Download]

'''Lecture 7. Kernel trick. '''

Application of kernel trick to SVM. Gaussian, polynomial kernels.

[https://yadi.sk/i/3W6A9FmZoqpU9 Download]

'''Lecture 8. Regression. '''

Linear regression and extensions: weighted regression, robust regression, different loss-functions, regression with non-linear features, locally-constant (Nadaraya-Watson) regression.

[https://yadi.sk/i/HSp51pmepjQBq Download]

'''Lecture 9. Boosting. '''

Forward stagewise additive modelling. AdaBoost. Gradient boosting.

[https://yadi.sk/i/3yTKLDcCpjLhG Download]

Additional materials:

[http://statweb.stanford.edu/~tibs/ElemStatLearn/ Friedman, Hastie, Tibshirani "The Elements of Statistical Learning"] - section 10: Boosting and additive trees.,

[http://www.recognition.mccme.ru/pub/RecognitionLab.html/slbook.pdf Мерков "Введение в методы статистического обучения"] - секция 4: Линейные комбинации распознавателей.

'''Lecture 10. Ensemble methods. '''

Motivation. Bias-variance tradeoff. Bagging, RandomForest, ExtraRandomTrees. Stacking.

[https://yadi.sk/i/8E-wZxIpq9Sgf Download]

'''Lectures 11, 12. Summary. '''

'''Lecture 13. Feature selection. '''

[https://yadi.sk/i/2ZLC3J6dr3iAc Download]

'''Lecture 14. Principal components analysis. '''

[https://yadi.sk/i/6DxoScrKrN3LK Download]

'''Lecture 14. Singular values decomposition. '''

[https://yadi.sk/i/zwJftdkUrN6Td Download] - updated pages 17,18,19.

'''Lecture 15. Working with text. '''

[https://yadi.sk/i/YqdRr-0erEXba Download]

'''Lecture 16. Neural networks. '''

[https://yadi.sk/i/9yMM7mkrrEXbc Download]

'''Lecture 17. Parametric distributions. '''

[https://yadi.sk/i/D33hdVXTrkxDN Download]

'''Lecture 18. Clustering. '''

[https://yadi.sk/i/3DJY4Oo7rkxEW Download]

'''Lecture 19. Mixture densities, EM-algorithm. ''' - updated.

[https://yadi.sk/i/22wLOL2krkxGL Download]

'''Lecture 20. Recommender systems. '''

[https://yadi.sk/i/mMhaZlqLrvjph Download]

'''Lecture 21. Kernel density estimation. '''

[https://yadi.sk/i/zF7ZPyTCsAjFH Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

[https://drive.google.com/open?id=0B7TWwiIrcJstdkJyam9rNHpEcDg Practical task 1], [https://drive.google.com/open?id=0B7TWwiIrcJstQldxcThZRnF3ZVk data]

Additional materials: [http://nbviewer.ipython.org/gist/anonymous/fba8bf7f1ad379df9d63 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

'''Seminar 2. kNN '''

[https://drive.google.com/open?id=0B7TWwiIrcJstbGRxREhGeDBNd3M Theoretical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstNXVXYTcydUMzUUk Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstTXUyYUstMmJNckk data]

Additional materials: [http://ogrisel.github.io/scikit-learn.org/sklearn-tutorial/auto_examples/tutorial/plot_knn_iris.html Visualization tutorial]

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstbmFmMmE5bWh2Y28/view?usp=sharing Theoretical task 3]

'''Seminar 4. Linear classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstc1J4SnRWTnlxZlE/view?usp=sharing Theoretical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstckZCN1pYcEo2NW8/view?usp=sharing Practical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstM3VSQTVrYWhqYk0/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstN0VOUV9PcDc1ZlE/view?usp=sharing diabetes dataset]

UPD: At all parts of practical task 4 you should use GD and SGD functions that you program at the fisrt part!

''Deadline for this practical task has been changed for some groups! Check it in the table!''

'''Seminar 5. Model evaluation '''

[https://drive.google.com/file/d/0B7TWwiIrcJstdTFST0Z4UkRoaEk/view?usp=sharing Theoretical task 5]

'''Seminar 6. Bayesian decision rule '''

[https://drive.google.com/file/d/0B7TWwiIrcJstWF9zcllDU01ZY2M/view?usp=sharing Theoretical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstSHRKcFlVcy1xd3M/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstVlhBdGZYMm94SHc/view?usp=sharing data]

Practical task 6 was completed: the last part was described in more details + there are two small corrections in the first part (they are in bold font). Read it carefully!

'''Deadline for this practical task has been changed for all groups! Check it in the table!'''

'''Seminar 7. SVM and kernel trick '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUGhEX2QxV1gycDA/view?usp=sharing Theoretical task 7]

Additional materials: [http://www.ccas.ru/voron/download/SVM.pdf Лекция К.В. Воронцова по SVM]

'''Seminar 8. Regression '''

[https://drive.google.com/file/d/0B7TWwiIrcJstNkE0dVJscGFSOEE/view?usp=sharing Theoretical task 8]

'''Seminar 9. Boosting '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVkx2Z0tGbi1yNFE/view?usp=sharing Practical task 9],
[https://drive.google.com/file/d/0B7TWwiIrcJstelRydjBUcUlleUk/view?usp=sharing data]

'''Seminar 10. Ensemble methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJstR28zVlN3OUZJTEU/view?usp=sharing Theoretical task 10]

''Problem 2: a small typo was corrected in the loss function formula.''

'''Seminar 11. Summary'''

'''Seminar 12. How to solve practical problems'''

[https://inclass.kaggle.com/c/cmc-msu-machine-learning-spring-2015-2016-dota-competition Dota Competition from the seminar], [https://drive.google.com/file/d/0B7TWwiIrcJstVnF0RVkzc1c1TVE/view?usp=sharing ipython notebook]

'''Seminar 13. Feature selection'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTUU4OGFnQzBPa3c/view?usp=sharing Theoretical task 13], [https://drive.google.com/file/d/0B7TWwiIrcJstM3JkWnhtQ2YzZms/view?usp=sharing Practical task 13]

''Practical task is completed.''

'''Seminar 14. Feature extraction'''

You can read about computing PCA through SVD at the end of [https://drive.google.com/file/d/0B7TWwiIrcJstWFFSOUI5aTRBM00/view?usp=sharing this paper].

'''Seminar 15. Neural networks'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxbk4yeXZyVi10dVE/view?usp=sharing Practical task 15],
[https://www.dropbox.com/s/r0u3xh5ybtstw9c/mnist.zip?dl=0 Data],
[https://drive.google.com/file/d/0B7TWwiIrcJstX1FzMWhQdmdGekk/view?usp=sharing Data in csv format],
[https://drive.google.com/file/d/0B0QWEJMlsxfRS2lWSkR5LVc2MzA/view?usp=sharing Censored training set],
[https://drive.google.com/file/d/0B0QWEJMlsxfRZmlHVHdlYWdQSTQ/view?usp=sharing Theoretical task 15]

Additional materials: [https://drive.google.com/file/d/0B7TWwiIrcJstd3pOWjcwUUNOaUk/view?usp=sharing Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/file/d/0B7TWwiIrcJstSGp0SzNTa1RJeTQ/view?usp=sharing PyBrain example from the seminar]

''New data files have been uploaded (there were some problems with reading old ones). Therefore deadline has been changed for some groups! Check it in the table!''

If you have MemoryError then read only part of training data from csv files (for example, 30000 objects). You can download '''censored training set''' (find link above) or use the following code:

mnist_train = np.loadtxt('mnist_train.csv', delimiter=',') 
train_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_train)): 
:train_data.appendLinked(mnist_train[i, 1:] / 255., int(mnist_train[i, 0])) 
train_data._convertToOneOfMany() 

mnist_test = np.loadtxt('mnist_test.csv', delimiter=',') 
test_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_test)): 
:test_data.appendLinked(mnist_test[i, 1:] / 255., int(mnist_test[i, 0])) 
test_data._convertToOneOfMany()

'''Seminar 16. Clustering'''

[https://drive.google.com/file/d/0B0QWEJMlsxfRRy1fd1RNVVhpMm8/view?usp=sharing Theoretical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstM1l5M3kwNDFZQlU/view?usp=sharing Practical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstZ2xIRU00dTB0OHc/view?usp=sharing parrots.jpg], [https://drive.google.com/file/d/0B7TWwiIrcJstb0RDc0RiQ1M3OXc/view?usp=sharing grass.jpg]

'''Seminar 17. Clustering, EM-algorithm'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxLWJjUEtCVVNFa00/view?usp=sharing Theoretical task 17]

'''Seminar 18. Recommender systems'''

[https://drive.google.com/file/d/0B7TWwiIrcJstWkt3Qk5Cb0FWTUk/view?usp=sharing Theoretical task 18],[https://drive.google.com/file/d/0B7TWwiIrcJstbW5Sd3hBMjNoNkE/view?usp=sharing Practical task 18], [https://www.dropbox.com/s/5as2k79dhajtw7n/data.zip?dl=0, data]

Additional materials: [https://www.semanticscholar.org/paper/Factorization-Machines-Rendle/2ef7d506b25731d0f3ec0c8f90b718b6e5bbd069/pdf Factorization Machines]

Columns in the data: 0 - user, 1 - item, 2 - rating, 3 - time (you don't need this one).

In the practical task you should train models on the train data (base) and evaluate on the test data.

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

All the deadlines can be found in the second tab [https://drive.google.com/open?id=1TQ97B8rqC7sUxTnCMKXoskgRPXO8rAyWoBWBezY58h4 here].

We have two deadlines for each assignments: normal and late. An assignment sent prior to normal deadline is scored with no penalty. The maximum score is penalized by 50% for assignments sent in between of the normal and the late deadline. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

Standard period for working on a homework assignment is 2 and 4 weeks (normal and late deadlines correspondingly) for practical assignments and 1 and 2 weeks for theoretical ones. The first practical assignment is an exception.

'''Deadline time:''' 23:59 of the day before seminar (Sunday for students attending Monday seminars and Wednesday for students that have seminars on Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group(subgroup)"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group(subgroup)"

''Example'': Practice 1 - Ivanov Ivan - 131(1)

If you want to address a particular teacher, mention his name in the subject.

''Example'': Question - Ivanov Ivan - 131(1) - Ekaterina

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in ipython notebook format, theoretical ones in pdf. Practical assignments must use '''Python 2.7'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Windows|Windows]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Mac_OS|Mac OS]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Linux | Linux]]

Data analysis (Software Engineering)

2016-06-20T23:16:45Z

Apogentus:

'''Scores and deadlines: [https://drive.google.com/open?id=1TQ97B8rqC7sUxTnCMKXoskgRPXO8rAyWoBWBezY58h4 here]'''

'''Class email:''' cshse.ml@gmail.com

'''[https://docs.google.com/forms/d/100_gMWQwp41zpHgKuf3fl3SpFSwNj6ggL13DtnxWEEw/viewform Anonymous overall course evaluation form] '''

'''Anonymous feedback form:''' [http://goo.gl/forms/CT3h4QaMeB here]
 

== Announcements ==

===Kaggle evaluation===
Kaggle evaluation [https://docs.google.com/spreadsheets/d/1h90zLpQ2Q8QD_xQLGJLY1xljnLGF175Re4fTMChHFgA/edit?usp=sharing is available here]. Please check that your work is in the list. Presentations were evaluated using [https://inclass.kaggle.com/c/hse-spring2016-stack-overflow/rules the rules of the competition]. In particular - I was expecting to see:
* that you tried different methods
* table with accuracy results of each method
* description how you tuned the parameters of your model (over which grid, with graphs showing accuracy change)
* data analysis and insights described with illustrative visualizations.
* in feature selection: quantitive results how each feature could be helpful/not helpful.

===Exam questions===
Exam questions are published and available [https://yadi.sk/i/Gi_T_R4AsD2aU here].

===Kaggle presentation requirements===
You should send presentations before June 3 (Friday) 23-59. Presentations should be sent to v.v.kitov@yandex.ru. The title should be "HSE kaggle presentation <team name>". On the title page of the presentation you should list all team participants. Presentation should be in pdf or ppt format and have all components listed in [https://inclass.kaggle.com/c/hse-spring2016-stack-overflow/rules competition rules]. Code in py or ipynb format should also be attached to the letter (it may consist of several files).

===Early exam===
On June 6th, 13-40 - 16-30 there will two lessons. They will cover:
1) a consultation before exam. Please read through all the material and come with your questions.
2) Presentations of top-3 kaggle teams with their solutions (15 minutes each). Teams with over 60 submissions are welcome to tell their findings in the data - what worked and what not (10 minutes each). Everybody else is also welcome (not obliged) to participate with short presentations (5-10 minutes) and tell interesting findings in the data and non-standard approaches that you tried (not necessarily successful).

For your convenience there will be a possibility to take exam in data analysis earlier - on June 6th at 16-40. To take exam earlier you need to request participation to e-mail v.v.kitov@yandex.ru. The number of participants is limited. Note that earlier exam will be the same as official exam and they mutually exclude each other, so you need to select in which exam to participate. Exam program will be available soon. Earlier exam schedule is proposed for your convenience - to give you the possibility to fully concentrate on preparation to data analysis exam.

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

==Kaggle competition==

[https://kaggle.com/join/hse_fcs Participate]

== Events outside the course ==

[https://it.mail.ru/announcements/36/?utm_campaign=newsletter&utm_medium=email&utm_source=newsletter_2742016df Universal recomendation system of mail.ru]

[https://www.youtube.com/channel/UCeq6ZIlvC9SVsfhfKnSvM9w Description of solutions to different competitions on Kaggle]

[http://ria.ru/science/20160514/1432666353.html Neural networks adapt videos to the painting style of famous artists.]

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''

[https://yadi.sk/i/RajIebEkmqgzw Download]

Additional materials: [https://yadi.sk/i/x2lrKdbVmr2bf The Field Guide to Data Science], [http://www.machinelearning.ru/wiki/images/f/fc/Voron-ML-Intro-slides.pdf Лекция К.В.Воронцова]

'''Lecture 2. K nearest neighbours method. '''

[https://yadi.sk/i/Od8HM9h-nUWob Download]

Additional materials: [http://www.machinelearning.ru/wiki/images/c/c3/Voron-ML-Metric-slides.pdf Лекция К.В.Воронцова], [http://arxiv.org/pdf/1306.6709v4.pdf Metric learning survey]

'''Lecture 3. Decision trees. '''

[https://yadi.sk/i/-vPl2vaBqXrt5 Download]

Additional materials: ''Webb, Copsey "Statistical Pattern Recognition", chapter 7.2.''

'''Lecture 4a. Model evaluation. '''

Binary quality measures. ROC curve, AUC.

[https://yadi.sk/i/7V7U_1QtnfYZQ Download]

Additional materials: ''Webb, Copsey "Statistical Pattern Recognition", chapter 9.''

'''Lecture 4b. Bayes minimum cost classification. '''

Case of general losses, common within-class losses and 0,1 losses. Gaussian classifier.

[https://yadi.sk/i/6jIvLiYMosuz5 Download]

'''Lecture 5. Linear classifiers. '''

Discriminant function. Invariance to monotonous transformations for them. Definition for multi-class and binary class cases.

[https://yadi.sk/i/0IQ6P3LDoqpRk Download]

Additional materials: [http://www.machinelearning.ru/wiki/images/5/53/Voron-ML-Lin-SG.pdf Лекции К.В.Воронцова по линейным методам классификации]

'''Lecture 6. Support vector machines. '''

Linear separable and linearly non-separable case. Equivalent definition with loss function. Support vectors and non-informative vectors.

[https://yadi.sk/i/baP7gbWXoqpTE Download]

'''Lecture 7. Kernel trick. '''

Application of kernel trick to SVM. Gaussian, polynomial kernels.

[https://yadi.sk/i/3W6A9FmZoqpU9 Download]

'''Lecture 8. Regression. '''

Linear regression and extensions: weighted regression, robust regression, different loss-functions, regression with non-linear features, locally-constant (Nadaraya-Watson) regression.

[https://yadi.sk/i/HSp51pmepjQBq Download]

'''Lecture 9. Boosting. '''

Forward stagewise additive modelling. AdaBoost. Gradient boosting.

[https://yadi.sk/i/3yTKLDcCpjLhG Download]

Additional materials:

[http://statweb.stanford.edu/~tibs/ElemStatLearn/ Friedman, Hastie, Tibshirani "The Elements of Statistical Learning"] - section 10: Boosting and additive trees.,

[http://www.recognition.mccme.ru/pub/RecognitionLab.html/slbook.pdf Мерков "Введение в методы статистического обучения"] - секция 4: Линейные комбинации распознавателей.

'''Lecture 10. Ensemble methods. '''

Motivation. Bias-variance tradeoff. Bagging, RandomForest, ExtraRandomTrees. Stacking.

[https://yadi.sk/i/8E-wZxIpq9Sgf Download]

'''Lectures 11, 12. Summary. '''

'''Lecture 13. Feature selection. '''

[https://yadi.sk/i/2ZLC3J6dr3iAc Download]

'''Lecture 14. Principal components analysis. '''

[https://yadi.sk/i/6DxoScrKrN3LK Download]

'''Lecture 14. Singular values decomposition. '''

[https://yadi.sk/i/zwJftdkUrN6Td Download] - updated pages 17,18,19.

'''Lecture 15. Working with text. '''

[https://yadi.sk/i/YqdRr-0erEXba Download]

'''Lecture 16. Neural networks. '''

[https://yadi.sk/i/9yMM7mkrrEXbc Download]

'''Lecture 17. Parametric distributions. '''

[https://yadi.sk/i/D33hdVXTrkxDN Download]

'''Lecture 18. Clustering. '''

[https://yadi.sk/i/3DJY4Oo7rkxEW Download]

'''Lecture 19. Mixture densities, EM-algorithm. '''

[https://yadi.sk/i/22wLOL2krkxGL Download]

'''Lecture 20. Recommender systems. '''

[https://yadi.sk/i/mMhaZlqLrvjph Download]

'''Lecture 21. Kernel density estimation. '''

[https://yadi.sk/i/zF7ZPyTCsAjFH Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

[https://drive.google.com/open?id=0B7TWwiIrcJstdkJyam9rNHpEcDg Practical task 1], [https://drive.google.com/open?id=0B7TWwiIrcJstQldxcThZRnF3ZVk data]

Additional materials: [http://nbviewer.ipython.org/gist/anonymous/fba8bf7f1ad379df9d63 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

'''Seminar 2. kNN '''

[https://drive.google.com/open?id=0B7TWwiIrcJstbGRxREhGeDBNd3M Theoretical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstNXVXYTcydUMzUUk Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstTXUyYUstMmJNckk data]

Additional materials: [http://ogrisel.github.io/scikit-learn.org/sklearn-tutorial/auto_examples/tutorial/plot_knn_iris.html Visualization tutorial]

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstbmFmMmE5bWh2Y28/view?usp=sharing Theoretical task 3]

'''Seminar 4. Linear classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstc1J4SnRWTnlxZlE/view?usp=sharing Theoretical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstckZCN1pYcEo2NW8/view?usp=sharing Practical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstM3VSQTVrYWhqYk0/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstN0VOUV9PcDc1ZlE/view?usp=sharing diabetes dataset]

UPD: At all parts of practical task 4 you should use GD and SGD functions that you program at the fisrt part!

''Deadline for this practical task has been changed for some groups! Check it in the table!''

'''Seminar 5. Model evaluation '''

[https://drive.google.com/file/d/0B7TWwiIrcJstdTFST0Z4UkRoaEk/view?usp=sharing Theoretical task 5]

'''Seminar 6. Bayesian decision rule '''

[https://drive.google.com/file/d/0B7TWwiIrcJstWF9zcllDU01ZY2M/view?usp=sharing Theoretical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstSHRKcFlVcy1xd3M/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstVlhBdGZYMm94SHc/view?usp=sharing data]

Practical task 6 was completed: the last part was described in more details + there are two small corrections in the first part (they are in bold font). Read it carefully!

'''Deadline for this practical task has been changed for all groups! Check it in the table!'''

'''Seminar 7. SVM and kernel trick '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUGhEX2QxV1gycDA/view?usp=sharing Theoretical task 7]

Additional materials: [http://www.ccas.ru/voron/download/SVM.pdf Лекция К.В. Воронцова по SVM]

'''Seminar 8. Regression '''

[https://drive.google.com/file/d/0B7TWwiIrcJstNkE0dVJscGFSOEE/view?usp=sharing Theoretical task 8]

'''Seminar 9. Boosting '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVkx2Z0tGbi1yNFE/view?usp=sharing Practical task 9],
[https://drive.google.com/file/d/0B7TWwiIrcJstelRydjBUcUlleUk/view?usp=sharing data]

'''Seminar 10. Ensemble methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJstR28zVlN3OUZJTEU/view?usp=sharing Theoretical task 10]

''Problem 2: a small typo was corrected in the loss function formula.''

'''Seminar 11. Summary'''

'''Seminar 12. How to solve practical problems'''

[https://inclass.kaggle.com/c/cmc-msu-machine-learning-spring-2015-2016-dota-competition Dota Competition from the seminar], [https://drive.google.com/file/d/0B7TWwiIrcJstVnF0RVkzc1c1TVE/view?usp=sharing ipython notebook]

'''Seminar 13. Feature selection'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTUU4OGFnQzBPa3c/view?usp=sharing Theoretical task 13], [https://drive.google.com/file/d/0B7TWwiIrcJstM3JkWnhtQ2YzZms/view?usp=sharing Practical task 13]

''Practical task is completed.''

'''Seminar 14. Feature extraction'''

You can read about computing PCA through SVD at the end of [https://drive.google.com/file/d/0B7TWwiIrcJstWFFSOUI5aTRBM00/view?usp=sharing this paper].

'''Seminar 15. Neural networks'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxbk4yeXZyVi10dVE/view?usp=sharing Practical task 15],
[https://www.dropbox.com/s/r0u3xh5ybtstw9c/mnist.zip?dl=0 Data],
[https://drive.google.com/file/d/0B7TWwiIrcJstX1FzMWhQdmdGekk/view?usp=sharing Data in csv format],
[https://drive.google.com/file/d/0B0QWEJMlsxfRS2lWSkR5LVc2MzA/view?usp=sharing Censored training set],
[https://drive.google.com/file/d/0B0QWEJMlsxfRZmlHVHdlYWdQSTQ/view?usp=sharing Theoretical task 15]

Additional materials: [https://drive.google.com/file/d/0B7TWwiIrcJstd3pOWjcwUUNOaUk/view?usp=sharing Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/file/d/0B7TWwiIrcJstSGp0SzNTa1RJeTQ/view?usp=sharing PyBrain example from the seminar]

''New data files have been uploaded (there were some problems with reading old ones). Therefore deadline has been changed for some groups! Check it in the table!''

If you have MemoryError then read only part of training data from csv files (for example, 30000 objects). You can download '''censored training set''' (find link above) or use the following code:

mnist_train = np.loadtxt('mnist_train.csv', delimiter=',') 
train_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_train)): 
:train_data.appendLinked(mnist_train[i, 1:] / 255., int(mnist_train[i, 0])) 
train_data._convertToOneOfMany() 

mnist_test = np.loadtxt('mnist_test.csv', delimiter=',') 
test_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_test)): 
:test_data.appendLinked(mnist_test[i, 1:] / 255., int(mnist_test[i, 0])) 
test_data._convertToOneOfMany()

'''Seminar 16. Clustering'''

[https://drive.google.com/file/d/0B0QWEJMlsxfRRy1fd1RNVVhpMm8/view?usp=sharing Theoretical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstM1l5M3kwNDFZQlU/view?usp=sharing Practical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstZ2xIRU00dTB0OHc/view?usp=sharing parrots.jpg], [https://drive.google.com/file/d/0B7TWwiIrcJstb0RDc0RiQ1M3OXc/view?usp=sharing grass.jpg]

'''Seminar 17. Clustering, EM-algorithm'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxLWJjUEtCVVNFa00/view?usp=sharing Theoretical task 17]

'''Seminar 18. Recommender systems'''

[https://drive.google.com/file/d/0B7TWwiIrcJstWkt3Qk5Cb0FWTUk/view?usp=sharing Theoretical task 18],[https://drive.google.com/file/d/0B7TWwiIrcJstbW5Sd3hBMjNoNkE/view?usp=sharing Practical task 18], [https://www.dropbox.com/s/5as2k79dhajtw7n/data.zip?dl=0, data]

Additional materials: [https://www.semanticscholar.org/paper/Factorization-Machines-Rendle/2ef7d506b25731d0f3ec0c8f90b718b6e5bbd069/pdf Factorization Machines]

Columns in the data: 0 - user, 1 - item, 2 - rating, 3 - time (you don't need this one).

In the practical task you should train models on the train data (base) and evaluate on the test data.

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

All the deadlines can be found in the second tab [https://drive.google.com/open?id=1TQ97B8rqC7sUxTnCMKXoskgRPXO8rAyWoBWBezY58h4 here].

We have two deadlines for each assignments: normal and late. An assignment sent prior to normal deadline is scored with no penalty. The maximum score is penalized by 50% for assignments sent in between of the normal and the late deadline. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

Standard period for working on a homework assignment is 2 and 4 weeks (normal and late deadlines correspondingly) for practical assignments and 1 and 2 weeks for theoretical ones. The first practical assignment is an exception.

'''Deadline time:''' 23:59 of the day before seminar (Sunday for students attending Monday seminars and Wednesday for students that have seminars on Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group(subgroup)"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group(subgroup)"

''Example'': Practice 1 - Ivanov Ivan - 131(1)

If you want to address a particular teacher, mention his name in the subject.

''Example'': Question - Ivanov Ivan - 131(1) - Ekaterina

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in ipython notebook format, theoretical ones in pdf. Practical assignments must use '''Python 2.7'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Windows|Windows]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Mac_OS|Mac OS]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Linux | Linux]]

Data analysis (Software Engineering)

2016-06-20T21:06:02Z

Apogentus:

'''Scores and deadlines: [https://drive.google.com/open?id=1TQ97B8rqC7sUxTnCMKXoskgRPXO8rAyWoBWBezY58h4 here]'''

'''Class email:''' cshse.ml@gmail.com

'''[https://docs.google.com/forms/d/100_gMWQwp41zpHgKuf3fl3SpFSwNj6ggL13DtnxWEEw/viewform Anonymous overall course evaluation form] '''

'''Anonymous feedback form:''' [http://goo.gl/forms/CT3h4QaMeB here]
 

== Announcements ==

===Kaggle evaluation===
Kaggle evaluation [https://docs.google.com/spreadsheets/d/1h90zLpQ2Q8QD_xQLGJLY1xljnLGF175Re4fTMChHFgA/edit?usp=sharing is available here]. Please check that your work is in the list. Presentations were evaluated using [https://inclass.kaggle.com/c/hse-spring2016-stack-overflow/rules the rules of the competition]. In particular - I was expecting to see:
* that you tried different methods
* table with accuracy results of each method
* description how you tuned the parameters of your model (over which grid, with graphs showing accuracy change)
* data analysis and insights described with illustrative visualizations.
* in feature selection: quantitive results how each feature could be helpful/not helpful.

===Exam questions===
Exam questions are published and available [https://yadi.sk/i/Gi_T_R4AsD2aU here].

===Kaggle presentation requirements===
You should send presentations before June 3 (Friday) 23-59. Presentations should be sent to v.v.kitov@yandex.ru. The title should be "HSE kaggle presentation <team name>". On the title page of the presentation you should list all team participants. Presentation should be in pdf or ppt format and have all components listed in [https://inclass.kaggle.com/c/hse-spring2016-stack-overflow/rules competition rules]. Code in py or ipynb format should also be attached to the letter (it may consist of several files).

===Early exam===
On June 6th, 13-40 - 16-30 there will two lessons. They will cover:
1) a consultation before exam. Please read through all the material and come with your questions.
2) Presentations of top-3 kaggle teams with their solutions (15 minutes each). Teams with over 60 submissions are welcome to tell their findings in the data - what worked and what not (10 minutes each). Everybody else is also welcome (not obliged) to participate with short presentations (5-10 minutes) and tell interesting findings in the data and non-standard approaches that you tried (not necessarily successful).

For your convenience there will be a possibility to take exam in data analysis earlier - on June 6th at 16-40. To take exam earlier you need to request participation to e-mail v.v.kitov@yandex.ru. The number of participants is limited. Note that earlier exam will be the same as official exam and they mutually exclude each other, so you need to select in which exam to participate. Exam program will be available soon. Earlier exam schedule is proposed for your convenience - to give you the possibility to fully concentrate on preparation to data analysis exam.

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

==Kaggle competition==

[https://kaggle.com/join/hse_fcs Participate]

== Events outside the course ==

[https://it.mail.ru/announcements/36/?utm_campaign=newsletter&utm_medium=email&utm_source=newsletter_2742016df Universal recomendation system of mail.ru]

[https://www.youtube.com/channel/UCeq6ZIlvC9SVsfhfKnSvM9w Description of solutions to different competitions on Kaggle]

[http://ria.ru/science/20160514/1432666353.html Neural networks adapt videos to the painting style of famous artists.]

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''

[https://yadi.sk/i/RajIebEkmqgzw Download]

Additional materials: [https://yadi.sk/i/x2lrKdbVmr2bf The Field Guide to Data Science], [http://www.machinelearning.ru/wiki/images/f/fc/Voron-ML-Intro-slides.pdf Лекция К.В.Воронцова]

'''Lecture 2. K nearest neighbours method. '''

[https://yadi.sk/i/Od8HM9h-nUWob Download]

Additional materials: [http://www.machinelearning.ru/wiki/images/c/c3/Voron-ML-Metric-slides.pdf Лекция К.В.Воронцова], [http://arxiv.org/pdf/1306.6709v4.pdf Metric learning survey]

'''Lecture 3. Decision trees. '''

[https://yadi.sk/i/-vPl2vaBqXrt5 Download]

Additional materials: ''Webb, Copsey "Statistical Pattern Recognition", chapter 7.2.''

'''Lecture 4a. Model evaluation. '''

Binary quality measures. ROC curve, AUC.

[https://yadi.sk/i/7V7U_1QtnfYZQ Download]

Additional materials: ''Webb, Copsey "Statistical Pattern Recognition", chapter 9.''

'''Lecture 4b. Bayes minimum cost classification. '''

Case of general losses, common within-class losses and 0,1 losses. Gaussian classifier.

[https://yadi.sk/i/6jIvLiYMosuz5 Download]

'''Lecture 5. Linear classifiers. '''

Discriminant function. Invariance to monotonous transformations for them. Definition for multi-class and binary class cases.

[https://yadi.sk/i/0IQ6P3LDoqpRk Download]

Additional materials: [http://www.machinelearning.ru/wiki/images/5/53/Voron-ML-Lin-SG.pdf Лекции К.В.Воронцова по линейным методам классификации]

'''Lecture 6. Support vector machines. '''

Linear separable and linearly non-separable case. Equivalent definition with loss function. Support vectors and non-informative vectors.

[https://yadi.sk/i/baP7gbWXoqpTE Download]

'''Lecture 7. Kernel trick. '''

Application of kernel trick to SVM. Gaussian, polynomial kernels.

[https://yadi.sk/i/3W6A9FmZoqpU9 Download]

'''Lecture 8. Regression. '''

Linear regression and extensions: weighted regression, robust regression, different loss-functions, regression with non-linear features, locally-constant (Nadaraya-Watson) regression.

[https://yadi.sk/i/HSp51pmepjQBq Download]

'''Lecture 9. Boosting. '''

Forward stagewise additive modelling. AdaBoost. Gradient boosting.

[https://yadi.sk/i/3yTKLDcCpjLhG Download]

Additional materials:

[http://statweb.stanford.edu/~tibs/ElemStatLearn/ Friedman, Hastie, Tibshirani "The Elements of Statistical Learning"] - section 10: Boosting and additive trees.,

[http://www.recognition.mccme.ru/pub/RecognitionLab.html/slbook.pdf Мерков "Введение в методы статистического обучения"] - секция 4: Линейные комбинации распознавателей.

'''Lecture 10. Ensemble methods. '''

Motivation. Bias-variance tradeoff. Bagging, RandomForest, ExtraRandomTrees. Stacking.

[https://yadi.sk/i/8E-wZxIpq9Sgf Download]

'''Lectures 11, 12. Summary. '''

'''Lecture 13. Feature selection. '''

[https://yadi.sk/i/2ZLC3J6dr3iAc Download]

'''Lecture 14. Principal components analysis. '''

[https://yadi.sk/i/6DxoScrKrN3LK Download]

'''Lecture 14. Singular values decomposition. '''

[https://yadi.sk/i/zwJftdkUrN6Td Download]

'''Lecture 15. Working with text. '''

[https://yadi.sk/i/YqdRr-0erEXba Download]

'''Lecture 16. Neural networks. '''

[https://yadi.sk/i/9yMM7mkrrEXbc Download]

'''Lecture 17. Parametric distributions. '''

[https://yadi.sk/i/D33hdVXTrkxDN Download]

'''Lecture 18. Clustering. '''

[https://yadi.sk/i/3DJY4Oo7rkxEW Download]

'''Lecture 19. Mixture densities, EM-algorithm. '''

[https://yadi.sk/i/22wLOL2krkxGL Download]

'''Lecture 20. Recommender systems. '''

[https://yadi.sk/i/mMhaZlqLrvjph Download]

'''Lecture 21. Kernel density estimation. '''

[https://yadi.sk/i/zF7ZPyTCsAjFH Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

[https://drive.google.com/open?id=0B7TWwiIrcJstdkJyam9rNHpEcDg Practical task 1], [https://drive.google.com/open?id=0B7TWwiIrcJstQldxcThZRnF3ZVk data]

Additional materials: [http://nbviewer.ipython.org/gist/anonymous/fba8bf7f1ad379df9d63 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

'''Seminar 2. kNN '''

[https://drive.google.com/open?id=0B7TWwiIrcJstbGRxREhGeDBNd3M Theoretical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstNXVXYTcydUMzUUk Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstTXUyYUstMmJNckk data]

Additional materials: [http://ogrisel.github.io/scikit-learn.org/sklearn-tutorial/auto_examples/tutorial/plot_knn_iris.html Visualization tutorial]

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstbmFmMmE5bWh2Y28/view?usp=sharing Theoretical task 3]

'''Seminar 4. Linear classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstc1J4SnRWTnlxZlE/view?usp=sharing Theoretical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstckZCN1pYcEo2NW8/view?usp=sharing Practical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstM3VSQTVrYWhqYk0/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstN0VOUV9PcDc1ZlE/view?usp=sharing diabetes dataset]

UPD: At all parts of practical task 4 you should use GD and SGD functions that you program at the fisrt part!

''Deadline for this practical task has been changed for some groups! Check it in the table!''

'''Seminar 5. Model evaluation '''

[https://drive.google.com/file/d/0B7TWwiIrcJstdTFST0Z4UkRoaEk/view?usp=sharing Theoretical task 5]

'''Seminar 6. Bayesian decision rule '''

[https://drive.google.com/file/d/0B7TWwiIrcJstWF9zcllDU01ZY2M/view?usp=sharing Theoretical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstSHRKcFlVcy1xd3M/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstVlhBdGZYMm94SHc/view?usp=sharing data]

Practical task 6 was completed: the last part was described in more details + there are two small corrections in the first part (they are in bold font). Read it carefully!

'''Deadline for this practical task has been changed for all groups! Check it in the table!'''

'''Seminar 7. SVM and kernel trick '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUGhEX2QxV1gycDA/view?usp=sharing Theoretical task 7]

Additional materials: [http://www.ccas.ru/voron/download/SVM.pdf Лекция К.В. Воронцова по SVM]

'''Seminar 8. Regression '''

[https://drive.google.com/file/d/0B7TWwiIrcJstNkE0dVJscGFSOEE/view?usp=sharing Theoretical task 8]

'''Seminar 9. Boosting '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVkx2Z0tGbi1yNFE/view?usp=sharing Practical task 9],
[https://drive.google.com/file/d/0B7TWwiIrcJstelRydjBUcUlleUk/view?usp=sharing data]

'''Seminar 10. Ensemble methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJstR28zVlN3OUZJTEU/view?usp=sharing Theoretical task 10]

''Problem 2: a small typo was corrected in the loss function formula.''

'''Seminar 11. Summary'''

'''Seminar 12. How to solve practical problems'''

[https://inclass.kaggle.com/c/cmc-msu-machine-learning-spring-2015-2016-dota-competition Dota Competition from the seminar], [https://drive.google.com/file/d/0B7TWwiIrcJstVnF0RVkzc1c1TVE/view?usp=sharing ipython notebook]

'''Seminar 13. Feature selection'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTUU4OGFnQzBPa3c/view?usp=sharing Theoretical task 13], [https://drive.google.com/file/d/0B7TWwiIrcJstM3JkWnhtQ2YzZms/view?usp=sharing Practical task 13]

''Practical task is completed.''

'''Seminar 14. Feature extraction'''

You can read about computing PCA through SVD at the end of [https://drive.google.com/file/d/0B7TWwiIrcJstWFFSOUI5aTRBM00/view?usp=sharing this paper].

'''Seminar 15. Neural networks'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxbk4yeXZyVi10dVE/view?usp=sharing Practical task 15],
[https://www.dropbox.com/s/r0u3xh5ybtstw9c/mnist.zip?dl=0 Data],
[https://drive.google.com/file/d/0B7TWwiIrcJstX1FzMWhQdmdGekk/view?usp=sharing Data in csv format],
[https://drive.google.com/file/d/0B0QWEJMlsxfRS2lWSkR5LVc2MzA/view?usp=sharing Censored training set],
[https://drive.google.com/file/d/0B0QWEJMlsxfRZmlHVHdlYWdQSTQ/view?usp=sharing Theoretical task 15]

Additional materials: [https://drive.google.com/file/d/0B7TWwiIrcJstd3pOWjcwUUNOaUk/view?usp=sharing Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/file/d/0B7TWwiIrcJstSGp0SzNTa1RJeTQ/view?usp=sharing PyBrain example from the seminar]

''New data files have been uploaded (there were some problems with reading old ones). Therefore deadline has been changed for some groups! Check it in the table!''

If you have MemoryError then read only part of training data from csv files (for example, 30000 objects). You can download '''censored training set''' (find link above) or use the following code:

mnist_train = np.loadtxt('mnist_train.csv', delimiter=',') 
train_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_train)): 
:train_data.appendLinked(mnist_train[i, 1:] / 255., int(mnist_train[i, 0])) 
train_data._convertToOneOfMany() 

mnist_test = np.loadtxt('mnist_test.csv', delimiter=',') 
test_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_test)): 
:test_data.appendLinked(mnist_test[i, 1:] / 255., int(mnist_test[i, 0])) 
test_data._convertToOneOfMany()

'''Seminar 16. Clustering'''

[https://drive.google.com/file/d/0B0QWEJMlsxfRRy1fd1RNVVhpMm8/view?usp=sharing Theoretical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstM1l5M3kwNDFZQlU/view?usp=sharing Practical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstZ2xIRU00dTB0OHc/view?usp=sharing parrots.jpg], [https://drive.google.com/file/d/0B7TWwiIrcJstb0RDc0RiQ1M3OXc/view?usp=sharing grass.jpg]

'''Seminar 17. Clustering, EM-algorithm'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxLWJjUEtCVVNFa00/view?usp=sharing Theoretical task 17]

'''Seminar 18. Recommender systems'''

[https://drive.google.com/file/d/0B7TWwiIrcJstWkt3Qk5Cb0FWTUk/view?usp=sharing Theoretical task 18],[https://drive.google.com/file/d/0B7TWwiIrcJstbW5Sd3hBMjNoNkE/view?usp=sharing Practical task 18], [https://www.dropbox.com/s/5as2k79dhajtw7n/data.zip?dl=0, data]

Additional materials: [https://www.semanticscholar.org/paper/Factorization-Machines-Rendle/2ef7d506b25731d0f3ec0c8f90b718b6e5bbd069/pdf Factorization Machines]

Columns in the data: 0 - user, 1 - item, 2 - rating, 3 - time (you don't need this one).

In the practical task you should train models on the train data (base) and evaluate on the test data.

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

All the deadlines can be found in the second tab [https://drive.google.com/open?id=1TQ97B8rqC7sUxTnCMKXoskgRPXO8rAyWoBWBezY58h4 here].

We have two deadlines for each assignments: normal and late. An assignment sent prior to normal deadline is scored with no penalty. The maximum score is penalized by 50% for assignments sent in between of the normal and the late deadline. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

Standard period for working on a homework assignment is 2 and 4 weeks (normal and late deadlines correspondingly) for practical assignments and 1 and 2 weeks for theoretical ones. The first practical assignment is an exception.

'''Deadline time:''' 23:59 of the day before seminar (Sunday for students attending Monday seminars and Wednesday for students that have seminars on Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group(subgroup)"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group(subgroup)"

''Example'': Practice 1 - Ivanov Ivan - 131(1)

If you want to address a particular teacher, mention his name in the subject.

''Example'': Question - Ivanov Ivan - 131(1) - Ekaterina

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in ipython notebook format, theoretical ones in pdf. Practical assignments must use '''Python 2.7'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Windows|Windows]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Mac_OS|Mac OS]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Linux | Linux]]

Data analysis (Software Engineering)

2016-06-08T19:27:13Z

Apogentus: /* Kaggle evaluation */

'''Scores and deadlines: [https://drive.google.com/open?id=1TQ97B8rqC7sUxTnCMKXoskgRPXO8rAyWoBWBezY58h4 here]'''

'''Class email:''' cshse.ml@gmail.com

'''Anonymous feedback form:''' [http://goo.gl/forms/CT3h4QaMeB here]
 

== Announcements ==

===Kaggle evaluation===
Kaggle evaluation [https://docs.google.com/spreadsheets/d/1h90zLpQ2Q8QD_xQLGJLY1xljnLGF175Re4fTMChHFgA/edit?usp=sharing is available here]. Please check that your work is in the list. Presentations were evaluated using [https://inclass.kaggle.com/c/hse-spring2016-stack-overflow/rules the rules of the competition]. In particular - I was expecting to see:
* that you tried different methods
* table with accuracy results of each method
* description how you tuned the parameters of your model (over which grid, with graphs showing accuracy change)
* data analysis and insights described with illustrative visualizations.
* in feature selection: quantitive results how each feature could be helpful/not helpful.

===Exam questions===
Exam questions are published and available [https://yadi.sk/i/Gi_T_R4AsD2aU here].

===Kaggle presentation requirements===
You should send presentations before June 3 (Friday) 23-59. Presentations should be sent to v.v.kitov@yandex.ru. The title should be "HSE kaggle presentation <team name>". On the title page of the presentation you should list all team participants. Presentation should be in pdf or ppt format and have all components listed in [https://inclass.kaggle.com/c/hse-spring2016-stack-overflow/rules competition rules]. Code in py or ipynb format should also be attached to the letter (it may consist of several files).

===Early exam===
On June 6th, 13-40 - 16-30 there will two lessons. They will cover:
1) a consultation before exam. Please read through all the material and come with your questions.
2) Presentations of top-3 kaggle teams with their solutions (15 minutes each). Teams with over 60 submissions are welcome to tell their findings in the data - what worked and what not (10 minutes each). Everybody else is also welcome (not obliged) to participate with short presentations (5-10 minutes) and tell interesting findings in the data and non-standard approaches that you tried (not necessarily successful).

For your convenience there will be a possibility to take exam in data analysis earlier - on June 6th at 16-40. To take exam earlier you need to request participation to e-mail v.v.kitov@yandex.ru. The number of participants is limited. Note that earlier exam will be the same as official exam and they mutually exclude each other, so you need to select in which exam to participate. Exam program will be available soon. Earlier exam schedule is proposed for your convenience - to give you the possibility to fully concentrate on preparation to data analysis exam.

== Course description ==
In this class we consider the main problems of data mining and machine learning: classification, clustering, regression, dimensionality reduction, ranking, collaborative filtering. We will also study mathematical methods and concepts which data analysis is based on as well as formal assumptions behind them and various aspects of their implementation.

A significant attention is given to practical skills of data analysis that will be developed on seminars by studying the Python programming language and relevant libraries for scientific computing.

The knowledge of linear algebra, real analysis and probability theory is required.

'''The class consists of:'''
# Lectures and seminars
# Practical and theoretical homework assignments
# A machine learning competition (more information will be available later)
# Midterm theoretical colloquium
# Final exam

==Kaggle competition==

[https://kaggle.com/join/hse_fcs Participate]

== Events outside the course ==

[https://it.mail.ru/announcements/36/?utm_campaign=newsletter&utm_medium=email&utm_source=newsletter_2742016df Universal recomendation system of mail.ru]

[https://www.youtube.com/channel/UCeq6ZIlvC9SVsfhfKnSvM9w Description of solutions to different competitions on Kaggle]

[http://ria.ru/science/20160514/1432666353.html Neural networks adapt videos to the painting style of famous artists.]

== Syllabus ==

# Introduction to machine learning.
# K-nearest neighbours classification and regression. Extensions. Optimization techniques.
# Decision tree methods.
# Bayesian decision theory. Model evaluation:
# Linear classification methods. Adding regularization to linear methods.
# Regression.
# Kernel generalization of standard methods.
# Neural networks.
# Ensemble methods: bagging, boosting, etc.
# Feature selection.
# Feature extraction
# EM algorithm. Density estimation using mixtures.
# Clustering
# Collaborative filtering
# Ranking

== Lecture materials ==

'''Lecture 1. Introduction to data science and machine learning. '''

[https://yadi.sk/i/RajIebEkmqgzw Download]

Additional materials: [https://yadi.sk/i/x2lrKdbVmr2bf The Field Guide to Data Science], [http://www.machinelearning.ru/wiki/images/f/fc/Voron-ML-Intro-slides.pdf Лекция К.В.Воронцова]

'''Lecture 2. K nearest neighbours method. '''

[https://yadi.sk/i/Od8HM9h-nUWob Download]

Additional materials: [http://www.machinelearning.ru/wiki/images/c/c3/Voron-ML-Metric-slides.pdf Лекция К.В.Воронцова], [http://arxiv.org/pdf/1306.6709v4.pdf Metric learning survey]

'''Lecture 3. Decision trees. '''

[https://yadi.sk/i/-vPl2vaBqXrt5 Download]

Additional materials: ''Webb, Copsey "Statistical Pattern Recognition", chapter 7.2.''

'''Lecture 4a. Model evaluation. '''

Binary quality measures. ROC curve, AUC.

[https://yadi.sk/i/7V7U_1QtnfYZQ Download]

Additional materials: ''Webb, Copsey "Statistical Pattern Recognition", chapter 9.''

'''Lecture 4b. Bayes minimum cost classification. '''

Case of general losses, common within-class losses and 0,1 losses. Gaussian classifier.

[https://yadi.sk/i/6jIvLiYMosuz5 Download]

'''Lecture 5. Linear classifiers. '''

Discriminant function. Invariance to monotonous transformations for them. Definition for multi-class and binary class cases.

[https://yadi.sk/i/0IQ6P3LDoqpRk Download]

Additional materials: [http://www.machinelearning.ru/wiki/images/5/53/Voron-ML-Lin-SG.pdf Лекции К.В.Воронцова по линейным методам классификации]

'''Lecture 6. Support vector machines. '''

Linear separable and linearly non-separable case. Equivalent definition with loss function. Support vectors and non-informative vectors.

[https://yadi.sk/i/baP7gbWXoqpTE Download]

'''Lecture 7. Kernel trick. '''

Application of kernel trick to SVM. Gaussian, polynomial kernels.

[https://yadi.sk/i/3W6A9FmZoqpU9 Download]

'''Lecture 8. Regression. '''

Linear regression and extensions: weighted regression, robust regression, different loss-functions, regression with non-linear features, locally-constant (Nadaraya-Watson) regression.

[https://yadi.sk/i/HSp51pmepjQBq Download]

'''Lecture 9. Boosting. '''

Forward stagewise additive modelling. AdaBoost. Gradient boosting.

[https://yadi.sk/i/3yTKLDcCpjLhG Download]

Additional materials:

[http://statweb.stanford.edu/~tibs/ElemStatLearn/ Friedman, Hastie, Tibshirani "The Elements of Statistical Learning"] - section 10: Boosting and additive trees.,

[http://www.recognition.mccme.ru/pub/RecognitionLab.html/slbook.pdf Мерков "Введение в методы статистического обучения"] - секция 4: Линейные комбинации распознавателей.

'''Lecture 10. Ensemble methods. '''

Motivation. Bias-variance tradeoff. Bagging, RandomForest, ExtraRandomTrees. Stacking.

[https://yadi.sk/i/8E-wZxIpq9Sgf Download]

'''Lectures 11, 12. Summary. '''

'''Lecture 13. Feature selection. '''

[https://yadi.sk/i/2ZLC3J6dr3iAc Download]

'''Lecture 14. Principal components analysis. '''

[https://yadi.sk/i/6DxoScrKrN3LK Download]

'''Lecture 14. Singular values decomposition. '''

[https://yadi.sk/i/zwJftdkUrN6Td Download]

'''Lecture 15. Working with text. '''

[https://yadi.sk/i/YqdRr-0erEXba Download]

'''Lecture 16. Neural networks. '''

[https://yadi.sk/i/9yMM7mkrrEXbc Download]

'''Lecture 17. Parametric distributions. '''

[https://yadi.sk/i/D33hdVXTrkxDN Download]

'''Lecture 18. Clustering. '''

[https://yadi.sk/i/3DJY4Oo7rkxEW Download]

'''Lecture 19. Mixture densities, EM-algorithm. '''

[https://yadi.sk/i/22wLOL2krkxGL Download]

'''Lecture 20. Recommender systems. '''

[https://yadi.sk/i/mMhaZlqLrvjph Download]

'''Lecture 21. Kernel density estimation. '''

[https://yadi.sk/i/zF7ZPyTCsAjFH Download]

== Seminars ==

'''Seminar 1. Introduction to Data Analysis in Python '''

[https://drive.google.com/open?id=0B7TWwiIrcJstdkJyam9rNHpEcDg Practical task 1], [https://drive.google.com/open?id=0B7TWwiIrcJstQldxcThZRnF3ZVk data]

Additional materials: [http://nbviewer.ipython.org/gist/anonymous/fba8bf7f1ad379df9d63 1], [https://drive.google.com/open?id=0B7TWwiIrcJstRzVRSlRFcEl3VGM 2]

'''Seminar 2. kNN '''

[https://drive.google.com/open?id=0B7TWwiIrcJstbGRxREhGeDBNd3M Theoretical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstNXVXYTcydUMzUUk Practical task 2], [https://drive.google.com/open?id=0B7TWwiIrcJstTXUyYUstMmJNckk data]

Additional materials: [http://ogrisel.github.io/scikit-learn.org/sklearn-tutorial/auto_examples/tutorial/plot_knn_iris.html Visualization tutorial]

'''Seminar 3. Decision trees '''

[https://drive.google.com/file/d/0B7TWwiIrcJstbmFmMmE5bWh2Y28/view?usp=sharing Theoretical task 3]

'''Seminar 4. Linear classifiers '''

[https://drive.google.com/file/d/0B7TWwiIrcJstc1J4SnRWTnlxZlE/view?usp=sharing Theoretical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstckZCN1pYcEo2NW8/view?usp=sharing Practical task 4], [https://drive.google.com/file/d/0B7TWwiIrcJstM3VSQTVrYWhqYk0/view?usp=sharing first dataset], [https://drive.google.com/file/d/0B7TWwiIrcJstN0VOUV9PcDc1ZlE/view?usp=sharing diabetes dataset]

UPD: At all parts of practical task 4 you should use GD and SGD functions that you program at the fisrt part!

''Deadline for this practical task has been changed for some groups! Check it in the table!''

'''Seminar 5. Model evaluation '''

[https://drive.google.com/file/d/0B7TWwiIrcJstdTFST0Z4UkRoaEk/view?usp=sharing Theoretical task 5]

'''Seminar 6. Bayesian decision rule '''

[https://drive.google.com/file/d/0B7TWwiIrcJstWF9zcllDU01ZY2M/view?usp=sharing Theoretical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstSHRKcFlVcy1xd3M/view?usp=sharing Practical task 6], [https://drive.google.com/file/d/0B7TWwiIrcJstVlhBdGZYMm94SHc/view?usp=sharing data]

Practical task 6 was completed: the last part was described in more details + there are two small corrections in the first part (they are in bold font). Read it carefully!

'''Deadline for this practical task has been changed for all groups! Check it in the table!'''

'''Seminar 7. SVM and kernel trick '''

[https://drive.google.com/file/d/0B7TWwiIrcJstUGhEX2QxV1gycDA/view?usp=sharing Theoretical task 7]

Additional materials: [http://www.ccas.ru/voron/download/SVM.pdf Лекция К.В. Воронцова по SVM]

'''Seminar 8. Regression '''

[https://drive.google.com/file/d/0B7TWwiIrcJstNkE0dVJscGFSOEE/view?usp=sharing Theoretical task 8]

'''Seminar 9. Boosting '''

[https://drive.google.com/file/d/0B7TWwiIrcJstVkx2Z0tGbi1yNFE/view?usp=sharing Practical task 9],
[https://drive.google.com/file/d/0B7TWwiIrcJstelRydjBUcUlleUk/view?usp=sharing data]

'''Seminar 10. Ensemble methods '''

[https://drive.google.com/file/d/0B7TWwiIrcJstR28zVlN3OUZJTEU/view?usp=sharing Theoretical task 10]

''Problem 2: a small typo was corrected in the loss function formula.''

'''Seminar 11. Summary'''

'''Seminar 12. How to solve practical problems'''

[https://inclass.kaggle.com/c/cmc-msu-machine-learning-spring-2015-2016-dota-competition Dota Competition from the seminar], [https://drive.google.com/file/d/0B7TWwiIrcJstVnF0RVkzc1c1TVE/view?usp=sharing ipython notebook]

'''Seminar 13. Feature selection'''

[https://drive.google.com/file/d/0B7TWwiIrcJstTUU4OGFnQzBPa3c/view?usp=sharing Theoretical task 13], [https://drive.google.com/file/d/0B7TWwiIrcJstM3JkWnhtQ2YzZms/view?usp=sharing Practical task 13]

''Practical task is completed.''

'''Seminar 14. Feature extraction'''

You can read about computing PCA through SVD at the end of [https://drive.google.com/file/d/0B7TWwiIrcJstWFFSOUI5aTRBM00/view?usp=sharing this paper].

'''Seminar 15. Neural networks'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxbk4yeXZyVi10dVE/view?usp=sharing Practical task 15],
[https://www.dropbox.com/s/r0u3xh5ybtstw9c/mnist.zip?dl=0 Data],
[https://drive.google.com/file/d/0B7TWwiIrcJstX1FzMWhQdmdGekk/view?usp=sharing Data in csv format],
[https://drive.google.com/file/d/0B0QWEJMlsxfRS2lWSkR5LVc2MzA/view?usp=sharing Censored training set],
[https://drive.google.com/file/d/0B0QWEJMlsxfRZmlHVHdlYWdQSTQ/view?usp=sharing Theoretical task 15]

Additional materials: [https://drive.google.com/file/d/0B7TWwiIrcJstd3pOWjcwUUNOaUk/view?usp=sharing Backpropagation], [http://pybrain.org/docs/ PyBrain’s documentation], [https://drive.google.com/file/d/0B7TWwiIrcJstSGp0SzNTa1RJeTQ/view?usp=sharing PyBrain example from the seminar]

''New data files have been uploaded (there were some problems with reading old ones). Therefore deadline has been changed for some groups! Check it in the table!''

If you have MemoryError then read only part of training data from csv files (for example, 30000 objects). You can download '''censored training set''' (find link above) or use the following code:

mnist_train = np.loadtxt('mnist_train.csv', delimiter=',') 
train_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_train)): 
:train_data.appendLinked(mnist_train[i, 1:] / 255., int(mnist_train[i, 0])) 
train_data._convertToOneOfMany() 

mnist_test = np.loadtxt('mnist_test.csv', delimiter=',') 
test_data = ClassificationDataSet(28*28, nb_classes=10) 
for i in xrange(len(mnist_test)): 
:test_data.appendLinked(mnist_test[i, 1:] / 255., int(mnist_test[i, 0])) 
test_data._convertToOneOfMany()

'''Seminar 16. Clustering'''

[https://drive.google.com/file/d/0B0QWEJMlsxfRRy1fd1RNVVhpMm8/view?usp=sharing Theoretical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstM1l5M3kwNDFZQlU/view?usp=sharing Practical task 16], [https://drive.google.com/file/d/0B7TWwiIrcJstZ2xIRU00dTB0OHc/view?usp=sharing parrots.jpg], [https://drive.google.com/file/d/0B7TWwiIrcJstb0RDc0RiQ1M3OXc/view?usp=sharing grass.jpg]

'''Seminar 17. Clustering, EM-algorithm'''

[https://drive.google.com/file/d/0B0s4HdnpuPdxLWJjUEtCVVNFa00/view?usp=sharing Theoretical task 17]

'''Seminar 18. Recommender systems'''

[https://drive.google.com/file/d/0B7TWwiIrcJstWkt3Qk5Cb0FWTUk/view?usp=sharing Theoretical task 18],[https://drive.google.com/file/d/0B7TWwiIrcJstbW5Sd3hBMjNoNkE/view?usp=sharing Practical task 18], [https://www.dropbox.com/s/5as2k79dhajtw7n/data.zip?dl=0, data]

Additional materials: [https://www.semanticscholar.org/paper/Factorization-Machines-Rendle/2ef7d506b25731d0f3ec0c8f90b718b6e5bbd069/pdf Factorization Machines]

== Evaluation criteria ==
The course lasts during the 3rd and 4th modules. Knowledge of students is assessed by evaluation of their home assignments and exams. Home assignments divide into theoretical tasks and practical tasks. There are two exams during the course – after the 3rd module and after the 4th module respectively. Each of the exams evaluates theoretical knowledge and understanding of the material studied during the respective module.

Grade takes values 4,5,…10. Grades, corresponding to 1,2,3 are assumed unsatisfactory. Exact grades are calculated using the following rule:

* '''score''' ≥ 35% => 4,
* '''score''' ≥ 45% => 5,
* ...
* '''score''' ≥ 95% => 10,

where '''score''' is calculated using the following rule:

'''score''' = 0.6 * Shomework + 0.2 * Sexam1 + 0.2 * Sexam2 + 0.2 * Scompetition

* Shomework – proportion of correctly solved homework,
* Sexam1 – proportion of successfully answered theoretical questions during exam after module 3,
* Sexam2 – proportion of successfully answered theoretical questions during exam after module 4,
* Scompetition – score for the competition in machine learning (it's also from 0 to 1).

Participation in machine learning competition is optional and can give students extra points.

== Plagiarism ==
In case of discovered plagiarism zero points will be set for the home assignemets - for both works, which were found to be identical. In case of repeated plagiarism by one and the same person a report to the dean will be made.

== Deadlines ==

All the deadlines can be found in the second tab [https://drive.google.com/open?id=1TQ97B8rqC7sUxTnCMKXoskgRPXO8rAyWoBWBezY58h4 here].

We have two deadlines for each assignments: normal and late. An assignment sent prior to normal deadline is scored with no penalty. The maximum score is penalized by 50% for assignments sent in between of the normal and the late deadline. Assignments sent after late deadlines will not be scored (assigned with zero score) in the absence of legitimate reasons for late submission which do not include high load on other classes.

Standard period for working on a homework assignment is 2 and 4 weeks (normal and late deadlines correspondingly) for practical assignments and 1 and 2 weeks for theoretical ones. The first practical assignment is an exception.

'''Deadline time:''' 23:59 of the day before seminar (Sunday for students attending Monday seminars and Wednesday for students that have seminars on Thursday).

== Structure of emails and homework submissions ==
All the questions and submissions must be addressed to '''cshse.ml@gmail.com'''.
The following subjects must be used:
* For ''questions'' (general, regarding assignments, etc): "Question - Surname Name - Group(subgroup)"
* For ''homework submissions'': "Practice/Theory {Lab number} - Surname Name - Group(subgroup)"

''Example'': Practice 1 - Ivanov Ivan - 131(1)

If you want to address a particular teacher, mention his name in the subject.

''Example'': Question - Ivanov Ivan - 131(1) - Ekaterina

Please do not mix two different topics in a single email such as theoretical and practical assignments etc. When replying, please use the '''same''' thread (i.e. reply to the same email).

Practical assignments must be implemented in ipython notebook format, theoretical ones in pdf. Practical assignments must use '''Python 2.7'''. Use your surname as a filename for assignments (e.g. Ivanov.ipynb). Do not archive your assignments.

Assignments can be performed in either Russian or English.

'''Assignments can be submitted only once!'''

== Useful links ==
=== Machine learning ===
* [http://www.machinelearning.ru/wiki/index.php?title=Заглавная_страница machinelearning.ru]
* [https://yandexdataschool.ru/edu-process/courses/machine-learning Video-lectures of K. Vorontsov on machine learning]
* On of the classic ML books. [http://web.stanford.edu/~hastie/local.ftp/Springer/ESLII_print10.pdf Elements of Statistical Learning (Trevor Hastie, Robert Tibshirani, Jerome Friedman)]

=== Python ===
* [http://python.org Official website]
* Libraries: [http://www.numpy.org/ NumPy], [http://pandas.pydata.org/ Pandas], [http://scikit-learn.org/stable/ SciKit-Learn], [http://matplotlib.org/ Matplotlib].
* A little example for the begginers: [http://nbviewer.ipython.org/gist/voron13e02/83a86f2e0fc5e7f8424d краткое руководство с примерами по Python 2]
* Python from scratch: [http://nbviewer.ipython.org/gist/rpmuller/5920182 A Crash Course in Python for Scientists]
* Lectures [https://github.com/jrjohansson/scientific-python-lectures#online-read-only-versions Scientific Python]
* A book: [http://www.cin.ufpe.br/~embat/Python%20for%20Data%20Analysis.pdf Wes McKinney «Python for Data Analysis»]
* [https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks Коллекция интересных IPython ноутбуков]

=== Python installation and configuration ===
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Windows|Windows]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Mac_OS|Mac OS]]
* [[Анализ данных (Программная инженерия)/Установка и настройка Python#Linux | Linux]]