FSD310 - Fundamentals of Decision Making

Statistical Foundations of Machine Learning🔗

This 15h course develops important aspects of statistical modelling, which are particularly related to machine learning.

Schedule
07/09 AM	Descriptive Statistics¹	3h
08/09 AM	Inferential Statistics¹	3h
09/09 AM	Linear models¹	3h
09/09 PM	Simple Linear regression and maximum likelihood	3h
15/09 PM	Multiple Linear regression	4h
23/09 PM	Mixed-effects models	3h
28/09 PM	Written Exam	1h
30/09 PM	Introduction to Extreme Value Statistics	2h
07/10 PM	Application to Imbalanced Classification	2h

Course 1 and 2 reviews the mathematical notions that will underlie machine learning. In particular, the notions of random variables, probability density and empirical estimation of model parameters will be rigorously defined and illustrated.

Course 3 is central in this course as it presents the linear regression model from a statistical point of view, but opens up questions that are essential in machine learning, such as overfitting and cross-validation.

These issues are developed in chapter 3, which deals with regularization and cross-validation but also develops the concepts of bias-variance trade-off and curse of dimensionality.

Chapters 4 and 5 push the statistical modeling aspects introduced in chapter 3 to make clear how the randomness modeled in different random variables allows to build a statistical test (chapter 4 on ANOVA) or to estimate the parameters of a relatively complex model from observed data (chapter 5 on mixed models).

Finally, chapter 6 makes two openings on two classic linear models, both in machine learning and in statistics, which are the logistic regression and the PLS method.