A probabilistic approach to ML🔗
Naive Bayes Classification🔗
This class provides a gentle introduction to the Bayesian approach to Supervised Learning. We will consider the question of estimating the distribution of a label \(y\) given an input \(x\) and some data. A first, very naive way to tackle this problem will involve a strong hypothesis of conditional independence, which we will call the naive Bayes assumption. This will open the door to the method of Naive Bayes Classification.
Pre-class refresher activities and solution
Summary card
Lecture notes
References
- Exploring conditions for the optimality of naive Bayes.
Zhang, H. International Journal of Pattern Recognition and Artificial Intelligence, 19(02), 183-198, (2005).
Gaussian Processes🔗
This class continues our exploration of the Bayesian approach to Machine Learning. Departing from the question of estimating \(P(y|x)\) and the hypothesis that a set of observations y is distributed as a Gaussian vector around its meand, we shall see how we can derive an explicit form for the distribution of \(y|x\). This explicit form will be called a Gaussian Process and will provide us with the likelihood that each possible function fits our data. In turn, this will provide an elegant and efficient way to estimate the most likely output \(y\) for a given \(x\) but also the confidence interval around this prediction.
Pre-class refresher activities and solution
Summary card
References
- Gaussian Processes in Machine Learning.
C. E. Rasmussen and C. K. I. Williams. MIT press, 2005.
Available for download at http://www.gaussianprocess.org/gpml.
Surrogate Modeling and Bayesian Optimization🔗
This practical session explores the use of Gaussian Processes in order to estimate a given physics phenomenon, say forces and torques on an aircraft's wing. Such a model is called a surrogate model of the actual phenomenon. This surrogate model is then used to optimize the mechanical characteristics of the airfoil, without resorting to costly numerical simulations or experiments. We explore methods for Bayesian optimization using surrogate models in the last part of the class.
Presentation
Notebook (colab)
Archive of notebook and solution files