Classical ML exercises

Exercises for a vast variety of classical ML techniques (i.e. Hierarchical Clustering, NMF, EM and many others)

These exercises were made in collaboration with Damien Chambon

The main topics approached in each of the files are the following:

1. Clustering

K-means, Agglomerative Clustering, DBSCAN, elbow method for parameter tuning

Application: Image processing for detecting outliers/points of interest

2. Classification

Logistic Regression, AUC (Area Under Curve), LDA (Linear Discriminant Analysis), Support Vector Machines, Kernels, Voting Classifier

Application: Bank marketing data

3. Non-negative Matrix Factorization (NMF)

Non-negative Matrix Factorization under random and NNSVD initialization

Application 1: Image processing for decomposing and recomposing facial features

Application 2: Topic segmentation under Natural Language Processing

4. Gaussian Mixture Models (GMM)

Implementation of the Expectation-Maximization (EM) algorithm from scratch enabling the creation of GMMs

Application 1: Sex detection from weight distribution (toy example)

Application 2: Learning the GMM model for the MNIST dataset + synthetization of new samples for each number from the fitted model

5. Model Order Selection

Akaike information criterion (AIC), Bayesian information criterion (BIC), application and discussions under an EM context

Application: Fitting a model under MNIST, measuring the interval of confidence of prediciton using the Central Limit Theorem

6. Dimensionality Reduction

Implementing PCA from scratch, mathematical considerations of PCA, tSNE

Application: Comparison of visualizations of MNIST features under PCA and under tSNE

These exercises were originally proposed by the Advanced Machine Learning course under MSc in Data Sciences and Business Analytics by CentraleSupélec and ESSEC Business School. All credit goes to the professors for the core structure of the exercises and notebooks.