Supervised Learning with scikit-learn: Regression and Classification
Supervised Learning: Regression and Classification is an intermediate-level course of 5 lessons (about 2 h 50 min), with 12 exercises and 15 quiz questions. Prerequisites: Introduction to Machine Learning (scikit-learn). Exercises run in the browser, with nothing to install.
- Data science
- Intermediate
- 5 lessons
- 2 h 50 min
- 12 exercises
- Exercises run in the browser
The course in brief
This course presents the main supervised learning algorithms and how to tune them with scikit-learn. You train linear and regularized regressions, logistic classifiers, trees, k-nearest neighbors, then random forests and gradient boosting. At each step, you learn to measure performance, control complexity, and choose the model that suits the problem.
What you will be able to do
- Train a linear regression and evaluate it with MAE, RMSE, and R² against a baseline
- Regularize a linear model with Ridge, Lasso, and the C parameter
- Use logistic regression, its probabilities, and its decision threshold
- Control the complexity of decision trees and k-nearest neighbors
- Train and compare random forests and gradient boosting
- Choose a supervised model based on performance and business constraints
Syllabus: 5 lessons
- Train a linear regression and read its coefficients
- Compute and interpret MAE, RMSE, and R²
- Compare a model to a naive baseline
- 02
Regularization: Ridge and Lasso
- Understand why a linear model can overfit
- Use Ridge and Lasso and tune alpha
- Use Lasso to select features
- 03
Classification with Logistic Regression
- Train a logistic regression and obtain probabilities
- Understand the decision threshold
- Tune regularization with C
Logistic regressionPredicted probabilitiesDecision threshold
- 04
Nonlinear Models: Decision Trees and k-Nearest Neighbors
- Train a decision tree and control its complexity
- Tune k in a k-nearest neighbors model
- Compare linear and nonlinear boundaries
- 05
Ensemble Methods: Random Forests and Gradient Boosting
- Explain the principle of bagging and boosting
- Train RandomForest and HistGradientBoosting
- Read variable importances and choose a model
Final project
Comparing supervised models to predict disease progression
Using the built-in load_diabetes (regression) and then load_breast_cancer (classification) datasets, build a complete approach for each problem: baseline, regularized linear model, constrained tree, random forest, and HistGradientBoosting. Compare them by cross-validation on the training set, select one model per problem, evaluate it only once on the test set, and justify your choice by taking performance and interpretability into account. For classification, propose a well-argued decision threshold.
Deliverables
- A reproducible script or notebook for each of the two problems
- A comparison table of the models (cross-validation mean and standard deviation, suitable metric)
- An analysis of the most influential variables of the selected model
- A one-page summary note explaining the choice of model and threshold
Assessment criteria
- Metrics suited to the type of problem and systematic comparison with a baseline
- Complexity hyperparameters justified (alpha, C, max_depth, number of trees)
- No data leakage (preprocessing in pipelines, test set used once)
- Careful interpretation of coefficients and importances
- Code clarity and reproducibility
Frequently asked questions
What is the difference between regression and classification?
Regression predicts a continuous numeric value (a price, a duration), while classification predicts a category (spam or not, customer type). Both are supervised learning, because you learn from examples whose answer is known.
Should I start with a simple model or with gradient boosting?
Always start with a naive baseline, then a linear model: they serve as a reference and are easy to interpret. Then try random forests and gradient boosting, and compare them by cross-validation.
Which version of scikit-learn is used?
The course follows the recent scikit-learn API (versions 1.8 and later): root_mean_squared_error for RMSE, l1_ratio to choose the logistic regression penalty, and HistGradientBoosting for boosting.
Does this course prepare me for a data scientist job?
Yes. It covers the supervised models used daily on tabular data and is part of the Data scientist track, before the courses on model evaluation and feature engineering.
This course, told for your job
Say what you want to be able to do: “Supervised Learning: Regression and Classification” takes its place in your path, with examples from your field. The concepts stay those of the course.