Module 03: Supervised Learning. Regression
Learn to predict continuous values using regression algorithms.
What You'll Learn
- Linear Regression
- Polynomial Regression
- Regularization (Ridge, Lasso, Elastic Net)
- Evaluation Metrics for Regression
- Real-world Regression Projects
ML for beginners curriculum map
| Topic | Where to study |
|---|---|
| Linear regression and evaluation metrics | Regression guide, Evaluation metrics |
| Multiple and polynomial regression | Linear regression, Polynomial regression |
| Data distribution analysis (percentiles, histogram, boxplot) | Data distribution analysis |
| Feature relationships (scatter, covariance, correlation) | Feature relationship analysis |
| End-to-end regression and saving a model | Train, evaluate, persist, Project tutorial |
| Descriptive stats / sampling | Introduction to ML |
Topics Covered
1. Linear Regression
- Simple Linear Regression
- Multiple Linear Regression
- Assumptions of Linear Regression
- Implementation with scikit-learn
2. Polynomial Regression
- When to use Polynomial Regression
- Overfitting concerns
- Implementation
3. Regularization
- Ridge Regression (L2): Reduces overfitting by penalizing large coefficients
- Lasso Regression (L1): Can eliminate features (feature selection)
- Elastic Net: Combines Ridge and Lasso
4. Evaluation Metrics
- Mean Squared Error (MSE): Average squared difference
- Root Mean Squared Error (RMSE): Square root of MSE
- Mean Absolute Error (MAE): Average absolute difference
- R² Score: Proportion of variance explained
- Adjusted R²: R² adjusted for number of features
Learning Objectives
By the end of this module, you should be able to:
- Implement linear and polynomial regression
- Understand how gradient descent works
- Apply regularization techniques (Ridge, Lasso, Elastic Net)
- Evaluate regression models using appropriate metrics
- Perform residual analysis and model diagnostics
- Handle outliers and multicollinearity
- Transform features for better performance
- Tune hyperparameters using cross-validation
- Build a complete regression project from scratch
Projects
- House Price Prediction: Predict house prices using features like size, location, etc.
- Stock Price Prediction: Predict stock prices (simplified version)
- Weather Prediction: Predict temperature or rainfall
Key Concepts
- Coefficients: Weights learned by the model
- Intercept: Bias term
- Residuals: Difference between actual and predicted values
- Multicollinearity: High correlation between features
- Feature Scaling: Important for regularization
Documentation & Learning Resources
Official Documentation:
Free Courses:
- Linear Regression (Coursera): Week 1-2 of Andrew Ng's course
- Regression Analysis (edX): Free audit available
- Linear Regression (Kaggle Learn): Free micro-course
Tutorials:
- Linear Regression Tutorial (Real Python)
- Regression Analysis Tutorial
- Ridge and Lasso Regression Explained
Video Tutorials:
Practice:
- House Prices Competition (Kaggle): Practice regression
- Regression Exercises (GitHub)
Additional Resources
- Advanced Regression Topics - Gradient descent, residual analysis, outliers, multicollinearity, feature transformations, model interpretation, and hyperparameter tuning
- Complete Regression Project Tutorial - Step-by-step walkthrough building a house price prediction model from scratch
- Regression Quick Reference - Quick reference guide with code snippets, algorithm selection, metrics, and troubleshooting
Previous Module: 02-introduction-to-ml
Next Module: 04-supervised-learning-classification