Module 04: Supervised Learning. Classification
Learn to classify data into categories using various classification algorithms.
What You'll Learn
- Logistic Regression
- Decision Trees
- Random Forests
- Support Vector Machines (SVM)
- K-Nearest Neighbors (KNN)
- Naive Bayes (Gaussian, Multinomial, Bernoulli)
- Multi-Class Classification Strategies (One-vs-Rest, One-vs-One)
- Evaluation Metrics for Classification
- Real-world Classification Projects
ML for beginners curriculum map
| Topic | Where to study |
|---|---|
| Logistic regression | Classification guide |
| K-Nearest Neighbours (distance-based) | KNN |
| Naive Bayes (probabilistic) | Naive Bayes |
| EDA and data cleaning | EDA guide |
| Feature relationships (scatter, covariance, correlation) | Feature relationship analysis |
| Feature engineering and preprocessing | Feature engineering |
| Projects implementing algorithms | Beginner projects |
Advanced machine learning curriculum map
| Topic | Where to study |
|---|---|
| Decision tree classification; entropy and information gain | Classification guide, Entropy and rules |
| Rule-based intuition; overfitting (tree depth) | Entropy section, Decision trees |
| Random Forest; tree vs forest | Random forests, Comparison |
| SVM; margin, hyperplane, kernels | SVM, Margins and kernels |
| Supervised model comparison; training and accuracy | Model comparison, Metrics |
Topics Covered
1. Logistic Regression
- Binary Classification
- Multinomial Classification
- Sigmoid Function
- Decision Boundaries
- When to use Logistic Regression
2. Decision Trees
- How Decision Trees Work
- Splitting Criteria (Gini, Entropy)
- Pruning to Prevent Overfitting
- Pros and Cons
3. Random Forests
- Ensemble Method (Bagging)
- How Random Forests Work
- Feature Importance
- When to use Random Forests
4. Support Vector Machines (SVM)
- Maximum Margin Classifier
- Kernel Trick (Linear, Polynomial, RBF)
- When to use SVM
5. K-Nearest Neighbors (KNN)
- Instance-based Learning
- Distance Metrics
- Choosing K
- Pros and Cons
6. Evaluation Metrics
- Accuracy: Overall correctness
- Precision: Of positive predictions, how many are correct
- Recall: Of actual positives, how many were found
- F1-Score: Harmonic mean of precision and recall
- ROC-AUC: Area under ROC curve
- Confusion Matrix: Detailed breakdown of predictions
Learning Objectives
By the end of this module, you should be able to:
- Implement various classification algorithms (Logistic Regression, Decision Trees, Random Forest, SVM, KNN)
- Choose appropriate algorithms for different problems
- Evaluate classification models using multiple metrics (Accuracy, Precision, Recall, F1, ROC-AUC, PR-AUC)
- Handle imbalanced datasets using resampling, class weights, and threshold tuning
- Calibrate probabilities for meaningful predictions
- Interpret model predictions and feature importance
- Build a complete classification project from scratch
Projects
- Iris Flower Classification: Classify iris species
- Spam Email Detection: Classify emails as spam or not
- Titanic Survival Prediction: Predict if passengers survived
- Customer Churn Prediction: Predict if customers will leave
Key Concepts
- Decision Boundary: Line/curve separating classes
- Probability Threshold: Cutoff for classification
- Class Imbalance: Unequal distribution of classes
- One-vs-Rest: Strategy for multiclass classification
- Feature Importance: Which features matter most
Documentation & Learning Resources
Official Documentation:
- Classification - Scikit-learn
- Classification Metrics - Scikit-learn
- Decision Trees Documentation
- Random Forest Documentation
Free Courses:
- Classification Algorithms (Coursera): Week 3-4 of Andrew Ng's course
- Classification (Kaggle Learn): Free micro-course
Tutorials:
- Logistic Regression Tutorial (Real Python)
- Decision Trees Tutorial
- Random Forest Explained
- SVM Tutorial
Video Tutorials:
- Logistic Regression (StatQuest)
- Decision Trees (StatQuest)
- Random Forest (StatQuest)
- SVM (StatQuest)
- KNN (StatQuest)
Practice:
- Titanic Competition (Kaggle): Classic classification problem
- Classification Exercises
Additional Resources
- Advanced Classification Topics - Handling imbalanced data, probability calibration, threshold tuning, precision-recall curves, model interpretation, multiclass strategies, feature engineering, hyperparameter tuning, and ensemble methods
- Complete Classification Project Tutorial - Step-by-step walkthrough building a customer churn prediction model from scratch
- Classification Quick Reference - Quick reference guide with code snippets, algorithm selection, metrics, and troubleshooting
Previous Module: 03-supervised-learning-regression
Next Module: 05-model-evaluation-optimization