Nabid In Motion Nabid In Motion

Study interactive :: Progress tools open in the Study Hub reader.

Machine Learning Terminology. Complete Reference

Glossary of machine learning terms with short explanations and examples.

Table of Contents

Core Concepts

Machine Learning (ML)

Definition: A method of data analysis that automates analytical model building, enabling computers to learn from data without being explicitly programmed.

Key Idea: Instead of writing rules, we show examples and let the computer learn patterns.

Example:

# Traditional: Write rules
if email_contains("free money"):
    mark_as_spam()

# ML: Learn from examples
model.fit(spam_emails, labels)  # Learn what spam looks like
model.predict(new_email)  # Use learned knowledge

Artificial Intelligence (AI)

Definition: The broader field of creating intelligent machines. ML is a subset of AI.

Relationship:

AI (Broad)
  └── Machine Learning
       └── Deep Learning

Deep Learning

Definition: A subset of ML using neural networks with multiple layers to learn complex patterns.

Example: Image recognition, natural language processing, speech recognition.

Data Terms

Features (X)

Definition: Input variables used to make predictions. Also called:

Example:

# Features for house price prediction
features = ['size', 'bedrooms', 'location', 'age']
X = df[features]  # Feature matrix

Labels/Targets (y)

Definition: The output variable we want to predict. Also called:

Example:

# Target for house price prediction
y = df['price']  # What we want to predict

Training Data

Definition: Data used to teach the model. The model learns patterns from this data.

Characteristics:

Example:

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
# X_train, y_train = Training data

Validation Data

Definition: Data used to tune hyperparameters and select models during development.

Characteristics:

Test Data

Definition: Data used for final evaluation. Model has never seen this data.

Characteristics:

Example:

# Test data - final evaluation only!
test_accuracy = model.score(X_test, y_test)

Dataset

Definition: Collection of data used for ML. Can be:

Sample/Instance

Definition: A single data point or row in the dataset.

Example:

# One sample (one house)
sample = {
    'size': 1500,
    'bedrooms': 3,
    'location': 'suburb',
    'price': 250000
}

Batch

Definition: A subset of training data processed together in one iteration.

Why use batches:

Example:

# Process 32 samples at a time
batch_size = 32
for batch in batches:
    model.train(batch)

Model Terms

Model

Definition: The learned function that makes predictions. It's the "brain" that learned from data.

Analogy: Like a student who studied examples and can now answer new questions.

Example:

# Model learns from data
model = LinearRegression()
model.fit(X_train, y_train)  # Learning phase

# Model makes predictions
predicti  # Prediction phase

Algorithm

Definition: The method or procedure used to learn from data.

Examples:

Note: Algorithm is the method, Model is what it learned.

Hyperparameters

Definition: Configuration settings for the algorithm (not learned from data).

Examples:

Key Point: Set by the user, not learned by the model.

# Hyperparameters (set by you)
model = RandomForestClassifier(
    n_estimators=100,  # Hyperparameter
    max_depth=10,      # Hyperparameter
    random_state=42    # Hyperparameter
)

Parameters

Definition: Values learned by the model during training.

Examples:

Key Point: Learned from data, not set by user.

# Parameters (learned by model)
model.fit(X_train, y_train)
weights = model.coef_  # Learned parameters

Weights

Definition: Parameters in a model that determine how features contribute to predictions.

Example:

# Linear regression: y = w1*x1 + w2*x2 + b
# w1, w2 are weights
# b is bias

Bias

Definition: A constant term added to predictions. Allows model to fit data that doesn't pass through origin.

Example:

# y = mx + b
# b is the bias (intercept)

Training Terms

Training

Definition: The process of teaching the model by showing it examples and letting it learn patterns.

Process:

  1. Initialize model
  2. Show training examples
  3. Model adjusts parameters
  4. Repeat until model learns

Example:

model.fit(X_train, y_train)  # Training phase

Learning

Definition: The process of improving performance through experience (seeing more data).

Epoch

Definition: One complete pass through the entire training dataset.

Example:

# Train for 10 epochs
for epoch in range(10):
    model.train_one_epoch(training_data)

Iteration

Definition: One update step of the model (processing one batch).

Relationship:

Loss Function / Cost Function

Definition: Function that measures how wrong the model's predictions are.

Purpose: Guides the model to improve by showing what's wrong.

Examples:

# Loss measures prediction error
predicti>
loss = loss_function(predictions, y_train)
# Lower loss = better model

Gradient

Definition: The direction and magnitude of steepest increase in loss. Used to update model parameters.

In Training:

Gradient Descent

Definition: Optimization algorithm that finds minimum of loss function by following negative gradient.

Process:

  1. Calculate gradient
  2. Update parameters: w = w - learning_rate * gradient
  3. Repeat until convergence

Learning Rate

Definition: Step size in gradient descent. Controls how much parameters change each iteration.

Too Small: Slow convergence Too Large: May overshoot or diverge Just Right: Fast, stable convergence

# Learning rate is a hyperparameter
model = SGDClassifier(learning_rate=0.01)

Overfitting

Definition: Model memorizes training data but fails on new data.

Signs:

Example:

# Overfitting
train_accuracy = 0.99  # Perfect on training
test_accuracy = 0.60   # Poor on test
# Gap = 0.39 (overfitting!)

Solutions:

Underfitting

Definition: Model too simple to capture patterns in data.

Signs:

Example:

# Underfitting
train_accuracy = 0.55  # Poor on training
test_accuracy = 0.53    # Poor on test
# Both low = underfitting

Solutions:

Generalization

Definition: Model's ability to perform well on new, unseen data.

Goal: Good generalization = model works in real world

Measures:

Regularization

Definition: Techniques to prevent overfitting by penalizing complex models.

Types:

# L2 regularization
model = Ridge(alpha=1.0)  # alpha controls regularization strength

Evaluation Terms

Accuracy

Definition: Proportion of correct predictions.

Formula: Accuracy = (Correct Predictions) / (Total Predictions)

Use Case: Balanced datasets

accuracy = accuracy_score(y_true, y_pred)
# Example: 85 out of 100 correct = 0.85 accuracy

Precision

Definition: Proportion of positive predictions that are actually positive.

Formula: Precision = TP / (TP + FP)

Use Case: When false positives are costly

Example: Spam detection - don't want to mark real emails as spam

Recall / Sensitivity

Definition: Proportion of actual positives correctly identified.

Formula: Recall = TP / (TP + FN)

Use Case: When false negatives are costly

Example: Disease diagnosis - don't want to miss sick patients

F1-Score

Definition: Harmonic mean of precision and recall. Balances both metrics.

Formula: F1 = 2 * (Precision * Recall) / (Precision + Recall)

Use Case: When need to balance precision and recall

Confusion Matrix

Definition: Table showing performance of classification model.

Structure:

                Predicted
              Negative  Positive
Actual Negative   TN      FP
       Positive   FN      TP

Components:

ROC-AUC

Definition: Area Under the ROC Curve. Measures classifier's ability to distinguish classes.

Range: 0 to 1 (higher is better)

MSE (Mean Squared Error)

Definition: Average squared difference between predictions and actual values.

Formula: MSE = (1/n) * Σ(predicted - actual)²

Use Case: Regression problems

RMSE (Root Mean Squared Error)

Definition: Square root of MSE. In same units as target variable.

Formula: RMSE = √MSE

MAE (Mean Absolute Error)

Definition: Average absolute difference between predictions and actual values.

Formula: MAE = (1/n) * Σ|predicted - actual|

R² (R-squared / Coefficient of Determination)

Definition: Proportion of variance in target explained by model.

Range: -∞ to 1

Algorithm Terms

Supervised Learning

Definition: Learning from labeled examples (input-output pairs).

Types:

Unsupervised Learning

Definition: Learning from unlabeled data (only features, no labels).

Types:

Reinforcement Learning

Definition: Learning through interaction, receiving rewards/penalties.

Ensemble

Definition: Combining multiple models to improve performance.

Types:

Cross-Validation

Definition: Technique to assess model performance by splitting data into folds.

K-Fold CV: Split data into k folds, train on k-1, test on 1, repeat k times.

Benefits:

Quick Reference

Common Abbreviations

Data Split Conventions

Model Performance Indicators

Good Model:

Overfitting:

Underfitting:

Resources

Try next: Define bias, variance, and leakage in your own words without opening the page.

Recall ::

In one sentence, what is overfitting?

The model fits training noise so well that it fails on new data.