Overview
Direct Answer
Online learning is a machine learning paradigm in which models are continuously updated incrementally as individual data points or small batches arrive, rather than retraining on the complete dataset at once. This approach enables systems to adapt dynamically to concept drift and non-stationary environments.
How It Works
Models process each incoming observation (or mini-batch) sequentially, updating internal parameters through algorithms such as stochastic gradient descent or adaptive learning rules. The system discards or downweights old data, allowing it to reflect recent patterns whilst maintaining computational efficiency by avoiding full retraining cycles.
Why It Matters
Organisations benefit from reduced memory overhead, lower latency in adapting to changing data distributions, and the ability to process unbounded data streams in real-time. This is critical for applications where retraining on historical data is impractical or where rapid response to emerging patterns directly impacts business decisions.
Common Applications
Practical deployments include recommendation systems that personalise suggestions as user behaviour evolves, fraud detection systems that adjust to new attack patterns, sensor monitoring in IoT networks, and stock price prediction in financial markets. Autonomous vehicle perception systems and web search ranking similarly exploit this capability.
Key Considerations
Trade-offs include potential instability from individual noisy samples, difficulty in tuning hyperparameters without cross-validation datasets, and the risk of catastrophic forgetting in neural networks. Practitioners must carefully balance learning rates and implement safeguards to prevent degradation on earlier learned concepts.
Cross-References(1)
Referenced By1 term mentions Online Learning
Other entries in the wiki whose definition references Online Learning — useful for understanding how this concept connects across Machine Learning and adjacent domains.
More in Machine Learning
Transfer Learning
Advanced MethodsA technique where knowledge gained from training on one task is applied to a different but related task.
Adam Optimiser
Training TechniquesAn adaptive learning rate optimisation algorithm combining momentum and RMSProp for efficient deep learning training.
Ensemble Methods
MLOps & ProductionMachine learning techniques that combine multiple models to produce better predictive performance than any single model, including bagging, boosting, and stacking approaches.
Model Registry
MLOps & ProductionA versioned catalogue of trained machine learning models with metadata, lineage, and approval workflows, enabling reproducible deployment and governance at enterprise scale.
K-Means Clustering
Unsupervised LearningA partitioning algorithm that divides data into k clusters by minimising the distance between points and their cluster centroids.
XGBoost
Supervised LearningAn optimised distributed gradient boosting library designed for speed and performance in machine learning competitions and production.
Model Calibration
MLOps & ProductionThe process of adjusting a model's predicted probabilities so they accurately reflect the true likelihood of outcomes, essential for risk-sensitive decision-making.
Class Imbalance
Feature Engineering & SelectionA situation where the distribution of classes in a dataset is significantly skewed, with some classes vastly outnumbering others.