Overview
Direct Answer
Online learning is a machine learning paradigm in which models are continuously updated incrementally as individual data points or small batches arrive, rather than retraining on the complete dataset at once. This approach enables systems to adapt dynamically to concept drift and non-stationary environments.
How It Works
Models process each incoming observation (or mini-batch) sequentially, updating internal parameters through algorithms such as stochastic gradient descent or adaptive learning rules. The system discards or downweights old data, allowing it to reflect recent patterns whilst maintaining computational efficiency by avoiding full retraining cycles.
Why It Matters
Organisations benefit from reduced memory overhead, lower latency in adapting to changing data distributions, and the ability to process unbounded data streams in real-time. This is critical for applications where retraining on historical data is impractical or where rapid response to emerging patterns directly impacts business decisions.
Common Applications
Practical deployments include recommendation systems that personalise suggestions as user behaviour evolves, fraud detection systems that adjust to new attack patterns, sensor monitoring in IoT networks, and stock price prediction in financial markets. Autonomous vehicle perception systems and web search ranking similarly exploit this capability.
Key Considerations
Trade-offs include potential instability from individual noisy samples, difficulty in tuning hyperparameters without cross-validation datasets, and the risk of catastrophic forgetting in neural networks. Practitioners must carefully balance learning rates and implement safeguards to prevent degradation on earlier learned concepts.
Cross-References(1)
Referenced By1 term mentions Online Learning
Other entries in the wiki whose definition references Online Learning — useful for understanding how this concept connects across Machine Learning and adjacent domains.
More in Machine Learning
Self-Supervised Learning
Advanced MethodsA learning paradigm where models generate their own supervisory signals from unlabelled data through pretext tasks.
Deep Reinforcement Learning
Reinforcement LearningCombining deep neural networks with reinforcement learning to enable agents to learn complex decision-making from raw sensory input.
UMAP
Unsupervised LearningUniform Manifold Approximation and Projection — a dimensionality reduction technique for visualisation and general non-linear reduction.
Gradient Boosting
Supervised LearningAn ensemble technique that builds models sequentially, with each new model correcting residual errors of the combined ensemble.
Bias-Variance Tradeoff
Training TechniquesThe balance between a model's ability to minimise bias (error from assumptions) and variance (sensitivity to training data fluctuations).
Feature Engineering
Feature Engineering & SelectionThe process of using domain knowledge to create, select, and transform input variables to improve model performance.
Semi-Supervised Learning
Advanced MethodsA learning approach that combines a small amount of labelled data with a large amount of unlabelled data during training.
DBSCAN
Unsupervised LearningDensity-Based Spatial Clustering of Applications with Noise — a clustering algorithm that finds arbitrarily shaped clusters based on density.