Overview
Direct Answer
Linear regression is a supervised learning algorithm that models the relationship between a dependent variable and one or more independent variables by fitting a straight line (or hyperplane in multiple dimensions) through observed data points. It assumes a linear relationship and estimates coefficients that minimise prediction error.
How It Works
The algorithm calculates optimal coefficients by minimising the sum of squared residuals—the differences between observed and predicted values. In simple regression, a single independent variable produces a 2D line; multiple regression uses the normal equation or gradient descent to solve for coefficients across n-dimensional space. The fitted model then makes predictions by applying the learned coefficients to new input data.
Why It Matters
Linear models are computationally efficient, interpretable, and require relatively small datasets, making them valuable for rapid prototyping and regulatory compliance in finance and healthcare. Their transparency—each coefficient's magnitude directly indicates variable importance—supports evidence-based decision-making where stakeholders must understand model behaviour rather than treat it as a black box.
Common Applications
Applications include sales forecasting based on historical trends, real estate price estimation from property features, medical outcome prediction (e.g., patient recovery time), and demand planning in supply chain operations. Financial institutions use it for credit risk assessment and cost-benefit analysis.
Key Considerations
The method assumes a genuine linear relationship; non-linear data produces poor predictions. Multicollinearity between independent variables, outliers, and heteroscedasticity (non-constant error variance) can degrade model performance and interpretability.
More in Machine Learning
Underfitting
Training TechniquesWhen a model is too simple to capture the underlying patterns in the data, resulting in poor performance on both training and test data.
Semi-Supervised Learning
Advanced MethodsA learning approach that combines a small amount of labelled data with a large amount of unlabelled data during training.
Curriculum Learning
Advanced MethodsA training strategy that presents examples to a model in a meaningful order, typically from easy to hard.
Principal Component Analysis
Unsupervised LearningA dimensionality reduction technique that transforms data into orthogonal components ordered by the amount of variance they explain.
Lasso Regression
Feature Engineering & SelectionA regularised regression technique that adds an L1 penalty, enabling feature selection by driving some coefficients to zero.
Class Imbalance
Feature Engineering & SelectionA situation where the distribution of classes in a dataset is significantly skewed, with some classes vastly outnumbering others.
Self-Supervised Learning
Advanced MethodsA learning paradigm where models generate their own supervisory signals from unlabelled data through pretext tasks.
K-Means Clustering
Unsupervised LearningA partitioning algorithm that divides data into k clusters by minimising the distance between points and their cluster centroids.