Naive Bayes — Technology Wiki

Overview

Direct Answer

Naive Bayes is a probabilistic classifier that applies Bayes' theorem under the assumption that all features are conditionally independent given the class label. Despite this independence assumption rarely holding in practice, the model provides computationally efficient classification with surprisingly robust performance across many domains.

How It Works

The classifier calculates the posterior probability of each class by multiplying the likelihood of observed features given that class and the prior probability of the class itself. Feature independence allows these likelihoods to be computed separately and multiplied together, avoiding the exponential complexity of estimating joint feature distributions. The algorithm assigns an input to the class with the highest posterior probability.

Why It Matters

Naive Bayes offers exceptional computational efficiency and minimal training data requirements compared to more complex models, making it valuable for resource-constrained environments and rapid prototyping. Its interpretability—probabilities directly indicate feature importance—supports compliance and auditing requirements in regulated industries.

Common Applications

The approach is widely deployed in email spam filtering, sentiment analysis of social media and customer reviews, document categorisation for content management systems, and medical diagnosis support tools. Text classification remains the dominant use case due to the model's natural alignment with discrete word-frequency features.

Key Considerations

The conditional independence assumption introduces systematic bias that can degrade performance when features are strongly correlated; practitioners should validate assumptions on domain-specific data. Probability estimates may become unreliable with sparse feature data, though calibration techniques can mitigate this issue.

Related in Supervised Learning

Boosting

An ensemble technique that sequentially trains models, each focusing on correcting the errors of previous models.

Random Forest

An ensemble learning method that constructs multiple decision trees during training and outputs the mode of their predictions.

Gradient Boosting

An ensemble technique that builds models sequentially, with each new model correcting residual errors of the combined ensemble.

XGBoost

An optimised distributed gradient boosting library designed for speed and performance in machine learning competitions and production.

Decision Tree

A tree-structured model where internal nodes represent feature tests, branches represent outcomes, and leaves represent predictions.

Support Vector Machine

A supervised learning algorithm that finds the optimal hyperplane to separate different classes in high-dimensional space.

K-Nearest Neighbours

A simple algorithm that classifies data points based on the majority class of their k closest neighbours in feature space.

Linear Regression

A statistical method modelling the relationship between a dependent variable and one or more independent variables using a linear equation.

Logistic Regression

A classification algorithm that models the probability of a binary outcome using a logistic function.

Polynomial Regression

A form of regression analysis where the relationship between variables is modelled as an nth degree polynomial.

Tabular Deep Learning

The application of deep neural networks to structured tabular datasets, competing with traditional methods like gradient boosting through specialised architectures and regularisation.

More in Machine Learning

Supervised Learning

MLOps & Production

A machine learning paradigm where models are trained on labelled data, learning to map inputs to known outputs.

Model Monitoring

MLOps & Production

Continuous observation of deployed machine learning models to detect performance degradation, data drift, anomalous predictions, and infrastructure issues in production.

Transfer Learning

Advanced Methods

A technique where knowledge gained from training on one task is applied to a different but related task.

Overfitting

Training Techniques

When a model learns the training data too well, including noise, resulting in poor performance on unseen data.

Active Learning

MLOps & Production

A machine learning approach where the algorithm interactively queries a user or oracle to label new data points.

Elastic Net

Training Techniques

A regularisation technique combining L1 and L2 penalties, balancing feature selection and coefficient shrinkage.

Self-Supervised Learning

Advanced Methods

A learning paradigm where models generate their own supervisory signals from unlabelled data through pretext tasks.

Ensemble Methods

MLOps & Production

Machine learning techniques that combine multiple models to produce better predictive performance than any single model, including bagging, boosting, and stacking approaches.