AI Bias — Technology Wiki

Overview

Direct Answer

AI bias refers to systematic disparities in model predictions or outputs that disadvantage particular groups or outcomes, stemming from non-representative training data, encoded human prejudices, or algorithmic design choices that amplify historical inequities. These errors are distinct from random model noise and propagate through downstream decisions.

How It Works

Bias emerges when training datasets reflect historical imbalances—for example, loan approval systems trained on decades of discriminatory lending practices. Algorithms optimise to minimise loss across aggregate populations, inadvertently learning to replicate or magnify disparities present in source data. Feature selection, sampling strategies, and loss function design further influence which groups experience worse performance or harmful outcomes.

Why It Matters

Organisations face regulatory exposure under anti-discrimination law, operational risk from public backlash, and accuracy degradation in underrepresented segments. Financial services, healthcare, recruitment, and criminal justice systems experience material harm when biased models deny loans, misdiagnose conditions, reject qualified candidates, or influence sentencing recommendations.

Common Applications

Facial recognition systems exhibit higher error rates on darker skin tones; hiring algorithms have screened out female candidates; medical risk scores underestimate disease burden in Black patients; credit scoring models perpetuate lending disparities across protected groups.

Key Considerations

Detecting and correcting bias requires multi-stage governance—auditing training data composition, validating performance across demographic segments, and accepting that mitigation often involves accuracy-fairness tradeoffs. No single metric captures bias comprehensively across all stakeholder perspectives.

Related in Training & Inference

Causal Inference

The process of determining cause-and-effect relationships from data, going beyond correlation to establish causation.

AI Feature Store

A centralised platform for storing, managing, and serving machine learning features consistently across training and inference.

Federated Learning

A machine learning approach where models are trained across decentralised devices without sharing raw data, preserving privacy.

AI Inference

The process of using a trained AI model to make predictions or decisions on new, unseen data.

AI Training

The process of teaching an AI model to recognise patterns by exposing it to large datasets and adjusting its parameters.

Hyperparameter Tuning

The process of optimising the external configuration settings of a machine learning model that are not learned during training.

AutoML

Automated machine learning that automates the end-to-end process of applying machine learning to real-world problems.

Reinforcement Learning from Human Feedback

A training paradigm where AI models are refined using human preference signals, aligning model outputs with human values and quality expectations through reward modelling.

Direct Preference Optimisation

A simplified alternative to RLHF that directly optimises language model policies using preference data without requiring a separate reward model.

Model Merging

Techniques for combining the weights and capabilities of multiple fine-tuned models into a single model without additional training, creating versatile multi-capability systems.

More in Artificial Intelligence

Recall

Evaluation & Metrics

The ratio of true positive predictions to all actual positive instances, measuring completeness of positive identification.

Artificial Intelligence

Foundations & Theory

The simulation of human intelligence processes by computer systems, including learning, reasoning, and self-correction.

Artificial General Intelligence

Foundations & Theory

A hypothetical form of AI that possesses the ability to understand, learn, and apply knowledge across any intellectual task a human can perform.

Tensor Processing Unit

Models & Architecture

Google's custom-designed application-specific integrated circuit for accelerating machine learning workloads.

AI Hallucination

Safety & Governance

When an AI model generates plausible-sounding but factually incorrect or fabricated information with high confidence.

AI Pipeline

Infrastructure & Operations

A sequence of data processing and model execution steps that automate the flow from raw data to AI-driven outputs.

Few-Shot Prompting

Prompting & Interaction

A technique where a language model is given a small number of examples within the prompt to guide its response pattern.

Artificial Narrow Intelligence

Foundations & Theory

AI systems designed and trained for a specific task or narrow range of tasks, such as image recognition or language translation.