Overview
Direct Answer
A capsule network is a neural network architecture that organises neurons into groups called capsules, each representing specific entity instantiation parameters such as pose, deformation, velocity, and texture. This structure enables the network to capture hierarchical spatial relationships and part-whole relationships more effectively than traditional convolutional approaches.
How It Works
Capsules function as small neural modules that output vector quantities rather than scalar activations, with vector magnitude representing the probability of entity presence and direction encoding entity properties. Dynamic routing algorithms iteratively determine which lower-level capsules should send their output to higher-level capsules based on agreement between predictions and actual outputs, creating a more sophisticated information flow than conventional pooling mechanisms.
Why It Matters
This architecture addresses fundamental limitations in standard CNNs, particularly their inability to handle spatial transformations and their reliance on massive training datasets. Organisations benefit from improved generalisation on transformed inputs, reduced data requirements, and more interpretable learned representations—critical factors for resource-constrained deployments and applications requiring robustness to viewpoint variations.
Common Applications
Applications include image classification tasks requiring transformation invariance, medical imaging analysis where hierarchical feature relationships prove important, and 3D object recognition. Research prototypes have demonstrated promise in handwritten digit recognition, traffic sign classification, and object detection scenarios involving occluded or rotated inputs.
Key Considerations
Computational overhead during routing procedures significantly exceeds standard convolutional networks, creating training and inference bottlenecks. The architecture remains primarily experimental at enterprise scale, with deployment on large-scale datasets presenting practical challenges compared to established CNN alternatives.
Cross-References(1)
More in Deep Learning
Self-Attention
Training & OptimisationAn attention mechanism where each element in a sequence attends to all other elements to compute its representation.
Fully Connected Layer
ArchitecturesA neural network layer where every neuron is connected to every neuron in the adjacent layers.
Data Parallelism
ArchitecturesA distributed training strategy that replicates the model across multiple devices and divides training data into batches processed simultaneously, synchronising gradients after each step.
Layer Normalisation
Training & OptimisationA normalisation technique that normalises across the features of each individual sample rather than across the batch.
Word Embedding
Language ModelsDense vector representations of words where semantically similar words are mapped to nearby points in vector space.
Softmax Function
Training & OptimisationAn activation function that converts a vector of numbers into a probability distribution, commonly used in multi-class classification.
Dropout
Training & OptimisationA regularisation technique that randomly deactivates neurons during training to prevent co-adaptation and reduce overfitting.
Adapter Layers
Language ModelsSmall trainable modules inserted between frozen transformer layers that enable task-specific adaptation without modifying the original model weights.