Overview
Direct Answer
A neural network architecture comprising an encoder–decoder framework that transforms an input sequence into an output sequence of different length and structure. Originally developed for machine translation, this design pattern has become foundational for sequence-to-sequence transformation tasks across natural language processing.
How It Works
The encoder processes an input sequence token-by-token, compressing information into a fixed-size context vector or attention mechanism. The decoder then consumes this representation to generate the output sequence autoregressively, predicting one token at a time based on the encoder state and previously generated tokens. Attention mechanisms allow the decoder to selectively focus on relevant input positions during generation, significantly improving translation quality and handling of long sequences.
Why It Matters
Organisations deploy this architecture to automate labour-intensive language tasks with high accuracy, reducing operational costs and latency in customer-facing systems. The pattern's flexibility enables handling variable-length inputs and outputs, which is essential for real-world applications where sentence structure and length differ between source and target languages.
Common Applications
Primary use cases include machine translation services, automated summarisation of documents and customer feedback, dialogue systems, and code generation. Question-answering systems and image captioning also leverage this architecture by adapting the encoder to process alternative modalities.
Key Considerations
The fixed context vector can become a bottleneck for very long sequences, though attention mechanisms mitigate this problem. Computational cost during inference—particularly beam search decoding—and exposure bias during training remain important tradeoffs requiring careful tuning and regularisation strategies.
Cross-References(1)
More in Natural Language Processing
Relation Extraction
Parsing & StructureIdentifying semantic relationships between entities mentioned in text.
Coreference Resolution
Parsing & StructureThe task of identifying all expressions in text that refer to the same real-world entity.
Machine Translation
Generation & TranslationThe use of AI to automatically translate text or speech from one natural language to another.
Speech Recognition
Speech & AudioThe technology that converts spoken language into text, also known as automatic speech recognition.
Text Generation
Generation & TranslationThe process of producing coherent and contextually relevant text using AI language models.
Multilingual Model
Semantics & RepresentationA language model trained on text from dozens or hundreds of languages simultaneously, enabling cross-lingual understanding and generation without language-specific fine-tuning.
Large Language Model
Semantics & RepresentationA neural network trained on massive text corpora that can generate, understand, and reason about natural language.
Question Answering
Generation & TranslationAn NLP task where a system automatically answers questions posed in natural language based on given context.