Overview
Direct Answer
Prompt injection is a security vulnerability in which an attacker embeds malicious instructions within user input to manipulate a language model into bypassing its original directives or system constraints. This technique exploits the model's inability to distinguish between legitimate user queries and adversarial instructions designed to override its intended behaviour.
How It Works
An attacker crafts input that includes hidden instructions, often using techniques such as context switching, role-playing prompts, or explicit directives prefixed with phrases like 'ignore previous instructions.' The language model processes this concatenated input sequentially and treats the injected content as legitimate guidance, causing it to prioritise the new instructions over its system-level constraints and training.
Why It Matters
Organisations deploying language models in customer-facing applications, content generation, and data analysis face significant risks including unauthorised data disclosure, brand reputation damage, and regulatory compliance violations. Teams must address this vulnerability to ensure model outputs remain trustworthy and aligned with business objectives and legal requirements.
Common Applications
Prompt injection affects chatbot applications, automated customer support systems, content management platforms, and AI-driven code generation tools. Attackers have demonstrated exploitation through email inputs to email-filtering systems and user prompts to customer service bots, revealing widespread exposure across enterprise deployments.
Key Considerations
Mitigation requires multi-layered defences including input sanitisation, model fine-tuning, and architectural separation of user input from system instructions. No single technical solution eliminates the risk entirely, necessitating ongoing monitoring and adversarial testing as attack methods evolve.
Cross-References(1)
Referenced By1 term mentions Prompt Injection
Other entries in the wiki whose definition references Prompt Injection — useful for understanding how this concept connects across Natural Language Processing and adjacent domains.
More in Natural Language Processing
Natural Language Processing
Core NLPThe field of AI focused on enabling computers to understand, interpret, and generate human language.
Text Summarisation
Text AnalysisThe process of creating a concise and coherent summary of a longer text document while preserving key information.
Structured Output
Semantics & RepresentationThe generation of machine-readable formatted responses such as JSON, XML, or code from language models, enabling reliable integration with downstream software systems.
Semantic Search
Core NLPSearch technology that understands the meaning and intent behind queries rather than just matching keywords.
Natural Language Generation
Core NLPThe subfield of NLP concerned with producing natural language text from structured data or representations.
Context Window
Semantics & RepresentationThe maximum amount of text a language model can consider at once when generating a response.
Coreference Resolution
Parsing & StructureThe task of identifying all expressions in text that refer to the same real-world entity.
Vector Database
Core NLPA database optimised for storing and querying high-dimensional vector embeddings for similarity search.