Prompt injection is not a bug you patch

Traditional application security rests on a separation that LLM applications do not have. In SQL injection, the query is code and the parameter is data, and parameterisation enforces the difference. In a language model, the system prompt, the user message and the retrieved document are all just tokens. There is no parameterised query for natural language.

Direct and indirect injection are different threats

Direct injection is a user typing instructions that override your intent. It is noisy, it is what most demos show, and it is the less dangerous of the two. Indirect injection is instructions hidden in content the model retrieves — a web page, a PDF, a support ticket, a calendar invite. The user never sees it, the model obeys it, and the attacker never touched your application.

This is why retrieval-augmented generation deserves the same scrutiny as an upload endpoint. If an attacker can get text into your corpus, they can address your model directly.

Defences that hold, in order of value

• Limit what the model can do. Scope tools narrowly, require approval for anything irreversible, and give the agent its own low-privilege identity. An injected instruction is only as dangerous as the capability behind it.
• Treat retrieved content as untrusted input. Filter and structurally delimit it, keep provenance, and never let corpus text be interpreted as configuration.
• Constrain the output path. If a model can emit HTML, links or code that a downstream system renders or executes, you have a second injection surface.
• Test adversarially and continuously. Injection payloads evolve; a one-off review ages badly. Build a regression suite of attacks and run it in CI.
• Filter as defence in depth, not as the defence. Input and output classifiers raise the cost of an attack. They do not close the class.

The agency question

The most consequential design decision in an LLM application is how much agency the model has. A summariser that reads a document and returns text has a small blast radius. An agent with mailbox access, a payment tool and the authority to act on what it reads has an enormous one. Excessive agency turns a prompt injection from an embarrassing output into an incident.

A useful exercise: for every tool you expose, write the sentence describing the worst thing an attacker could accomplish if the model called it with parameters of their choosing. If the sentence is alarming, the control belongs outside the model.

The OWASP Top 10 for LLM Applications is the most useful shared vocabulary the field has right now. Working through it with labs beats reading it as a list.