Protocol

Validate model output for the system that will consume it

Readable text becomes a different risk when the next component treats it as code.

About 1 min read

Before you use it

Use when
  • LLM output will be executed, rendered or interpreted by another component.

Keep this limit in mind

  • Validation rules must match the actual sink and can themselves contain vulnerabilities.

An idea to explore.

Try this idea.

Inspect the example, try the idea in your situation, then check your next move.

Keep reading

Why it matters

Treat model output as untrusted input at the downstream boundary. Validate allowed fields and ranges, use parameterized interfaces where relevant, encode for the destination context, and reject output that requests effects outside the contract. A generic 'valid JSON' check is not enough when the payload later becomes SQL, HTML, a path or a shell argument.

Steps

  1. Output becomes SQL or another query language.: Use parameterized operations and allowlisted semantics rather than string execution.
  2. Output is rendered in a browser or message.: Apply the required context-aware encoding and content controls.
  3. Output controls a tool call.: Validate the function, arguments, resource scope and authorization before execution.

An example

Parse an agent's file-edit request into an allowlisted path and operation instead of interpolating its text into a shell command.

Check your result

Malformed or out-of-contract model output is rejected before the downstream component acts on it.

Save and practise this idea

My learning

Read. Recall. Repeat.

Save this idea and return to it on any device.

Use email
Keep exploring

Connected ideas

You are readingValidate model output for the system that will consume it

Evidence and sources
Supports

OWASP recommends treating model output as untrusted input to downstream components and validating or encoding it for the destination context.

Validation must match the sink; a JSON shape check does not sanitize a shell command, SQL fragment or HTML payload.

LLM05:2025 Improper Output Handling · Prevention and Mitigation Strategies

All sources (1)
Edition and review

Version 0.1.0

No review details are supplied for this edition.

Read this idea as JSON · Release details