Prompts That Return Valid JSON
Ask for a named key contract rather than "JSON", state that the response must contain nothing but the object, decide up front whether a missing value is null or an absent key, and validate before you trust it. Those four things close the five failures that actually break a parse: prose wrappers, code fences, renamed keys, invented keys, and silent truncation.
Asking a model for JSON usually works. Asking a hundred times and having a hundred responses parse is a different problem, and it fails in a small number of specific ways. This guide takes each failure in turn and shows the wording that closes it.
By Andrei Bădulescu, Founder at VantagePrompt·Updated
A JSON response is either usable by the next step or it is not — there is no partial credit in a parser. That makes JSON the output shape where prompt wording pays off most directly, because every failure mode has a phrase that prevents it.
How does an LLM JSON response actually fail?
| Failure | What arrives | What closes it |
|---|---|---|
| Prose wrapper | "Here is the JSON you requested:" followed by the object. | "Respond with the JSON object and nothing else." |
| Code fence | The object wrapped in a fenced block. | "Do not wrap the response in code fences." Or strip fences before parsing. |
| Renamed keys | user_name where you asked for userName. | List the exact keys, verbatim, in the prompt. |
| Invented keys | Helpful extras you never asked for. | "Include exactly these keys and no others." |
| Truncation | A valid prefix that ends mid-array. | Cap the result count; check the finish reason before parsing. |
Why is "return JSON" not enough?
Because it names a syntax, not a contract. The model is free to choose the keys, their casing, their nesting, and whether an absent value is null, an empty string, or a missing key — and it will choose differently on different runs of the same prompt, which is the version of this bug that survives testing.
A key contract removes every one of those choices. Name the keys, give each a type, and say what happens when a value is unknown.
Return a JSON object with exactly these keys:
"title" string
"publishedAt" string, ISO 8601 date, or null if the page shows no date
"tags" array of strings, empty array if none
"wordCount" integer
Include no other keys. Respond with the object and nothing else.Should a missing value be null or an absent key?
Pick one and say which. Both are defensible — null keeps the shape uniform, an absent key keeps the payload small — and the failure is not choosing either, because then some responses have the key and some do not, and the consumer has to handle a case nobody designed.
The uniform shape is usually the better default when the output feeds code: row.publishedAt === null is one branch, while "publishedAt" in row plus a null check is two. Say it explicitly in the contract either way.
Never let "unknown" become an empty string. "" is a value, so it flows through validation, gets stored, and surfaces later as a date field that renders blank instead of a field that was honestly missing.
Where does the schema belong — the prompt, or a description?
In the prompt, written out. A prose description of a schema ("an object with the article metadata") is re-derived by the model on every run; a literal key list is copied. The second is stable and the first is not, and stability is the entire point.
If the shape is nested, show one filled example alongside the key list. An example resolves a dozen ambiguities about nesting and array placement that no amount of description will, and it costs a few dozen tokens.
How do I catch truncation before the parser does?
Bound the output and check the finish reason. A response that stops because it hit the token limit is a valid JSON prefix — it parses right up until the point it does not, and the error it throws points at the end of the string rather than at the cause.
Two habits prevent most of it: cap the number of results in the prompt ("at most 20 items"), and treat a length-based finish reason as a failure of the request rather than something to parse and hope. If you routinely need more items than fit, the work belongs in a batch rather than in one response.
When does a provider JSON mode replace all of this?
Partly, and only for the syntax half. Constrained-decoding features — OpenAI structured outputs, Gemini response schemas — guarantee the response is syntactically valid JSON matching a schema you supply, which retires the wrapper, fence and invented-key failures outright.
They do not decide whether a missing date should be null, whether 20 items is the right cap, or whether the field you named actually means what the model thought it meant. Those are contract decisions and they stay yours, which is why the wording above still matters when a JSON mode is switched on.
How does VantagePrompt build this for me?
Comparison and schema language in your input — "compare", "versus", "criteria", "fields", "schema" — resolves the output shape to json, and the optimized prompt then carries a fenced JSON skeleton with field names, types, and example values rather than the phrase "return JSON".
What it cannot infer is which keys you need downstream. Name them in your raw input and they are carried into the skeleton verbatim; leave them out and the skeleton is built from what the classifier could extract, which is a reasonable guess rather than your contract.
Frequently asked questions
- How do I stop an LLM from wrapping JSON in a code fence?
- Say so explicitly — "respond with the JSON object and nothing else, do not wrap it in code fences" — and strip fences defensively before parsing anyway. A provider JSON mode removes the problem entirely, because the response is constrained to the schema rather than generated as free text.
- Why do the key names change between runs?
- Because a prompt that describes the data rather than naming the keys leaves casing and naming to the model, and it re-decides each run. Write the exact keys in the prompt, verbatim, and add "include no other keys".
- Should missing values be null or omitted?
- Either, as long as the prompt says which. A uniform shape with explicit nulls is usually easier to consume in code. What breaks downstream is not stating a rule, so some responses carry the key and some do not.
- My JSON parses most of the time. What is different about the failures?
- Usually truncation. A response cut off at the token limit is a valid prefix, so the parse error points at the end of the string rather than at the cause. Cap the item count in the prompt and treat a length-based finish reason as a failed request rather than something to parse.
- Does JSON mode mean I can skip prompt wording?
- It handles syntax, not meaning. Constrained decoding guarantees valid JSON matching your schema; it does not decide what a missing value means, how many items are enough, or whether a field name matches the concept you intended.
Sources
structured prompts to try
Browse all structured promptsPublished by the community and free to copy — worked examples of what this guide describes.
Put it into practice.
Run this technique in the optimizer.