The Instruction Position Problem: Where You Place Things in Your Prompt Is an Architecture Decision

You wrote a clear system prompt. You tested it in the playground and it worked. You deployed it. Three weeks later, a user figures out that your safety constraint doesn't reliably fire — not because of a clever jailbreak, but because you placed the constraint after a 400-token context block that you added in the last sprint. The model just… forgot it was there.

This is the instruction position problem, and it's not a bug in your prompt. It's a structural property of how transformer-based models process sequences. Every token in your prompt does not receive equal attention. Where you place an instruction determines, in a measurable way, whether the model will follow it.

Most teams discover this through unexplained regressions. A prompt that performed fine starts misbehaving after a routine update that "only added more context." The actual cause — instruction position shift — rarely appears in post-mortems because engineers don't have the mental model to look for it.

The U-Shaped Attention Curve

Research on long-context LLMs has consistently identified what's now called the "lost in the middle" phenomenon: when relevant information is placed at the beginning or end of a prompt, models use it correctly; when it's placed in the middle, performance degrades significantly.

In multi-document question answering tasks, moving the answer-relevant document from position 1 to a middle position in a 20-document context causes accuracy to drop by 30% or more. This wasn't a niche finding. It held across GPT-3.5, GPT-4, and purpose-built long-context models. Extended context windows didn't fix it — they just moved the cliff further out.

The root cause sits in how position embeddings work. Rotary Position Embeddings (RoPE), used in most modern LLMs, introduce a distance decay: attention between two tokens weakens as the distance between them increases. This is an intentional design choice — it makes local context more salient than distant context, which is useful for language modeling. But it has a structural side effect: tokens in the middle of a long sequence receive weaker aggregate attention than tokens at the endpoints.

The result is a U-shaped performance curve. Beginning and end are reliable. The middle is not.

Instruction Compliance Degrades with Position

The lost-in-the-middle effect applies to retrieval tasks. What about instructions?

The same positional bias shows up in instruction-following benchmarks. Research measuring reliability across nuanced prompt variations found that modern models show up to 61.8% performance variance when instructions are reworded or repositioned, even when the semantic intent stays constant. The degradation is highest for behavioral constraints — the instructions that say what the model must never do.

Critical rules buried in the middle of a long system prompt degrade compliance rates by 30–50% compared to the same rules placed at the beginning. The model isn't ignoring them consciously. Its attention mechanism is structurally underfunding them relative to content that appears closer to the sequence endpoints.

This creates a particular failure mode for system prompts that grew organically. A prompt that started at 150 tokens and worked fine gradually accumulated feature-specific instructions, role descriptions, domain knowledge, and format specifications until it reached 800 tokens. The safety constraint that was originally first is now at token position 200 out of 800 — squarely in the middle. Its compliance rate dropped without anyone changing it.

Primacy Beats Recency, but Both Beat Middle

Across studies measuring serial position effects in LLMs, primacy dominates: information at the beginning of a prompt is used correctly in roughly 73% of test cases where positional bias is detectable. Recency matters too, but is less consistent — the model reliably attends to the last few hundred tokens, but this is more exploitable by adversarial inputs that appear at the end of the user turn.

What this means practically:

• Place your hardest constraints first. Behavioral guardrails, non-negotiable rules, output format requirements — these belong at the top of the system prompt, before any context or examples.
• Exploit recency for task framing. The final instruction before the user input is well-attended. Use this position for task-specific cues that should shape the immediate response.
• Middle of the system prompt is for content the model needs to know, not rules it must follow. Domain knowledge, role descriptions, and few-shot examples can live in the middle. Instructions with compliance consequences cannot.

The Instruction Hierarchy Problem

A subtler version of this problem appears when you have multiple instructions in the same prompt and they implicitly conflict. Research on instruction hierarchy compliance found that GPT-4o-mini fails to correctly resolve conflicting constraints 47.5% of the time; Claude handles it better at around 23.6% failure rate, but it's far from reliable.

The failure mode usually isn't that the model rejects the conflict — it's that the model silently picks one instruction to follow and discards the other. Which one it picks depends partly on position (earlier instruction typically wins) and partly on framing.

The instruction sandwich pattern emerged as one response to this. For safety constraints that must hold regardless of user input, placing them at both the beginning and the end of the system prompt — with user input in between — exploits both primacy and recency simultaneously. The version at the beginning establishes the constraint; the version at the end reinforces it before processing the user's request. It's redundant by design.

This is not a complete solution. It doesn't work against prompt injection from retrieved content that appears after the final safety block. But it materially improves compliance for well-defined behavioral constraints in non-adversarial contexts.

What "Prompt Architecture" Actually Means

Once you accept that position is a design variable, the question becomes: how do you structure a system prompt?

A workable ordering for most applications:

1. Core identity and role (brief, ~20 tokens) — establishes what the system is before anything else
2. Non-negotiable behavioral constraints — safety rules, privacy requirements, absolute prohibitions
3. Output format and structure requirements — JSON schema, tone, length limits
4. Domain context and knowledge — background information the model needs but doesn't need to prioritize
5. Few-shot examples — demonstrations of expected behavior
6. Task-specific instructions for this request (repeated at end if critical)

The first three categories contain compliance-sensitive instructions and belong at the top. Categories 4 and 5 contain content the model should use but isn't required to follow as rules — the middle is appropriate for them.

A second principle: keep the base system prompt small. Long monolithic system prompts bloat the KV cache, reduce effective context for user input, and distribute attention across a larger space where middle-positioning effects compound. The recommended practice is to keep a base prompt at 200–300 tokens and load feature-specific instruction blocks conditionally. If the user is performing task A, append instruction block A. If they're on the B flow, append B. This approach keeps critical instructions close to the sequence endpoints and avoids burying them under irrelevant context.

Measuring Your Prompt's Positional Sensitivity

Most teams discover prompt positional issues in production, which is too late. The fix is to build sensitivity testing into your evaluation process before deployment.

A basic positional sensitivity test reorders the same set of instructions across multiple prompt variants and measures compliance consistency. You take your critical instructions, place them at positions 0–20%, 40–60%, and 80–100% of the total prompt length, and compare how often the model follows them in each configuration. If your compliance rate on a safety constraint is 95% when it's in the first quartile and drops to 65% when it's in the middle quartile, that constraint is position-sensitive and needs to be pinned near the top.

For RAG pipelines, the analogous test retrieves multiple documents, places the answer-relevant document at different positions across runs, and measures retrieval accuracy by position. The U-shaped performance curve will appear in virtually every sufficiently long-context scenario. The shape of the curve tells you where your accuracy cliff is.

LIFBench, a benchmark from ACL 2025, formalizes this kind of evaluation with 2,766 instructions across long-context scenarios, measuring compliance at beginning, middle, and end positions separately. The finding most directly applicable to system design: models often perform better when constraints are ordered from most-difficult to least-difficult rather than the intuitive reverse. Counter to human documentation conventions, leading with the hardest rule tends to improve overall compliance across all constraints.

The benchmark to build for your own system is simpler. Pick your five most compliance-critical instructions. Test each one at three positions in a representative prompt. If any instruction shows more than 15% compliance variance across positions, treat that as a structural issue requiring prompt architecture changes, not prompt wording changes.

Position Sensitivity Is a CI Failure Mode

The last thing to internalize: prompt updates can introduce positional regressions silently.

A change that adds 200 tokens of context to your system prompt changes the position of every instruction that follows it. The instructions didn't change. Their position did. If your compliance-critical rules were in the second quartile, they're now in the third. You won't see this in a standard output comparison because the model still generates plausible responses — it just follows the buried constraint less reliably.

This is why prompt testing needs to include positional checks, not just output quality checks. Treating prompts as code means tracking how positional changes affect compliance, not just whether the outputs look reasonable. A prompt CI suite should include a positional sensitivity regression test that fires when a change shifts critical instructions past a position threshold.

The instruction position problem won't go away. Positional attention bias is a structural property of transformer architectures, and the fix isn't to wait for a model update — it's to design your prompts knowing that position matters, test the constraints that cannot fail, and treat any prompt update as a potential compliance regression until proven otherwise.

---

The "lost in the middle" finding originated in a Stanford study; instruction reliability variance data comes from ICLR 2025 work on nuanced instruction following; serial position effect quantification uses the SPEM methodology from a 2024 arXiv study covering 104 model-task combinations.