How to Write a System Prompt That Actually Works

Most engineers who write system prompts are essentially guessing. They write something, run it a few times, decide it looks roughly right, and move on. The result is prompts that work in the demo, break in production, and nobody can debug because nobody really understands why the prompt was written the way it was.

This is fixable. System prompt writing is a learnable skill with clear patterns. Here's a practical framework for writing prompts that produce consistent, reliable output.

What a system prompt actually does

A system prompt sets the context for every conversation that follows. It tells the model who it is, what it should do, what format it should respond in, and what constraints it should respect.

The model doesn't "remember" your system prompt between turns in a way that degrades over time — it re-reads it at the start of every request. This means the system prompt has full influence on every response. A weak system prompt means every response is weaker than it could be.

The failure mode isn't usually a prompt that produces wrong output. It's a prompt that produces unpredictable output — sometimes right, sometimes not, with no clear pattern. That's the sign of a vague prompt.

Start with role, task, and output format

Every effective system prompt has three components at minimum:

Role. Tell the model explicitly what it is. Not just "you are a helpful assistant" — that's too vague to be useful. Be specific: "You are a meeting note processor. Your only job is to extract action items from meeting transcripts." The more precisely you define the role, the more reliably the model stays in it.

Task. Describe exactly what the model should do with the user's input. What transformation is it applying? What decision is it making? What information is it extracting? Vague tasks produce vague output. "Summarise this meeting" will produce something different every time. "Extract every action item from this meeting, including the person responsible and the deadline" produces something measurable.

Output format. Tell the model exactly how to structure its response. If you want a numbered list, say so. If you want JSON, give a schema. If you want three sentences, say three sentences. Leaving the format open means the model will choose — and it will choose differently on different days.

Be specific about what to include and what to ignore

One of the most common prompt mistakes is telling the model what to do without telling it what not to do.

In a meeting transcript, there might be side discussions, off-topic comments, and things that sound like action items but aren't ("we should probably look into that at some point"). A prompt that says "extract action items" will often include these. A prompt that says "extract action items — only include items where someone is explicitly assigned responsibility. Do not include vague suggestions or general discussion." will not.

Negative constraints — explicit statements of what to exclude — are often the difference between a prompt that works and one that sort of works.

Handle edge cases explicitly

Real input is messy. Meeting transcripts have interruptions. Documents have ambiguous passages. Queries don't always have clear answers in the available context.

A good system prompt specifies what the model should do in these cases. "If no action items are present, respond with: No action items found." is better than hoping the model figures it out. "If the relevant information is not in the provided context, say so explicitly rather than speculating." is better than getting a confident-sounding hallucination.

Think about the inputs that will break your prompt — the edge cases, the ambiguous cases, the empty cases — and handle them in the prompt itself.

Keep it shorter than you think

There's a tendency to make system prompts longer as more edge cases emerge. The result is a prompt that contradicts itself, buries the most important instructions in the middle, and causes the model to lose focus.

A well-structured 150-word prompt will outperform a chaotic 800-word prompt almost every time. If your prompt is getting long, ask: what is the single most important thing this prompt needs to accomplish? Make that the first paragraph. Cut everything else to supporting detail.

The model pays more attention to instructions at the start and end of the prompt than in the middle. Put your critical constraints where they'll be read.

Test against adversarial inputs

The easiest way to test a system prompt is to give it inputs that should work. The more valuable test is inputs that should fail cleanly — edge cases, off-topic inputs, ambiguous inputs — and checking that the model handles them the way you intended.

A prompt that produces good output for easy inputs is a starting point. A prompt that handles hard inputs gracefully is one you can actually rely on.

Concretely: after writing a prompt, generate five inputs that should work and five that should challenge it. Run all ten. Look for where the output deviates from what you wanted. Fix the prompt, not the examples.

What separates good prompt writers from great ones

The difference isn't vocabulary or familiarity with prompting tricks. It's discipline about specificity. Great prompt writers write exactly what they mean. They don't use "helpful" when they mean "concise". They don't use "summarise" when they mean "extract the three most important decisions". They don't leave the output format open when they need a specific structure.

Vague prompts produce vague output. The fix is always more specificity, not more length.

---

Want to test your system prompt writing against a real task? TryCrucible's prompt challenges give you a fixed task, a real test input, and an AI-scored result in under 10 minutes — no setup needed. Try the Meeting Action Items challenge →