Enterprise prompt engineering treats prompts as part of the software: each prompt lives in version control, follows a set structure, returns output that matches a schema, is scored against a test set before every change, and is rolled out in stages with monitoring. That discipline separates a prompt that works in a demo from one that holds up for a year of real inputs. Remolda builds it into production features in the six-week AI Pilot Sprint, a fixed price of $9,800 CAD plus applicable taxes, and teaches it in our prompt engineering training.
Building or fixing an AI feature: send a written request with the task, the model you use and where the output goes, or call +1 (613) 867‑7767. We reply within one business day.
Prompts are code
Once a prompt drives a production feature, a change to it changes how the system behaves. Manage it like any other code change.
- Version control. Prompts and their examples live in the repository. Each change has a commit, a reviewer and a history, so a regression can be traced and rolled back.
- Review. Prompt changes go through code review. In regulated work they also go through the review that covers how the system decides: for federally regulated financial institutions, OSFI Guideline E-23 on model risk management, in effect from May 1, 2027, includes AI models.
- Tests before release. Every change runs against the test set, and the scores decide; "it seemed better" is an impression.
- Monitoring after release. Production output is tracked so drift shows up in days.
The LLM integration guide shows the thin integration layer that owns prompts, model calls, validation and logging in one module.
The five parts of a system prompt
The system prompt is the standing instruction for every request an application sends. In production it usually has five parts.
| Part | What it contains |
|---|---|
| 1. Role and purpose | What the application does, for whom, and its limits |
| 2. Rules | What it must always do and must never do |
| 3. Output format | Fields, types, allowed values, how to mark missing data |
| 4. Domain context | Your categories, terms, thresholds, approval matrix |
| 5. Escalation | Out-of-scope inputs, low confidence, when to hand to a person |
Role and purpose. "You are a helpful assistant" gives the model nothing to work with. "You classify incoming contracts for the legal department by type, extract the listed key terms and route each contract to a practice group" does.
Rules. State them as rules, with no softening:
- Always return a JSON object matching the schema.
- Always include a confidence score from 0.0 to 1.0 for each extracted field.
- Always set
requires_human_review: truewhen confidence is below the threshold you set. - Always cite the document section each extraction comes from.
- Never put names, addresses or account numbers in free-text fields.
- Never mark a document for automatic approval above the value limit in your approval matrix.
Output format. Spell out every field:
Return a JSON object with these fields:
- document_type: one of ["NDA", "MSA", "SOW", "employment", "lease", "other"]
- parties: array of strings (full legal names)
- effective_date: ISO 8601 date string or null
- termination_provisions: short summary or null
- key_risks: array of strings, one risk per item
- confidence: number between 0.0 and 1.0
- requires_human_review: boolean
- review_reason: string if requires_human_review is true, else null
If a field cannot be determined from the document, set it to null. Do not guess.
Domain context. The model knows general language; your taxonomy, your terms and your thresholds have to be in the prompt.
Escalation. Models tend to answer confidently when they should say they are unsure, unless the prompt sets escalation rules. Name what is out of scope, the confidence threshold and the wording for handing a case to a person.
Examples in the prompt
A few input-output examples (few-shot) make structured extraction and classification more consistent.
- Cover the hard cases. Include incomplete documents, unusual layouts, borderline categories and inputs that should trigger human review.
- Show near misses. An agreement that looks like an NDA but is a consulting contract, with a note on the features that tell them apart.
- Scale to the task. As working numbers, we start with three to five examples for simple classification and eight to fifteen for extraction with several linked fields, then adjust from test results.
- Keep examples in their own file. They are data, maintained alongside the prompt and inserted when the prompt is built, so they can change without touching the instructions.
Structured output
When another system reads the model's output, the format is an interface contract.
- Use the API's schema support. OpenAI Structured Outputs (
strict: truewith a JSON schema) and Claude structured outputs (output_config.format) constrain the response to your schema. OpenAI recommends Structured Outputs over plain JSON mode, which only guarantees valid JSON. - Describe the schema in the prompt as well. Writing it out in plain language exposes ambiguous fields before production does.
- Validate before use. Anthropic documents that a refusal or reaching the token limit can return output that does not match the schema, and no schema checks business rules. Failed items go to a review queue with the raw output attached.
- Decide what "missing" looks like.
null, an empty string and"unknown"mean different things downstream. Pick one per field.
Test sets and metrics
Building the test set
- Coverage. Every document type, every category, every edge case you know of.
- Real proportions. If most real inputs are NDAs, most test cases are NDAs.
- Extra edge cases. Unusual inputs appear more often in the test set than in real traffic, because that is where failures hide.
- Hostile inputs. Prompt-injection attempts, ambiguous cases, truncated documents.
As working numbers, we start with about 100 real, anonymized cases with known good answers and grow the set from production feedback. Higher-stakes uses, such as decisions about credit or health, need more cases and a domain expert's sign-off.
Metrics by task
| Task | Metrics |
|---|---|
| Classification | Accuracy and F1 per class; a confusion matrix |
| Extraction | Field accuracy, completeness of required fields, whether confidence predicts accuracy |
| Open-ended text | Expert review against a short rubric |
Track results per category. An average can hide a category that fails every time.
Expert review
Have a domain expert review a sample of production outputs every week. Their findings update both the metrics and the test set.
Prompt injection
Prompt injection is the first risk in the OWASP Top 10 for LLM applications (LLM01:2025). It comes in two forms: direct, when a user's input changes the model's behaviour, and indirect, when a web page or file the model reads carries the instruction. Any system that reads untrusted documents or user input faces it.
OWASP lists seven controls, and we apply all of them in production features:
- Constrain model behaviour in the system prompt.
- Define and validate expected output formats.
- Filter inputs and outputs.
- Give tools least privilege: the model proposes an action, your code checks permissions.
- Require human approval for high-risk actions.
- Separate and label external content so the model treats it as data.
- Run adversarial tests and attack simulations before release and after changes.
Releasing and monitoring prompt changes
A prompt change follows the same path every time:
- Write down the reason for the change.
- Run the full test set; compare with the current version.
- Release to a small share of traffic and compare output distributions with the baseline.
- Release fully only when the staged run shows no regression.
- Watch closely for two weeks before calling it stable.
Track in production: the share of outputs that pass schema validation, the distribution of categories, the distribution of confidence scores and the human-review rate. A sudden change in any of them points to a prompt regression, a model update or new kinds of input.
Pin model versions. Use dated model versions where the provider offers them, and treat a model upgrade like a prompt change: full test set, staged release, two weeks of monitoring.
How Remolda helps
- Prompt engineering training: a 3–4 hour workshop for daily users and a one-day builder track for staff who write system prompts or build assistants, on your own tasks; quoted after a 30-minute call. See prompt engineering training.
- AI Pilot Sprint ($9,800, six weeks): one production feature with prompts, examples, test set, validation and monitoring in your repository; see custom AI development and AI API integration.
- AI Readiness Review ($490, one week): which tasks are ready for a production prompt, and the data rules around them.
Prices are in CAD plus applicable taxes and listed on the pricing page. When prompts alone cannot reach the accuracy you need, our comparison of RAG and fine-tuning covers the next options, and staff rules for everyday AI tools belong in an AI use policy.
Get a second review of a system prompt or test set. Send a written request describing the feature and its current error rate. We reply within one business day.
Sources
- OWASP GenAI Security Project: LLM01:2025 Prompt Injection
- OpenAI: Structured Outputs
- Anthropic: Structured outputs
- OSFI: Guideline E-23, Model Risk Management (2027)

