Test the application, not just the model

01

Start with the job and the harm

IMDA's starter kit for testing LLM-based applications frames testing as a pre-deployment discipline for safety and reliability. For a small team, the first move is to define the job precisely enough that a test can pass or fail.

• State the user, input, allowed output, prohibited output, and downstream action.
• List the credible harms from a wrong, unsafe, delayed, or unavailable result.
• Set the review level based on consequence. Drafting an internal outline and changing a payment instruction should not share the same release gate.
• Choose a manual fallback and the point where the workflow must use it.

02

Build a test set that resembles the real work

• Include ordinary examples, rare edge cases, incomplete inputs, conflicting instructions, and deliberately hostile inputs.
• Represent the languages, formats, terminology, and local context the real users will bring.
• Keep a separate holdout set that prompt authors do not tune against.
• For changing facts, store the expected source and date instead of treating one answer as permanently correct.
• Add cases from actual failures and human overrides after launch, with sensitive details removed.

03

Measure more than answer quality

Select measures that match the job. A workflow may need factual grounding, instruction adherence, correct refusal, stable formatting, secure tool use, low variance, or a reliable handoff to a person. Capture latency and cost when they affect whether the system can be operated consistently.

• Use deterministic checks for structure, required fields, citations, calculations, and tool results where possible.
• Use a written scoring rubric for judgment calls and calibrate reviewers on a sample before the full run.
• Track severe failures separately. A good average can hide one unacceptable outcome.
• Repeat a sample to expose variance, especially when the workflow can take actions or produce external communication.

04

Create the release record

• Record the model, prompt or workflow version, connected tools, dataset version, test date, reviewer, and results.
• List known limitations and the controls that reduce their impact.
• Require explicit approval from the person accountable for the business consequence.
• Set monitoring signals and a review date before the workflow goes live.
• Keep the ability to roll back to a known version or switch to the manual path.

05

What to run this week

Before release, demonstrate a normal run, an edge case, a refusal, a downstream failure, and a human takeover. The point is not to prove the system cannot fail. It is to prove the team can detect, contain, and recover from the failures that matter.