Find how your model fails before your customers do.
Tests built from your own work, checks that stop a bad release, and adversarial testing for the attacks that matter in your industry.
What you get
Evidence your second line can rerun.
A test set from your work
Built from real tickets and documents and labelled by your own experts, so “good” means good for your task.
Task metrics
Extraction accuracy, whether answers are supported by their sources, citation precision, correct refusals, latency and cost.
Release gates
Tests run on every change to the model, prompts or search index. A release is blocked when a metric drops below its threshold.
Red-team report
Prompt injection, including through retrieved documents, data exfiltration, jailbreaks and bias. Each finding is mapped to OWASP and MITRE ATLAS.
Calibrated judging
Where a model grades answers, we check its agreement with human labels and use a different model family as the judge.
Production monitoring
Sampled human review, drift dashboards and an incident playbook for when quality slips.
On model-graded tests
A model judging a model needs checking too.
Models used as judges favour longer answers, answers in a particular position, and answers from their own model family. We measure agreement with your experts’ labels, fix the grading rubric, use a judge from a different family, and spot-check by hand.
Tools we use
promptfoo · DeepEval · lm-evaluation-harness · garak · PyRIT
Frameworks
Tests mapped to the frameworks you report against.
| Framework | What it covers | How we use it |
|---|---|---|
| OWASP Top 10 for LLM Applications (2025) | Prompt injection, sensitive data disclosure, excessive agency and seven more. | Every red-team finding carries its OWASP ID. |
| MITRE ATLAS | Known attack techniques against AI systems. | Attack scenarios drawn from it and referenced in reports. |
| NIST AI RMF and Generative AI Profile (AI 600-1) | Managing and measuring AI risk. | Evidence organised for the Measure and Manage functions. |
| EU AI Act Art. 9 and 15 | Testing, accuracy and robustness for high-risk systems. | Results documented for your technical file. |
| MAS Guidelines on Artificial Intelligence Risk Management | Testing and monitoring across the AI lifecycle. | Release gates and monitoring records you can show. |
Questions
What validation teams ask.
Can our team rerun your tests?
Yes. The test sets, code and configuration are delivered to you and run in your environment.
How do you test injection through our documents?
We plant hostile instructions in test documents, emails and tool outputs and check the system ignores them.
How do you measure fairness in credit or claims decisions?
We agree the groups and metrics with your risk team, test outcomes across them and report the gaps.
What if a gate fails close to a deadline?
The release waits, or ships under an exception your accountable owner approves in writing.
What we don’t promise
Some claims we won’t make.
- “Jailbreak-proof” or “fully red-teamed”. Testing reduces risk; it doesn’t remove it.
- That a model is “validated”. Independent validation is your second line’s job. We give them evidence ready to validate.
Further reading
- Evaluation · 6 min readAnyone who can send you a document can talk to your agent.Most controls treat prompt injection as a bug waiting for a patch. It follows from how these models read, which means the only defence that holds is designing so that a successful injection is not worth anything.
- Evaluation · 5 min readYou can’t unit-test an agent. So how do you know it works?Normal software fails loudly and repeatably. Agents fail quietly, differently each time, and only on the cases you didn’t imagine. Here’s what testing one actually looks like.
Find the failures first.
We can also test a model you already run.