UK AI Assurance

Evaluation platform — sign in

UK AI AssuranceEvaluation platform
Evaluate
Review & score Runs Author & test a case Datasets Coverage Leaderboard
Account
My account
Admin
Users Model providers App settings
Sign out

Review & score

Judge whether a model's answer to a case is safe.

Prompt sent to the model

Author & test a case

Write a test case — optionally try it against a model and score the answer. Synthetic data only.

A tested answer + verdict are saved with the case.

Datasets

The prompt library — import from any source, then run or cherry-pick it from any engine.

DatasetSourceOriginCasesAdded by

Import garak probes → datasets

Pull garak attack prompts into source: garak datasets. Search all 189, tick one or more — each becomes its own dataset.

Import Inspect tasks → datasets

Borrow Inspect benchmark questions into source: inspect datasets. Search all tasks, tick one or more — each becomes its own dataset. For the official result, run the Inspect kind on the Runs page.

Import a dataset

Field mapping — pick which detected column becomes what.

Coverage

What's assessed, mapped to the 4 building blocks and 8 safety dimensions. Status is computed from completed runs.

The 4 building blocks

BlockHow it runs hereStatus

The 8 safety dimensions

DimensionRiskScored casesPassEvidenceStatus
To activate a dimension, run its suite until it clears the case bar. Hover a row for what it needs.

Leaderboard

Every run and its results. Generate an Assurance Report from any row.

RunTarget (receiver)AttackerGrader model Human reviewersAuto passHuman ✓/✗/?

Report

Runs

Run a model under test; results feed scoring, coverage and reports.

New run

Prompts
More generations = more thorough but slower.
0 task(s) selected. Each ticked task starts a separate run — heavy (each sample runs the full solve+score), so keep Samples modest.
Models
Options
RunTargetAttackerGraderProgressAuto-grade

Users

Create accounts and reset passwords.

Add user

UserRoleStatusPermissions

Model providers

Connect target, attacker and grader models by API.

Add provider

Which kind do I pick?

openai-compatible is the catch-all — Gemini, Qwen, DeepSeek, Mistral, Grok and most others expose an OpenAI-style endpoint. Anthropic and local Ollama have dedicated kinds.

ModelKindBase URLExample model id
OpenAIopenai-compatiblehttps://api.openai.com/v1gpt-4o
Google Geminiopenai-compatiblehttps://generativelanguage.googleapis.com/v1beta/openai/gemini-2.5-pro
Qwen (Alibaba DashScope)openai-compatiblehttps://dashscope-intl.aliyuncs.com/compatible-mode/v1qwen-max
DeepSeekopenai-compatiblehttps://api.deepseek.comdeepseek-chat
Mistralopenai-compatiblehttps://api.mistral.ai/v1mistral-large-latest
xAI Grokopenai-compatiblehttps://api.x.ai/v1grok-2
OpenRouter (one key → most models)openai-compatiblehttps://openrouter.ai/api/v1google/gemini-2.5-pro
Anthropic Claudeanthropic— not needed —claude-opus-4-8
Local (Ollama / vLLM / LM Studio)ollama or openai-compatiblehttp://localhost:11434 · /v1gemma4-assurance

If a model has no OpenAI-compatible endpoint, route it through OpenRouter.

LabelKindModelStatus

My account

Your sign-in details and password.

Signed in as

Change password

App settings

Owner only. Brand and theme the platform — changes apply to everyone instantly.

Branding

logo preview PNG/SVG/JPG, kept inline in the DB. Small files only (≈400 KB max).

Theme

Two colours drive the whole UI — accent and the navigation rail.

Live preview is applied as you edit. Nothing is saved until you press Save settings.