Open-source verification infrastructure for AI systems · v0.2.0
Declare.
Observe.
Verify.
Know whether your AI system does what you said it would, before release and in production.
Verify a single API response or a multi-hour agent coding session. Check a support agent's answers, an uploaded document against your policy, or every step of a long-running workflow.
Core platform
One loop, three steps.
- 01Declare
Describe the system in YAML, and bind the Verifiers that check them.
- 02Observe
Record what the system does using Wyrd's purpose-built high-throughput data plane.
- 03Verify
Verify behavior in real time or in the background using Wyrd's verification engine.
Get started
From intent to evidence.
One support agent and one Verifier, one step per tab: declare what a ticket must contain, hand the ticket to the verification engine, and read the verdict.
The Service binds one Verifier. The Verifier declares the evidence it takes and the requirement it checks.
apiVersion: wyrd/v1
kind: Service
metadata:
name: support-desk
version: 1.0.0
space: default
spec:
service_type: agent
verified_by:
- verifier: ./ticket-complete.yaml# A code check: sandboxed Python reads the ticket and names every field the
# help desk needs that is missing. No model.
apiVersion: wyrd/v1
kind: Verifier
metadata:
name: ticket-complete
version: 1.0.0
space: support
spec:
description: A support ticket carries the fields the help desk needs.
inputs:
ticket: {type: json}
requirements:
- id: ticket-complete
statement: The ticket has an answer and an order id
check:
kind: code
entrypoint: check
inputs: {ticket: inputs.ticket}
source:
inline: |
def check(ticket):
missing = [field for field in ("answer", "order_id") if not ticket.get(field)]
return {
"outcome": "failed" if missing else "passed",
"reason": "missing " + ", ".join(missing) if missing else "all fields present",
"citations": [{"slot": "ticket", "locator": {"json_pointer": ""}}],
}Inside a Run, hand the agent's ticket to the verification engine. verify waits for the verdict; submit queues the check and returns at once.
from pydantic import BaseModel
class Ticket(BaseModel):
question: str
answer: str
order_id: str | None = None
run = state.run()
with run:
ticket = Ticket(question=question, answer=answer, order_id=order_id)
# Wait for the verdict.
result = run.observe.verify("ticket-complete", ticket=ticket)
# Or queue it and return at once.
execution = run.observe.submit("ticket-complete", ticket=ticket)let run = state.run();
let ticket = json!({ "question": question, "answer": answer, "order_id": order_id });
// Wait for the verdict.
let result = run.observe().verify("ticket-complete", ticket.clone()).await?;
// Or queue it and return at once.
let submission = run.observe().submit("ticket-complete", ticket).await?;const run = state.run();
const ticket = { question, answer, order_id: orderId };
// Wait for the verdict.
const result = await run.observe.verify("ticket-complete", { ticket });
// Or queue it and return at once.
const submitted = await run.observe.submit("ticket-complete", { ticket });The verdict cites the evidence it judged. A failed check is a result, not an exception.
{
"execution_id": "019a3c2e-7b41-7d2a-9c1e-4f6b8a2d0e13",
"run_id": "019a3c2e-7a90-7f11-8b3d-2e5c9d1a6f47",
"verifier": {"kind": "Verifier", "name": "ticket-complete", "version": "1.0.0", "space": "support", "uid": "019a3c2e-6f12-7c4d-8e31-5a9b0c7d2e64"},
"subject": {"kind": "Service", "name": "support-desk", "version": "1.0.0", "space": "default", "uid": "019a3c2e-6f0a-7b22-9d4e-1c8f3a5b6d70"},
"outcome": {
"verdict": "failed",
"requirements": [
{
"id": "ticket-complete",
"statement": "The ticket has an answer and an order id",
"outcome": "failed",
"reason": "missing order_id",
"citations": [
{
"slot": "ticket",
"locator": {
"json_pointer": ""
}
}
],
"check": {
"kind": "code",
"source": {
"inline": "def check(ticket):\n missing = [field for field in (\"answer\", \"order_id\") if not ticket.get(field)]\n return {\n \"outcome\": \"failed\" if missing else \"passed\",\n \"reason\": \"missing \" + \", \".join(missing) if missing else \"all fields present\",\n \"citations\": [{\"slot\": \"ticket\", \"locator\": {\"json_pointer\": \"\"}}],\n }\n"
},
"entrypoint": "check",
"inputs": {
"ticket": "inputs.ticket"
}
}
}
],
"steps": [],
"evidence": {
"ticket": {
"kind": "json",
"value": {
"question": "Where is my order?",
"answer": "Your order A-1001 ships tomorrow.",
"order_id": null
}
}
}
}
}{
"execution_id": "019a3c2e-7c08-71b5-a2f4-9d3e6b1c8a50",
"status": "completed",
"result": {
"execution_id": "019a3c2e-7c08-71b5-a2f4-9d3e6b1c8a50",
"run_id": "019a3c2e-7a90-7f11-8b3d-2e5c9d1a6f47",
"verifier": {"kind": "Verifier", "name": "ticket-complete", "version": "1.0.0", "space": "support", "uid": "019a3c2e-6f12-7c4d-8e31-5a9b0c7d2e64"},
"subject": {"kind": "Service", "name": "support-desk", "version": "1.0.0", "space": "default", "uid": "019a3c2e-6f0a-7b22-9d4e-1c8f3a5b6d70"},
"outcome": {
"verdict": "failed",
"requirements": [
{
"id": "ticket-complete",
"statement": "The ticket has an answer and an order id",
"outcome": "failed",
"reason": "missing order_id",
"citations": [
{
"slot": "ticket",
"locator": {
"json_pointer": ""
}
}
],
"check": {
"kind": "code",
"source": {
"inline": "def check(ticket):\n missing = [field for field in (\"answer\", \"order_id\") if not ticket.get(field)]\n return {\n \"outcome\": \"failed\" if missing else \"passed\",\n \"reason\": \"missing \" + \", \".join(missing) if missing else \"all fields present\",\n \"citations\": [{\"slot\": \"ticket\", \"locator\": {\"json_pointer\": \"\"}}],\n }\n"
},
"entrypoint": "check",
"inputs": {
"ticket": "inputs.ticket"
}
}
}
],
"steps": [],
"evidence": {
"ticket": {
"kind": "json",
"value": {
"question": "Where is my order?",
"answer": "Your order A-1001 ships tomorrow.",
"order_id": null
}
}
}
}
},
"dispatches": []
}Also in the platform
Evidence others can check.
Role-based access, OIDC sign-in for people, and API keys for workloads. Delegated tokens (RFC 8693 token exchange) keep calls made on someone's behalf attributed to them. Every access decision is written to the audit log.
Route LLM calls through the Wyrd AI Gateway. Each call is authorized for the calling agent, attributed to it, and recorded for end-to-end verification.
Python, TypeScript, and Rust SDKs, OpenTelemetry (OTLP) ingest, HTTP, CLI, and MCP. You run the server yourself, on Postgres and object storage.