A decision and a ledger record at every stage
Thulpar sees all nine stages of an agent run. At each stage it runs the rails the policy pack assigns to it. Below are the parts of the engine, each with a simulation that runs in your browser.
Each streamed sentence is checked and can be retracted.
The gateway splits the model’s stream into sentences and checks each one at the output_chunk stage. When a sentence must be masked or stopped, the client receives a retraction frame and the masked text. Existing OpenAI clients keep working.
Streaming simulation →The pack sets the rule, the engine applies it.
A policy pack is a JSON document: the rails and their settings, tool groups, sink classes, Cedar policies, approval, retention and evidence profiles. Packs layer in org, tenant, app and agent order; a locked setting cannot be changed at a lower layer. Every version is signed.
Policy sandbox →A hold pauses the run until a person approves.
@advice("hold:<profile>") in Cedar or onViolation: hold in the sink policy opens an approval record. The profile names the approvers, the four-eyes requirement, the SLA and the escalation target. Approval is given in the console, over a webhook or on Aginies’ approval card; every approval is its own ledger event.
End-to-end run →A hash-chained ledger and a signed evidence pack
Every event joins the tenant’s chain with hash = SHA-256(previous hash ‖ canonical JSON). The ledger stores the hash of the content, never the content itself; when a PII rail ran, a redacted sample of at most 2 KB sits beside it. The chain is anchored daily and verified nightly.
Ledger explorer →Nine stages, seven possible decisions
The caller names its stage and sends the content and context. Thulpar runs that stage’s rail graph. The response carries the decision, the findings, the actions to apply and the ledger id. Actions are redaction spans, a rewrite or a prepend.
Stage
- input
- User or caller text before the model
- retrieval
- RAG chunks before they enter a prompt
- tool_call
- Tool name and arguments before execution
- tool_result
- Tool output before it re-enters the model
- output
- The complete model response
- output_chunk
- Every sentence of a stream
- memory_write
- What is about to be persisted for later turns
- file
- Bytes and metadata before storage
- agent_trace
- A whole trajectory, asynchronous
Decision
- allow
- No findings; content passes as is
- allow_with_findings
- Findings, no action; written to the ledger (the forced outcome of shadow mode)
- redact
- Content returned with masks applied; actions[] carry the spans
- rewrite
- Content replaced, for example a refusal template
- prepend
- Text is added before the answer, for example an AI notice
- deny
- Content is stopped
- hold
- An approval record opens; the caller pauses, the run resumes on approval
- escalate
- Allowed now, an async judge queued; the result goes to the ledger and the webhook
Mode
- enforce
- Default; the decision is applied
- shadow
- Full evaluation, nothing is blocked; the caller is told what enforce would have decided
- audit
- Asynchronous: 202 + ledgerId; checks the caller cannot wait for
- off
- Rail off, per pack and scope
Latency budget (p50, design)
- input
- 150 ms · sync; speculative ticket in parallel with generation when the caller supports it
- tool_call
- 50 ms · Cedar plus deterministic validators
- output
- 250 ms · sync or async with a retraction frame
- output_chunk
- 40 ms · per sentence
- retrieval
- 100 ms · per batch
32 built-in rails in six groups
Every rail declares the stages it runs at, the detector class it binds to and how it fails. Security rails ship with a closed posture. When such a rail cannot run, the check is denied. Quality rails ship with an open posture. The pack can change the posture per rail. Built-in rails run in-process and need no detector.
Input rails
| rail | stages | class | |
|---|---|---|---|
| injection.direct | input | injection | Direct prompt injection: rules plus an mmBERT classifier together; Turkish and English at the same quality |
| jailbreak | input | jailbreak | Jailbreak patterns and role-play escapes; a small guard LLM, a larger one on the GPU tiers |
| content.input / content.output | input / output, output_chunk | content_safety | Content safety: AILuminate categories plus Turkish extensions; per-category thresholds, sentence by sentence on streams |
| toxicity | input, output | toxicity | Toxicity and offensive language; a separately trained Turkish model |
| pii.detect / pii.redact | every text stage | pii | Personal data: TCKN, IBAN, card, plate and phone with checksum validation; KVKK Art. 6 special categories flagged separately; span-based masking |
| pii.health | input, output, retrieval, file | pii | Health data: disease, medication, procedure, symptom; TR/EN lexicon plus NER; the tenant adds its own terms |
| secretsbuilt-in | input, output, tool_result, file | rules | API keys, tokens, PEM private keys, credentials in URLs; both directions |
| languagebuilt-in | input, output | rules | Sentence-level language detection; allowed languages; routes TR and EN detector variants |
| topic | input, output | topic | Allowed and denied topics: embedding similarity, built-in politics and four advice kinds, counter anchors |
Output rails
| rail | stages | class | |
|---|---|---|---|
| injection.indirect | retrieval, tool_result, file | injection | Indirect injection from tool output, RAG chunks and files; the content is labelled untrusted and cannot become a new tool call without policy |
| injection.indirect.melon | tool_call | injection | MELON masked re-execution sampling: the agent’s intent is independently verified on write/exec calls |
| groundedness | output | groundedness | Groundedness: every sentence of the answer against the supplied context.sources; an unsupported sentence is a finding, optionally rewritten |
| system_prompt.leakbuilt-in | output | rules | System-prompt leak: canary tokens and fuzzy matching |
| exfilbuilt-in | output, tool_call | rules | Markdown image and link exfiltration patterns, PII in URL parameters, denied domains |
| conversationbuilt-in | input, output | rules | Mandatory AI disclosure (EU AI Act Art. 50), forbidden promises, persona and dialogue state across turns |
| code / code.tool_result | output, tool_result | code | Insecure patterns in generated code (CodeShield, Semgrep) with CWE ids; insecure code a tool returns is denied |
Tool and agent rails
| rail | stages | class | |
|---|---|---|---|
| tool.authzbuilt-in | tool_call, memory_write | cedar | Cedar policies: default deny; permit / forbid / @advice("hold:<profile>"); tenant → app → agent inheritance |
| tool.argsbuilt-in | tool_call | rules | Argument facets, size caps, a JSON schema per tool, SSRF classes, the egress allowlist |
| tool.args.injection | tool_call | rules | SQL, command, path-traversal, RFI and XSS injection in tool arguments: libinjection plus an OWASP CRS subset, by JSON path |
| tool.trajectory | agent_trace, tool_call | agent_trace | Agent trajectory: goal drift, privilege creep, chained calls; on the GPU tiers |
| tool.schema_pinbuilt-in | MCP tools/list | rules | MCP tool schemas are hashed and pinned; a changed schema is quarantined; description hygiene |
| budgetbuilt-in | input, tool_call | rules | Per-run token, cost, tool-call and depth caps; circuit breaker |
Information flow and memory
| rail | stages | class | |
|---|---|---|---|
| labels.sinkbuilt-in | tool_call | labels | Information-flow labels (confidentiality, integrity, tenant, PII class) enforced at sensitive sinks: egress, messaging, export, write, exec |
| memory.writebuilt-in | memory_write | rules | Memory poisoning: provenance required; untrusted content and instruction-like text cannot persist; memory authorisation with Cedar |
Files and images
| rail | stages | class | |
|---|---|---|---|
| file.analyze / file.type | file, input (ekler) | file | Type sniffed from the bytes; name/type mismatch, bombs, encrypted files; PDF, Office, images, QR; OCR |
| file.active_content / file.hidden_text | file | file | Macros, JavaScript, OLE, external templates, embedded files; invisible text and hidden instructions |
| file.metadata_pii / file.qr | file | file | EXIF/XMP location, device, author; QR URLs to denied domains or carrying PII |
| image.content_safety | file, input | image | Image moderation: a multimodal guard on the GPU tiers |
| yara | text stages, file bytes | rules | YARA rule sets: the tenant’s rules plus a shipped baseline (web shells, download chains, private keys); matched bytes are never logged |
Quality rails
| rail | stages | class | |
|---|---|---|---|
| schemabuilt-in | output | rules | Structured output: JSON schema, regex, validators; reask on failure |
| self_check / fact_check | input / output | policy_judge | LLM-as-judge with tenant-written criteria; asynchronous on lower tiers, the finding written to the ledger afterwards |
| banlist / competitorsbuilt-in | output | rules | Banned terms (Turkish suffixes, İ/ı folding) and competitor names |
The pack sets the rule, the engine applies it.
Activation refuses a closed rail that nothing implements. A failing red-team benchmark can block activation.
The sandbox below loads the banking example pack. Raise the amount above 1,000 TRY. Type an e-mail address as the recipient. Point the URL at a private address. Each of the four rails shows its decision.
The pack on the left is the banking example: Cedar policies, tool groups, sink policies and approval profiles. Edit it; the tool call on the right is evaluated through the tool.args → pii.detect → labels.sink → tool.authz rails.
Tool call
Decision
Rail graph
Cedar policies
Facets (context)
- argumentBytes
- 66
- stage
- tool_call
- isDeployed
- true
- currency
- TRY
- amount
- 2500
A deterministic gate before every tool call
tool.args validates arguments against the schema, size caps and SSRF classes. tool.args.injection catches SQL, command and path-traversal patterns with libinjection and OWASP CRS. labels.sink enforces the confidentiality and integrity labels carried with the content at the sink class. tool.authz permits, forbids or holds with Cedar over default deny. The four rails decide together within milliseconds. The model judge runs only when asked, and mostly asynchronously.
Changed tools are quarantined before the agent sees them.
On every tools/list answer the proxy hashes the schemas and compares them with the pinned version. A changed tool is quarantined. The diff waits for approval in the console. Instructions such as “always call” in descriptions are stripped. tools/call arguments pass the same tool rails. Results pass the indirect-injection and PII rails. The proxy is an OAuth 2.1 resource server. The audience is verified and no token is passed through.
The proxy receives every tools/list answer, hashes each tool’s schema over canonical JSON and compares it with the pinned version. A changed tool is quarantined; instruction patterns in the description are stripped; arguments are scanned with tool.args.injection.
Tools (tools/list)
Try a tools/call · orders:list
Proxy log
Each streamed sentence is checked and can be retracted.
One frame type is enough. Existing OpenAI clients keep working.
As the model streams, every sentence passes the pii.redact and content.output rails at the output_chunk stage. When a sentence carries a customer secret the stream frame is retracted in band and the masked sentence sent; the client needs to understand one frame type.
What the client sees
Stream frames
A hash-chained ledger and a signed evidence pack
The evidence pack is a signed ZIP per period and profile. It holds the control catalogue with statuses, the summary, the decisions, the chain verification, the approvals and the policy versions in force.
Twelve events from one tenant’s day: every hash is SHA-256(previous hash ‖ canonical JSON), computed in your browser. Pick an event and change its content; the chain breaks from that point and verification shows it.
Try the injection rail’s rule layer in your browser.
The rule layer below uses the product’s own patterns and Turkish folding. In the product, an mmBERT classifier runs beside it. The GPU tiers add guard LLMs.
The product’s rule layer (rules-injection) runs in your browser: the same patterns, the same Turkish folding. In the product an mmBERT classifier runs beside it and guard LLMs on the GPU tiers; here only the rules.
Decision · injection.direct
Findings (1)
crosswalk
An agent run from start to finish
Two scenarios play out step by step. They show the rails, the decisions, the approval and the ledger entries. The sector pages carry forty more like these.
A customer pays card debt from the mobile channel; the IBAN is masked and a transfer above the threshold passes four-eyes approval.
See it in your own environment.
A one-hour technical demo: your scenario, your policy pack, on a closed-circuit install. Then the install kit and a trial licence for a pilot.