Validation record
LIMEN13 0.10.1: technical validation
Software version 0.10.1. Verified on 14 November 2026 (America/Toronto). Run by the project itself: this is not an independent audit.
In plain words. Before each release we run more than a thousand automatic tests and a library of simulated attacks. This page gives the exact results for release 0.10.1, including the two attacks it does not stop.
Results
Verification results
| Gate | Result |
|---|---|
| Full verification script | 1,086 checks passed, 0 failed |
| Python test suites | 29 suites |
| Console JavaScript checks | 7 passed (included in the total) |
| Reference acceptance test | 42 of 42 passed |
| Attack library | 73 of 73 expected blocks; 2 documented limits |
| Security scan | Clean: Bandit, dependency advisories, supply-chain check, Gitleaks, Semgrep |
| SBOM | Regenerated from the locked dependencies (CycloneDX) |
| Container build and smoke test | Built; the health endpoint, the console and the simple view answered; the image health check passed |
| Runtime hardening | Non-root, read-only root filesystem, all capabilities dropped, no new privileges |
Categories
What the checks cover
- Engine and gateway decisions
- Console and access control, route by route
- Threat cases and hardening
- Model gateway and budgets
- Alert delivery and recovery after restart
- Discovery, collectors and bypass probes
- MCP client and server sides
- Cryptography and data at rest
- Customer isolation with two engines
- Supply-chain checks
- Kill switch and containment
- Evidence coverage and reporting
Definitions
How results are defined
- Check passed
- An automated assertion produced the expected result.
- Expected block
- An attack case that LIMEN13 refused, held or contained, as the case defines.
- Documented limit
- An attack case that LIMEN13 does not stop. It is kept in the library on purpose and described below.
- Failed
- An assertion that did not produce the expected result. Count in this release: 0.
Limits
The two documented limits, and the others
- Rewriting an unsigned lab log. In lab mode without a signing key, a log can be rewritten. This is not a bypass of the signed, anchored production design.
- An agent making a false promise. An agent can tell a customer something untrue inside an allowed message. LIMEN13 checks actions, not truthfulness.
- Calls already admitted to an external service may finish after containment.
- An alert accepted just before a crash can be delivered again.
- Budget accounting is estimated; it is not a provider billing cap.
- These tests establish correctness for the cases tested, not a sustained high-volume capacity.
Corrections
Issues found and corrected in this review
- Invalid budget or delivery state no longer resets silently: startup refuses it with a recovery error.
- A containment that happens during a slow tool check or an approval re-check now refuses the action before it executes.
- Interrupted model streams keep conservative usage accounting instead of counting missing output as zero.
- Alert delivery progress no longer skips an earlier event whose record arrives late.
Measured
Real-engine demonstration and local latency
Run on 15 November 2026 with the real engine (release 0.10.1) and the fictional reference client, on one development laptop. This is not the browser simulation, and it is not a production measurement.
| Step | What the engine answered |
|---|---|
A test agent asks for a tool in its role (list_new_orders) | HTTP 200, decision: allow |
The same run asks for a tool outside its role (delete_customer) | HTTP 403, decision: deny, rule: role, reason: tool 'delete_customer' is not in this agent's role |
| Evidence written | Two signed records for the run: one allow, one deny, each with its rule and reason |
| Chain verification | ok: true, signed: true |
| Workload | Requests | p50 | p95 | p99 | Throughput |
|---|---|---|---|---|---|
| Allowed calls, 1 client | 1,000 | 4.95 ms | 8.49 ms | 9.93 ms | 177 per second |
| Refused calls, 1 client | 300 | 4.71 ms | 5.62 ms | 6.44 ms | not measured |
| Allowed calls, 4 clients at once | 400 | 36.18 ms | 43.07 ms | 63.57 ms | 109 per second |
| Allowed calls, 16 clients at once | 1,600 | 177.46 ms | 234.57 ms | 265.72 ms | 86 per second |
- Local development figures. Client and engine on one laptop (AMD Ryzen 5 4600H, 12 threads, 18 GB, Linux, Python 3.14), over the loopback interface, with a test tool that answers from local files. A real network and a real business system add their own time.
- One engine handles calls one at a time. Throughput did not rise with more clients, and waiting time grew. Plan one engine per workspace and measure with your own load.
- Not measured: memory and processor use, behaviour during a dependency failure, long runs, and model-gateway calls.
- Reproduce
.venv/bin/python deploy/bench_gateway.py- Evidence artefacts
- The raw results are given to pilot participants on request.
Reproduce
How to reproduce
The software is delivered as source, so pilot participants can run the same scripts on their own machine.
- All checks
./deploy/verify.sh- Acceptance test and attack library
./deploy/verify_attacks.sh- Security scans
./deploy/security_scan.sh- Evidence artefacts
- The verification report and the SHA-256 list of the reviewed files are given to pilot participants on request. The source repository is private.
Pending
Not yet done
- An independent penetration test.
- Validation with a customer's own single sign-on, SIEM, vault and MCP servers.
- Load tests on a deployment.
- Backup and restore objectives.
- Published support commitments.