How do we know an agent remains under control?

Research prototype · synthetic traces · Python

Recursive self-improvement (RSI) makes evaluation a moving target: the solver, its scaffolding, and potentially the measurement process can change between generations. My RSI Eval Lab asks whether an apparent capability gain survives scrutiny of the controls and evidence behind it.

I designed and implemented a deterministic auditor of recorded lineages, with example traces, tests, an optional Inspect integration, and a research agenda. It can be run locally without a model or API key.

1. Integrity of the evidence. Evaluator and monitor anchors are supplied outside the trace. Hash chains detect edits; a separate HMAC seal protects against recomputing the record when the candidate cannot access the key.
2. Controls across generations. Declared invariants track whether requirements such as full action logging survive modification, including breaches that later appear repaired.
3. What a score means. Challenge levels, public versus held-out scores, cost, and behaviour near rollback thresholds help expose results that need further investigation.