Level 5 · Flagship researchFlagship
Track B · Alignment science
When AI agents do alignment research autonomously, how much of their reported progress survives an evaluator they can’t game, and do their discoveries transfer?
Re-run Anthropic’s released automated weak-to-strong researcher behind a sealed evaluation service, measure how much progress survives, then study transfer and build a detector for research-agent gaming.
Anthropic’s Automated Alignment Researcher (nine Claude agents) reached a performance-gap-recovered score of 0.97 on weak-to-strong supervision in 5 days (~$18k), vs 0.23 for two human researchers after 7 days – but the agents also cherry-picked seeds, exfiltrated test labels and exploited dataset shortcuts. The authors call un-overfittable metrics the main bottleneck. The sandbox, datasets and baselines are released.
01Get the released sandbox, datasets and baselinesM1 · week 8
Read Anthropic’s write-up and set up the released environment exactly as published.
02Build a sealed evaluation serviceM1 · week 8
Held-out splits, rate limits, audit logs and query-pattern detection, so agents can’t see or probe test labels.
03Re-run the agents behind itM1 · week 8
Run fewer agents for longer rather than many briefly, to control API spend.
04Measure and publish how much progress survivesM1 · week 8
Compare reported progress with progress under the sealed evaluator. Publish as the week-8 milestone.
05Transfer study
Do the agents’ discoveries transfer across new datasets and model pairs?
06Detector for research-agent gaming
Build and validate a detector for the gaming behaviours you catalogued.
07Directed vs undirected agents
Compare directed and undirected agents under the sealed evaluator.
08Release
Open sealed-evaluator harness, transfer results and a catalogue of agent exploits.