Level 2 · Small-model research on one GPU
Does a published sparse-autoencoder result hold on a different open model – and exactly where does it stop holding?
Train sparse autoencoders with SAELens on a model the original authors didn’t use, reproduce one specific published finding, then push until it breaks and characterise the failure.
Reproductions on new models are what interpretability teams actually need, and ‘where it breaks’ is the novel part. Lab researchers have said publicly that improving or stress-testing existing papers is an excellent way to prove yourself.
01Pick one finding, not a whole paper
Choose a single, checkable claim from an SAE paper (for example, how features split as dictionary size grows, or a specific feature type existing). Write the claim in one sentence.
02Install SAELens and do the official tutorials
SAELens v6 restructured training code, so use the current tutorials (loading pre-trained SAEs, training an SAE), not old blog snippets.
$ pip install sae-lens03Reproduce on the original model first
Load a pre-trained SAE for the original model (browse on Neuronpedia) and confirm you can see the finding at all.
04Train SAEs on your new model
Pick a different open model of similar size. Train SAEs at 2–3 dictionary sizes on one layer. Track reconstruction loss, L0 and dead features.
05Test the claim on the new model
Run the same analysis. Does the finding hold, partly hold, or not hold? Quantify it.
06Push it until it breaks
Vary layer depth, dictionary size, training tokens or input domain. Find the condition where the finding stops holding and characterise it.
07Write it up with negative results
Clear claim, what replicated, what didn’t, and your best explanation. Negative results are fine if they’re rigorous.