🔬 Circuit Visualizations

Interactive circuit diagrams from sparse pretrained models

IOI

1024-dim
Click to view circuit →
3072-dim
Click to view circuit →

Pronoun

1024-dim
Click to view circuit →
3072-dim
Click to view circuit →

Question

128-dim
78 nodes →
1024-dim
Click to view circuit →
3072-dim
Click to view circuit →

Never Found Important (d1024)

Nodes that never appeared in any circuit across 100 seeds

Pronoun - Zero Ablation
14 nodes →
Pronoun - Mean Ablation
11 nodes →
IOI - Zero Ablation
7 nodes →
IOI - Mean Ablation
14 nodes →
Question - Zero Ablation
8 nodes →
Question - Mean Ablation
15 nodes →