§4.2 — Causal / interventional methods

B1. Activation patching (causal mediation / interchange intervention)

• Key papers: origins: Vig et al., NeurIPS 2020, https://arxiv.org/abs/2004.12265 ; best practices: Zhang & Nanda, ICLR 2024, https://arxiv.org/abs/2309.16042 ; Heimersheim & Nanda 2024, https://arxiv.org/pdf/2404.15255
• Reveals: which components causally mediate a behavior (clean/corrupted runs; noising vs denoising directions differ in meaning).
• Cost: 2 forwards per (component, prompt-pair); exhaustive sweeps O(#components). Time-heavy, memory-light — well-suited to Macs 0.5B–8B; 14B with patience/quantization. Hook overhead MLX vs MPS = Experiment H.
• Implementations: TransformerLens (MPS opt-in), NNsight https://github.com/ndif-team/nnsight , pyvene, baukit.
• Failure modes: metric (logit-diff vs prob) and corruption type (Gaussian vs symmetric token swap) change localization conclusions; backup/hydra compensation; dormant-pathway subspace illusions; multi-mediator interactions (Curse of Multiple Mediators, 2026, https://arxiv.org/html/2606.27510v1).
• Epistemic status: Established core method; hyperparameter sensitivity well-documented. methodology.md must pin corruption method + metric per map type, before scanning.

B2. Path patching

• Key papers: Wang et al. (IOI), ICLR 2023, https://arxiv.org/abs/2211.00593 ; Goldowsky-Dill et al. 2023, https://arxiv.org/abs/2304.05969 ; accelerated: APP 2025, https://arxiv.org/html/2511.05442
• Reveals: causal importance of specific edges/paths, not just nodes.
• Cost: combinatorially heavier than node patching. Practical ≤3B–7B on Mac; 14B painful.
• Failure modes: metric sensitivity + hypothesis-space explosion; faithfulness not guaranteed (Miller et al. 2024, https://arxiv.org/pdf/2407.08734).
• Epistemic status: Established; IOI replicated (even in Mamba, https://arxiv.org/html/2407.14008v1) but faithfulness swings with ablation details.

B3. Attribution patching (AtP/AtP*) and gradient-based screening

• Key papers: Nanda 2023, https://www.neelnanda.io/mechanistic-interpretability/attribution-patching ; Syed, Rager, Conmy, BlackboxNLP 2024, https://aclanthology.org/2024.blackboxnlp-1.25.pdf ; AtP*: Kramár et al. 2024, https://arxiv.org/abs/2403.00745 ; EAP-IG: Hanna et al., COLM 2024, https://arxiv.org/abs/2403.17806
• Reveals: first-order approximation of every patch effect in ~2 forwards + 1 backward; AtP* fixes false negatives (QK softmax linearization + GradDrop).
• Cost: cheapest per-component causal estimate, but backward ≈ 2–3× weight memory: ≤3B fp16 on 16–32 GB, ≤7–8B on 64 GB. Gradients through 4-bit quantized models are awkward — a real Mac constraint. MLX autodiff is the native path worth benchmarking (Experiment H).
• Failure modes: bad approximation for large activations (residual, MLP0, saturated attention); systematic false negatives (quantified in AtP*); second-order failures (When Attribution Patching Lies, 2026, https://arxiv.org/pdf/2606.09899); LRP alternative RelP (2025, https://arxiv.org/html/2508.21258v1).
• Epistemic status: Established as screening; never a substitute for real patching on final claims.

B4. Causal tracing / ROME-style locating

• Key papers: Meng et al., NeurIPS 2022, https://arxiv.org/abs/2202.05262 (code https://rome.baulab.info/) ; critique: Hase et al., NeurIPS 2023, https://arxiv.org/abs/2301.04213
• Reveals: where factual associations appear "stored" (noise subject tokens, restore states; mid-layer MLPs at last subject token).
• THE central cautionary tale for modelmap: Hase et al. — edit success essentially uncorrelated with tracing-identified layers (ρ≈−0.13 at GPT-J layer 6); editing works almost anywhere; localization ≠ editability. Gaussian noising also distorts activations off-distribution.
• Epistemic status: Contested / debunked-in-part (the locating→editing inference); the tracing observation itself replicates.

B5. Ablations (zero, mean, resample)

• Key analyses: Zhang & Nanda (B1); Miller et al. 2024 (ablation choice flips faithfulness scores).
• Reveals: necessity (not sufficiency) of a component.
• Cost: cheapest intervention class; 1 forward per ablation. Ideal for Experiment B (localization vs diffusion).
• Failure modes: zero-ablation is off-distribution (can create signal); mean-ablation destroys within-distribution variance; resample > mean > zero is the emerging norm; hydra effects mask importance; unit scans → multiple-comparison inflation.
• Epistemic status: Established, with distribution-dependence caveats.

B6. Causal scrubbing

• Key sources: Chan et al. (Redwood) 2022, https://www.alignmentforum.org/posts/JvZhhzycHu2Yd57RN/causal-scrubbing-a-method-for-rigorously-testing ; pitfalls: https://www.alignmentforum.org/posts/DFarDnQjMnjsKvW8s/practical-pitfalls-of-causal-scrubbing
• Reveals: quantitative faithfulness score for a full interpretability hypothesis via all claimed-behavior-preserving resample ablations.
• Cost: expensive; fine ≤1B on Mac, research-grade beyond.
• Note: Redwood's rust_circuit is archived; no maintained macOS-friendly implementation exists — a genuine gap.
• Epistemic status: Established conceptually, little-used; succeeded in spirit by circuit-faithfulness metrics.

B7. DAS / interchange intervention accuracy / pyvene

• Key papers: Geiger et al., CLeaR 2024, https://arxiv.org/abs/2303.02536 ; Boundless DAS: Wu et al., NeurIPS 2023, https://arxiv.org/abs/2305.08809 ; pyvene: https://arxiv.org/abs/2403.07809 , https://github.com/stanfordnlp/pyvene
• Reveals: whether a high-level causal model is implemented in learned subspaces (train a rotation; IIA = fraction of counterfactuals matched).
• Cost: moderate–high; Boundless DAS ran on 7B — Mac-feasible ≤7B (gradients only through rotation + partial graph).
• The headline dispute: Makelov, Lange, Nanda 2023, https://arxiv.org/abs/2311.17030 (trained subspaces can work via dormant causally-disconnected pathways) vs Wu, Geiger et al. reply 2024, https://www.alphaxiv.org/abs/2401.12631 . Genuinely unresolved — trained-subspace claims need convergent evidence and a distinct flag in confidence.md.
• Epistemic status: Framework established; subspace-illusion question contested. RAVEL provides standardized evaluation.

B8. Steering vectors (ActAdd, CAA, ITI)

• Key papers: ActAdd: Turner et al. 2023, https://arxiv.org/abs/2308.10248 ; CAA: Rimsky et al., ACL 2024, https://arxiv.org/abs/2312.06681 ; ITI: Li et al., NeurIPS 2023, https://arxiv.org/pdf/2306.03341
• Reveals: behaviors as linear residual-stream directions — sufficiency-style causal evidence.
• Cost: very cheap; among the most Mac-friendly causal techniques. repeng + llama.cpp applies control vectors to GGUF quantized models.
• Critiques (the 2024–2026 evaluation wave): Tan et al., NeurIPS 2024, https://arxiv.org/abs/2407.12404 (steerability input-heterogeneous, sometimes anti-steers); Steering off Course, ACL 2025, https://aclanthology.org/2025.acl-long.974/ (fails to transfer across 36 models); AxBench 2025, https://arxiv.org/abs/2501.17148 (prompting/finetuning beat representation steering); non-identifiability (2026, https://arxiv.org/pdf/2602.06801); geometric predictors of failure (2026, https://arxiv.org/pdf/2602.17881).
• Epistemic status: Steering can work — established; reliability contested. Always report against prompting baselines + random-direction controls. Opportunity: publish steerability maps with variance.

Cross-cutting 2024–2026 signals

1. Faithfulness (not circuit overlap) became the standard comparison metric (EAP-IG) — then shown non-robust itself (Miller et al.).
2. Patchscopes unifies lens-style readouts with interventions.
3. Field syntheses: Open Problems in Mechanistic Interpretability (Sharkey et al. 2025, https://arxiv.org/abs/2501.16496); Bereska & Gavves review 2024, https://arxiv.org/pdf/2404.14082 ; Dead Salmons (2025, https://arxiv.org/pdf/2512.18792) — null-model controls, aligned with charter §8.4.
4. Tooling: NNsight/NDIF paper https://arxiv.org/abs/2407.14561 benchmarks NNsight vs TransformerLens/pyvene/baukit. TransformerLens MPS remains opt-in. Plan: baukit-style raw hooks on MPS + native MLX hooks, benchmark both (Experiment H). No mainstream interpretability library targets MLX — publishable gap.