§4.10 — Visualization and atlas design

Prior atlases and browsers

• Neuroscope https://github.com/neelnanda-io/Neuroscope — max-activating examples for every MLP neuron in 25 small models. The canonical suggestive-but-unfalsifiable atlas: max-activating examples are actively misleading (Bolukbasi illusion); no controls, no uncertainty, no versioned provenance; unmaintained. Generation itself is cheap and Mac-feasible.
• Neuronpedia https://www.neuronpedia.org/ (open-sourced 2025: https://github.com/hijohnnylin/neuronpedia) — the field's central artifact host: 50M+ latents, 5+ TB, per-feature dashboards (activation histograms, logit effects, auto-interp scores), steering, attribution-graph hosting, full REST API. The modelmap gap: no per-entry reproducibility contract — no commit/config/hardware manifest that regenerates a dashboard, no confidence taxonomy; hosted data changes without versioned releases. A local Neuronpedia instance is a plausible serving layer for modelmap viewers rather than rebuilding dashboards.
• OpenAI Microscope (2020, vision) https://openai.com/index/microscope/ — exhaustive per-unit coverage + canonical "model organisms": both ideas modelmap inherits. No uncertainty/provenance.
• Activation Atlases (Distill 2019) https://distill.pub/2019/activation-atlas/ — the literal "atlas" ancestor; UMAP layouts are seed/hyperparameter-sensitive with no stability analysis — the beautiful-but-unfalsifiable failure mode charter §2 warns about. Never done for LLMs with replication controls.
• Anthropic Scaling Monosemanticity interface https://transformer-circuits.pub/2024/scaling-monosemanticity/ — activation spectra across strength deciles = an explicit anti-cherry-picking device worth copying; but features hand-picked, underlying model closed, artifact unversioned.
• circuit-tracer + attribution-graph frontend https://github.com/decoderesearch/circuit-tracer ; hosted graphs https://www.neuronpedia.org/graph/info ; library paper https://aclanthology.org/2025.blackboxnlp-1.14.pdf — first mainstream browser with a built-in verification affordance (feature interventions to test graph hypotheses). Limitations: graphs describe a replacement model; faithfulness bounded, not displayed per-edge; pruning thresholds shape what you see; no confidence levels or provenance manifests.
• Transluce Monitor / neuron descriptions https://transluce.org/observability-interface ; https://github.com/TransluceAI/observatory — LLM-generated descriptions of all 458,752 neurons of Llama-3.1-8B + live steer/ablate UI. Auto-interp descriptions unreliable (metrics can fail to distinguish trained from random transformers: https://arxiv.org/pdf/2501.17727); no causal-verification labels, no versioned provenance.
• BertViz https://github.com/jessevig/bertviz — most-used interp viz ever; attention ≠ explanation; a caution: high adoption of a low-epistemics tool shaped years of over-claiming.
• exBERT https://arxiv.org/abs/1910.05276 — early corpus-level aggregation (anecdote → statistics). Ecco https://github.com/jalammar/ecco — NMF factors without stability analysis. LM-Debugger https://arxiv.org/abs/2204.12130 — early observation+intervention coupling; single interpretive frame presented as ground truth.
• Gemma Scope / Llama Scope https://arxiv.org/abs/2408.05147 ; https://huggingface.co/google/gemma-scope ; https://arxiv.org/abs/2410.20526 — best existing example of interpretability artifacts released with reproducible identity (fixed weights, named hookpoints, versioned HF artifacts). Still: no per-feature confidence labels; seed-replication not surfaced. Running (not training) Gemma Scope SAEs on a 2B fits 16 GB — natural third-party input for modelmap cross-checks.

Provenance templates

• Datasheets for Datasets (Gebru et al. 2018) https://arxiv.org/abs/1803.09010 ; Model Cards (Mitchell et al. 2018) https://arxiv.org/abs/1810.03993 — direct templates for modelmap's "map cards": commit hash, config, model hash+quantization, promptset checksums, hardware manifest, seeds, controls run, replication stats, confidence level. No interpretability artifact host ships anything equivalent (verified across Neuronpedia, Neuroscope, Transluce, circuit-tracer).

Uncertainty encoding — cross-cutting finding

Across all tools surveyed: uncertainty is essentially never encoded visually. Partial exceptions: Neuronpedia auto-interp scores (a scalar of simulation fit, not a CI, not causal validity); Anthropic activation-spectrum deciles (anti-cherry-picking, not uncertainty); circuit-tracer prunable thresholds (manipulable, not summarized). Nobody displays seed-replication rates, bootstrap CIs, or correlational-vs-causal status on the artifact. The Level 0–3 badges have no prior art to copy — and none to compete with.

Novelty check — "model atlas" naming collision

• Horwitz et al. 2025, Charting and Navigating Hugging Face's Model Atlas, https://arxiv.org/abs/2503.10633 ; https://horwitz.ai/model-atlas — maps the inter-model graph of HF (which checkpoint fine-tuned from which), not model internals. Name collision to record in novelty_check.md, not a scope overlap.
• Verdict (2026-08-12): the project's bet is confirmed — no existing atlas of LLM internals combines versioned entries + regeneration recipes + explicit confidence levels + published negative results. Closest partials: Gemma Scope (versioning without confidence), Neuronpedia (hosting/API without reproducibility contracts), circuit-tracer (verification affordance without provenance).

Design conclusions for modelmap.io

1. Copy: exhaustive per-unit coverage (Microscope), activation-spectrum deciles (Anthropic), intervention affordances (circuit-tracer, LM-Debugger), corpus-level aggregation (exBERT).
2. Avoid: max-activating-example-only views (Neuroscope), unstable DR layouts without stability analysis (Activation Atlases), auto-interp labels presented as ground truth (Transluce).
3. Differentiate: map cards (datasheet-style provenance), visible Level 0–3 badges, seed-replication rates and ablation-scheme curves rendered on the artifact, negative-results section.
4. Consider a local Neuronpedia instance as serving infrastructure; keep modelmap.io as the provenance-first publication layer.