modelmap.io

Independent interpretability research · Apple Silicon

An internal cartography of
local large language models

Given an already-trained open-weight model, what can actually be known, measured, localized, and mapped about its internal organization — where knowledge lives, how computation is distributed, which structures are stable across inputs, layers, scales, and model families — using only a consumer-grade Mac? Every map published here is versioned, provenanced, confidence-labeled, and regenerable.

what a probe reports ≠ what the model computes ≠ what is stable across methods ≠ what survives intervention

269sources reviewed
24research gaps identified
1/12experiments completed
0atlas maps published

The evidence standard

A claim enters the atlas only at the confidence level its evidence supports. Correlational maps and causally-verified maps are never conflated — visually or textually. The two metrics tracked for every map are its replication rate (does it reproduce under resampling of seeds and data?) and its causal confirmation rate (what fraction of localized claims survive intervention?).

  1. L0
    anecdotalsingle run, no controls — never published alone
  2. L1
    correlationalcontrolled, replicated ≥3 seeds, ≥2 datasets
  3. L2
    method-robustLevel 1 + agreement across ≥2 independent techniques
  4. L3
    causalLevel 2 + intervention confirms the claim

Negative results — techniques whose maps do not replicate — are first-class, publishable findings here.

Research phases

  • Phase 1 — Ultra-deep literature research (§4.1–4.10)10 theme notes · 269 sources
    in progress
  • Phase 2 — State-of-the-art maptechnique taxonomy with epistemic status
    done
  • Phase 3 — Research gaps24 falsifiable candidate directions
    done
  • Phase 4 — Candidate ranking10-axis scoring, 3–5 prototype candidates
    done
  • Phases 5–6 — Framework & micro-experiments A–H1/12 experiments completed
    in progress
  • Phases 7–9 — Prototypes → methodologyevidence-driven; the methodology is a primary deliverable
    pending
  • Phase 10 — Atlas pipeline & this platformplatform live · 0 maps published
    in progress
  • Phase 11 — Novelty verificationassume not novel until evidence suggests otherwise
    pending

Latest from the research log

2026-08-12 05:30 EDT — Phase 5 substrate + expH run #1 (FIRST MEASURED RESULT)

Question. Build the substrate; then: which activation-store format wins on APFS, and what does capture actually cost on MLX vs PyTorch-MPS?

Built.

  • src/modelmap/atlas/mapcard.py — map-card schema v0 (G23): provenance + per-level requirements (L1: ≥3 seeds, ≥2 promptsets, replication_rate; L2: ≥2 methods; L3: interventions + featurizer class). tools/publish.py is the gate: refuses any atlas entry without a valid card.
Full research log →

2026-08-12 03:50 EDT — Phase 4: candidates ranked and selected

Question. Which of the 24 gaps become prototype candidates, and in what order does everything else get built?

Action. Wrote research/candidate_ranking.md: full 24×10 score table (convention: 10 always favorable, including complexity and risk), grouped written reasoning, and selection. Updated experiments/candidate_01–04 READMEs with assignments.

Full research log →

2026-08-12 03:20 EDT — Phase 3: 24 research gaps registered

Question. Which substantially different directions are (a) genuinely open per the Phase 1/2 evidence, (b) falsifiable with a Mac-scale experiment, (c) worth ranking in Phase 4?

Action. Wrote research/research_gaps.md: 24 gaps (G01–G24) in seven clusters — quantization×internals (G01–G05), weight-only pre-screens (G06–G09), replication/method-agreement science (G10–G13), cross-model coordinates (G14–G16), localization science (G17–G19), systems/tooling

Full research log →