Hypothesis — expH run #1 (storage formats + hook overhead)

Capture cost frontier — what can you map on which Mac (macOS-specific). Run #1 scope: (A) activation-store formats on APFS, (B) capture/hook overhead in MLX vs PyTorch-MPS on a synthetic transformer. Registered 2026-08-12 before the first benchmark execution.

Hypothesis              : (A) For the SAE-shuffle access pattern (random row
                          batches), raw np.memmap sustains ≥ 2× the read
                          throughput of chunked zarr on APFS (warm cache),
                          because dense fp16 activations gain little from
                          compression while chunk decode adds latency.
                          (B) Activation capture (retain per-layer outputs)
                          costs < 2× plain inference on both MLX and
                          PyTorch-MPS at the 12-layer/d1024 scale, and the
                          device→CPU copy + disk write — not the hook/retain
                          mechanism — dominates capture cost.

Falsification criterion : (A) dies if zarr (any codec) matches or beats mmap
                          on random-batch reads (ratio < 1.25×).
                          (B) dies if retain-only capture overhead exceeds
                          2× plain inference on either backend; the "capture
                          is cheap on consumer Macs" premise dies at > 3×
                          including copy+write.

Method                  : Storage: 100k rows × 4096 dims fp16 (~0.8 GB) per
                          format (raw mmap, safetensors-mmap, zarr zstd, zarr
                          uncompressed); sequential write; sequential scan;
                          random-batch reads (4096 rows/batch, SAE-shuffle
                          pattern); 3 repeats/cell. Compute: synthetic
                          pre-norm transformer (12 layers, d_model 1024,
                          4 heads, seq 512, batch 8, fp16) implemented
                          identically in torch-MPS and MLX; modes = plain /
                          retain-on-device / retain+CPU-copy+mmap-write;
                          20 timed forwards (10 for write mode), 3 repeats,
                          explicit device synchronization.

Baseline / null         : plain forward (no capture) per backend; sequential
                          scan as the reference for random-read degradation.

Result                  : (pending)

Interpretation          : (pending — with explicit confidence level)

Next experiment         : (pending)

Known limitations, declared in advance. Warm-cache regime (the 0.8 GB store fits in the unified buffer cache): read numbers upper-bound the cold case; a purge/F_NOCACHE cold pass is a registered follow-up (run #2), and the sister project's cold-cache NVMe ceiling (localvm expH: ~13.1 GB/s at ≥256 KiB, QD≥4) bounds what run #2 can find. Synthetic model ≠ real checkpoint: run #3 will repeat mode timings on a real 0.5B model via mlx-lm.

---

Hypothesis — expH run #2 (cold-cache storage, second hardware)

Registered 2026-08-12 before the run. Host: MacLustr node M3U96a (Mac Studio, 32 cores, 96 GB), where sudo purge empties the unified buffer cache between repetitions — also our first cross-hardware replication point.

Hypothesis              : The warm-cache format ordering survives cold:
                          raw mmap sustains ≥ 1.5× zarr-uncompressed on
                          random-batch reads with a purged cache; absolute
                          throughputs drop for all formats; zarr-zstd's
                          relative gap narrows (decode overlaps disk IO)
                          but does not close.
Falsification criterion : Dies if any zarr variant matches or beats mmap
                          cold (ratio < 1.25×) — that would rehabilitate
                          chunked stores and overturn run #1's design call.
Method                  : Same grid as run #1 Part A, store enlarged to
                          200k×4096 fp16 (~1.6 GB), `sudo purge` before
                          every timed read repetition, 3 repeats/cell,
                          writes measured warm (unchanged claim).
Baseline / null         : run #1 warm numbers on M5 Max; sequential scan
                          as the random-read reference.
Result                  : (pending)
Interpretation          : (pending)
Next experiment         : (pending)

---

Hypothesis — expH run #3 (real quantized checkpoint via mlx-lm)

Registered 2026-08-12 before the run. Local M5 Max, model = mlx-community Qwen3-0.6B (4-bit), capture by wrapping decoder layers.

Hypothesis              : On a real 0.6B 4-bit model under MLX, per-layer
                          residual retention costs ≤ 1.15× plain prefill
                          (lazy-graph retention, as in run #1's synthetic
                          result), and retention + NumPy conversion + mmap
                          write stays ≤ 1.5×.
Falsification criterion : Dies at retain > 1.5× or retain+write > 2× —
                          would mean synthetic overheads do not transfer to
                          real checkpoints and all run #1 Part B numbers
                          must be re-derived on real models.
Method                  : Prefill of a fixed 1024-token prompt, batch 1,
                          10 timed forwards after 3 warmups, 3 repeats;
                          modes = plain / retain-all-layers / retain+write;
                          mx.eval barriers around each forward.
Baseline / null         : plain prefill throughput (tok/s).
Result                  : (pending)
Interpretation          : (pending)
Next experiment         : (pending)

Analysis — expH run #1 (storage formats + hook overhead)

Run: results/expH_capture_cost_frontier/20260812T052441Z/results.json Hardware: Apple M5 Max, 48 GB unified memory, macOS 27.0 · torch 2.13.0 (MPS) · MLX 0.32.0 · Python 3.14.4. Hypothesis registered before the run in hypothesis.md; config, seed, commit and full manifest embedded in the JSON.

Hypothesis              : (A) raw mmap ≥ 2× zarr on random-batch reads (warm);
                          (B) retain-capture < 2× plain inference on both
                          backends; copy+write dominates capture cost.
Falsification criterion : (A) dies at ratio < 1.25×; (B) dies at retain > 2×
                          or total > 3×.
Method / Baseline       : as registered (see hypothesis.md) — 3 repeats/cell,
                          plain-forward baselines, sequential-scan references.
Result                  : (A) CONFIRMED. Random-batch reads: raw-mmap 4.09 GB/s,
                          safetensors(mmap) 4.26, zarr-uncompressed 1.27,
                          zarr-zstd 0.38 → mmap/zarr ratio 3.2× (uncompressed)
                          and 10.8× (zstd), both past the 2× bar.
                          Writes: safetensors 8.10 GB/s (single large write),
                          raw-mmap chunked 2.82, zarr-uncompressed 1.49,
                          zarr-zstd 0.54. Seq scans: 3.9–7.3 GB/s all formats.
                          (B) CONFIRMED, asymmetrically. torch-MPS: plain
                          46.7 ms/fwd (±1.6) → retain 57.2 (1.22×) →
                          retain+copy+write 68.8 (1.47×). MLX: plain 36.2
                          (±2.6) → retain 36.9 (1.02×) → +copy+write 40.2
                          (1.11×). Retain never exceeds 1.25×; totals never
                          exceed 1.5× — far under the 2×/3× kill lines.
                          The "copy+write dominates" sub-claim holds on MLX
                          (+3.3 ms vs +0.7 ms retain) but on torch-MPS the
                          two costs are comparable (+10.5 ms retain,
                          +11.6 ms copy+write) — retain itself is expensive
                          on MPS, matching its eager hook materialization.
                          Bonus observation (not registered, Level 0): MLX
                          baseline is 1.29× faster than torch-MPS on the
                          identical architecture, and MLX capture is nearly
                          free (2%), consistent with lazy-graph retention.
Interpretation          : Level 0–1. Within-machine replication only
                          (3 repeats/cell, tight variance ≤ ±4 ms); single
                          hardware, single size class, warm cache, synthetic
                          model — all declared in advance. Design decisions
                          this licenses NOW: (1) activation stores are
                          mmap-backed flat files (raw or safetensors container
                          for self-description) — zarr is eliminated for the
                          SAE-shuffle pattern unless cold-cache reverses the
                          ordering; (2) MLX is the preferred capture backend,
                          with torch-MPS as the compatibility path; (3)
                          capture-cost is NOT the bottleneck at this scale —
                          the 4 TB/500M-token storage wall (notes §4.9)
                          remains the real constraint.
Next experiment         : run #2 = cold-cache pass (purge/F_NOCACHE) to test
                          whether the mmap>zarr ordering survives; run #3 =
                          same capture modes on a real 0.5B checkpoint via
                          mlx-lm vs transformers-MPS; then replicate the whole
                          grid on ≥2 cluster Macs (M3U96a, M2U64) to promote
                          the cost table toward Level 1 cross-hardware.

Notes

• zarr numbers use chunk = one write-block (4096×4096); different chunk geometries could narrow the gap — a registered caveat, not tested here.
• The torch-MPS retain overhead (22%) is per-layer-output retention of ~100 MB/forward; capture pipelines that subsample layers will scale it down roughly linearly.
• All timing cells and raw seconds are in the results JSON with the hardware manifest; the figure-ready aggregation is deferred to the atlas pipeline.

---

Analysis — expH run #2 (cold cache, second hardware) — HYPOTHESIS FALSIFIED

Run: results/expH_capture_cost_frontier/*_run2_M3U96a/results.json Hardware: Apple M3 Ultra (Mac Studio, MacLustr M3U96a), 96 GB, macOS. Store enlarged to 200k×4096 fp16 (~1.6 GB); sudo purge before every timed read repetition. Hypothesis registered before the run.

Hypothesis              : warm ordering survives cold — mmap ≥ 1.5× zarr on
                          random-batch reads.
Falsification criterion : dies if any zarr variant beats mmap (ratio <1.25×).
Result                  : FALSIFIED — the ordering INVERTS cold.
                          Random-batch (cold): zarr-uncompressed 0.62 GB/s,
                          safetensors(mmap) 0.32, zarr-zstd 0.19,
                          raw-mmap 0.14 → zarr-uncompressed beats mmap 4.4×.
                          Sequential (cold): zarr-uncompressed 3.47,
                          zarr-zstd 3.18, raw-mmap 0.64, safetensors 0.53.
Interpretation          : Level 0–1, and the mechanism is legible: cold mmap
                          reads are page-fault-driven — effectively ~8–16 KiB
                          random reads at queue depth 1, which localvm's
                          cold-cache SSD grid already showed run at
                          ~0.07–0.14 GB/s. zarr instead read()s whole 32 MiB
                          chunks, exactly the ≥256 KiB / high-QD regime where
                          Apple NVMe delivers. The warm-cache run #1 numbers
                          measured memory, not disk. Cross-project agreement
                          (modelmap run #2 ↔ localvm expH) is strong
                          convergent evidence for the real rule:
                          ***IO granularity decides, not the container.***
                          Design decision REVISED: for stores larger than
                          RAM, either use chunked containers or access mmap
                          in chunk-aligned batched reads; never row-level
                          random access. Caveat: run #1 was M5 Max, run #2
                          M3 Ultra — the warm-vs-cold contrast within run #2's
                          own machine is what carries the claim; hardware
                          confound noted for the cross-run comparison.
Next experiment         : run #4 (registered idea, not yet run): mmap with
                          chunk-sorted, ≥1 MiB batched reads + F_RDAHEAD vs
                          zarr-uncompressed, cold, same machine — decides the
                          final store design. Also replicate run #2 on the
                          M5 Max laptop for the hardware axis.

---

Analysis — expH run #3 (real 4-bit checkpoint via mlx-lm) — CONFIRMED

Run: results/expH_capture_cost_frontier/20260812T061742Z/results.json Model: mlx-community/Qwen3-0.6B-4bit (28 layers, d_model 1024), prefill of a 901-token prompt, M5 Max 48 GB. Hypothesis registered before the run.

Hypothesis              : retain ≤ 1.15× plain prefill; retain+write ≤ 1.5×.
Result                  : CONFIRMED, stronger than predicted.
                          plain 28.5 ms/prefill (31,644 tok/s) →
                          retain 28.6 ms (1.004× — capture is FREE) →
                          retain+copy+write 36.4 ms (1.28×, 24,733 tok/s).
Interpretation          : Level 0–1 (3 repeats, single machine/model). The
                          synthetic run #1 MLX result (1.02×) transfers to a
                          real quantized checkpoint. This run is also the
                          project's first QUANTIZED-model activation capture
                          in Python — the capability the Phase 1 survey found
                          nowhere (notes §4.9). Engineering note: quantized
                          embeddings pack weight shapes, so d_model must be
                          inferred from a runtime activation, and bf16
                          activations must be cast in MLX before NumPy
                          conversion (numpy has no bfloat16).
Next experiment         : same three modes at 1.7B/4B to trace the overhead
                          curve vs model size; then wire this Tap pattern
                          into src/modelmap/capture as the standard MLX
                          capture layer (expA depends on it).

expH_capture_cost_frontier

Capture cost frontier — what can you map on which Mac (macOS-specific)

Status: scaffolded — hypothesis to be registered before any benchmark runs (charter §10).