Apple compute report
Verified locally on 2026-09-07. The recommended reproduction path is PyTorch MPS, original Qwen3-1.7B bf16 weights, and the existing 466-prompt lens. Applying the lens and intervening at block outputs worked without changing the reference library. MLX was faster on one short prompt, but the experiment's correctness checks and full evaluation use MPS.
Machine and environment
| Item | Observed value |
|---|---|
| Mac | MacBook Pro, Mac16,8 |
| Chip | Apple M4 Pro, 12 CPU cores (8 performance, 4 efficiency) |
| GPU | 16 cores; system reports Metal 4 |
| Unified memory | 24 GiB (25,769,803,776 bytes) |
| macOS | 26.5.1, build 25F80 |
| Initial available disk | Approximately 15 GiB by df; 14.4 GB by later Python query after setup began |
| Near completion | Approximately 5.9 GiB free; no unrelated files removed |
| Python used | Homebrew 3.13.7, arm64, isolated .venv313 |
| Packages | torch 2.14.0; transformers 5.16.1; mlx 0.32.2; mlx-lm 0.31.3; numpy 2.5.3; jlens 0.1.0 at pinned commit |
| MPS | Built and available; recorded tensors on mps:0 |
| MLX | GPU 0, Apple M4 Pro; recommended working set 19,069,665,280 bytes |
Full environment, hardware and package records are in provenance/. Hardware identifiers unnecessary for reproduction were omitted from the saved hardware record. requirements.lock.txt pins every installed Python dependency.
Routes evaluated
| Route | Executed checks | Decision |
|---|---|---|
| PyTorch MPS | bf16/fp16/fp32 matrix operations; full original model; exact no-op; block hooks; reference lens parity; causal patch; cache positioning and reset; CPU comparison; 32-case evaluation | Use for reproduction |
| MLX / MLX-LM | Same original local checkpoint, bf16, same 12 input tokens; block wrapper captures residuals without changing output; an additive block intervention changes logits; same-activation J transport; synchronized timing | Feasible alternative, not used for evaluation |
| CPU | Full same-checkpoint bf16 forward, and independent float64 norm/pseudoinverse algebra | Correctness reference; no full evaluation |
| Core ML / Neural Engine | Official documentation inspection only | Conversion and explicit intermediate input/output boundaries would add work; no claim of empirical incompatibility, no execution attempted |
Apple's MPS documentation describes GPU execution. MLX also uses the GPU here. Neither run uses the Neural Engine. Core ML's prediction API supports converted models and compute-unit selection; that alone does not establish arbitrary residual-stream intervention access. A converted graph could potentially expose/split intermediate states, but we did not investigate that route further because MPS already met the requirement.
Numerical agreement before speed
Same prompt: Fact: The currency used in the country containing Lyon is the. All models used the original checkpoint in bf16, with no quantization. Input IDs are saved. PyTorch used eager attention; MLX used its own Qwen3 attention/normalization kernels.
- CPU versus MPS: same top token; last-token distribution total variation 0.00124. Maximum absolute last-token logit difference 0.4453; maximum over all positions 0.8281. Not bitwise equivalent. These tolerances were tested on one prompt, not a population.
- MLX versus MPS: same top token; last-token distribution total variation 0.02976. All-position maximum logit difference 1.59375; block-13 activation relative L2 difference 0.00622.
- Given the same saved MPS activation, the MLX lens readout differed from the PyTorch reference by at most 0.03125 logits. End-to-end activation differences are therefore a separate issue from lens transport arithmetic.
Different bf16 kernel accumulation and fused normalization/attention paths are plausible sources of these discrepancies, supported by inspection of the two implementations; we did not isolate one kernel as the cause. Do not mix backend results or assume token-by-token equivalence from matching top-1 on this prompt. No claim of speed comparison at float32, other sequence lengths, or generation throughput.
Measured resources and timing
| Measurement | Result |
|---|---|
| MPS warm 12-token prefill | Median 36.4 ms, five synchronized repeats, range 36.1–38.9 ms |
| MLX warm same prefill | Median 20.5 ms, five evaluated/synchronized repeats, range 20.4–20.8 ms |
| CPU same prefill | 670 ms, one run, 4 PyTorch threads |
| MPS model/lens setup | Approximately 4.6–5.7 s across recorded loads |
| MLX model load/evaluation | 0.715 s, warm file cache; not directly comparable to MPS setup, which also loads the lens |
| Evaluation | 153.3 s summed recorded readout/condition runtimes, plus setup and logging overhead |
| Per evaluation condition | Median 0.45 s, including two full continuation scores and 12 greedy output tokens |
| Evaluation MPS driver high-water observation | 4.02 GB (3.74 GiB), sampled at condition boundaries |
| Evaluation process peak RSS | 1.74 GB, OS-reported process accounting; not total unified GPU memory |
| MLX reported peak allocation | 3.48 GB |
| Model and lens files | Approximately 4.0 GiB on disk |
| Environment | Approximately 1.2 GiB |
GPU allocator figures and process RSS are different accounting views and must not be added as independent memory. Boundary sampling can miss transient GPU peaks. The correctness process reached 3.78 GB RSS while moving the full model to CPU for comparison. A system memory query during the run reported 35% free; this is a snapshot, not continuous pressure monitoring. No sustained memory-pressure failure was observed. Other applications were active; these are practical local timings, not isolated hardware benchmarks.
Timing uses torch.mps.synchronize() and MLX eval plus synchronize, accounting for MLX lazy evaluation. Large model copies were loaded sequentially. MPS was released before MLX loaded.
Adaptations and failures
- Explicitly selected Homebrew Python 3.13.7. An initial shell selected Xcode Python 3.9.6, whose older pip rejected editable installation. Both failure and successful installation logs are retained. No system/global Python configuration changed.
- Replaced the walkthrough's CUDA placement with
.to('mps'); used eager attention and no compilation. Weight precision remains bf16, as in the fitting config. Applying a precomputed lens needs only forward computation; no fitting/autodiff capability was claimed or required. - J matrices load from fp16 storage and are promoted by the reference class to float32. Transport and intervention arithmetic use float32. The original final norm and unembedding cast transported residuals back to bf16 exactly as the reference does.
- MPS has no float64 support: an attempted conversion failed visibly. The independent algebra check and pseudoinverse explicitly copy small matrices to CPU before converting to float64; float32 results return to MPS.
PYTORCH_ENABLE_MPS_FALLBACK=0makes unsupported MPS kernels fail instead of silently falling back. - Saved directions include Qwen's learned final RMSNorm gain. This is stated in the mathematical method and checked against an independent double-precision calculation. No full library port, quantization, or replacement inference engine was built.
The Qwen3.5-4B walkthrough route was verified at the artifact/config level, not executed. Its weights alone require about 9.32 GB on disk, versus 4.06 GB for the selected Qwen3-1.7B artifacts. The smaller model satisfied the development capability gate, so downloading a competing larger model would not have been justified on this disk budget.