[rasa04]

Related discussion: #4025.

Problem. The Metal decode kernels use tiles tuned on the M4 and M5. On an M1 Max (24-core GPU) the dense GEMVs of a Qwen3.8-Flash-Next Q2_0 decode window take 33.1 ms on the default tiles and 20.7 ms on each shape's best one (gemv_parity --one … --ntok 2). setup.sh --calibrate is not ported to the Mac.

Change (opt-in; with neither variable set the dispatches are unchanged):

• STRATA_METAL_CALIBRATE=<file>: the engine times every instantiated tile of the dense GEMVs, gr_down and the routed experts on the device, outputs into a scratch buffer, and keeps a tile only if it is ≥ 3% faster.
• STRATA_METAL_TILES=<file>: use the tiles, never measure. A shape the file lacks, a file from another GPU, or an expert tile that does not divide the model's rows keeps the default.
• setup.sh --calibrate on a Mac (tools/calibrate.py, like the HIP STRATA_SH_STREAM step): measures, races the tiles against the defaults over 3 interleaved rounds, and writes STRATA_METAL_TILES into the config only above MIN_GAIN. One file per model.

Result (M1 Max, 64 GB, Q2_0 resident, MTP; tools/ab_engine.py, 3 rounds, medians, tok/s):

	default	tiles
decode, short chats	25.1-28.3	32.6-36.8 (+30-32%)
decode after a 6K-token prompt	23.2	29.2 (+26%)
short prompt read	39.1-40.2	52.2-53.8
6K-token prompt read	251.1	251.9

Quality (tools/quant_quality.py, 4995 positions): same top token 98.4%, mean KL 0.003, perplexity 4.439 → 4.431. Near-ties can flip, since a tile changes the order of a sum.

Validation (3b5736ca on 7dd5cf18; speed and quality measured on f53e57a4, which lacks only the row-divisibility guard):

• Passed: build; gemv_parity (20 tiles), gr_parity (6 variants), expert_parity 507/507 (every gate/up and down format × tile × ntok 1, 2, 4); python -m unittest test_calibrate test_setup_mac (57, incl. the new test_metal_tiles); default path byte-identical to 7dd5cf18 (quant_quality, 4995/4995 rows).
• Failed: test_failed_calibration_says_the_engines_reason, which fails on a Mac without this change too (metal/setup_mac.py is missing from the test's temp tree).
• Not run: other Macs (M2-M5, Pro / Max / Ultra), SSD streaming, the IQ3 / Unsloth / Gyro packs, CUDA / HIP / SYCL (not touched).

Limits. The gain is confirmed only on the Mac and --spec the calibration ran on. Window sizes it did not see and the MTP layer keep the defaults. Re-run after an update that changes the kernels.

Individual runs (tok/s)

request          tiles    decode (rounds 1/2/3)    prompt read (rounds 1/2/3)
chat 1           default  30.7 / 27.6 / 28.3       42.3 / 38.1 / 39.1
chat 1           tiles    36.8 / 36.5 / 41.8       51.4 / 52.2 / 58.6
chat 2           default  27.0 / 24.8 / 25.1       42.9 / 39.4 / 40.2
chat 2           tiles    33.2 / 32.6 / 37.0       53.6 / 53.8 / 59.6
chat 3           default  27.0 / 25.0 / 25.1       42.3 / 39.1 / 38.8
chat 3           tiles    32.6 / 32.2 / 33.7       51.3 / 52.6 / 58.6
6K-token prompt  default  23.6 / 23.2 / 22.7       268.4 / 251.1 / 248.6
6K-token prompt  tiles    29.2 / 28.8 / 30.0       251.9 / 251.6 / 253.3

setup.sh --calibrate: default 25.9 / 26.1 / 27.8, tiles 35.0 / 35.1 / 37.3.

Local changes to tools/ab_engine.py (whole answers kept, the die cooled before each start)

--- tools/ab_engine.py
+++ the local copy
@@ -22,7 +22,7 @@
 import urllib.request
 from pathlib import Path
 
-ROOT = Path(__file__).resolve().parents[1]
+ROOT = Path("<the Strata checkout>")
 PROMPTS = (
     "Write a Python function that merges two sorted lists into one sorted list, with a docstring and two tests.",
     "Explain in two paragraphs how a refrigerator moves heat from inside to outside.",
@@ -64,6 +64,19 @@
     raise TimeoutError("server not ready")
 
 
+def tdie() -> float:
+    d = json.load(open("<the die temperature file>"))
+    return next(g["avg"] for g in d["temps"]["groups"] if "die" in g["name"].lower())
+
+
+def cool(limit: float) -> None:
+    """Each arm starts on a cool chip: a hot M1 Max throttles and the arm that runs later loses."""
+    t0 = time.time()
+    while (t := tdie()) >= limit and time.time() - t0 < 900:
+        time.sleep(15)
+    print(f"tdie {t:.1f} C", flush=True)
+
+
 def main() -> int:
     ap = argparse.ArgumentParser()
     ap.add_argument("config")
@@ -83,9 +96,12 @@
     reqs = [(f"chat{i}", p) for i, p in enumerate(PROMPTS)] + [("long", long_prompt(a.long_words))]
     out = open(a.log, "a")
     res: dict = {n: {} for n, _ in arms}
+    cool_limit = max(50.0, tdie() + 1.0)   # the idle chip now, +1 C: equal starts without waiting for a cold room
+    print(f"cool limit {cool_limit:.1f} C", flush=True)
     for rnd in range(a.rounds):
         order = arms if rnd % 2 == 0 else arms[::-1]
         for name, env in order:
+            cool(cool_limit)
             cfg = dict(base)
             e = dict(env)
             if "exe" in e:
@@ -110,7 +126,7 @@
                     row = {"round": rnd, "arm": name, "req": tag, "decode": t.get("predicted_per_second"),
                            "prompt": t.get("prompt_per_second"), "prompt_n": t.get("prompt_n"),
                            "n": r.get("usage", {}).get("completion_tokens"),
-                           "text": r["choices"][0]["message"].get("content", "")[:80]}
+                           "text": r["choices"][0]["message"].get("content", "")}
                     out.write(json.dumps(row) + "\n")
                     out.flush()
                     res[name].setdefault(tag, []).append(row)