[Engine] conversation cache: parked conversations are never restored on Linux/HIP (gfx1100) — parks accumulate, 'session restored' never logs #4081

Environment

• Strata app 0.2.4.2, engine 0.1.49.2 (BUILD.json: source=local-hip, backend=hip, archs=[gfx1100], src=b9993e25b7e78735, built 2026-10-24)
• 2× AMD RX 7900 XTX 24 GB (gfx1100), EPYC 7402 (48 T), 160 GiB RAM
• Linux, Ubuntu 24.04 host, ROCm userspace from the _rocm_sdk_* wheels (hipBLASLt 1.4.1), engine started via serve/server.py --engine strata
• Model: Qwen3.8-Flash-Next UD-IQ4_XS (pack), --serve --pack ... --native ... --kv int8 --kv-resident 32768 --batch 2 --conversation-cache-mib 8192 --max-context 262144 --prefill auto:32768 --spec 4 --mtp ... --prompt-cache 4 (also tried --conversation-cache-mib 16384 --conversation-cache-slots 4/16, single replica, dual replicas)
• Config carried the same env block the working CUDA box uses (STRATA_KV_PREFETCH=1, STRATA_CACHE_MESSAGE_BOUNDARY=1, …)

Observation

serve parks conversations on every turn and never restores any of them. Over ~30 alternating-conversation requests the engine log shows:

strata serve: conversation cache: parked 2757 tokens in 190.5 ms; parked=7 bytes=1940762268 evictions=0 snapshot_bytes=278281516 reused_kv_bytes=0
strata serve: prompt 2741 tokens = 0 reused + 2741 read in 4612 ms (594.3 tok/s), 16 generated in 205 ms

grep -c "session restored" over every engine log line written since this engine version was installed: 0. /metrics → conversation_cache: parks=22 restores=0 reused_tokens=0 while parked climbs to the slot limit and evictions follow.

The next request that is an exact continuation of a parked conversation (same tokens, same images, temperature 0, the client echoes the model's own previous reply back as the assistant turn) reports cache_n=0 and re-reads the whole prompt (2,741 tokens ≈ 4.3–4.6 s). This never changes:

• not with 1 conversation or with 2–3 alternating conversations (turn 2+ of the same conversation always 0 reused once the request has left the hot batch slot);
• not with --conversation-cache-mib 8192 or 16384, --conversation-cache-slots unset / 4 / 16;
• not on one replica, on two replicas, with --batch 2 or without;
• the same prompt mix run on the CUDA engine 0.1.44.1 (2× RTX 4090D) restores from the first park: prompt 6658 tokens = 6642 reused + 16 read in 150 ms, conversation_cache: restores=115, 14× TTFT improvement — the client script is byte-identical.

Two reuse paths do work here, neither goes through the conversation cache:

• a second copy of the same request, served back-to-back, reuses from the hot batch slot's resident KV (= 2611 reused + 5 read in 245 ms, no session restored line);
• short requests share a 34-token template prefix.

So parking writes snapshots the read-back path never consults on Linux/HIP.

Reproduction

Python 3, no dependencies beyond urllib; two conversations sharing one 2,624-token system prompt, alternating, each turn echoing the previous real reply back:

import json, urllib.request
URL = "http://127.0.0.1:8080/v1/chat/completions"; MODEL = "<your-model>"
def call(msgs):
    b = json.dumps({"model": MODEL, "messages": msgs, "max_tokens": 16,
                    "temperature": 0, "stream": False}).encode()
    d = json.loads(urllib.request.urlopen(urllib.request.Request(
        URL, b, {"Content-Type": "application/json"}), timeout=600).read())
    m = d["choices"][0]["message"]
    reply = (m.get("content") or "") or (m.get("reasoning_content") or "") or "OK"
    return d["timings"].get("cache_n", 0), reply[:200]
sysmsg = {"role": "user", "content": "You are a coding agent.\n" +
    "\n".join(f"Section {j}: the interplay of quanta and glaciers shows up as an energy gradient and a feedback loop." for j in range(140))}
hist = {i: [] for i in range(2)}
for r in range(4):
    for i in range(2):
        msgs = [sysmsg] + hist[i] + [{"role": "user", "content": f"session {i} turn {r}: answer OK"}]
        cache, reply = call(msgs)
        print(f"r{r} s{i} cache={cache}")
        hist[i] += [{"role": "assistant", "content": reply},
                    {"role": "user", "content": f"session {i} turn {r}: answer OK"}]

Expected: from r1 on, every turn hits its parked snapshot (cache_n ≈ prompt-16, ~0.2 s, one session restored line in the engine log).
Actual on Linux/HIP gfx1100 0.1.49.2: cache=0 and 4+ s on every turn after the conversation leaves the hot slot; restores=0.

Ruled out

• client history mismatch (the reply is echoed verbatim; the CUDA control hits);
• budget/slots (evictions=0 while the miss happens; 16 GiB / 16 slots same);
• replica routing / DP2 (single replica, single GPU same);
• STRATA_KV_PREFETCH=0, STRATA_CACHE_MESSAGE_BOUNDARY=0 (both make no difference);
• memory guard: parked lines print with real snapshot_bytes (~278 MB for 2.7k tokens, ≈ 103 KB/token, as documented).

Related

• [Engine] macOS: conversation cache parks and restores (Metal checkpoints, K/V description, memory guard) #3231 (merged today) fixed "never parks / never restores on macOS" with four Metal-specific causes — this looks like the same class of per-backend breakage, but on HIP/Linux, where the park side succeeds and the restore side never runs.
• [Engine] Windows/WDDM: parked-conversation snapshot copies corrupt at 262144 -> all-logits NaN (#879 symptom, separate trigger) #3036 / [Engine] conversation snapshot: stage parked-state copies through pinned memory on Windows/WDDM #3288 is HIP but Windows/WDDM-only (copy-classification at park time; that PR notes "Linux is unaffected").
• [Engine] sm_80 (GA100) tuning from #3712, all five features in one PR: one commit each, to split or cherry-pick as you prefer #3779 contains "the conversation-cache copy paths … are CUDA only. On HIP … they are off" — is the restore side gated on a CUDA-only path on Linux as well?

Question: is the parked-snapshot read-back expected to work at all with --backend hip on Linux, and if so, which switch or code path decides it? Happy to run candidate builds or traces on this 2× gfx1100 rig.

Not covered

• Other AMD archs (only gfx1100 tested), Windows/HIP (see [Engine] conversation snapshot: stage parked-state copies through pinned memory on Windows/WDDM #3288), SYCL, Metal.
• The engine was built locally (local-hip); the previous engine on this box (0.1.49, kept as strata.0.1.49.backup) was not re-tested with this script, so "regression in 0.1.49.2" is not established — restore may never have worked on Linux/HIP.