You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
This project is very impressive. Can you support Deepseek v4 Flash？ #479
Thank you! Strata is built around Qwen3.8-Flash-Next's architecture (Gated DeltaNet + sparse attention, its MoE layout, its MTP head). DeepSeek V4 Flash differs in attention (MLA), router and layout, so it would need a new loader, packer, kernels and drafter. That's not on the roadmap now; fine-tunes of the same model come first. Closing; the request stays here.
What it does on a 2xL20 (96 GiB VRAM, 125 GiB RAM) with DeepSeek-V4-Flash-0731-NVFP4 (164 GB GGUF -> 145.13 GiB NVFP4 expert pool, 57.5 % resident):
prefill argmax and three decode steps match a CPU reference forward exactly (9544 / 734 / 260 / 4654);
multi-row verify R=2..6 and a window-wrapped long context are bit-identical to single-row decode;
compute-sanitizer memcheck: 0 errors; Qwen3.8-Flash-Next on the same binary stays in its usual band (109.24 / 96.99 / 116.94 t/s);
~30 t/s single decode stream, ~47 t/s with six rows in one forward.
Not included: the DSpark/MTP draft head (measured at < 1.5x net on this pool, and it costs resident expert memory), IQ2/IQ4 packs, and Hadamard-folded v5 packs.
The scope question is the maintainer's call: it is a separate family (src/dsv4, STRATA_HAVE_DSV4) and the Qwen path is untouched, but it is ~11 k lines. Happy to split it into loader/geometry, attention and MoE PRs if that reviews better. Leaving #3589 as a draft until there is agreement on that.
@JUNQINGV587 Hey ! Yesterday I have done nearly the same thing in a different way :)
I launch myself in the port of DeepSeek v4 on top of Strata.
Saying to myself that deepseek v4.1 is out of sight because it's way too large for my hardware and I don't want my NVMe drive to be hot all time, but deepseek v4 0731, I know it can work and I have just enough resource.
I know it because I tested it previously with llamacpp at ~7 token/s decode.
Huge speedup and memory consumption using Strata, using Deepseek v4 0731 Q2, I got 12 tok/s decode with Ryzen 9 3900X, 128 GiB RAM, RX 9070 XT 16 GiB using a Quant 2 model.
And the ability to use less memory and stream expert from SSD but at the cost of ~10tok/s.
Activity
hendrikp commentedon Oct 2, 2026
lets first get qwen4, that one should be same architecture ;)
Niko1221 commentedon Oct 2, 2026
Thank you! Strata is built around Qwen3.8-Flash-Next's architecture (Gated DeltaNet + sparse attention, its MoE layout, its MTP head). DeepSeek V4 Flash differs in attention (MLA), router and layout, so it would need a new loader, packer, kernels and drafter. That's not on the roadmap now; fine-tunes of the same model come first. Closing; the request stays here.
JUNQINGV587 commentedon Oct 21, 2026
New evidence on this request: there is now a working port, not just a design.
Branch: https://github.com/JUNQINGV587/Strata/tree/dsv4-mvp-rebase (rebased onto
main011c9e6, one commit)Draft PR: #3589
What it does on a 2xL20 (96 GiB VRAM, 125 GiB RAM) with
DeepSeek-V4-Flash-0731-NVFP4(164 GB GGUF -> 145.13 GiB NVFP4 expert pool, 57.5 % resident):Not included: the DSpark/MTP draft head (measured at < 1.5x net on this pool, and it costs resident expert memory), IQ2/IQ4 packs, and Hadamard-folded v5 packs.
The scope question is the maintainer's call: it is a separate family (
src/dsv4,STRATA_HAVE_DSV4) and the Qwen path is untouched, but it is ~11 k lines. Happy to split it into loader/geometry, attention and MoE PRs if that reviews better. Leaving #3589 as a draft until there is agreement on that.yoyz commentedon Oct 25, 2026
@JUNQINGV587 Hey ! Yesterday I have done nearly the same thing in a different way :)
I launch myself in the port of DeepSeek v4 on top of Strata.
Saying to myself that deepseek v4.1 is out of sight because it's way too large for my hardware and I don't want my NVMe drive to be hot all time, but deepseek v4 0731, I know it can work and I have just enough resource.
I know it because I tested it previously with llamacpp at ~7 token/s decode.
Huge speedup and memory consumption using Strata, using Deepseek v4 0731 Q2, I got 12 tok/s decode with Ryzen 9 3900X, 128 GiB RAM, RX 9070 XT 16 GiB using a Quant 2 model.
And the ability to use less memory and stream expert from SSD but at the cost of ~10tok/s.
@JUNQINGV587 @Niko1221 The branch is here if you wana take a look :
https://github.com/yoyz/Strata/tree/f-deepseek-v4-flash-0731
And here is a first PR for the tokenizer, to make the merge possible.
https://github.com/yoyz/Strata/tree/f-tokenizer-joyai-pre
@Niko1221 I think this project deserve a Deekseek v4 0731 port !