This reads like you didn’t read the post.
z-lab runs BF16 on B200 (54+ GB). There is no z-lab path that fits on a 24 GB 3090. That is literally the entire point of our work, and it is stated in the second paragraph. If you had checked the HF model card you linked before posting, you would see the same thing. Before this repo, there was no path to run this... SGLang's GGUF path for this model is broken. llama.cpp doesn't have DFlash speculative decoding at all. If you wanted to run this hybrid model fast on a 24 GB consumer card, there was nothing...
That took weeks of real engineering.
Calling that "vibecoded" because we used a bit of AI in the README is clean is the laziest possible critique. An LLM reading the DFlash paper does not catch verify_logits_buf being sized vocabq_len when DDTree reads vocab(budget+1). That is hours of debugging with nvidia-smi and memory sanitizers, not prompting.
The 207 and 129.5 numbers are both in the second sentence of the post and again in the TL;DR. 207.6 is peak tok/s in the linked demo video, 129.5 is the HumanEval 10-prompt mean at DDTree budget=22. We specify both just behind the title.
On the Q4 KV cache: the tradeoff is disclosed with actual numbers. AL 8.56 -> 8.33 at short context (3% drop), dramatically better at long context. It’s the only way 128K allocates on 24 GB. The binary is env-selectable, you can run BF16 KV if you don’t need 128K. Both are benchmarked.