HNGlance

Top Best New Show

Run Deepseek from fast NVMe drives

https://github.com/BlinkDL/fast.c

by ironbound • 1 year ago

5 3 1 year ago

GI

Related Stories

Loading related stories...

Comments

3

ironbound• 1 year ago

Testing extreme NVME offload (4 x Gen5x4) for DeepSeek R1Because PCI-E 5x16 (~60GB/s) is close to dual channel DDR5 bandwidth, this is the cheapest method to run huge models. Code: https://github.com/BlinkDL/fast.c

fspeech2• 1 year ago

Do you have any benchmark run yet? I am interested in knowing how many tokens/sec you can get to. Though in the end it should be more efficient to run the model on distributed server clusters.

fspeech2• 1 year ago

Deepseek's open source inference code, while correct, may not be fully efficient. For example the MLA needs the right associative matrix multiplication order to be efficient.

Source preview

github.com

HNGlance

© 2024-2026 HN Glance. Story content belongs to its original sources.

Privacy
Terms