$6k to run full DeepSeek R1 LLM with reasoning at home

https://twitter.com/carrigmat/status/1884244369907278106

by tzury • 1 year ago

26 8 1 year ago

dathinab• 1 year ago

a right the link below you can't read without loggin in because X penalizes links in posts no matter how much sense they make :shaking_head: (and because it got infected with the same kind of toxic design so common with apps today where it tries to keep you as much as possible in the app, instead of helping you to get what you want as fast done as possible so that you can focus on relevant things in life).

__MatrixMan__• 1 year ago

https://nitter.poast.org/carrigmat/status/188424436990727810...

teleforce• 1 year ago

Blimey! An X/Twitter you cannot even read the contents if you're not registered with X.

andy_ppp• 1 year ago

All the people robbing blind creatives for their AI job destruction machine are also attempting to hide any content from each other at this point. Instagram and X and probably YouTube are more limited in what you can do in a logged out state because of this.

croes• 1 year ago

Didn’t Twitter start to limit the view before the AI hype?

DanAtC• 1 year ago

Flagged for not being readable without an X account

guntars• 1 year ago

Since it's a MoE model with 37B active params, I imagined you don't even need all of that ram to keep the whole model in memory, just the active bits.

rahimnathwani• 1 year ago

The active bits may change with each token. You need the whole model in memory, even though, for any single token, only a subset of that memory will have been used in computation. The memory efficiency comes when you have multiple sessions in parallel.

$6k to run full DeepSeek R1 LLM with reasoning at home

Related Stories