Ask HN: Best local LLM under 2B paramater and consuming RAM less than 3gb

https://news.ycombinator.com/item?id=48708262

by adithyaharish • 24 days ago

2 4 24 days ago

news.ycombinator.com

For my local project, I have been finding a good LLM or a SLM that consumes less than 2gb ram, can you get me any models that would aid me?

big-chungus4• 24 days ago

LFM 2.5 1.2B instruct, it's good at instruction following as long as the instruction is short and simple.

Gemma 4 E2B is even better than LFM but I don't know if you can fit it in 3GB.

Qwen 3.5 0.8B and 2B are smart but very inconsistent for instruction following.

If you need absolutely tiny models, check Gemma 3 270m and FunctionGemma, LFM 2.5 230m and Granite 4.0 350m. However those are very stupid models and whether they are feasible for you will depend on your use case.

adithyaharish2• 24 days ago

Thanks for taking out your time and listing out the models, just tried qwen 3.5 2B works well for my use case

macwhisperer• 24 days ago

qwen3 1.7b- q4_k_m is your best best for that size

adithyaharish2• 24 days ago

just checked it out, has mixture of experts, mit fit my use case!

Ask HN: Best local LLM under 2B paramater and consuming RAM less than 3gb

Related Stories