HNGlance

Top Best New Show

GemMoE: An 8x8 Mixture Of Experts based on Gemma

https://huggingface.co/Crystalcareai/GemMoE-Beta-1

by zopper • 2 years ago

4 1 2 years ago

HU

Related Stories

Loading related stories...

Comments

1

unraveller• 2 years ago

That's Gemma-7Bx8

He goes on about the journey, the future, the collaboration that happens somewhere. Nothing about the current result, like is it any good or unexpectedly great at anything?

I suppose it was more of an exercise in readying the tools to make it possible and cheap to fine tune and it is now left up to the community to tune and push the benchmarks. I also wonder if it could benefit from today's latest paper drop (from meta) on dirt cheap MoE creation. https://huggingface.co/papers/2403.07816

Source preview

huggingface.co

HNGlance

© 2024-2026 HN Glance. Story content belongs to its original sources.

Privacy
Terms