HNGlance

Top Best New Show

Mamba: New SSM arch with linear-time scaling that outperforms Transformers

https://github.com/state-spaces/mamba

by xenova • 3 years ago

6 2 3 years ago

GI

Related Stories

Loading related stories...

Comments

2

xenova• 3 years ago

Paper: https://arxiv.org/abs/2312.00752 Models: https://huggingface.co/state-spaces

cuuupid• 3 years ago

This is promising as the future of modeling. Outperforming transformers doesn’t catch my eye because there is so much variation in performance based on size, training data, training method etc.

However the greater (5x) inference bandwidth makes this super appealing especially for democratizing AI and enabling the GPU poor. This could very well be a watershed moment for SSMs, similar to how Transformers boasted improvements to training and inference speed in 2019.

Source preview

github.com

HNGlance

© 2024-2026 HN Glance. Story content belongs to its original sources.

Privacy
Terms