HNGlance

Top Best New Show

Continuous Autoregressive Language Models

https://arxiv.org/abs/2510.27688

by guybedo • 9 months ago

3 1 9 months ago

AR

Related Stories

Loading related stories...

Comments

1

guybedo• 9 months ago

Abstract:

we introduce Continuous Autoregressive Language Models (CALM), a paradigm shift from discrete next-token prediction to continuous next-vector prediction.

CALM uses a high-fidelity autoencoder to compress a chunk of K tokens into a single continuous vector, from which the original tokens can be reconstructed with over 99.9\% accuracy.

This allows us to model language as a sequence of continuous vectors instead of discrete tokens, which reduces the number of generative steps by a factor of K

Source preview

arxiv.org

HNGlance

© 2024-2026 HN Glance. Story content belongs to its original sources.

Privacy
Terms