HNGlance

Top Best New Show

Ask HN: Why aren't there Open source embedding models with context length > 512?

https://news.ycombinator.com/item?id=37477241

by rawsh • 3 years ago

3 2 3 years ago

NE

news.ycombinator.com

Related Stories

Loading related stories...

Comments

2

caprock• 3 years ago

There are some:

https://huggingface.co/spaces/mteb/leaderboard

james-revisoai• 3 years ago

There are as mentioned, but additionally, for many models, you can split content up into several vectors (say one for each sentence or paragraph depending on how the model is trained) and pool the vectors together to get a representation that will span the content overall well.

Since the models trained to work on single sentences (like Mini-V2, the SBERT default) work worse at length, pooling representations of sentences is typically more useful.

For deliberately longer representations, generative model embeddings or document embeddings are the right answer sometimes.

Source preview

news.ycombinator.com

HNGlance

© 2024-2026 HN Glance. Story content belongs to its original sources.

Privacy
Terms