Visualizing GPT-OSS-20B embeddings

https://melonmars.github.io/LatentExplorer/embedding_viewer.html

by melonmars • 11 months ago

89 24 11 months ago

melonmars.github.io

kingstnap• 11 months ago

It's an interesting looking plot I suppose.

My guess is its the 2 largest principle components of the embedding.

But none of the points are labelled? There isn't a writeup on the page or anything?

jablongo2• 11 months ago

Usually PCA doesn't look quite like this so this is likely done using TSNE or UMAP, which are non parametric embeddings (they optimize a loss by modifying the embedded points directly). I can see labels if I mouseover the dots.

terhechte• 11 months ago

I can see the labels when I hover with the pointer

graphviz2• 11 months ago

What do people learn from visualizations like this?

What is the most important problem anyone has solved this way?

Speaking as somewhat of a co-defendant.

jablongo2• 11 months ago

I lets you inspect what actually constitutes a given cluster, for example it seems like the outer clusters are variations of individual words and their direct translations, rather than synonyms (the ones I saw at least).

minimaxir4• 11 months ago

Not everything has to be directly informative or solve a problem. Sometimes data visualization can look pretty for pretty's sake.

Dimensionality reduction/clustering like this may be less useful for identifying trends in token embeddings, but for other types of embeddings it's extremely useful.

diwank• 11 months ago

Agreed. The fact that it has any structure at all is fascinating (and super pretty). Could signal at interesting internal structures. I would love to see a version for Qwen-3 and Mistral too!

I wonder if being trained on significant amounts of synthetic data gave it any unique characteristics.

TuringNYC• 11 months ago

What do people learn from visualizations like this?

Applying the embeddings model to some dataset of yours of interest, and then a similar visualization, is where it gets cool because you can visually look at clusters and draw conclusions about the closeness of items in your own dataset

ethan_smith• 11 months ago

Embedding visualizations have helped identify bias in word embeddings (Word2Vec), debug entity resolution systems, and optimize document retrieval by revealing semantic clusters that inform better indexing strategies.

graphviz2• 11 months ago

Interesting, glad to know it's been useful for some specific contributions. (Not questioning that interesting-looking, appealing displays as overviews for general awareness are also worthwhile.)

_def• 11 months ago

I have the suspicion that this is how GPT-OSS-20B would generate a visualization of it's embeddings. Happy to learn otherwise.

eddywebs• 11 months ago

Cool ! Would it possible to generate visualizations of any given open weight model out there ?

minimaxir4• 11 months ago

Yes, it's just yoinking the weights out of the embeddings layer.

numpad0• 11 months ago

Is this handling Unicode correctly? Seems like a lot of even Latin alphabets are getting mangled.

int_19h• 11 months ago

It looks like it's not handling UTF-8 at all and displaying it as if it were Latin-1

mkl• 11 months ago

I don't think it's actually UTF-8. The data is at https://melonmars.github.io/LatentExplorer/embeddings_2d.jso... and contains things like

  "\u00e0\u00a7\u012d\u00e0\u00a6\u013e"

with some characters > 0xff (but none above 0x0143, weirdly).

ashvardanian• 11 months ago

Any good comparisons of traditional embedding models against embeddings derived from autoregressive language models?

minimaxir4• 11 months ago

They are incomparable. Token embeddings generated with something like word2vec worked well because the networks are shallow and therefore the learned semantic data can be contained solely and independently within the embeddings themselves. Token embeddings as a part of an LLM (e.g. gpt-oss-20b) are conditioned on said LLM and do not have fully independent learned data, although as shown here there still can be some relationships preserved.

Embeddings derived from autoregressive language models apply full attention mechanisms to get something different entirely.

lawlessone• 11 months ago

what does it mean that some embeddings are close to others in this space?

That they're related or connected or it arbitrary?

Why does it look like a fried egg?

edit: must be related in some way as one of the "droplets" in the bottom left quadrant seems to consist of various versions of the word "parameter"

minimaxir4• 11 months ago

Typically these algorithms cluster by similarity (either euclidian or cosine).

The density of the clusters tend to have trends. In this case, the "yolk" has a lot of bizarre unicode tokens.

esafak• 11 months ago

Without a way to tune it, this visualization is as much about the dimensionality reduction algorithm used as the embeddings themselves, because trade-offs are unavoidable when you go from a very high dimensional space to a 2D one. I would not read too much into it.

promiseofbeans• 11 months ago

This demo is a lot more useful for comparing word embeddings: https://www.cs.cmu.edu/~dst/WordEmbeddingDemo/index.html

You can choose which dimensions to show, pick which embeddings to show, and play with vector maths between them in a visual way

It doesn't show the whole set of embeddings, though I am sure someone could fix that, as well as adapting it to use the gpt-oss model instead of the custom (?) mini set it uses.

voodooEntity• 11 months ago

@Author i would recommend you to give

https://github.com/vasturiano/3d-force-graph

a try, for the text labels you can use

https://github.com/vasturiano/three-spritetext

its based on Three.js and creates great 3D graph visualisations GPU rendered (webgl). This could make it alot more interresting to watch because it could display actual depth (your gpu is gonne run hot but i guess worth it)

just a suggestion.

suprjami• 11 months ago

Why does it look like an image of an asteroid hitting a planet?

https://stock.adobe.com/images/asteroid-hitting-the-earth-ai...

Visualizing GPT-OSS-20B embeddings

Related Stories