rain1

I've used Google Antigravity to write scripts to download and produce architecture diagrams for various LLMs from huggingface. It's pretty useful so I thought I'd share it.

There's also a model comparison spreadsheet that you can compare sizes and such https://weavers.neocities.org/architecture-encyclopedia/mode...

If you'd like any additional models to be added I can add them in.

The Gemma models are too small to be included in this list.

You're right the T5 stuff is very important historically but they're below 11B and I don't have much to say about them. Definitely a very interesting and important set of models though.

Yes but just purely in terms of entropy, you can't make a model better than GPT-4 by training it on GPT-4 outputs. The limit you would converge towards is GPT-4.

This is kind of related to the jack morris post https://blog.jxmo.io/p/there-are-no-new-ideas-in-ai-only he discusses how the big leaps in LLMs have mostly come - not so much from new training methods or arch. changes as such - but the ability of new archs. to ingest more data.

It's extremely interesting how powerful a language model is at compression.

When you train it to be an assistant model, it's better at compressing assistant transcripts than it is general text.

There is an eval which I have a lot of interested in and respect for https://huggingface.co/spaces/Jellyfish042/UncheatableEval called UncheatableEval, which tests how good of a language model an LLM is by applying it on a range of compression tasks.

This task is essentially impossible to 'cheat'. Compression is a benchmark you cannot game!

I think that one thing that this chart makes visually very clear is the point I about GPT-3 being such a huge leap, and there being a long gap before anybody was able to match it.

This is really awesome. Thank you for creating that. I included a screenshot and link to the chart with credit to you in a comment to my post.

I can correct mistakes.

it somehow merged Llama 4 Maverick's custom Arena chatbot version with Behemoth

I can clarify this part. I wrote 'There was a scandal as facebook decided to mislead people by gaming the lmarena benchmark site - they served one version of llama-4 there and released a different model' which is true.

But it is inside the section about the llama 4 model behemoth. So I see how that could be confusing/misleading.

I could restructure that section a little to improve it.

Llama 405B was also trained on more than 15 trillion tokens[1],

You're talking about Llama 405B instruct, I'm talking about Llama 405B base. Of course the instruct model has been traiend on more tokens.

why is there such a focus on token training count?

I tried to include the rough training token count for each model I wrote about - plus additional details about training data mixture if available. Training data is an important part of an LLM.

I have corrected that. It was supposed to say "None of this document was written by AI."

Thank you for spotting the error.

Take care of your mental health

How?

todsacerdoti is a spambot btw

I don't understand this. Please can you point me to information about it?

The people that are astonished by this just need to learn why.

It's not the function that is wrong, it's those people.

This is incorrect, the goats and car are behind doors. They are not inside cardboard boxes.

This is an example of hallucination.

An LLM doesn't know anything about itself - it can be pre-prompted with facts about itself, but this is going to be an example of it just making plausible text up.

tell me you're posting from an armchair without telling me you're posting from an armchair

what the actual fuck were they thinking uploading dolphin to steam??

Why don't they let us edit what the bot says? Could be useful.

This is the future of linux syscalls. Get on board with this or get left behind.

so list a few known to work models and their requirements

I feel like this isn't healthy for a human being

There are no solutions

This looks incredible. Wow.

Does this do one query per {{}} thing?

unfortunately that chip is proprietary and undocumented, it's very difficult for open source programs to make use of. I think there is some reverse engineering work being done but it's not complete.

wonderful! thank you

1. Download the weights for the model you want to use, e.g. gpt4-x-vicuna-13B.ggml.q5_1.bin

I think you need to quantize the model yourself from the float/huggingface versions. My understanding is that the quantization formats have changed recently. and old quantized models no longer work.

That is a crazy speedup!!

I don't think the integrated GPU on that supports CUDA. So you will need to use CPU mode only.

Tell us how it goes! Try different numbers of layers if needed.

A good place to dig for prompt structures may be the 'text-generation-webui' commit log. For example https://github.com/oobabooga/text-generation-webui/commit/33...