HNGlance

Top Best New Show

LLM-speed-benchmark results (GPT-4o mini slower than GPT-4o?)

https://github.com/coder543/llm-speed-benchmark/blob/main/results/README.md

by coder543 • 2 years ago

2 1 2 years ago

GI

Related Stories

Loading related stories...

Comments

1

coder543• 2 years ago

Once GPT-4o mini launched, I noticed that it didn't really perform any better than GPT-4o. I thought this might change over time, but it still hasn't, so I finally sat down to do some more comprehensive benchmarking of different LLM APIs to see how they compare. The vast gulf in pricing between GPT-4o and GPT-4o mini would usually indicate a speed gulf too, but it is oddly missing... and the data actually indicates the smaller model is slower.

I didn't search for any similar benchmarks today, but I have searched in the past, and I've never been able to find any good reference for the tok/s that people are getting out of different hosted models. I hope other people will find this data valuable.

Source preview

github.com

HNGlance

© 2024-2026 HN Glance. Story content belongs to its original sources.

Privacy
Terms