Its worthwhile to have a quote from the article as some comment without reading:
"...I’ve been running Kimi K3 alongside Claude on my normal coding work, and for all practical purposes I can’t tell them apart. Same tasks, same quality of output, and near identical token counts to get there. I expected an open model to be sloppier or to grind through more tokens on the way to the same answer, and neither turned out to be true.
The prices are nowhere near each other. K3’s API runs $3 per million input tokens and $15 per million output. Claude’s top model costs $10 and $50 for the same units. The subscription side is even more lopsided..."
Thanks for posting here. Keep expanding and improving your study. Correct where it deserves correction.
The fact that HN decided to downvote the author of the study, shows how these people cant stay classy, and the mods stay silent...just shows what this is all about.
I agree you dont owe anyone a reproduction, but also you dont owe anyone an effort to discredit the study and you did it.
> I don't think I need to spend more time on this than I have.
How pious of you. I am still looking into the credibility of the study. It will take me more than 25 min...but I am really looking forward to see what this means for this 10 trillion industry.
I can however notice you had enough urgency to publicly critique the study within 25 minutes, and your comments carry weight, but when asked about checking whether the headline result actually holds, the answer is “why would I?”
"... the situation with compressed sensing can be compared with the current situation with modern large language models and related AI tools. Here, the field is almost entirely dominated by empirical research, often from industry rather than from academics. As such, there is a lot less clarity on what the key ingredients are to make a given AI technology suitable for a given use case; there are spectacular successes that cannot be replicated, next to seemingly promising uses that hit an unexpected wall (or, conversely, unlikely applications for which AI tools are far more effective than anticipated). With the notable exception of the mathematics of optimization and numerical linear algebra which are both somewhat mature, most of the theoretical mathematical framework needed to explain the strengths and weaknesses of AI is still in its infancy. (Though I would say here that the main bottleneck is not exactly lack of funding in basic research in the foundations of AI, but more that the mathematics itself is not understood to anywhere near the level we would like.)..."