anthonypasq

And neither do Lecun or Sutskever or Sutton! They are all focused on human intelligence. None of them are even slightly concerned about an AI which is intelligent before it learns any language.

??? https://www.youtube.com/watch?v=GvibIstOn_E his arguemtn here is clearly built around using some sort of sensory data to build a model of the world like humans (animals) do. also you clearly decline to mention Lecun who has made this point ad-infinitum

This isn't true either! Mathematica / Maple / etc are "old-fashioned AI" and they obviously work. The Lisp expert systems were also useful, though less so than an LLM.

i personally find it very strange that non-deep learning AI approaches which essentially boiled down to a giant bundle of if statements, or some very simple statistical modeling were called AI in the first place.

If AI researchers cared about scientific thinking, they would be intensely focused on the brains of bees.

Basically every academic AI researcher in history was doing what you described. The AI industrialists stopped caring 6 years ago once they realized LLMs seem to have been the only thing in 80 years that actually seems to work at any useful level.

There are plenty of pioneering scientists who are either returning to actual AI research (Yann Lecun, Ilya, etc), and plenty who never left (Richard Sutton) who are doing exactly what you are talking about.

brother thats not the comment you replied to

lmao, you know all Anthropic models are trained on TPU right?

The reason people can make a lot of money building houses is that it’s really hard and requires a lot of specialized and localized knowledge to figure it out.

I fundamentally don't believe this is true. This shit isnt hard at all. its a fucking building. Creating EUV lithography machines is hard, creating cutting edge vaccines is hard. Building a building is not hard.

if Dario and the CEO of Moonshot switched places, Mythos would have been generally available to the public 4 months ago. As a statement of fact, it wasnt, because of Project Glasswing and Dario thinking they created a superweapon that the rubes shouldnt have access to.

the person you were replying to says absolutely nothing about using it to write code.

Logan Kilpatrick said on an interview not too long ago that flash 3 and 3.5 are the same pre-train. all gains on top of 3 flash are post-training

because its a matter of public record that the release of Fable was artificially delayed

multimodal + latency

ah yes, because i said something factually accurate and vaguely positive about Anthropic I must be a shareholder which would mean I either run a venture capital firm or am a current employee of Anthropic...

I wish.

Love to see these crazy kinds of experiments going on. Even if this doesn't 100% work or is prohibitively expensive for now, these are glimpses into the future in the same way people were talking about coding agents in 2023 when we just had tab complete.

Kimi K3 is still worse than Fable and Fable was trained >4 months ago.

this is pure cope, and the realm of things you are better than it at will continue to shrink, and you need to be mentally and economically prepared for this.

how are you coming to that conclusion? If anything this tells me Fable and Sol and probably smaller, unless you think the Chinese have better data mix, learning algorithms, architecture etc?

how is this idea still so persistent? The fact people are able to run open models with about the same performance at 1/10th the cost should make it glaringly obvious that Anthropic has massive inference margins at api pricing.

theres plenty of people making 6 figures at 25 that still have their parents pay their phone bill because they get a family discount. i dont get your point

I think the word "rely" is doing some heavy lifting here. Still being on your parent's family phone plan doesnt mean youd be destitute if they werent helping you out.

Does anyone have any heuristics on how scaling parameter count actually scales cost to serve? Also im assuming we dont really know the sparsity here?

Is them pricing at Sonnet level actually give us any information at all at how big Sonnet is or is there too much opacity around inference margins?

99% of code ever written has always been shit. Most of it has been thrown out or not in use anymore, or EVER used. Most people arent going anything particularly unique or complicated or performance focused.

what exactly is technical about personal attacks?

But that hit piece would be an answer to the (multi-billion dollar) studio saying how much better the result is after the rewrite to Unreal

this in fact happens all the time and engine creators dont come out with bitter blog posts about it

none of this justifies a nasty mean spirited blog post.

Yet we somehow managed to build the interstate highway system?

This is just so defeatist and annoying. Building houses isnt inherently hard. we've intentionally made it hard for no good reason.

I suspect there are massive selection effects the are skewing the results here. I can imagine this being quite effective for intelligent, curious, and self-motivated kids and a complete and utter disaster for the average child.

I love how you think that is my primary argument. Want to address the openrouter point?

I don't make that mistake. I actually suspect that the actual costs may be higher than the API prices. I think those may still be subsidized.

how are you still under this delusion? so you think that all the hosting companies on openrouter are just burning money selling GLM 5.2 tokens at 1/6th the cost of opus 4.8 api pricing?

Mark Zuckerberg literally tweeted today that they are pricing Muse Spark 1.1 much lower because other companies have excessive margin.

if all the competition pushes down margins on tokens to 10-20%, i dont see how the inherent scale advantages of cloud inference wont be way more to account for the 20% cheaper tokens youd be getting running locally. i dont see how local will ever be more economical

... i dont think internal iteration counts dude. thats just called in-development.

As far as i remember, the entire AI org was essentially gutted and replaced with whoever Wang wanted to hire, and tbh that org completely failed to train llama 4 and I honestly doubt whatever techniques they used to ship llama 3 are at all relevant now. That was before reasoning models and the heavy emphasis on RL/post-training.

so yeah, this is essentially their first try with a completely new org.