atleastoptimal

Closed-source models have to deal with the current frontier being heavily regulated. Fable, at its old level, was "too good" to be released and they had to add an additional safety layer to sanitize the outputs. Lowering the quality of the models so they are safer and more steerable has been something all the closed-source models have been doing for a while, a requirement that many open source models don't need to deal with.

If Kimi k3 really were above Fable 5 then there invariably the USG would have to consider their restrictions on model capabilities excessive, or one would have to admin closed source models are held to more restrictive safety standards than open source models.

Although I will reiterate the fact that distillation is not the primary reason why these models are performing so competitively.

How would you know this? How could you ascertain exactly how much performance is attributable to their unique engineering/research? If they really were so competitive they could surely make a model that isn't dependent on distilling Fable or other frontier models.

It matters because everyone imagines the inevitable "closing of the gap" between closed and open source, but the rate at which open source catches up with closed source seems to depend on being able to train on and distill the outputs of open source models. As long as performance of open source models is at least partially dependent on frontier-model outputs, then that gap will remain in place by definition.

Claiming in any shape or form that fable disillation is one of the primary reasons why kimi k3 is so competitive is slandering the work of other labs that cooperatively push the open-source models forward.

If the distillation is irrelevant to why it is competitive, why do they do it then? Obviously is helps improve their benchmarks/performance to some degree, otherwise they wouldn't need to do it.

The VC money is only contingent on the strategy eventually bearing fruit. I do imagine going open source -> closed source could work for some model companies who get enterprise/ecosystem buy-in but the probability of ROI is lower.

AI models cost tens of millions to train. Offering them for free won’t justify the upfront costs.

The Chinese model of model training/open sourcing only makes sense in the context of the overall strategy of undercutting American frontier labs’ profit margins.

You went through 5 million tokens in an hour? You would be significantly benefitted by using fable as an orchestrator for other models

I wonder what percentage of the US GDP is "Fooling old people"

No matter how bad the climate crisis gets, those who have the power to change it (heads of state/capital) will never abandon their lifestyles which expend 1000x more energy and fossil fuels than the average person. Pushing for voluntary, individual reductions in consumption is pointless and insulting to the average person.

Do people actually believe that the revenue for these companies is "fake"? Do they think financial analysts don't have anything but revenue to look at to determine the substance of their growth?

So much of neurodegeneration has turned out to be the effects of latent viruses in the body. HSV-1 is correlated with developing dementia along with many respiratory viruses.

Playing whack-a-mole with AI tests like this will never work, specifically because there is nothing AI has been proven to do better at than specific tasks with verifiable correctness

Hilarious that the article is framed as a humanities vs sciences thing even though the caliber of philosophers who can get these jobs at labs are the top 0.1% in their field, and wouldn't have trouble finding a job elsewhere, whereas you could get a good-paying job as an engineer at a relatively lower percentile.

If you exercise your resting heart rate will be lower, so athletic people have fewer total heart beats. However I'm not sure number of heart beats is a relevant metric to cardiovascular wear and tear.

A claim that LLM's can in a theoretical sense be 100% accurate all the time is not the same as the claim that scaling models with more compute/params will reduce hallucination. The former is a far stronger claim and I agree with the paper in that it probably isn't the case, but we don't rely on general reasoners (a.k.a. humans) to be 100% accurate all the time either.

No, it can hold more floating point numbers.

Fallacy of composition. Just because an LLM is made up of floating point numbers doesn't mean its capabilities are limited to that of bare floating point numbers, in the same way that the individual faculties of a neuron don't preclude the human brain from emergent properties born from the synthesis of its synapses.

I think the use of the word "hallucination" with respect to AI confidently making errors has led a lot of people astray, including the author.

He claims that his company has "solved" hallucination by creating a verifiable fact-finding system, which is like saying that a person has solved plan crashes by creating a plane that never leaves the ground.

When an LLM says something incorrect, it often is due to that LLM reaching the limits of its abilities, but it doesn't "know" (for lack of a better term) what being wrong feels like, so it will try its best to fit the information it has into a compelling story. The reason why scaling leads to fewer hallucinations is that the model can hold more abstractions, more facts about the world, it can work through the complex, vague machinery of reason with more scaffolding, and more of a buffer (via its weights) to reason with nuance. This is why LLM's are useful, not because they can be fed into a fact-retrieval system, but because they can produce new information via the association of things they know.

The point is, we want LLM's to actually produce new information and work out things via their thinking, not be limited to citing facts that already exist and avoid veering into the limits of its abilities. In that sense hallucination is really just exposing the limits of scale, which would necessitate scaling models further.

Scaling is the only way we have gotten to this interesting, emergent property of LLM's. Further, the best way to make small models which don't hallucinate (that we've found so far) is to train a big model first, then distill it, or use it as a teacher to a smaller model. Either way, pursuing scale is the most defensible strategy, and a more robust solution to hallucination.

I feel this could turn into a patronage system.

Want frontier intelligence? Better not defy the current administration, or your competitors will have access to a better model you could never use.

Thinking and consciousness are not the same thing imo. At least in that something can show all the faculties of thought without necessarily being conscious. An LLM can still be useful and give you the useful results you want without being conscious.

Like lets say i gave a human and an LLM a tech spec and the LLM created a software product as good as the human, does it matter that there was consciousness behind the human thinking?

LLMs can’t think

What is your definition of thinking?

These companies biggest source of revenue is per-token pricing though, not subscriptions. On tokens they make a good margin.

Yes, sadly the most principled will have to contend with the world passing them by when it turns out those principles are dead weight to capital momentum. You could have made the same point during the dawn of the industrial revolution: that purchasing any product of a machine would be betraying your friends who have spent years honing their craft, but either way, people will still opt for the machine because it's 10x cheaper and faster.

AI is good enough now that you can't claim that you aren't using it because you're upholding some higher standard of quality. It is simply a matter of it offending your sensibilities.

And it is why I will call for his arrest, speedy trial and imprisonment every time I have the misfortune of reading his name. I wish to see this person suffer from the bottom of my heart.

RAM prices were going to go up no matter what given the demand for AI. To so strongly want someone to suffer for fulfilling his fiduciary responsibility to the company he runs seems a bit disproportionate.

It is undeniable that the entire AI sphere is a bubble, artificially propped up by circular investments and wash trades, and that is now poised to raid pension funds.

I honestly can't take your claims that AI is a huge artificial bubble seriously when you make it very clear how much you want it to be a bubble. The mark of a mature mind is being able to admit that people you personally dislike actually may be right about a few things.

That these people whom you hate so much are also cartoon villains who have taken the entire world economy for a ride is a very satisfying narrative because it implies a future point when all of their crimes will be revealed and they will be held accountable. It is however not a realistic thing to hope for. This is all because of one very real but hard to admit reality: AI is a very useful technology and is worth the billions of dollars that are being paid for it. To deny this reality at this point, when AI models have already proven unsolved Erdos problems and are driving such huge demand on both the consumer and enterprise side, is unreasonable, and consequentially smart people who would otherwise take your points seriously are going to see right through your claims as just being emotional self-soothing.

What evidence do you have that Altman, Amodei, Hassabis, etca re hateful?

I think in the short term much of the concern for AI replacing jobs was overblown, largely due to two factors

1. Benchmark performance of LLM's and AI models did not fully represent skill in real-world domains

2. Most jobs span far more requirements than their specific job descriptions, many of which lie, even in simple jobs, in the realm of highly adaptive, context rich multi-modal information processing that most humans do still better than AI

However, there is nothing fundamental that prevents LLM's from scaling and improving, aided by better scaffolding, to the point of replacing many white-collar jobs, especially ones which have limited, specific requirements and output parameters. This is an enormous chunk of the white collar work force, and displacement is already happening in limited sections, and will surely continue as AI capabilities diffuse.

It seems however deeply entrenched in many people's identity to deny this fact, because to accept it requires accepting that many of the essential claims of AI CEO's are somewhat true to a degree, and that LLM's are a genuinely useful technology.

in part because of the huckster-like triangulations of scumbags like Sam Altman and Dario Amodei

I would happily read an AI-critical blogpost if it weren't clearly motivated by a strange, specific hatred of the prominent AI figureheads.

At this point I automatically dismiss writing like this, the motivated reasoning is palpable. Their distaste for the character and general vibes of the AI industry trap them in blatant denials of reality, like claiming that AI is a completely worthless technology or surely the bubble will pop any minute now.

I am all for well-researched criticisms of these companies and their claims, but please start at the facts and use them to derive your conclusions, rather than the other way around.

Extremely strict copyright laws were intended to give the rich and powerful the outcomes they want, allowing corporations to rich themselves on IP gatekeeping. Until the last couple years, it was firmly corporation and wealth aligned to be extremely litigious and offer little leeway for using copyrighted works.

Let’s say I created a robot that walked around the world and gathered data from its environment. On its way it heard 20 copyrighted songs, looked at 40 copyrighted works. Should I owe royalties to the creators of those 60 works if I were to sell my robot?

Yes of course, but why do you believe it's not fair use?

The essence of fair use is that it is ok if the use of the copyrighted material is transformative. LLM training I believe is transformative, as it is taking the data (as in a work of text, an image, video), and feeding it through the layers of a neural net to marginally update its weights. It is factoring each work into a very very small fraction of the model's overall sense of the world.

Now, AI models are capable of reproducing copyrighted works to a degree of high accuracy, especially if those works recur very frequently in the training data, but I believe that's a different issue. Any video camera is capable of taking a photo of a copyrighted work, but that isn't essential to the value of the camera, though it is undeniably what certain cameras are used for. The exact reproduction of copyrighted works is a likewise something that LLM's can do, as any intelligent person could recite song lyrics or a work if they memorized it, but each individual work is only marginal to the overall effectiveness and value proposition of the model.

WHy would there be riots in the streets?

It's funny how worked up people get about copyright with respect to to AI training when using copyrighted material for training an AI model is fair use. We have a concept of fair use in copyright because economic growth is essentially tied to the free proliferation of information.

I can't really trust any anti-AI argument when it feels more of a tribal grievance than a rational explanation of concern. Especially with the overuse of the "techbro" pejorative, it seems more a lament against a certain type of attidude in the tech world and a hatred that that attitude has translated into massive material wealth.

Everyone's financial literacy seems to evaporate when discussing AI companies. They assume that companies need to be profitable or they're a bubble waiting to burst.

The whole point of the company is that they are investing a huge amount of money upfront in order to make models that are better and better, and thus have a higher productivity multiplier.

They are very profitable on inference, they just know that the race to AGI requires a huge amount of investment, compute, getting the best researchers, etc.

They are having no problems raising capital. People are offering up their homes to get Anthropic stock.

It is not ok to lobby and manipulate governments for fiscal policy changes to write off the costs and risks on the general public.

How are they doing this? What are you referring to?