int_19h

Yes. A mixture of experts is a single model that activates different routes though the same weights, with the route possibly changing literally on every token. It's not experts as in a bunch of standalone models that are good at specific high-level tasks.

By this definition most of our culture isn't art because it's made not to "communicate some emotional experience" but to induce one in order to make money. 90% of pop music is like that, for example. And LLMs can do that just as well - better, probably, since they know all the psychological tricks from their training data.

I would argue that LLMs are still magic, because, unlike the rest of this list, we still don't actually know how exactly they do what they do. We know how to build and train them, and we have general ideas about why e.g. attention is important, but if we actually knew how they tick we could do the same thing in a much smaller package (by handcoding everything).

MoE is just activating fewer weights per token than the whole model. It will continue to make sense for as long as compute is more expensive than memory (at scale).

Is it? To me, the notion that a superintelligence (by which I'm assuming you mean the more sensible "something smarter than us", not "literally a godlike entity") automatically means that sky is going to fall is sillier.

We're supposedly getting Mac Studio with 1.5Tb RAM in 2 years. That would be enough to run an Opus-level model.

Of course, it will also probably cost somewhere around $50k...

But if local AI really does become pervasive, maybe it'll be one of the things people buy on credit, like cars.

I'd say that's more indicative of what kind of things people in your bubble generally work on.

I vibe-coded a semantic parser for Lojban.

A friend of mine is using it to work on dev tooling.

Another friend, a mathematician, has recently used it to prove a conjecture he published 15 years ago.

Elon saying that it is "Opus level" doesn't actually make it so.

The good thing is that the competition in the field is very cutthroat with so many contenders, so if there are gains to be made they will be made, and then broadly adopted by others.

Higher subscription brackets are likely worse for them. I recall seeing someone calculate that a fully maxed out highest subscription bracket is something like $15K in tokens?

And people paying $100 or $200 are much more likely to max it out for purely psychological reasons - it crosses that threshold where I want to see my money's worth in full. Whereas people on $20 subs are more likely to be there just to get access to better models and features, and are not necessarily even doing any substantial work.

They didn't ban people from using Claude, though. They banned them from their flat-fee subscription and required that you pay per token.

It's still questionable but I don't think it's in the same ballpark as what you describe.

That is just wrong. In many places it literally results in a brain drain. Your position literally hurts already poor countries just because you want to see less white faces in the streets.

Countries are not entitled to keep their citizens by force to prop up their economy. If the person doesn't want to stay, why should they be forced to?

You might want to contemplate the fact that you're literally rehashing the same arguments that Soviets and East Germany used to justify their exit visas. Except you're doing it from the other end of the pipe.

Ultimately it doesn't matter what the constitution says if a large enough supermajority is unhappy about it. Constitutions by themselves are pieces of paper, they only work when society as a whole chooses to treat them as more than that.

Service providers like say Apple and Google can push an update to your phone which will intercept all that data after it has been decrypted.

Any hierarchical system of governance ends up with sociopaths on top. It's just the nature of hierarchy and bureaucracy that is necessary for it.

Your sons and daughters will still be wide eyed zombies, it's just that now your government will decide what mush to turn their brains into.

"The right of equal suffrage among the States is another exceptionable part of the Confederation. Every idea of proportion and every rule of fair representation conspire to condemn a principle, which gives to Rhode Island an equal weight in the scale of power with Massachusetts, or Connecticut, or New York; and to Deleware an equal voice in the national deliberations with Pennsylvania, or Virginia, or North Carolina. Its operation contradicts the fundamental maxim of republican government, which requires that the sense of the majority should prevail. Sophistry may reply, that sovereigns are equal, and that a majority of the votes of the States will be a majority of confederated America. But this kind of logical legerdemain will never counteract the plain suggestions of justice and common-sense. It may happen that this majority of States is a small minority of the people of America; and two thirds of the people of America could not long be persuaded, upon the credit of artificial distinctions and syllogistic subtleties, to submit their interests to the management and disposal of one third. The larger States would after a while revolt from the idea of receiving the law from the smaller. To acquiesce in such a privation of their due importance in the political scale, would be not merely to be insensible to the love of power, but even to sacrifice the desire of equality. It is neither rational to expect the first, nor just to require the last."

Have you seen Wickard v. Filburn?

Germany is one prominent example for obvious historical reasons.

But https://en.wikipedia.org/wiki/Entrenched_clause are not all that uncommon in general.

Fable is great as a "manager" model (writing specs, opening issues, doing PRs, verifying fixes) while Codex cranks out the code. Especially if you tell it to roleplay as an Eastern European software engineer and "tell it like it is without consideration for anyone's feelings".

You don't have to choose between Elm and TypeScript though, there are so many other options now - Haskell, F#, Rust, OCaml, just to name a few.

These days, though, why not just use Haskell itself? Upstream ghc cancompile to wasm for several major releases now and there are several actively developed web frameworks, some of them very Elm-inspired, e.g. https://haskell-miso.org

It doesn't matter how good a language is in theory if people using it for practical purposes end up having to abandon it.

Fable is also very expensive, unfortunately.

It will be interesting to see how cheap they can make it long term.

It was true every time though. The capacity of frontier models to tackle complicated issues has improved immensely. I still remember the first time I saw a model do a non-trivial issue end to end, and that was less than two years ago. Now they can genuinely do whole projects with human only as a supervisor / quality checker.

Do they still make mistakes? Sure. So do humans, though, so it would be unrealistic to expect perfection. The question is: does Fable make fewer mistakes than the median human coder? And at this point I'm genuinely not sure anymore.

If you just keep throwing feature requests at an LLM, then yes, this happens. However it can self-correct if you specifically give it engineering debt / code cleanup as a task. And Fable in particular is very good at this exact thing.

With DS4 it would have a lot more bugs, too.

And they are welcome to crowdfund for them. But they need to be explicit about what exactly the money is funding.

Fail fast is a feature, not a bug. It's much better to get clear and actionable feedback rather than the page silently rendering incorrectly in some subtle manner.

Yes, that's correct. I am calling into question the democratic processes of any so-called "representative democracy".