kaycey2022

What is routing? How do they decide which is better? The only way to come up with a routing model for your workload is to send queries to all the models and then come up with a way to say which solution was better, often trying out multiple times for the same model + query to account for other statistical errors.

This makes you, an ai inference user an unwitting AI company with a non scalable product.

The biggest mental trap people have fallen for is the notion of “best” and, always using the frontier model. Instead you should just bite the bulet and choose the cheapest or the best. This routing dance is just tokenmaxxing in disguise.

Edit: Another thing concerning me is that the models themselves are not concrete behind their endpoints. Models can be arbitrarily dumbed down by reducing their inference resources. Once Anthro/OAI release a new great model, they are fully incentivised to dumb down their current models to drive traffic to the new shiny more expensive one. In fact they can do this for any reason. Once they switch things behind the API interface, how useful is your meticulously tuned router? Not much at all.

Is that different from mixture of experts?

Does a sculptor enjoy the act of chiseling stone more than be driven and desperate enough to uncover the statue within. Both are valid ways of thinking. Who am I to say otherwise? One just has more social rewards associated.

Blaming it on “culture” makes me feel good about myself and my in-group.

"The cost of fragmented (sovereign) systems is ... sanctions evasion"

My man, that is the prize. Article is clearly written for US government elite to exhort them to bully other countries into staying in the system.

Just install steam on Linux. I think nearly every windows game is playable now. Gaming was the only thing keeping me in windows. The irony is I think Linux gaming us only feasible because of the win32 target. Otherwise the user space in Linux afaik is still a poop show.

I love it. Put the guardrails on the human. Your self driving car crashes? Straight to jail with you Mr Musk. Your AI is generating CSAM, straight to jail with all the pedos!

Recognise the safety and guardrails BS for what it is. An elaborate smoke and mirrors to get out of responsibility and liability.

I haven’t seen loops work for anything beyond simple tasks. And for complex tasks the loop itself is so complex half the time it would just fail and need you to intervene. Why would you go through all that effort of engineering a loop when a descriptive prompt followed by inspection is more than enough. It’s like idle hands looking for something to do.

And Claude or gpt can make websites just fine. So can the open weight models.

There is no skill involved in using AI. Most of the stuff people do on top of frontier models is just cope. It’s literally the bitter lesson repeated.

Got fired. I’ve no idea what I’ll do after 10 years of backend dev. I’m thinking of making video games. It looks rough but I want to do it.

My read was: "The first 90% of the work takes the first 10% of the time, and the remaining 10% of the work takes the other 90% of the time." And that we are now squarely in the remaining 10% of the work.

To first open source AI, you must first open source GPU compute

Is abliteration even necessary. While “playing around” I have noticed that most models are very strict only in the first prompt. The moment you get past that with a good turn, the next turn on you can get them to do _anything_.

I just ask whatever model I'm using, mostly GPT these days, to do the thing. First I discuss a plan of action and then go back and forth with it to settle on one. This is usually a long process and for complex features may even take a day or two. And then once I am satisfied, I ask it to implement. I skim over the code, it often generates too much code for my liking. I sometimes ask it to change things that I dont like. This is almost always to do with bad tests or not using existing code paths. I just work on one thing at a time.

I dislike it because it feels like I’m doing someone else’s work for them, reviewing what they should have reviewed in the first place.

We carry the seeds of our own destruction they say. So this is a good thing. America should take care of their own citizens first. It is a good thing. Also there is a school of thought that says prosperity of any country is primarily a function of the kind of citizens they have, and how the country is able to leverage the intellect of its citizens. An important thesis of recent discussions of American prosperity is that a lot of it has been built up by the immigration of enterprising people into the US. So it can be argued that American prosperity is at the cost of prosperity of the rest of the world. And most of these things have compounding effects. The more intellect gathers in the US, that country can leap frog into the future at a far more vigorous pace than other could if most of these people were left inside their own countries. But in any case, these current events give us an opportunity into testing the thesis of American prosperity. Either balance will be brought to how global prosperity is distributed, or finally a country will be able to take care of its own citizens first. Either way this is all a great thing.

Last 6 months vibecoding, I feel I have learned very little. You learn things like a high level executive or a manager. I can describe things at a very high level, but remain clueless how things work at the implementation level. It is just whatever code quality, architecture, common sense I learned earlier is carrying me through. Platforms like Codecrafters will be badly missed.

Those are not red flags. Those are the actual thing. I think red flag is a heuristic that warns you about a course of action.

I was in the same boat. You should try running an AI agent to solve your problems. Works like a charm. Most of the times. The times it doesn't, it wasn't worth it anyway.

We can't even say something in public these days. All your comment needs is the trigger words or constructions

- That's why X works. - Not X but Y.

And some moron will be in the replies, saying "LLM comment". I hate this world. But probably yeah. llm comment.

Arent they just on the hook for trillion dollars

you can do way more than just coding with the coding agents.

No. It's still shit. It can do some well contained tasks, but it is very less usable on production codebases than gpt or claude models. Mainly because of the usage limits and the lack of good environments for us to use it on. Anthropic gets away with this because claude code, as bad as it is, is still quite functional. Gemini cli and antigravity are utter trash in comparison.

scaling these solutions will prove to be counter productive. If you are thinking of scaling you are still trapped in the current paradigm. This plumber guy is only unique in that he read the news and pushed a little harder to see what the actual f is going on. In the days that come, every single person who is serious about their job will do and experience the same thing.

I'm not saying this particular individual is wrong in trying to build his solution to the market. Maybe there is some VC money to be made in this moment. But as AI in the workplace gets normalised, most people will either come up with solutions for their problems, or they will ask someone they know to help them with this.

scale will only matter if you are explicitly building a platform. That will still require real software engineering skills.

As for hardware interfacing, if I am not mistaken, almost all companies selling hardware right now still behave like babies when it comes to users getting access to the software inside it. They void warrantees, sue them, so on etc. For ambitious user driven software innovations in the hardware space the companies should open up their interfaces. I don't see this happening at all not only because of the companies' greed but also for regulatory and safety reasons.

sry just a joke man

there is nothing about claude code that prevents you from using it for non coding use cases. nothing that happens in open code or any harness for that matter is hidden from anthropic. neither does open code allow access in some nefarious use case that claude code does not.

the difference is not like the difference between gmail and gmailfs like you seem to be misunderstanding. a more accurate comparison would be the difference between curl, or httpie vs postman.

I don't think it's any of that. It's plain as day to see that as the owners of the API they can see every request, collect every metric they need. The routing can easily be done in the API gateway layer and doesn't depend on the harness at all. Good harnesses are not performing acrobatics around user behaviour to improve anything. They just understand the provider API very well and translate tool calls really well. Then they get out of the way to let the magic happen.

This is about controlling user behaviour, resisting standardisation efforts, to keep subscribers hooked to their models. If they keep their users on claude code, they don't have to adapt to evolving harness standards when shown increased usage across different harnesses. They will try to keep everything on their terms, on their poor quality of engineering, and the users accustomed to claude behaviour will be left with no quick way to switch to other models.

This strategy will keep working as long as they own a top tier frontier model with enough marketing hype behind it to blind unsophisticated users.

This is clearest reason for us to accustom ourselves to using open weight models on open source harnesses. Whatever advantages the frontier closed models offer, this will turn into ash in the mouth, when the enshittification cycle begins. And don't be mistaken, it will begin. There is no precedent which can claim otherwise.

I am sure the models themselves are being RLHF tuned to work very well with the proprietary agent harnesses. This is all turning into a huge trap right in front of our eyes and the target is not just programmers but also companies whose core product involves software production.

You honestly don't know of Moritz Hardt?

Agile Intelligence