big-chungus4

Qwen3.7 pro is meh, but 3.7 max is a very good model

I'd say it's going to be very hard to come up with a method that works on general nonconvex functions while not working on convex functions

I'm not sure if the page is not fully loaded for me, because there are a lot of gaps in it, but from what I can see, I still have no idea what "Ode" is. The entire page is just empty marketing without saying anything.

Adam doesn't use the second derivatives matrix, it uses second moments of the gradient, which is the diagonal of the uncentered covariance matrix, but neither of them are directly related to SVD or singular values anyway.

There is a slight connection where Adam approximates full-matrix Adagrad which computes inverse square root of the convariance matrix, which you usually do using eigendecomposition, but on the covariance matrix SVD and eigendecomposition are equivalent (can easily be converted to each other), so you could use SVD to compute the inverse square root.

Last statement is a bit sus... Muon computes matrix sign function which can be defined as setting singular values to 1, though you can also define it without SVD. Muon itself doesn't use SVD because it uses a faster method to compute matrix sign. Adam doesn't do anything related to SVD or singular values. Also not sure what you meant by "second order singular values"

I am sure they also use Google search and Google maps, and it gives them an even greater advantage

A better solution might be not to constrain the generation, but to remove invalid fields from the tool call in the assistant message. So on the next turn, the model receives chat history which contains it's tool call, but without extra arguments. You can do that in OpenAI chat completions / responses, not sure about Anthopic API.

There is still a downside, sometimes the model really wants to include an additional argument for whatever reason it reasoned towards, and it needs the error message to understand that the argument doesn't exist. Otherwise if the argument is manually removed from it's tool call, the model will think that it accidentally left out the argument and start retrying and might go into a loop.

You can take the output of the matrix LSTM, which is going to be matrix for each token, and compute the SVD. To get better storage, we want U and V to be the same for all tokens, so that we can operate on the diagonal S matrix. But LSTM is likely highly nonlinear, U and V will be vastly different for different tokens.

That's useful, but wouldn't help with this particular experiment because they orthogonalize activations, not weights

LFM 2.5 1.2B instruct, it's good at instruction following as long as the instruction is short and simple.

Gemma 4 E2B is even better than LFM but I don't know if you can fit it in 3GB.

Qwen 3.5 0.8B and 2B are smart but very inconsistent for instruction following.

If you need absolutely tiny models, check Gemma 3 270m and FunctionGemma, LFM 2.5 230m and Granite 4.0 350m. However those are very stupid models and whether they are feasible for you will depend on your use case.

I can run Qwen3.6-35B-A3B at 20 TPS on my laptop with RTX 5070 Ti, with partial offloading to RAM. But the most I do is mess with it when I'm bored. I do coding by hand, but I often run autoresearch loops using free models, right now it's MiMo code. Autoresearch often requires my GPU, so it wouldn't be feasible to do when all of my GPU is used up by a local model. For mundane tasks like extracting and formatting specific structured text, I use Gemini in Google search

Just like me! I like AI because of how smart I am.

I am interested to know why people use those graph based agentic frameworks. Why not just define the behavior in python?

Or attach an LSP server with a type checker and require the model to produce strict strongly typed python code

we would never conclude that the LLM has conjured up digital re-creations of Julius Caesar and Genghis Khan

At some point the AI might become so powerful that whatever it reasons through isn't any less real than a computer simulation. If we assume that a person in a perfect computer simulation is conscious, then if it reasons about how people might suffer, it might simulate the outcomes, and there will be a conscious experience of those people suffering.

I've been running stuff like this too. I ran one "benchmark" where there are 10 agents, each agent initially only knows the name of the next agent in the list, and the goal for them is that each agent has a unique order 1 to n to assigned to them, where n is number is agents (also initially unknown to them). They are invoked in a random other and can only message one other agent per step.

Qwen3.5-9B was able to do this after a lot of time.

Qwen3.6-35B-A3B failed because it kept insisting that it needs to know n, but didn't try to figure it out by messing other agents.

Granite 4.1 9B failed completely because it was just writing non-descriptive massages like "Request to know the order" to the other agents and not replying to anyone.

I use the one in Google search

Running local agents 24/7, I get that it's a powerful CPU or GPU or whatever it is, but still, isn't it going to be constantly loud and 95C hot, that can't be good for the laptop if it's like that 24/7

MCP is a way to define tools that works with many apps and has a lot of extra functionality built in, it's not the only way, but it's popular because many apps support it. You can also make tools using the opencode API or any other API, and you can give them large descriptions that take up a lot of context. No matter how you define the tools, they are injected into the context of the model using the same chat template provided by the developers of the model.

Isn't MCP just a way to give agent tools? When you are building your own agent, you can define the tools manually, but if you're using something existing like opencode, how do you add new tools as a user? You use the API for that which is currently MCP. Saying MCP is dead is kind of like saying tools are dead, which is definitely not true because all modern LLM agents are trained for tool use and you wouldn't have agents without it.

The problems listed on the article are problems with specific tools that have large tool descriptions. This has nothing to do with MCP. There is nothing in MCP that would cause the tool descriptions to use more context than they would otherwise.

This article was generated from the original Qwen3.7-Max release blogpost and contains nothing new https://qwen.ai/blog?id=qwen3.7

It optimized the Extend Attention operator in triton. All models were optimizing the same operator

Out of curiosity, how much does it cost to daily drive Claude like this?

Why sloptember when it's may

How does x.cos().cos() work faster than doing two cos calls separately? Like the first cos call returns a tensor either way, the only difference is that it's not assigned to a variable. But how is it even possible know that difference in python?

Can you please train Qwen 3.5 like 0.8B to 9B using the same training techniques

They don't use the old model, they use EMA of the trained model weights as teacher

This article is more than 6 years old

seven

But, you have been using your mouse and keyboard for many years, so you know well how to use them quickly. I think that you shouldn't expect to be able to be quick with a new input type when you only tried it for a few minutes