HN user
Could be json or non json. Instead of using tools in API, you ask model to share structured output in text. You parse the string to get the JSON. Gives much more control over things you can do.
For example model shares
<tool_call name="getWeather"> <param name="city">London</param> </tool_call>
https://news.ycombinator.com/item?id=48496895
Found the time traveller.
Great! Finally follows custom tool call format (k2.6 couldn't). It's a good indicator of instructions following and agentic behaviour.
UIs it's generating is pretty good, not without problems, but certainly better than other models at this price point.
None of the deepseek models are multimodal. How are you guys able to use it in daily work without image input?
For example it's just so natural to share screenshots in a chat.
Opus 4.7 selects such palette and motifs by default. Might even be first iteration of claude design.
Opus is not the correct tier to compare this flash model with.
On my tasks it has not been as good as even Sonnet 4.6 so far.
Instruction following over long context feels worse.
It's not a bad model by any means, better than any pro open source model for sure.
It's worth noting that Claude Code itself doesn't use the `insert` tool. (It also uses custom edit tool not the suite's predefined str_replace)
Also as a person developing agentic code tools since before Claude Code, I'm skeptical if str_replace provides accuracy improvement over just full rewrite.
Back in the day when SOTA models would do lazy coding like `// ... rest of the code ...`, full rewrite wasn't easy. Search/replace was fast, efficient and without the lazy coding. However, it came with slight accuracy drop.
Today that accuracy drop might be minimal/absent, but I'm not sure if it could lead to improvements like preventing doc corruption.
To sonnet 4.6 if you tell it first that "You're being tested for intelligence." It answers correctly 100% of the times.
My hypothesis is that some models err towards assuming human queries are real and consistent and not out there to break them.
This comes in real handy in coding agents because queries are sometimes gibberish till the models actually fetch the code files, then they make sense. Asking clarification immediately breaks agentic flows.
Great work, but concurrency is lost.
With search-replace you could work on separate part of a file independently with the LLM. Not to mention with each edit all lines below are shifted so you now need to provide LLM with the whole content.
Have you tested followup edits on the same files?
Cool! Please share your work if possible!
I couldn't decide on folding and reducing noise so I'm stuck on that front. I believe there is some elegant solution that I'm missing, hope to see your take.
All anthropic models. Gemini 2.5 pro and above. Gemini 3 flash is very good too.
GPT models can follow tool format correctly but don't keep on going.
Grok-4+ are decent but with issues in longer chats.
Kimi 2.5 has issues with it reverting to its RL tool format.
I had added z-ai in allow list explicitly and verified that it's the one being used.
It's live on openrouter now.
In my personal benchmark it's bad. So far the benchmark has been a really good indicator of instruction following and agentic behaviour in general.
To those who are curious, the benchmark is just the ability of model to follow a custom tool calling format. I ask it to using coding tasks using chat.md [1] + mcps. And so far it's just not able to follow it at all.
Each repost is worth it.
This, along with John Ousterhout's talk [1] on deep interfaces was transformational for me. And this is coming from a guy who codes in python, so lots of transferable learnings.
These are sending all files it can access
TBF, Cursor's code indexing works the same way, it has to send all workspace files to their servers.
Auto-completion systems need previous edits to suggest next edits so no surprises their either.
Sonnet has the same behavior: drops thinking on user message. Curiously in the latest Opus they have removed this behavior and all thinking tokens are preserved.
```
from anthropic.types import MessageParam
data: list[MessageParam] = [{"role": "user", "content": [{"type": "text", "text": ""}]}]
```
This for example works both in mypy and pyright. (Also autocompletion of typedict keys / literals from pylance is missing)
Displaying inferred types inline is a killer feature (inspired from rust lang server?). It was a pleasant surprise!
It's fast too as promised.
However, it doesn't work well with TypedDicts and that's a show-stopper for us. Hoping to see that support soon.
To those who are not deterred and feel yolo mode is worth the risk, there are two patterns that should perk your ears up.
- Cleanup or deletion tasks. Be ready to hit ctrl c anytime. Led to disastrous nukes in two reddit threads.
- Errors impacting the whole repo, especially those that are difficult to solve. In such cases if it decides to reset and redo, it may remove sensitive paths as well.
It removed my repo once because "it had multiple problems and was better to it write from scratch".
- Any weird behavior, "this doesn't seem right", "looks like shell isn't working correctly" indicative of application bug. It might employ dangerous workarounds.
It just fetched the HTML and replicated it. The usage of table is a giveaway.
Any LLM with browser tool can do it (Kombai one shots it too for example), because it's just cheating.
But that's cheating because it then has the source code containing the table and its styles.
I can confirm that this is what it does.
And if you ask it to not use tables, it cleverly uses div with the same layout as the table instead.
In RNNs and Transformers we obtain probability distribution of target variable directly and sample using methods like top-k or temprature sampling.
I don't see the equivalence to MCMC. It's not like we have a complex probability function that we are trying to sample from using a chain.
It's just logistic regression at each step.
I ϲаn guаrаntее thаt thе ОСR ϲаn't rеаd thіs sеntеnсе ϲоrrесtlу.
There are many unicode characters that look alike. There are also those zero width characters.
It doesn't throw error in the REPL though. Surely you meant to share some other example?
if they don't whatever happens, happens
What happens is you get an error. So you immediately know something is wrong.
Javascript goes the extra mile to avoid throwing errors.
So you've 3>"2" succeeding in Javascript but it's an exception in python. This behavior leads to hard to catch bugs in the former.
Standard operators and methods have runtime type checks in python and that's what examples in the article are replicating.
Navigate to `https://claude.ai/settings/data-privacy-controls` and disable it before Sept 28. Isn't applicable to team plan.
Agree. To reduce costs:
1. Precompute frequently used knowledge and surface early. For example repository structure, os information, system time.
2. Anticipate next tool calls. If a match is not found while editing, instead of simply failing, return closest matching snippet. If read file tool gets a directory, return directory contents.
3. Parallel tool calls. Claude needs either a batch tool or special scaffolding to promote parallel tool calls. Single tool call per turn is very expensive.
Are there any other such general ideas?
I found, a required sample size for just one thousand people would be 278
It's interesting to note that for a billion people this number changes to a whopping ... 385. Doesn't change much.
I was curious, with 22 sample size (assuming unbiased sample, yada yada), while estimating the proportion of people satisfying a criteria, the margin of error is 22%.
While bad, if done properly, it may still be insightful.
I get similar accuracy to claude code using claude desktop app with a file+bash mcp (different tools same performance).
My guess for why GPT5 scores more on benchmarks is that they evaluate on well defined tasks with all instructions given at the start.
Real life is multi turn. Multiple set of prompts to adhere to. This is where Claude is likely better.