ANTML: Anthropic’s Markup Language

https://karashiiro.leaflet.pub/3m4gf7geefs2l

by ko_pivot • 9 months ago

54 9 9 months ago

karashiiro.leaflet.pub

mudkipdev• 9 months ago

have model providers not found a better way to provide tool calling other than adding "respond with special json" to the system prompt?

avandekleut• 9 months ago

Using context-free grammars and sampling techniques.

otabdeveloper4• 9 months ago

The grammar still describes JSON though, right?

nl• 9 months ago

Despite the comments here, this isn't a special language. It's namespaced XML which is exactly what XML namespaces were designed for.

dainiusse• 9 months ago

And thank god. I saw the title and thought - yes, the world needs just another markup standard...

BoorishBears• 9 months ago

I think there's actually a wrong assumption here: that the tags aren't part of training

The model can't introspect itself to know this, but the latest hybrid thinking models do think noticeably differently when prompted the right way between these tags, suddenly shifting to an extremely verbose mode of output that looks more like raw reasoning traces the earliest versions of reasoning models were showing

I have a prompt that used the original <antThinking> tag (it wasn't namespaced with 3.5). Every release eroded how steerable that output was and now 4 and up start spewing massive amounts of tokens in an extremely freeform manner that's also more effective

nano-banana/Gemini Flash 2.5 Image similarly does something like this: when prompted to think carefully will generally write a paragraph or two, but once coaxed a bit harder will suddenly output massive sprawling markdown formatted planning like the original un-masked thinking traces from Gemini Thinking.

It's like there's a sort of "mode collapse" for reasoning in these models that's going to continue getting more noticeable as providers lean harder into RL, which is ironic given how much ceremony there is around hiding reasoning traces in these APIs to avoid distillation

est• 9 months ago

looks like slowly re-invent XML

zvr• 9 months ago

No, they're just using XML.

smahs• 9 months ago

The author doesn't explain (or is ignorant about) why this happens. These are special tokens that the model is trained on, and are part of its vocab. For example, here are the <think> and </think> tokens defined in the [Qwen3 tokenizer config](https://huggingface.co/Qwen/Qwen3-235B-A22B-Thinking-2507/bl...).

The model runtime recognizes these as special tokens. It can be configured using a chat template to replace these token with something else. This is how one provider is modifying the xml namespace, while llama.cpp and vllm would move the content between <think> and </think> tags to a separate field in the response JSON called `reasoning_content`.

ANTML: Anthropic’s Markup Language

Related Stories