runeblaze

tbh the summarized thinking with encrypted raw thinking is there for many purposes; it is there to:

1. make distillation much harder

2. safety: prevent modifications to the thinking leading to injection attacks.

3. also honestly sometimes the model raw thoughts can be deranged and is not a good user experience (consider the varied audience in the market, etc.)

also often the mass underestimate/the model makers over-estimate how people love distilling models

links to two papers with at least enough apparent quality and novelty to get into ICLR 2026

So basically... openrouter

:skull:

i now really wonder how many people of the public understood my thesis defense lol

visual similarity

SigLIP 2

Maybe visual-semantic similarity is more appropriate? Nonetheless the design is fantastic

I mean I used to work on model reliability with my little PhD degree and the models i manage go down all the time.

Some profs have a team of PhDs and things go to shit all the time. I don’t know why we expect $FRONTIER_LLM to do better

1. openrouter is API usage. There is obviously consumer side

2. people often use openrouter for the sole purpose of using a unified chat completions API

3. OpenAI invented chat completions; if you use openrouter for chat completions often you can just switch your endpoint URL to point to the OAI endpoint to avoid the openrouter surcharge!

4. Hence anyone with large enough volume will very likely not use openrouter for OpenAI; there is an active incentive to take the easy route of changing the endpoint URL to OAI’s

Schemas can get pretty complex (and LLMs might not be the best at counting). Also schemas are sometimes the first way to guard against the stochasticity of LLMs.

With that said, the model is pretty good at it.

Resouce-affording, if you are chasing the frontier of some more niche task you redo your training regime on the new-gen LLMs

Is it though? There is a reason gpt has codex variants. RL on a specific task raises the performance on that task

Sure never again is totally fair and I am sure a lot of people hate it. I was mostly objecting to the radioactivity of it. Your friends will be more like “I am looking to sell my Tesla in 3 months” if it is truly radioactive.

Let’s be realistic in our portrayal here.

I think radioactive is a strong word here… I have talked to a lot of people in tech

Reading what you wrote scares me

And if for some ungodly reason you had to do it in Python

I literally invoke sglang and vllm in Python. You are supposed to (if not using them over-the-network) use the two fastest inference engines there is via Python.

Agreed, I am surprised he is happy to stay this long. He would have been on paper a far better match at a place like pre-Gemini-era Google

I don't know the data distribution, but are you sure that's generated by an Adobe model? I can only see that it is in Stock + it is tagged as AI generated (that is, was that image generated by some other model?)

Disclaimer: I used to work at Adobe GenAI. Opinions are of my own ofc.

Emmmm sure, but throw this to a human artist who has not heard of Indiana Jones and see if they draw something alike.

I work in this space. In traditional diffusion-based regimes (paired image and text), one can absolutely check the text to remove all occurrences of Indiana Jones. Likewise, Adobe Stock has content moderation that ensures (up to human moderation limit) no dirty content. It is a world without Indiana Jones to the model

My personal mantra (that I myself cannot uphold 100%) is that every dev should at least do the exercise of implementing binary search from scratch in a language with arbitrary-precision integers (e.g., Python) once in a while. It is the best exercise in invariant-based thinking, useful for software correctness at large

each text token is often subword unit, but in VLMs the visual tokens are in semantic space. Semantic space obviously compresses much more than subword slices.

disclaimer: not expert, on top of my head

beats me. I spent so much time learning what a fundamental group is and I still cannot tell ppl what a fundamental group is convincingly.

I can’t even make stuff with fundamental groups.

Ummm guys when we talk about memory access in theory can we just be rigorous and talk about the computational model?

The real RAM model “in theory” tells me that memory access is O(1). Of course real RAM is a spherical cow but like we could use a bit more rigor

Of course it seems to be the same person to have done minecraft inside minecraft using redstone :)). peak!! done it again

Yea experience is very useful (it is very hard to exert influence without power and be the tech lead of a team unless you have some corporate experience) but at some point one needs to realize that if someone was writing Haskell at the age of 12 and they got decent scores at USACO they would make quite a decent engineer...

Python types - all the onus of static types, with none of the power of calculus of constructions. /s

sure of course one does not need types to do natural deduction… if you throw what you showed me to your average maths undergrad in the US they will get confused — I truly don’t see how proofs are simple

How proofs work is really simple

idgi. If you do your 101 logic class often you learn natural deduction, and how do you formalize natural deduction in a computer system? (Hint: type theory is "natural" for this).

Also how proofs work is far from simple.

idgi tho. The average working mathematician seldomly thinks about proof theory or how proofs "work". Proof theory if must be put in a single box it is an logician's art, and at that point it is niche enough in both CS and maths that it becomes meaningless to say mathematicians know more about it. Logicians are better at thinking about proofs than either maths people or CS people.

I mean I don't know what you mean by right, but given how type theory/ZFC/categorical foundations there is no clear winner (unless you can make a good argument for set theory as our foundation, because again, the average mathematician seldomly thinks in ZFC), I'd say type theory is quite nice in its computational interpretations so let's go with that.

I also don't want to go there, but obviously research drains manpower and money. If we can get CS people to write PAs we really should do that. They're filthy rich in the grand scheme of research.

ZF(w or w/o C)/type theory/other foundations of maths are "equally" right? My argument is that even if we are trying to build PAs from scratch, type theory provides tangible benefits to the working mathematician because we can just borrow CS research in many stuffs like proof checking and synthesis.

Of course that is a benefit in the "meta" level, but it is not like ZFC is a better foundation than topoi/types/etc..

I mean sure we can do ZFC or ZF as the foundation, and I am sure with enough effort we can make metamath or some modern derivative great again. At some point though I like having years of type-directed program synthesis research helping me write proofs; I like type-checking research helping me check my proofs, and I like my proof assistants to work well for verified program extraction if I also happen to write code because the mathematics academic job market is quite shit for your average pure maths PhD, and if I were to do logic then it might be even shittier.

Admittedly the above (except program extraction I suppose) can all be achieved with ZFC + a layer of type theory on top, and again that's a reasonable thing to do, but I hope this makes a strong enough case for the type theoretical foundation.

That story reads like what happens when the average senior engineer tries to do a hardish usaco problem; turns out algorithm engineering is different from your average enterprise engineering; turns out there are people in both camps

The coin problem is like the intro to it. Try some of the codeforces one :skull: