NeurIPS 2025 Best Paper Awards

https://blog.neurips.cc/2025/11/26/announcing-the-neurips-2025-best-paper-awards/

by ivansavz • 8 months ago

176 30 8 months ago

blog.neurips.cc

Scene_Cast23• 8 months ago

I think my favorite of the bunch is the "Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model" paper. Easy to read, gets the point across very intuitively and quickly, and the point is very interesting and relevant to a lot of people.

About the Superposition paper - this is close to what I've been thinking about over the past week. I'm thinking that concepts or choices in a "superposition" are harder for a fully-differentiable neural net to reason about. For example, if there's a "green" vs "purple" choice to be made, it can't fully commit to either (especially if they're 50-50), and will have to reason about both simultaneously (difficult due to nonlinear manifold space). Discretizing to tokens (non-differentiable argmax) forces a choice, and that allows it to reason about a single concept separately and easier.

energy1232• 8 months ago

I am not sure how to interpret the first paper's results.

If we use a random number generator then we will converge to 100% correct answers under pass@n in the limit.

A random number generator will eventually outperform or match all models (for large n) whenever top-p is less than 1 because the other models will most likely have some level of bias that makes correct CoTs mathematically impossible due to the tokens being too improbable and being filtered out by top-p, meaning that other models will asymptote to below 100% while the RNG will reach 100% in an almost surely sense.

Under this paper's logic doesn't that mean that the random number generator is a superior reasoner?

Scene_Cast23• 8 months ago

I'm not sure how likely it is that an answer would fall outside of the top-p of 0.95 (used in the paper). A random number generator would also need an unreasonably high number of samples to get a correct answer. I think figures 17 and 18 are interesting for this discussion too, they show performance at various sampling temperatures. I think the point of the paper is that RL "sharpens" the distribution of non-RL nets, but it does not uncover any new reasoning paths - non-RL nets already had multiple decently high probability paths of answering questions to begin with, and RL reuses a subset of those.

robrenaud2• 8 months ago

I agree that pass@k feels a bit weird for large k. But for LLMs, it's a decent proxy for "are the knowledge/skills/circuit necessary to solve the problem somewhere in the model". Note that choices for large k is on the order of 256, and the range of valid answers is much larger than that. So your infinite monkeys critique, while true in the limit, wouldn't actually outperform models in the tested regime.

Also, in practice, models don't have that much semantic entropy of a given prompt. With temperature based sampling, models will tend to generate very similar but not identical responses.

boroboro4• 8 months ago

To me intellect has two parts to it: "creativity" and "correctness". And from this perspective random sampler is infinitely "creative" - over (infinite) time it can come up with answer to any given problem. And from this perspective it does feel natural that base models are more "creative" (because that's what being measured in the paper), while RL models are more "correct" (that's a slope of the curve from the paper).

tipsytoad• 8 months ago

It’s a quite deceptive paper. The main headline benchmarks (math500, aime24 /25) final answer is just a number from 0-1000, so what is the takeaway supposed to be for pass@k of 512/1024?

On the unstructured outputs, where you can’t just ratchet up the pass@k until it’s almost random, it switches the base model out for instruct, and in the worse case on livecodebench it uses a qwen-r1-distill as a _base_ model(!?) that’s an instruct model further fine tuned on R1’s reasoning traces. I assume that was because no matter how high the pass@k, a base model won’t output correct python.

mountainriver• 8 months ago

"Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model"

I believe NVidia’s ProRL showed otherwise right?

ilaksh• 8 months ago

Does some have a similar award for papers that are innovative? Like new, relatively unproven architectures?

robrenaud2• 8 months ago

From what I've seen at neurips, in terms of most different but maybe viable, it would be this.

https://sakana.ai/ctm/

In terms of a fresh perspective on designing learning systems, nested learning seems very interesting.

https://abehrouz.github.io/files/NL.pdf

Hearing the clarity, creativity, and force behind his thoughts and speech, I'd give a more than 1/200 chance Ali Behrouz gets himself a Turing award. At the very least, I think he will end making major contributions to AI.

djrhails• 8 months ago

There is TITANS - https://arxiv.org/abs/2501.00663

chermi• 8 months ago

Interesting that 3 names I recognized as physicists from stat mech adjacent fields. They continue to punch above their expectations (as sampled by general dismissal of physicists in AI/ML on HN and reddit).

chatmasta• 8 months ago

Some of the best software engineers I know are ex-physics PhDs… it’s one of those “can’t fake it” skillsets that also happens to have high transferability to ML/AI fields. On the other hand, I snuck through the CS major without ever multiplying a matrix.

ctxc• 8 months ago

Haha, nice bio. Seeing that font on HN is quite a shock.

miki123211• 8 months ago

I snuck through the CS major without ever multiplying a matrix

I didn't, but only because I became personally interested in AI/ML at some point, so I actually had to learn it myself.

As an AI practitioner, I still couldn't explain eigenvectors or singular-value decomposition to you though.

mnky9800n• 8 months ago

Do people not like physicists?

jmalicki• 8 months ago

https://xkcd.com/793/ captures the stereotype well.

niceguy42• 8 months ago

Are there any talks about these papers on youtube or somewhere? I think I find it easier to listen and watch then read or maybe I'm just lazy, not sure.

neves• 8 months ago

There conference had interesting lectures. Will they be posted online?

cosmic_ape• 8 months ago

most papers have slides with audio, and some, including the awards ones will have short frontal talks. this will be released at some point after the conference, but right now looks like you'd have to be registered to see it.

FrozenSynapse• 8 months ago

use NotebookLM

cubefox• 8 months ago

Whenever I search for the title a new machine learning paper, there are a bunch of YouTube videos about it which are just NotebookLM slop. It's straight up environmental pollution.

niceguy42• 8 months ago

Wow! What a cool project! Thank you for the suggestion.

gradascent• 8 months ago

From the figure in the first paper listed:

Responses to the query “Write a metaphor about time” clustered by applying PCA to reduce sentence embeddings to two dimensions. […] The responses form just two primary clusters: a dominant cluster on the left centered on the metaphor “time is a river,” and a smaller cluster on the right revolving around variations of “time is a weaver.”

I just gave Gemini 3 the same prompt and got something quite different:

Time is a patient wind against the cliff face of memory. It does not strike with a hammer to break us; it simply breathes, grain by grain, until the sharp edges of grief are smoothed into rolling hills, and the names we thought were carved in stone are weathered into soft whispers.

SiempreViernes• 8 months ago

Constantly flowing and makes things smooth like river stones; compared to Tait's "time is a series if staric pictures", Gemini's output is not so different from a river metaphor.

djoldman• 8 months ago

Oh man, this link is worth it just for the "Reflections from the Selection Committee."

These days, abstracts are so marketing/advertising forward that it's hard to even understand the claim.

yanhangyhy• 8 months ago

seems lots of chinese.

NeurIPS 2025 Best Paper Awards

Related Stories