srush

Awesome to hear, I will share with the team.

Roughly, we had Cursor software engineers record real questions they were asking models, and then had them record the PR that they made that contained the result. We then cleaned these up. That is the benchmark.

A lot of people use it! It scores very well on our benchmarks, significantly better than Composer-1.

We use Ray data for our map-style processing jobs. For example one tool have runs over all the rollouts from the RL system and collects qualitative statistics to understand which type of agent trajectories are being reward, and what types of searches and terminal commands are being made.

Oh good question. Actually speaking at the Ray Summit next week in SF so we will talk more about it. We used Ray throughout the pipeline for running evals, for the RL controller, for data collation, and for visualizations. One tool we found helpful was Ray Data which let us easily scale over data and run logs.

We train with a single agent. is that the question?

neat!

Our view is that there is a now a minimal amount of intelligence that is necessary to be productive, and that if you can pair that with speed that is awesome.

Let us know how you like it.

Straight up untrue.

There are lots of good models we like here. But we agree that getting the right point on the smart+fast graph can make agentic coding feel really good.

(Cursor researcher)

Thanks! Yeah, been working here for 9 months now. Fascinated byt agentic coding both as a researcher and user.

Totally agree that "smart model" is the table stakes for usefulness these days.

We will do our best. Luckily I don't think there are major telecom companies called Composer-2.

Unfortunately not, as we used our own internal code for the benchmark. We would also like to see more benchmarks that reflect the day-to-day agentic coding use.

The blog talks about the training process. Specifically we trained with RL post-training on coding examples.

Thanks! I do like the labs blog posts as well though, OpenAI and Anthropic have some classics.

Our primary focus is on RL post-training. We think that is the best way to get the model to be a strong interactive agent.

We did a lot of internal testing and thought this model was already quite useful for release.

Cheetah was an earlier (and dumber) version of this model that we used to test production speed. They are both developed in-house. If you liked Cheetah, give this model a try.

We like the name Composer and were sad to see it go. Excited to bring it back. (Agree Cheetah is a cool name too.)

There is a footnote that should help with the models. Training is a harder thing to report on, but roughly our finding here is that RL scales.

We don't wear shoes [1].

[1] https://www.businessinsider.com/no-shoes-policy-in-office-cu...

Agree that Sonnet 4.5 is an excellent model. Would be curious to hear your experience using Composer though, it's quite good.

We also are big Tab users here at Cursor. In the blog we talk about the motivation for this project came from thinking about a Tab-like agent.

Hi everyone,

I am an ML researcher at Cursor, and worked on this project. Would love to hear any feedback you may have on the model, and can answer question about the blog post.

A recent tutorial video from one of the authors featured in this article:

Evaluating AI's World Models (https://www.youtube.com/watch?v=hguIUmMsvA4)

Goes into details about several of the challenges discussed.

Actually realize this is to the modern version not the original. So props to Austin Huang, Suraj Subramanian, Jonathan Sum, Khalid Almubarak, and Stella Biderman who rewrote this one.

Oh hey, I wrote this. Been a long time. I had the lucky of break of working in machine translation / parsing when the most important invention of the century happened in my niche field.

I'm pretty interested in the intersection of code / ML. If that's your thing here are some other writing you might be interested in.

* Thinking about cuda: http://github.com/srush/gpu-puzzles

* Tensors considered harmful: https://nlp.seas.harvard.edu/NamedTensor

* Differentiating SVG: https://srush.github.io/DiffRast/

* Annotated S4: https://srush.github.io/annotated-s4/

Recently moved back to industry, so haven't had a chance to write in a while.

I've tried to build sort of model several times, but could never get it to work. The challenge is that small perturbations in encoder space lead to removing semantically important details (e.g. dates). You really want these to mess up syntax instead to get something more analogous to a lossy video encoder.

For problems that require multi-step reasoning, standard LLMs seem to be stuck. The field is increasingly interested in models like o1 that output many "guesses" to find the right one. Currently open-source does not know how to do this, but we are reimplementing several possible directions to try. This replicates one important path using search and a verifier model.