HN user

usaar333

4,075 karma
Posts17
Comments1,756
View on HN
Claude Opus 4.7 3 months ago

page is updated to state:

MCP-Atlas: The Opus 4.6 score has been updated to reflect revised grading methodology from Scale AI.

Claude Opus 4.6 6 months ago

Openai has; they don't even mention score on gpt-5.3-codex.

On the other hand, it is their own verified benchmark, which is telling.

Claude Opus 4.6 6 months ago

i'd interpret that as rounding error. that is unchanged

swe-bench seems really hard once you are above 80%

Gemini 3 8 months ago

claude 4.5 gets 82% on their own highly customized scaffolding. (parallel compute with a scoring function). That beats Doubao

Sora 2 10 months ago

Physics seems better than veo 3 at least from demo videos

GPT-5 12 months ago

No it doesn't. If it were even linear compared to o1 -> o3, we'd be at 2.43 hours. Instead we're only at 2.29.

Exponential would be at 3.6 hours

GPT-5 12 months ago

At this point the prediction for SWE bench (85% by end of this month) is not materializing. We're actually quite far away.

Claude Opus 4.1 12 months ago

No obvious gains I feel from quick chats, but too early to tell.

These benchmark gains aren't that high, so I doubt it is that obvious.

Firstly, if your prior is that every previous startup failed, what does that say about your future chances of success?

The prior is the market. It isn't sane to use your own prior experience. (Works both ways -- if your last startup did great, shouldn't assume next will).

4% of YC companies become unicorns. How many startups do you need to work for before you become part of the 4%? That number is not a feasible number of jobs for one lifetime.

The bar (and what the model is calculating) is Series A from top VC, not YC Seed funding. That significantly increases odds. Specifically, ~45% YC companies get Series A, so it's more like 10% chance of a YC Series A funded company becoming a unicorn (https://www.lennysnewsletter.com/p/pulling-back-the-curtain-...).

Model is change jobs every 18 months if not booming. A 1 in 10 chance is quite reasonable over a career.

I agree there is an issue with the event being too rare, but you can't just look only at modal returns. 2/3 chance of $0 (the modal return) and 1/3 chance of $10 million profit is still pretty good odds to work with.

It's a probabilistic model. It assumes (correctly) that the low probability of a home run times the home run's valuation is quite large ("expected returns" in the probabilistic sense).

this argument reads to me like "the returns on a Powerball win are so much higher than your projected lifetime earnings that playing the lottery is a smart financial move".

That's stronger claim than it is making, but yes in a sense it is saying the lottery can be a good move because the expectation is large - that's what VCs do after all.

Note that all the model aims to do is value the equity package. If a public company is offering more than what this model values the startup equity package as (and this often is the case!), it isn't worth it financially to work at that startup.

The value of the equity package is 4x higher than the FAANG equivalent equity package (at preferred/market pricing) - that's not the same as saying the shares themselves are worth that.

To sum up the arguments:

* Employment packages allow things a shareholder cannot do (functionally recall their investment), so the high volatility leads to higher package returns.

* FAANG equity grants (RSUs) are taxed at much higher rates

* Expected return is in fact higher on startup equity than FAANG equity (and you generally have no way to invest in the good startups directly aside from working for them).

I don't see why the market cap proves whether she is correct or not. You'd have to compare it to the counter-factual of what the value of a Figma subsidiary would be under Adobe today.

This is not obvious at all to me. Instagram (bought for $1B) is probably worth ~700 B of Meta's market cap.