soVeryTired

No way is vocab size zipfian. Word counts from a corpus follow zipf's law, but not vocab sizes themselves.

Otherwise the most common vocab size would be equal to one.

Can anyone explain to me why Q and K are both needed? They only ever appear as a pair, so why can’t you just define a matrix A = QK and learn that directly?

On March 12, 2025, a search warrant was executed at Sohaib’s home in Alexandria. Agents grabbed plenty of tech gear but also turned up seven firearms and 370 rounds of .30 caliber ammunition. Given his former crimes, Sohaib should have had none of this.

For god's sake, don't commit crimes while you're committing crimes.

I have a feeling taxes are possible to circumvent only because a government tends to have one arm that wants to collect taxes, and another that wants to reduce them to encourage certain outcomes (like having a business setting up shop within its borders).

The US may have this dual incentive structure since it wants to build its tech giants while limiting their control, but the EU doesn't. The arrival of a foreign tech social media giant might make the legislation a bit more palatable to pass.

It will undoubtedly be complex to regulate all dark patterns away. But there are a few obvious, easy wins. It'd be a shame to make perfect the enemy of good.

Us consumers don't want moats. But to understand the investment you need to understand the moat: no sensible investor will throw this much money around without one. If you don't understand what is keeping new players from entering the market, then either you've missed something or we're in bubble territory.

That's why it's interesting to try to pick apart where the moat is.

Whenever people bring this up I like to remind them that linear interpolation is a universal function approximator.

The company is named after the evil telepathic orbs from lord of the rings. Wasn't that the first clue that everything might not be hunky dory?

I'm sure some python devs care about cache misses too. I guess my point was that the big results will be picked over again and again to understand _exactly_ which conditions are needed for them to hold.

In my experience those debates are usually between experts who deeply understand the difference between ABC and XYZ widgets (the example I'm thinking of in my head is whether manifolds should be paracompact). The decision between the two is usually an aesthetic one. For example, certain theorems might be streamlined if you use the ABC definition instead of the XYZ one, at the cost of generality.

But the key is that proponents of both definitions can convert freely between the two in their understandings.

IMO it's not far off how most python or javascript devs don't care about registers or cache misses. Someone's thought deeply about those things so you don't have to.

Mathematicians do care about how much "black magic" they're invoking, and like to use simple constructions where possible (the field of reverse mathematics makes the central object of study). For example, Wiles' initial proof of Fermat's last theorem used quite exotic machinery called "inaccessible cardinals", which lie outside of ZFC. Subsequent work showed they weren't needed.

Another good example of mathematicians caring which 'house of cards' their results are built on is the search for an "elementary" proof of the prime number theorem (i.e. showing it doesn't rely on complex analysis).

Edit: here's a great related discussion on MathOverflow, bringing in analogies from CS: https://mathoverflow.net/questions/90820/set-theories-withou...

Anyone have a good explanation for why elliptic curves have a 'natural' group law? I've seen the definition of the group law in R before, where you draw a line through two points, find the third point, and mirror-image. I feel like there's something deeper going on though.

As far as I've seen, the group law is what makes elliptic curves special. Are they the _only_ flavour of curve that has a nice geometric group law? (let's say aside from really simple cases like lines through the origin, where you can just port over the additive group from R)

The average density of matter in the universe is one proton per five cubic meters or so. We're very much the outlier!

That's more like the false positive rate and false negative rate.

If we're being literal, accuracy is (number correct guesses) / (total number of guesses). Maybe the folks at turnitin don't actually mean 'accuracy', but if they're selling an AI/ML product they should at least know their metrics.

The word “some” in the quote from Box is doing a lot of heavy lifting.

If a model is useful, I’d like to see it being used (outside academia, where there’s minimal penalty for complexity and a high emphasis on novelty).

If models like these are widely adopted at social media companies or news agencies, it’s fair to say OP’s take isn’t valid. Otherwise they may have a point.

There's no place for you here you lunatic

I didn't say it wasn't alarming, I said it wasn't civilization- or nation-ending. Unless you're a tiny island nation, in which case I'll happily retract what I'm saying.

There are degrees of awfulness between "the end of all mankind" and "nothing to see here", but it seems like there's a taboo on calling those shades of grey out when it comes to climate change.

Ports get retrofitted, redesigned, and rebuilt. The AMOC collapsing is a serious thing, but I'm not saying climate change isn't real or isn't a threat. My original point is that three feet of sea level rise is manageable, if expensive. Simply that, nothing else.

If you draw the line at the year 2100, things are uncomfortable but maneagable. If your horizon is 2300 or 2500, you get a different story. But you would hope that in tha sort of time frame, we have time to adapt.

A metre of sea-level rise is painful for a rural cottage by the sea. But if you're in a city - particularly a wealthy city - it's something that can be engineered around.

An expensive liability? Definitely. A civilization or nation ending event? Unlikely.

He's comparing AGW, which drives a trend, with weather-based events, which are noise around the trend. He conveniently cuts his analysis off at the year 2100, by which we'll all probably be dead. But he's probably right that the trend itself doesn't cause insurmountable problems by that point.

But what about the year 2200, or 2300? At three degrees warming per century, the earth looks like a pretty hostile place to live in a few centuries.

"A society grows great when old men plant trees in whose shade they will never sit", and all that...

It confused me a lot - I cancelled the denominators in my head too.

But then I realised they're just plotting

y/(x^2+y^2) - (x+1)/(x^2+y^2) = c

and colouring by c (i.e. a heatmap, as others have mentioned in the thread).

That's why you get a more interesting image than you'd get with y - (x + 1) = c

If it's a win / lose outcome and both win and lose have a probability of 1/100, I'll make 99x my stake by betting on both win and lose at the same time.

If both outcomes have a probability of 0.999 (summing to almost 2), I'll barely make any money if I'm right, and lose my money if I'm wrong.

So when probabilities sum to less than 1, it's good for the gambler, and when they sum to more than 1 it's bad for them (and good for the bookkeeper).

In one sense, winning bets. If you lose, you lose: your money is gone either way. If you win, the fact that the probabilities sum to about 1.05 means you win less than you would have in a fair game. The state just takes a cut of that extra 0.05.

Have you used PaddleOCR? I'm surprised they're claiming SOTA without comparing against Amazon Textract or Azure doc intelligence (LayoutLM v3 under the hood, as far as I know).

I've played around with doc recognition quite a bit, and as far as I can tell those two are best-in-class.

Siméon Denis Poisson would like a word with you.

But in seriousness, I agree.

There's a very interesting video about the "1.5 language problem" in Julia [0]. The point being that when you write high-performance Julia it ends up looking nothing like "standard" Julia.

It seems like it's just extremely difficult to give fine-grained control over the metal while having an easy, ergonomic language that lets you just get on with your tasks.

[0] https://www.youtube.com/watch?v=RUJFd-rEa0k

I have to say, I don't really understand this. Can anyone shed some light?

I won't get into the history of why people take so much acetaminophen despite how crappy a drug it is.

I'd love if you did. What's the story there?

Computers can already do a lot of things that no human can though. They can reliably find the best chess or go move better than a human.

It's conceivable (though not likely) that given training enough training in symbolic mathematics and some experimental data, an LLM-style AI could figure out a neat reconciliation of the two theories. I wouldn't say that makes it AGI though. You could achieve that unification with an AI that was limted to mathematics rather than being something that can function in many domains like a human can.

Roaming a labyrinth and savaging young Athenians might seem like a positive change in the short term, but ultimately it’s probably just as unfulfilling as corporate IT.

IMO that's the fundamental difference between statistics and ML. The culture of stats is about fitting a model and interpreting the fit, while the culture of ML is to treat the model as a black box.

That's one of the reasons that multicollinearity is seen as a big deal by statisticians, but ML practitioners couldn't give a hoot.