brrrrrm

you are basically looking at a whole system prompt just describing the new language

whats wrong with this? You may be over-indexing on the need for large quantities of examples. These days self-play through RL is far more effective and data (not compute) efficient.

it's very much an in-domain term for folks in machine learning. heavily used when pipeline parallelism caught on in training https://alband.github.io/doc_view/pipeline.html

it has paddle shifters - what are those for?

what's MRT?

same can be said for a lot of things tho. e.g. nature used to be fun but then we discovered it all :’( I miss when ships literally sailed into the unknown and found surprising and novel things like hot peppers and pineapples

I agree fully. Hyundai has a mockup that starts to get there (different era, but same concept) called the N vision 74[1], but I doubt we'll see it in market anytime soon. The unfortunate reality IIUC is that modern cars (electric vehicles) have certain aero restrictions (for mileage) that heavily limit design options.

[1] https://www.hyundai-n.com/en/models/rolling-lab/n-vision-74

meta.ai in instant mode gets it first try too (I think?)

``` 2x + y = \operatorname{eml}\Big(1,\; \operatorname{eml}\big(\operatorname{eml}(1,\; \operatorname{eml}(\operatorname{eml}(1,\; \operatorname{eml}(\operatorname{eml}(L_2 + L_x, 1), 1) \cdot \operatorname{eml}(y,1)),1)\big),1\big)\Big) ```

for me Gemini hallucinated EML to mean something else despite the paper link being provided: "elementary mathematical layers"

you're right, this is actually correctly placed! I was confusing the orientation. I live right around there and recognize the M&T bank in the photo on the left, so it can't be down by 9th

I checked 3 spots I'm familiar with and 1 is wrong

https://www.oldnyc.org/#707133f-a this is supposed to be here https://www.oldnyc.org/#702487f-a

also, if folks are interested in these old depictions of NYC, check out https://1940s.nyc/ as well!

looks cool! one bit of feedback: make your demo gif get to the point faster. either practice typing a bit quicker or speed it up 2x for the typing section

on Bun's website, the runtime section features HTTP, networking, storage -- all are very web-focused. any plans to start expanding into native ML support? (e.g. GPUs, RDMA-type networking, cluster management, NFS)

we've discovered some kind of differentiable computer[1] and as with all computers, people have their own interests and hobbies they use them for. but unlike computers, everyone pitches their interest or hobby as being the only one that matters.

[1] https://x.com/karpathy/status/1582807367988654081

a recent wave of interest in bitwise equivalent execution had a lot of kernels this level get pumped out.

new attention mechanisms also often need new kernels to run at any reasonable rate

theres definitely a breed of frontend-only ML dev that dominates the space, but a lot novel exploration needs new kernels

one thing I've learned in my career is that escape hatches are one of the most important things in tools made for building other stuff.

dropping down into the familiar or the simple or the dumb is so innately necessary in the building process. many things meant to be "pure" tend to also be restrictive in that regard.

None of which are they currently capable

what makes you say this? modern LLMs (the top players in this leaderboard) are typically equipped with the ability to execute arbitrary Python and regularly do math + random generations.

I agree it's not an efficient mechanism by any means, but I think a fine-tuned LLM could play near GTO for almost all hands in a small ring setting

what about "test time scaling"?

downshifting? these are all electric vehicles IIUC

sure, numerous examples can be shown to say smart play does help. but, would you argue the net benefits of smart play are identical between a sport like basketball and racing?

I wouldn't deny that (and probably should have caveated this in my OP), but compared to a basketball or football team, the benefit of smart play doesn't seem as significant compared to doping up and pressing hard.

endurance sport which is a natural target for doping

This makes a lot of sense to me. A very singular goal of "maximum output" without much need for fine motor skills and strategizing. I'd guess sprinting/marathons might have similar issues?

emergent behavior. These things are surprisingly good at generalizing

what's going to be interesting is to see the large space of fused kernels being tackled by AI generated code. that might include gemm + relu + gemm + a norm of some kind - which would be annoyingly exhaustive to 1. sweep with a tuner and 2. handwrite as a human

Open-source use of closed source models?

wait sorry, can you explain how this works? I thought gemma3 used siglip, which can output all 256 embeddings in parallel

(also, would you mind sharing a code pointer if you have any handy? I found this https://github.com/ggml-org/llama.cpp/blob/master/tools/mtmd... but not sure if that's the codepath taken)

hmm, I'm getting the same results - but I see on M1 with a 7b model we should expect ~10x faster prompt processing

https://github.com/ggml-org/llama.cpp/discussions/4167

I wonder if it's the encoder that isn't optimized?

When using least squares, a zero derivative always marks a minimum. But that's not true in general ... To tell the difference between a minimum and a maximum, you'd need to look at the second derivative.

It's interesting to continue the analysis into higher dimensions, which have interesting stationary points that require looking at the matrix properties of a specific type of second order derivative (the Hessian) https://en.wikipedia.org/wiki/Saddle_point

In general it's super powerful to convert data problems like linear regression into geometric considerations.

how is this data collected?

a premise*

Do super computers run in fp64 mostly? At fp8 an h100 hits 2 petaflops, and with only 1000 of them you’ve got more compute power than el capitan (in raw flop count)

Paper https://arxiv.org/abs/2501.00663