cshimmin

Interesting, I wonder if the rugby thing is a common bias. I did find myself in the weights, as the top result. But apparently there are also Australian rugby versions of me!

That is simply not true. The naive “glorified auto-complete / stochastic parrot” argument may have some merit when applied to generic pre-trained models, which only learn from unsupervised next-token prediction. But the post training through reinforcement learning the frontier models undergo is very sophisticated and they genuinely learn to do novel things that are purely the work of the model being trained (and the work of the GPUs they burn along the way of course).

https://xcancel.com/pj_lambert/status/2057477629528150369

It's on the page, if you click the little info icon in the upper-right. Here's the text but there's some nice graphics there too:

  Snake Game, training entirely in the browser. Built on tinygrad: the rollout / targets / train graphs are TinyJits authored in Python, then compiled once to WGSL and replayed here under WebGPU.

  Observation: flat 10×10 board (100) + 4-dim prev-action one-hot = 104 dims. fc_pi.weight is zero-init so the opening policy is uniform over the legal actions; fc_v uses tinygrad's default Kaiming init.

  Per rollout: T=24 × N=384 parallel snakes (9,216 transitions), then K=3 epochs × 4 mini-batches of PPO updates. GAE γ=0.99, λ=0.95; AdamW wd=0.01; ratio clip ε=0.1; grad-norm 0.5; Huber value β=1, val_coef=1; entropy bonus 0.008333333333333333.

  Action mask + value clip + KL early stop. The 4-dim prev_a obs tail lets fc_pi zero the U-turn logit (the env silently overrides same-axis reversals anyway). Value loss is max(huber(v_new−td), huber(v_clip−td)) at ε=0.2. Approx-KL is sampled after each epoch and breaks the loop at 1.5·kl_target.

Very helpful! Naïve question (I haven’t had a chance to read TFA at all and diffusion/flow models are not my area of expertise). Doesn’t learning the integral/solution of the diffusion process in a single pass just take us back to like OG generative CNN that we had before diffusion models took over? Surely the answer is “no” but would love to hear your framing as to why.

Not gonna lie, the first thing I noticed was that the first author was in an anesthesiology department. Your guidelines for sniff-testing are not unreasonable, and can definitely be helpful to people who are unfamiliar with the research area. But I quite intentionally did not appeal to any of those. As a (somewhat) subject matter expert, it's important to _ignore_ things like ad hominem judgement, and instead address the paper on its self-contained merits. And more importantly, to share my assessment of those with the lay public.

When I was a research physicist I spent a lot of time looking at the effects of ionizing radiation in pictures, although mostly in the context of digital images. The mechanisms are a bit different for photo emulsions, but to me the reason I'd discount radiation is because they're specifically filtering for features that exhibit the expected point spread function (which is a geometric property of the telescope's optical assembly itself). I guess you could test by exposing emulsion plates to ionizing radiation and seeing how often you get PSF-like images by chance. Also, their search is for +/- 1 day of nuclear testing, which seems weird. Certainly radiation from fallout wouldn't make sense on the day before testing. It would have been useful to see +1 day and -1 day separately. Or 0-2 days. The way it's chosen makes me suspect they couldn't find a signal in those windows, and therefore it's probably just statistical noise that they've massaged out of the data.

But to me the biggest flag is that these images are from 50 minute exposures. The objects don't appear as streaks, so they are either very, very short flashes (much shorter than 50 min), or they are very far away. The authors interpret this to mean the objects should be in geosynchronous orbit, which doesn't make sense; objects in geosync would still appear to move relative to the star background over the course of 50 min. Yet this is the entire basis for their "shadow deficit" window calculation. You could constrain the duration vs distance by looking at the effect it would have on smearing the PSF, which would be interesting.

Overall it seems pretty unscientific. If you go looking through enough statistically noisy data for signals in enough places, you'll eventually find it.

It kinda sounds like a post-doc, in that it provides an on-ramp to working in the industry/institution. But without having to waste your time getting a PhD.

The 6 is part of 3.6, the model version. 35B parameters, A3B means it's a mixture of experts model with only 3B parameters active in any forward pass.

Incidentally, I recently learned the origin of the term. Cyber - short for cybernetic - is from the greek κυβερνήτης (kybernetes), meaning helmsman. The original use of cybernetics is in the context of automated control systems, so steering a rudder was a good analogy. It is also the origin for the name k8s.

Yeah perhaps a better term for Loser is Abstainer. Because the Sociopaths also can certainly lose at the game of maximum capitalist profit. Loser/Abstainer just chooses not to play the game.

It's basically just a way for the LLM to lazy-load curated information, tools, and scripts into context. The benefit of making it a "standard" is that future generations of LLMs will be trained on this pattern specifically, and will get quite good at it.

Was this after you and two zany friends made a scheme to divert a fraction of a penny from each of your employer's transactions into a bank account that you control? And then you gave it all back but the building burned down and Milton made off with the cash?

So… your argument is that it’s not counterculture unless it’s mainstream culture? And that one should only credit derivative works once they become mainstream, rather than the original inspiring works because they were too obscure?

I don’t think anyone is trying to “gotcha” you. You’ve just got a bad take.

If I understood correctly, the global lock is so that notify events are emitted in order. Would it make sense to have a variant that doesn't make this ordering guarantee if you don't care about it, so that you can "notify" within transactions without locking the whole thing?

P-1 AI | Hybrid (SF) and Remote (US) roles | Full time

P-1 is trying to do for physical engineering (mechanical, aerospace, electrical), what Devin, Cursor et al. are doing for the software world. Founded by ex Airbus execs and DARPA program managers, we've just raised a $23 million seed round. Our goal is to solve engineering AGI to enable humans to scale engineering complexity by orders of magnitude.

We have several roles open including:

* Research Scientist - Applied AI

Looking for creative people to innovate on agentic AI systems for specialized engineering domains.

* ML Engineer - Training and AI Infra (ad should post tomorrow)

We're doing a lot of LLM fine-tuning and reinforcement learning. We need someone who can make our GPU cluster sing!

* Software Engineer - AI Evals and Test

We believe it's essential develop rigorous, domain-specific evaluations in order to ensure we're making progress on real world issues -- not just hill-climbing on general purpose benchmarks.

... and more!

Apply online at: https://jobs.ashbyhq.com/P-1%20AI

some of us have to wear glasses anyway :/

Thank you for finally making this make sense to me.

almost as though the AIs were trained on a corpus of text written by... humans

It’s not unreasonable to suspect they are doing the same. The article starts with a description of a lawsuit NY Times brought against OpenAI for similar reasons. The big difference is that research presented here is only possible with open weight models. OAI and Anthropic don’t make the base models available, so it’s easier to hide the fact that you’ve used copyrighted material by instruction post-training. And I’m not sure you can get the logprobs for specific tokens from their APIs either (which is what the researchers did to make the figures and come up with a concrete number like 42%)

My favorite example, which was an honest translation error from a non-native speaker friend: Hand job (he meant to say manual labor)

There's a board-game version of Tetris that I've been playing with my toddler, and is pretty much the same low-stress approach. It's also multiplayer (the next piece everyone must use is decided by drawing cards from a deck). The only shortcoming is that you can't do the maneuver where you slide a piece sideways just as it hits the bottom to slot it into place under another piece. Highly recommend!

“I've moved around a lot - and I realized I'm losing track of friends across a lot of apps, locations, and time-zones.”

From the first sentence, sounds like this isn’t an option. I can relate.

You don’t have to convince me that this is a bad thing for America. Yes it’s exactly how turkey and russia and other failed/failing democracies got to where they are.

One thing you need to keep in mind is that in the US things are stacked against people who would want to protest or engage in any kind of activism.

Take a day off work to go to a rally or peaceful protest? “At will” employment means you can be fired the next day, no reason given. You got fired? Virtually all workers in the US get their health insurance through their employer, so now you and your family just lost access to medical care. It’s a really rough job market in many sectors, so it could take a few months to get a job. But since you got fired without cause, you can at least try to claim some unemployment benefits. In California, that maxes out at something like $450 a week.

Meanwhile in France if they want to fire you they have to give like 3 months notice (or pay you out for that time). Healthcare is socialized so no worries there. And if you still can’t find a job in a few months IIRC there’s fairly reasonable social benefits available.

Interesting that they changed the author order to put Kip Thorne first... marketing?

Yep, I mostly interact with these AIs through Cursor. When I want to ask it a question, there's a little dropdown box and I can select openai/anthropic/deepseek whatever model. It's as easy as that to switch.

Oof, wow. I'm very much in the tech space and I literally had never heard his name before your post.

They did start planning around then. I don’t remember the exact date of when the FCC working group officially convened but they’ve been working on it at least since I was a grad student in the early 2010s.

If you're making technical parts, I highly recommend just biting the bullet and learning parametric cad with fusion 360. There's a ton of learning resources on youtube etc. OnShape seems like a quite promising alternative but I assume there's less material since it's newer.