salamo

Possible reasons:

- They might be dynamically adjusting these at inference time [1]. For example, start with a low temperature and generate samples with increasingly high temperatures until one of them passes some quality gate.

- They don't want you to fine-tune on high temperature completions (rejection fine-tuning). You could call this "rejection fine-tuning rejection".

[1] https://rlhfbook.com/c/09-rejection-sampling#related-best-of...

Yeah, the model is small enough that inference is already basically instant for my usecase (only 6 transformer layers for the blog search).

Yeah, I also found that for ultra low footprint models ORT is a big portion of the total payload, because it contains logic for general ONNX graph operations. In my case I found that ORT alone was 3.4MB over the wire, so I swapped it out for a tiny wasm that was 850x smaller and only contained the operations I needed: https://blog.lukesalamone.com/posts/creating-tiny-semantic-s...

On the one hand, compacting at 300k saves OpenAI 40%. That's great.

On the other hand, $10 for 1M tokens still seems really high? It's not too hard to blow through that in an hour or two.

I come at it from a slightly different angle.

I write technical blog posts with visualizations and live demos. That usually means embedding a bit of custom javascript in the page for the demo. Or shipping custom wasm to enable extreme semantic model compression.

I do this by pushing content from my machine to github pages which is wired up to my subdomain.

If github pages stops being a good, free option for this, I will find another. Not sure I would call this "hardcore" really.

$1.7 billion is small potatoes. My bill is over $155 billion and growing. I'm worried if the trend continues I'll have depleted my rainy day fund.

An alternative algorithm which would probably converge faster than 100 questions would be something like Elo or Glicko 2.

A word's "difficulty" would be some function of how rare it is. Once you have a reasonable estimate of the user's "skill" you can infer that a user won't know more difficult words. The benefit of this is you're not spending time asking the user about words they probably know.

Of course it's possible at an individual level, difficulty does not monotonically increase as a function of how rare the word is. A person might be very familiar with a domain-specific subset of English. But the "stratified sampling" approach will also have this problem.

There is a similar problem in chess, where players have ratings which really only change on one dimension. So there can theoretically be a mismatch when puzzles are also scored on a single axis, since a "harder" puzzle that contains a motif a player is familiar with will actually be easier for the player.

I’m working on an iOS app, One Million Checkmates [1]. It scratches an itch I had of chess puzzles for a long plane ride. This app has a functionally unlimited number of puzzles, all offline.

There was a decent amount of work involved in getting the download size reasonable since we need to store all valid moves in a position. There are puzzles with over 40 million valid move sequences, so I had to aggressively prune and compress the move trees.

[1] https://apps.apple.com/us/app/one-million-checkmates/id67625...

I'd really wouldn't on the machine.

I'll second this. Much better to set up a second machine you can ssh/tailscale into. If a training run takes down your training machine, you don't want it to also take down your home entertainment server.

See, that's why I have to post these things. Someone will inevitably reply with something more clever.

You'll also have some fun pinning down the difference between an "inaccuracy", a "mistake", and a "blunder". These are meaningful delineations for humans but not for a chess algorithm. Objectively, any amount of centipawn loss either changes the best possible outcome for the player or it does not.

So in practice, a drop in win probability greater than 14% is considered a blunder on Lichess.

For reference, lichess uses the following function to map centipawn advantage to the probability bar, derived from observed outcomes: https://github.com/lichess-org/lila/pull/11148

From an ML perspective, this is basically logistic regression with a single feature. However, once we leave the realm of theoretical centipawn value and begin to optimize predictive power, we could imagine adding in other things like the players' ELOs or time remaining per player, etc.

I think there are some interesting theoretical differences between predicted win probability derived from Stockfish CP and actual outcomes. As in, you could even imagine predicting positions where certain players struggle and steering them towards those positions. [0]

[0] https://www.youtube.com/watch?v=KgOC1D8wkyE

Really happy to see additional solutions for on-device ML.

That said, I probably wouldn't use this unless mine was one of the specific use cases supported[0]. I have no idea how hard it would be to add a new model supporting arbitrary inputs and outputs.

For running inference cross-device I have used Onnx, which is low-level enough to support whatever weights I need. For a good number of tasks you can also use transformers.js which wraps onnx and handles things like decoding (unless you really enjoy implementing beam search on your own). I believe an equivalent link to the above would be [1] which is just much more comprehensive.

[0] https://ai.google.dev/edge/mediapipe/solutions/guide

[1] https://github.com/huggingface/transformers.js-examples

I mainly blog for myself in the future, but in a slightly different flavor than the author mentions. If there's a complicated ML concept that I'd really like to understand, explaining it to an audience (even if that audience is myself in the future) is a great way to understand it.

That goes double for visuals, which is another reason I use a custom static site. Can't run JS on Medium. I even built out a client-side search and learned a good amount from that too.

Ok, I emailed you.

I live in the South Bay and would be willing to "host". But there's no way to announce a meetup or even reach out to some people.

I believe that observation is borne out in the statistics too, but traditional chess training usually centers around finding the best, hard-to-find move in a position rather than avoiding blunders. I think it would be great if there was more blunder-avoidance training. In other words, a normal position where a blunder looks attractive but the player needs to avoid it.

Sure, go ahead!

I'm happy you enjoyed it! There are definitely a few rough edges, yes.

Since the whole thing is executed in the browser (including the model) there aren't a ton of secrets for me to keep. Essentially it is expectation maximization: the bot tries to find the move with the highest value. What is "value"? Essentially, it is the dot product between the probability distribution coming out of the model and the centipawn evaluations from Stockfish.

In other words if the model thinks you will blunder with high probability, it will try to steer you towards making that mistake.

The issue is that humans and computers don't evaluate board positions in the same way. A computer will analyze every possible move, and then every possible response to each of those moves, etc. Human grandmasters will typically only analyze a handful of candidate moves, and a few possible replies to those moves. This means human search is much narrower and shallower.

If you want a computer that plays like a human, you will probably need to imitate the way that a human thinks about the game. This means for example thinking about the interactions between pieces and the flow of the game rather than stateless evaluations.

I built something like this. It works as long as you're not too high-rated: chessmate.ai. Once players get higher rated it is more difficult to predict their moves because you need to model their search process, not just their intuitive move choice. It's also possible to train on one player's games only so that it is more personalized.

It uses a similar approach to Maia but with a different neural network, so it had a bit better move matching performance. And on top of that it has an expectation maximization algorithm so that the bot will try to exploit your mistakes.

I personally prefer a UI most of the time. It's higher fidelity and cuts through the inherent ambiguity of language.

The exception for me would be situations where I can't use my hands, like driving. I don't want to have to look at a screen. If a voice agent could replicate the functionality of CarPlay, that would be really useful.

I found Cosine Club recently and it's pretty cool. They have a model which just finds the top ~50 most similar tracks to the one you're searching for. They use a model trained from this paper [0] which was trained based on four types of similarity: 1) two samples from the same song, 2) two samples from the same album, 3) two samples from the same artist, 4) two samples from the same record label.

I get pretty good recommendations, and they're different from Spotify which is nice.

[0] https://repositori.upf.edu/bitstream/handle/10230/54158/Alon...

CATL has a 650 mile battery, which would definitely put Tesla on the ropes if e.g. Toyota could use it. Range and cost are probably the main barriers to electric cars.

The stove linked in the article is also bonkers. It would have been more impressive with a thermometer readout before they turned it on though. I checked other videos [0] and it seems to be real.

[0] https://www.youtube.com/watch?v=YdawGen0QPc

Is there any info on how to diagnose this problem? Having just put together a computer with the 14900KF, I really don't want to swap it out if not necessary.

By far the easiest if you just want to go from markdown to online blog would be a static site on Github pages. I use Hugo [0] with a custom theme but there are even easier options if you don't even care about style. In particular, I have used Grip [1], which can generate HTML from markdown.

[0] https://lukesalamone.github.io

[1] https://github.com/joeyespo/grip

Towards Data Science. They reached out to me with some kind of profit-sharing offer after I wrote something on Medium a while back. But I have way too much interactive content on my personal site that wouldn't translate to Medium, and I don't do it for money anyways.

It's like if GTA V ended your game as soon as you jaywalked, and showed you a moralizing lecture about why breaking the law is bad.

There was a game called Driving in Tehran which was exactly that. If you speed or crash, you get fined. If you hit someone, it tells you "don't play games with people's lives" and then exits entirely.

https://www.youtube.com/watch?v=hFDYvlm7geU

I spent a ton of time on Glest growing up. The fact that it could be modded with simple XML files made it really accessible and was a gateway for me to get into programming. So even if seems a bit dated today Glest will always be special to me.

Switching my smart lights to red works pretty well for me, although from the street it definitely looks like something unholy is happening in the apartment.

Reminds me of Dan Luu's article I read this morning: https://danluu.com/sounds-easy/

I think part of why we underestimate difficulty is, it's hard to see anything but the happy path. The complications are harder to anticipate. And for each of the solutions to a problem, we also only think of the happy path to solving it. And so on.