namr2000

You don't have to train from scratch but you can. Distillation ends up being somewhere in the ballpark of 1000x faster to train [1]. It also comes with the huge advantage of not needing to create RLHF datasets, since you can just copy the behavior of the teacher model. This saves an enormous amount of labeling money at the cost of making the model behave similarly to the teacher. If you are training from scratch, you can look at LLM scaling laws to figure out roughly the compute budget you need to optimally train a model [2].

Based on [2] a 30B model needs something like 2e+23 FLOPS to train from scratch whereas a 1.6T model needs something like 1e+27 FLOPs to train. So DeepSeek v4 Pro was roughly 5000x more expensive to train than this model. I'm not totally sure how MOE affects scaling laws, so these numbers might be different in reality, but it gives you a good ballpark estimate of the difference in training scale.

[1] https://arxiv.org/abs/2505.12781 [2] https://arxiv.org/abs/2203.15556

It depends on the facilities the language offers to you by default right?

C++ offers much higher level primitives out of the box compared to Zig, so I'd say its a higher level language. Of course you can ignore all the features of C++ and just write C, but that's not why people are picking the language.

I agree with you that package management has nothing to do with how low-level a language is.

That being said Rust is definitely a much higher level language than either C or Zig. The availability of `Arc` and `Box`, the existence and reliance on `drop`, and all of `async` are things that just wouldn't exist in Zig and allow Rust programmers to think at higher levels of abstraction when it comes to memory management.

Having a rich standard library isn't just a pure positive. More code means more maintenance.

I would argue it's much worse to rely on packages that are not in the standard library since its harder to gain trust on maintenance and quality of the code you rely on. I do agree that more code is almost always just more of a burden though.

The languages trade complexity in different areas. Rust tries to prevent a class of problems that appear in almost all languages (i.e two threads mutating the same piece of data at the same time) via a strict type system and borrow checker. Zig won't do any of that but will force you to think about the allocator that you're using, when you need to free memory, the exact composition of your data structures, etc. Depending on the kind of programmer you are you may find one of these more difficult to work with than the other.

Rust is a world away from Zig as far as being low-level. Rust does not have manual memory management and revolves around RAII which hides a great deal of complexity from you. Moreover it is not unusual for a Rust project to have 300+ dependencies that deal with data structures, synchronization, threading etc. Zig has a rich std lib, but is otherwise very bare and expects you to implement the things you actually want.

This has not been my experience with Waymo. I drove a total of about ~3.5 hours in Waymos in LA when I was visiting and their robustness to very unusual situations absolutely floored me.

I am sure you can find truly out-of-distribution cases where the car will make a mistake, but the data shows that this is more rare than a human driver making a mistake.

GPT2 was definitely a risk, just not of the same magnitude. It would have (and did!) make social media bot farms way more convincing and widespread. There was specific worry about that being used to sway elections, which is why they held back the model.

Real median wages have been increasing pretty consistently since the FRED started recording it: https://fred.stlouisfed.org/series/MEPAINUSA672N

I don't really understand what you mean by this. The claim is that the same prompt with the same question produces worse results when it's queried in a model that has more than 200k tokens in its context. That doesn't have to do much with the "skillfulness" of using a model.

Does anyone know the font & colorscheme being used in the code snippets?

Wow, I was going to make something almost exactly like this! Really cool work and thank you for sharing

I think that's a fair assessment. To your point `cbindgen` makes the mechanics of the whole thing painless & linking was trivial. That's worth a lot especially when compared to other languages.

I have to disagree here a little bit. Calling C functions from Rust is a very pleasant experience, but the other way around is not so nice. You usually have to manually create types that will unpack rust collections into C compatible structures (think decomposing `Vec` into ptr, len, capacity) & then ensure that memory passed between the two sides is free'd with the appropriate allocator. Even with `cbindgen` taking care of the mindless conversions for you, you still have to put a lot of thought into the API between the two languages.

I am currently working on a fairly involved C & Rust embedded systems project and getting the inter-language interface stable and memory-leak free took a good amount of effort. It probably didn't help that I don't have access to valgrind or gdb on this platform.

Likewise Ken Liu (the English translator for the Three Body Problem) has a really good short story "The Perfect Match" about the same concept, which you can read here: https://www.lightspeedmagazine.com/fiction/the-perfect-match... It was the first thing that came to mind when I read this announcement.

There are still a HUGE number of games you can play with Intel integrated graphics all of which are extremely good. Many of the greatest games of all time are quite old or 2D which means you don't need anything special to run them. At most I'd invest in a controller (both Xbox & PS5 controllers are compatible) but you don't even need that to play almost all games on Steam. Some cheap but great games: Hollow Knight, Celeste, Hades, Outer Wilds, Undertale, Rimworld.

This is a really impressive project, but I wish the Github page had more information about the design of the system, e.g

1. Is it intending to be a unix-like system?

2. is libc supported? I see that you have XECLib which looks like a custom libc impl?

3. What are the principles behind IPC? I see that there's "PostBox IPC" and that's how windows communicate with the window manager, but from a quick glance I'm not sure how the window manager communicates with the video driver.

4. What's the object format? I see there's docs for XELoader but it doesn't get into how it works or how the linker produces the object files that it loads.

This clearly took a ton of effort and it's a cool project!