mmastrac

Is it just me or does this post not mention how much RAM they had? I would love to know - I have a dual-Xeon 1U screamer with 96GB of DDR4 RDIMM just sitting around...

FWIW I'm getting a hardware max of 20 tok/s (approx topping out the GPU's compute) on my custom local diffusiongemma port running on an M3.

There's a bot user that seems to crosspost every lobsters story here if it isn't already.

This sort of thing is a lot of fun.

I've been going smaller.. I have a custom-quantized Rust port of DiffusionGemma (26B) that seems to perform better (in responses) than benchmarks seemed to indicate and reasonably fast for its model size. Works really well on a 36GB mac as well for both prefill and generation.

It's been interesting learning about the balance of factors for performant metal kernels on unified memory.

Should have a repo up on github in the next few weeks.

Author here - if anyone has any contacts at Cloudflare to get the proxied domains (at least roadpay[.]cc) taken down, that would be great. I wasn't able to get an abuse report to stick. Ditto for the related LinkedIn profile and Twitter accounts.

The C2 IP (89.124.107.161) and malware-serving git repo (144.124.244.92) are both hosted on VDSINA in Russia, so not sure if there's anything to do there.

They are long-lived if you're a mayfly.

But I think the point is that they should be cheap to set up, and because of the short life, never really contain anything except the potential to compute when needed, not important data.

Om Malik was the guy who had the biggest influence on the direction of my life, by far. It was through him I met Naval Ravikant in 2007, and then through Naval I met my co-founder that led to my startup exit in the '10s.

Luck surface area. I really owe so much to Om. I really can't imagine where I would be without that chance.

One of the most fascinating things to fall out of the AI apocalypse is seeing how abundant AI access amplifies the qualities of a company.

IMO, Deno has always been more methodical, more focused (maybe too focused?) on standards. But now the Deno team is on the right track: using Claude extensively to improve the node.js compat which was absolutely herculean if not impossible before AI. [+]

On the other hand, Bun has always played a bit fast and loose, chasing metrics at the cost of stability. Access to abundant AI has sent that project off the rails.

Disclaimer: Former Deno engineer - I'm obviously going to have some biases. All IMO of course, but if you ask me I'd still bet on Deno in the long term, and I personally still use it for any .ts projects.

[+] There might be a dozen people in the world that know how sensitive and subtle the timing and ordering in the JS event loop is and how meticulous just this single part needs to be for major node.js projects not to completely crap themselves.

The consequences of plugging the wrong voltage or polarity into a barrel jack, most of which are compatible enough, is pretty bad. Ask me how I know.

FWIW this post has both a "thanks" section for the human reviewers, and numerous footnotes linking to more authoritative sources.

I'd love it if WASI modules could introspect their own custom sections (potentially even more introspection than that), but I've never been able to figure out a good way to do this. Seems like a fairly useful feature for a few use cases.

Author here, happy to answer any questions. I've been working on building some higher-level abstractions on link sections (specifically, link-time optimized collections like maps (1) and sorted slices (2)) and wanted to share the hard-fought knowledge from the last couple of months.

There's a decent amount of knowledge around pre-main work in Rust, but I think this is one of the first attempts to walk through mutable link sections, which open up a pretty wide world of optimization, IMO. Even without mutability, I figured there isn't nearly enough documentation on these approaches out there.

(1) https://docs.rs/scattered-collect/0.20.0/scattered_collect/m...

(2) https://docs.rs/scattered-collect/0.20.0/scattered_collect/s...

Looks great, but why isn't there a checkbox to turn off the chunk of pork fat? I use avocado oil, personally.

Types and generics are hard to parse in regular macros without a tt muncher. Ditto for fn args. If you need to do actual matching of types, you can't capture them as $xxx:ty because Rust will not allow a larger matched token to be broken up again (unless you use a paste! hack to roundtrip it back into ungrouped tokens).

I wrote https://crates.io/crates/type-mapper as a way to work around those limitations but it is _very_ painful TBH.

I like the idea.

I think I'd feel more comfortable if I could drop an .rs file into the eBPF dir instead of a .c one. It's already a Rust project! :)

And for some reason I was expecting this to be a kernel-accelerated webserver - if that could be done safely using eBPF that would be amazing!

Also, single-threaded? Forking and sharing an incoming connection queue is basically trivial on Linux, that should be literally just a few lines, even with Rust. Use SO_REUSEPORT and the kernel will do the rest.

FWIW, if you're going to push for io_uring, you should also be pushing kTLS IMO, you'll drastically simplify your design if you can avoid pumping userspace SSL after the handshake.

Great writeup! Apologies if it came across as a criticism of the writeup itself, more of a frustration of years in the proc_macro space.

It's surprising how little information exists out there about proc_macros in general.

As someone who has spent a _lot_ of time writing declarative and procedural macros, the important thing to ask before digging into a macro is whether you need a procedural macro at all.

Complex proc macros absolutely do slow builds down. In many cases, a proc macro only need to be a stub that can delegate to a declarative macro.

You may not need to use syn/quote, but if you are doing any sort of processing/parsing of Rust code you pretty much need to.

FWIW, I really hope that the Rust project focused on finer-grained token matching in declarative macros so we can migrate most proc_macro code away. The macro system is powerful, but nowhere near where it needs to be.

I seriously looked at this as a way to improve the RAM situation in a QNAP 2U unit that I was having trouble sourcing RAM for. It's somewhat annoying that legit memory-over-PCIe is gated on PCIe5 and chipset support.

In the end I just had to bite the bullet and take a gamble on finding ECC DDR4 RAM that would work with the ancient AMD chipset...

This particular implementation seems to be running over too many layers to be particularly performant. Why not a custom block driver instead?

CyberChron. If you don't need to know, don't ask.

And you're also assuming that all the pings are being returned by this box.

I've weirdly found that I like the Google AI mode in specific cases, and I find that the hybrid is the worst of the two worlds. There are some cases where I don't know exactly what I'm looking for and I want the AI to curate results. In other cases, I know what I'm looking for and I want to read the OG source.

The AI popup is the worst and will hallucinate answers from Reddit comments. I specifically had it ask me a nonsense question which was literally just someone's Reddit comment suggesting a follow-on topic B to the search topic A. The AI mode will _sometimes_ be useful enough to prompt into doing the search and summarization for me and get me just enough info and some links to continue the work myself.

The most effective rule for me is no addictive apps on my phone or laptop - browser apps only. The browser apps are _far_ less addictive and just enough friction to keep me off them for extended periods of time. As well, infinite scroll just isn't as effective in a browser and there's a real feeling of limited content running out.

Last update seems to be "lawsuit was filed" with zero updates since then. That stuff tends to move slowly.

Dav2d doesn't have the same nice ring to it. I hope there's someone with a decent repo-name punning skill who'll contribute before that.

avi2ude? av2go?

_NSAKEY part deux

I am certain that the max sustainable boost from AI use -- with code review and otherwise all-in -- is approximately 20% with the appropriately skilled senior engineering talent, and the token budget for any engineer should not exceed that.

I do not believe that engineers who are tokenmaxxing are truely productive and I have not seen any evidence whatsoever (perhaps the opposite).

I've personally found that with the right flow and codebase knowledge, that's achievable with sustainable levels of effort.

That looks vaguely breadboardish too

Did the microcode disassembly find any useful backdoors to read microcode without decapping?

How would that be technically feasible? Would we get IP bans?

Remember kids, unsafe is not undefined behaviour. It just means you need to prove via unchecked English statements that your code is not UB.

I previously worked at Deno and even with all that tbh, I am not sure the http deps were the right way to go. I've really wanted to like them but package managers really have advantages.

I would not say npm was the right direction. I actually was a fan of JSR (didn't work on it but all my experience with it was great)

Apparently more than one are in hiding, TIL. Seems appropriate for the kind of folk that do this.