HN user

celrod

1,132 karma

SIMD and performance enthusiast. https://github.com/JuliaSIMD https://spmd.org/

Posts1
Comments377
View on HN

When claude goes down a wrong path, I tend to clear context and write a new prompt that helps guide it down the correct path. Whatever thinking or context that led it there has inertia and tends to be sticky, otherwise.

Pretty annoying when it brings those up again later from memory...

Exactly. How can "we" develop and encourage benchmarks for multi-turn user assistance? That is what I want. I feel like the models and harnesses push much too hard against this workflow -- that they push you towards letting go and vibe coding, with only your discipline (and desire for a quality and maintainable product) holding it back.

What quant do you run it at? 32GB seems like cutting it close on the rtx 5090 if going 8b, but other commenters are saying 4b lobotomizes the model.

Every Byte Matters 2 months ago

Yes. I think one of the big advantages of SoA is that you only pay for the fields you're currently using. If you need a field somewhere, you can add it and only pay the cost of iterating it where you need it.

foot also offers a client/server architecture. If you start a foot server (e.g. with a systemd service), you can use `footclient -N`. This may reduce the memory pressure of running many terminals.

This is similar to the `kitty --singleinstance` mentioned in another comment by amarshall.

Yeah, for now. I'd like it to be open, but I also want to potentially be able to make money/a living off of it. My dream would be that it can be open while hardware vendors pay me to optimize for their hardware. For how, being closed gives me more options. It's a lot easier to open in the future than to close, so it's just keeping options open.

I've thought a lot more about the engineering than any sort of marketing or businesses plan, so I just want to defer those.

I'm still working on it. I'm currently working on a cache tile-size optimization algorithm that should (a) handle trees (a set of loops can be merged at some cache levels and split at others, e.g. in an MLP it may carry an output through the L3 cache, while doing sub-operations in the L2/L1/registers) (b) converge reasonably quickly so compile times are acceptable.

This is the last step before I move to code generation and then generating a ton of test cases/debugging.

My goal is some form of release by the end of the year.

I was once a bit of a Julia performance expert, but moved toward c++ for hobby projects even while still using Julia professionally.

I wrote a blog post at the time with exactly that punchline (not explicitly stated, but just look at the code!): https://spmd.org/posts/multithreadedallocations/ The example was similar to a real production-critical hot path from work.

Maybe things changed since I left Julia, but that was December 2023, for years after this blog post.

In my experience, llms don't reason well about expected states, contracts, invariants, etc. Partly because that don't have long term memory and are often forced to reason about code in isolation. Maybe this means all invariants should go into AGENTS.md/CLAUDE.md files, or into doc strings so a new human reader will quickly understand assumptions.

Regardless, I think a habit of putting contracts to make pre- and post-conditions clear could help an AI reason about code.

Maybe instead of suggesting a patch to cover up a symptom, an AI may reason that a post-condition somewhere was violated, and will dig towards the root cause.

This applies just as well to asserts, too. Contracts/asserts actually need to be added to tell a reader something.

I netted huge performance wins out of AVX512 on my Skylake-X chips all the time. I'm excited about less downclocking and smarter throttling algorithms, but AVX512 was great even without them -- mostly just hampered by poor hardware availability, poor adoption in software, and some FUD.

I use niri and footclient -N, so builtin window and tab completion don't appeal to be.

Foot feels fast, but I've not actually measured the latency. It also seems to use less CPU than GPU accelerated terminals (which it isn't) from just glancing at btop. So I'm not sold on GPU-acceleration as a feature unless I see benchmarks demonstrating the value in improved latency and reduced CPU use compared to foot

I love that foot's scrollback search, selection expansive, and copy can be entirely keyboard driven. Huge QoL feature for me that often seems neglected to me in other terminals.

I use kakoune, and don't understand why helix seems to be taking off while kakoune (which predated and inspired helix) remains niche.

Kakoune fully embraces the unix philosophy, even going so far as relying on OS (or terminal-multiplexer, e.g. kitty or tmux) for window management (via client/sever, so each kakoune instance can still share state like open buffers).

A comparison going into the differences (and embracing of the unix philosophy by kakoune) by someone who uses both kakoune and helix: https://phaazon.net/blog/more-hindsight-vim-helix-kakoune

Sensible defaults and easy setup are a big deal. No one wants to fiddle with setting up their lsp and tree-sitter. There's probably more to their differences in popularity than just this, though.

Is including batteries the main reason helix seems to have started taking off, while kakoune hasn't?

I use kakoune, because I like the client/server architecture for managing multiple windows, which helix can't do. The less configuring I do the better, but I've hardly done any in the past year. It's nice to have the option.

I do use kakoune-lsp and kak-tree-sitter.

I agree.

The problem with eager diagnostics and templates is that the program could define a `Base<int>` specialization that has a working copy constructor later. [0]

I think if you define an explicit instantiation definition, it should type check at that point and error. [1] I find myself sometimes defining explicit instantiations to make clangd useful (can also help avoid repeated instantiations if you use explicit declarations in other TUs).

[0] https://en.cppreference.com/w/cpp/language/template_speciali...

[1] https://en.cppreference.com/w/cpp/language/class_template.ht...

I don't want my web browser or video player to be resized because I open a new program

I've been using niri (a tiling WM) recently. This is their very first design principle: https://github.com/YaLTeR/niri/wiki/Design-Principles Maybe other PaperWM-inspired WMs are similar. niri is the first I've used.

If your windows within a workspace are wider than your screen, you can scroll through them. You also have different workspaces like normal. I'll normally have 1 workspace with a bunch of terminals, and another for browsers and other apps (often another terminal I want to use at the same time as browsing, e.g. if I'm looking things up online).

Do you not often quickly look between files? If so, odds are you're using tiles within tmux, vim, emacs, vscode, or something.

I use kakoune, which has a client/server architecture. Each kak instance I open within a project connects to the same server, so it is natural for me to use my WM (niri) to tile my terminals, instead of having something like tmux or the editor do the tiling for me. I don't want to bother with more than one layer of WM, where separate layers don't mix.

I've defined a few pretty printers, but `operator[]` doesn't work for my user-defined types. Knowing it works for vectors, I'll try and experiment to see if there's something that'll make it work.

  (gdb) p unrolls_[0]
  Could not find operator[].
  (gdb) p unrolls_[(long)0]
  Could not find operator[].
  (gdb) p unrolls_.data_.mem[0]
  $2 = {
`unrolls_[i]` works within C++. This `operator[]` method isn't even templated (although the container type is); the index is hard-coded to be of type `ptrdiff_t`, which is `long` on my platform.

I'm on Linux, gdb 15.1.

How feasible would it be for something like gdb to be able to use a C++ interpreter (whether icpp, or even a souped up `constexpr` interpreter from the compiler) to help with "optimized out" functions?

gdb also doesn't handle overloaded functions well, e.g. `x[i]`.

Skymont little cores have 4x 128-bit execution. They could quadruple-pump.

But looks more like they're giving up on people writing code for wide vectors, instead settling on trying to make the existing code faster.

Zen5 appears to officially support up to DDR5 5600, but unfortunately all of the ASRock Rack or Supermicro boards I looked at only supported DDR5 5200.

I may wait for new Zen5 boards, or maybe take a gamble on something like the Asus ProArt, where I saw comments online indicating that ECC is (unofficially?) supported.

Looking forward to Ryzen 9000 ECC benchmarks.