rapidlua

For go specifically, I find ko-build handy. It builds on the host (leveraging go crosscompilation and taking advantage of caches) and outputs a Docker image.

How to manage pointer+offset address integrity/legality inside the kernel (for instance) has a proof by examples a-plenty in the other code

Let me provide some context here. These annotations aren’t there to help the compiler/linter. They exist to aid external tooling. Kernel can load BPF programs (JIT-compiled bytecode). BPF can invoke kernel functions and also some kernel entities can be implemented or augmented with BPF.

It is paramount to ensure that types are compatible at the boundaries and that constraints such as RCU locking are respected.

Kernel build records type info in a BTF blob. Some aspects aren’t captured in the type system, such as rcu facet, this is what the annotations are used for. The verifier relies on the BTF.

Go build is fundamentally better? How so? Go build is so light on features that adding generated files to source control is a norm in go land.

Packages may not circularly reference each other.

Actually possible with go:linkname.

I do occasionally attempt to edit patch files produced by git-format-patch. Frequently I end up with corrupt patch. Still curious how to fix those numbers.

Is it just me, or the piece doesn’t explain how to make edits without messing up the hunk?

Re nuclear reactor: higher tier virtualization products work flawlessly. It is a shame what a garbage virtual box turned into over the years.

It was fun to read, but it would’ve probably been easier to rely on seccomp filters instead.

Hardly. For starters, wasm doesn’t guarantee that a piece of code terminates in bound time. There are further security guarantees in ebpf such as any lock acquired must be released.

Great writeup, thoroughly enjoyed! The provided "lab" is especially appreciated.

I'm curious if you had reasons to not use veth in noarp mode.

Thank you for bpftrace! It was a vital aid for kernel spelunking. Very excited to see vmtest. I did a similar tool in the past [1] but never achieved this level of polish.

[1] https://github.com/mejedi/vmwrap

The BPF instrumentation is quite cool! I wonder if uprobes have a performance impact. Does it roughly compare to a single syscall?

https://github.com/keyval-dev/opentelemetry-go-instrumentati...

Docker only uses namespaces and cgroups.

How is that not isolation?

There was an entry from one of the Chrome early developers on NH recently: https://neugierig.org/software/blog/2022/12/chrome.html

“Chrome 12 [an old version] was the best Chrome", but since I’ve come to see underneath that for me what I wanted was a fast tight browser, and once I had that, pretty much by definition most subsequent work was just gonna make it worse.

Safari is today’s fast tight browser I dare say.

Sorry, but I’m failing to see how rootless containers are relevant.

Having multiple user accounts is a legit use case - think e.g. a shared device. A typical Linux package manager’s install is a privileged operation. Homebrew on the other hand gives the user that had originally installed it write permissions to certain directories. This strategy enables sudo-less brew install for the specific user and prevent it from being used by anyone else.

Homebrew the package manager is quite popular but that doesn’t indicate that it is engineered well. For instance, it doesn’t work with multiple user accounts which is such a trivial thing!

This is an impressive effort. I would love to see ignore rules for the particular editors and ideS added to project .gitignore files less frequently though.

Which editor someone uses is a private matter. Don’t commit it to public repositories. There’s local gitignore for that.

As of today, vector’s iterator is just a pointer. You need more than just a pointer to detect out of bounds accesses. It makes old binaries incompatible with new binaries. It also means that you can’t mix and match, it must be either a or b.

Hence you can’t make the change incrementally, which is a big risk.

Concerning Linux distributions, I seriously doubt if they have resources to assess the performance impact across the wide range of software.

Concerning performance impact, we end up with more instructions and more memory accesses and increased register pressure. This has a chance to make things slower.

But c’mon there’s no such thing as safe brain surgery. Would you then market brain surgeries as unsafe?

This is unfortunately ABI-altering, therefore it won’t be possible to selectively enable for projects that want it. It’s a distribution that has either to bite the bullet and say “we are going to pessimize all C++ software”, or err on the safe side and leave it disabled.

The warning about pointer arithmetic is most likely suppressable per compilation unit (for things like tagged pointers).

This is mostly due to ease of distribution. The binary is self-contained, no need to ship anything but the binary itself. You can also compile locally and then run it elsewhere that is a completely different flavor of Linux. Quite handy when experimenting. Finally, no dependency on libc enables easy cross-compilation. You might not even have libc installed for the architecture you are targeting!

This is fake. Sells at 105 right now. Edit: I have an account in a Russian bank and can purchase usd at this rate right now.

I strongly suspect that the buffer size is largely irrelevant. The article linked from readme estimates the amount of syscalls needed with the default buffer size as 16K. That’s peanuts, can do it in under a second, considering just syscall overhead.

The likely culprit is stat - dentry doesn’t have all the attributes inline, so a stat call is needed for EVERY file. That’s a lot of syscalls. Even worse, in a large directory the information won’t be in a cache, so every single stat hits the disk, dominating the time spent.

Thank you for the feedback! A new calling convention could probably nail it for many use cases. Sometimes you want something very special though. E.g. LuaJIT pre-decodes instruction prior to dispatch to squeeze some work into branch misprediction delay. There are limited callee save registers available hence it is better to use volatile registers for instruction arguments. It should be possible to reuse the implementation on different architectures and only tweak the register assignment.

I’ve opened an issue in LLVM bugzilla concerning jump not being folded with address computation on x86 with a proposed fix. Would love if it gets some attention. https://bugs.llvm.org/show_bug.cgi?id=50042

Also been working on a C language extension to enable guaranteed tail calls along with explicit control over registers used for argument passing. Provided that callee-save registers are used for arguments, calling fallback functions incurs no overhead.

https://github.com/rapidlua/barebone-c

Do openssl speed sha256 to get the idea how high the latency for a cache hit would be. I see a throughput of ~300MiB/sec. This can be parallelised easily but still we are burning lots of CPU cycles for nothing. Bad for battery life.

https://gist.github.com/nonylene/d08977e8952c83d20d2c5f7cbaf...

I wonder if they do indeed compute a checksum of the binary to come up with a aot-translation cache key. Must be quite ineffficient.

FlameGraph [1] renders profiling results into svg. Profile is essentially hierarchical; a user can zoom into subtree by clicking an element. Probably not an inspiring use of script in svg; but boy, how handy this feature is!

https://gist.githack.com/mejedi/90eac4f11f794f43808f90e38b1a...

[1] http://www.brendangregg.com/flamegraphs.html

With random keys updates are going to touch way more pages than with sequential ones. It increases IO due to more dirty buffers and journaling.

True, but for a cpu bound task like this VM typically causes only a 10% slowdown. You aren’t going to see an order of magnitude faster execution on real hardware.

The article makes it look like compilation is almost instantaneous. It’s not. Cached artifacts are rather hefty as well.

V8 fares surprisingly well given that it’s not a specialized WebAssembly runtime. Wasmer should probably watch competitors more closely.