HN user
no
Exactly. I do not know the specifics, but for example if libraries you call into liberally spawn_blocking under the expectation that it is okay, you will be in trouble.
Says it right there actually:
It’s recommended to not set this limit too low in order to avoid hanging on operations requiring spawn_blocking.
So a total like 6 - reasonable for a web backend - would be way too low.
Funnily enough, spawn_blocking is not the right tool here. It is meant for blocking I/O, such as DNS lookups, where your platform might not give you anything better.
For genuine CPU-bound work, submitting to a Rayon worker pool is the way to go. It solved a runtime starvation issue for us at work, spawn_blocking did not work.
The reason for all this is spawn_blocking having a very large underlying thread pool, in the hundreds. That is okay if you assume work will yield those threads and mostly sleep/wait. It is not okay if the work never yields, like pure data crunching. (Go solves this by forcefully preempting loops, no such thing in Rust without a language runtime)
Our solution shape was: multi-threaded Tokio (2 threads), then give the rest of available_concurrency to a Rayon thread pool. If you grant 6 vCPU you should see a thread pool of 4, and a maximum CPU consumption of about 400%, as the Tokio threads sit mostly idle (under low load single-threaded runtime should also suffice).
You inject the thread pool using an Arc.
Then, when work comes in, just spawn Tokio tasks liberally (cheap) and submit to the thread pool. Rayon will internally queue and limit concurrency and parallelism to 4 (this is the important bit compared to spawn_blocking: no way your system can hog all 6 threads with non-yielding work and starve Tokio runtime threads).
We use one-shot channels to submit results back, they are designed for exactly this. The tx aka sender end is sync, as there is never a wait (cannot block). The rx aka receiver side is async and can be awaited normally on the async side. This is a cheap operation, similar to Go.
Optionally you can reach for semaphores to also limit I/O concurrency. You probably want to do this for more control and avoiding resource exhaustion loudly (that is, not silently accidentally peg thousands of FDs, database connections, …).
It ended up working beautifully for our purposes and relatively simply. No lifetime woes, Arc solves those. Oneshot channels just transfer ownership etc.
Perhaps this is what TFA talks about, I have not read it.
One caveat: to reach all the above conclusions and designs, we had help from some genuine Rust experts. As much as I dislike Go, it "just works" there even if one writes naive code.
NAT busting is a great point.
I do not understand this rebuttal.
I also run self-hosted Wireguard. Initially on a Debian box, nowadays it is integrated into my router (admittedly, this is closed source). For around 6 years at this point.
The whole thing could not be easier and simpler. It has never randomly broken on me. It is fast. It is free. There is no middle man, no vendor.
I never understood the popularity of Tailscale, though that is on me. I'm sure it is a great product, I just never tried it, do not seem the target audience.
What confuses me is the often accompanying, sometimes aggressive anti-selfhosting stance in these sorts of threads. I do not see this in other topics, e.g. someone mentioning they run Jellyfin isn't met with "why not Plex?". Where does that come from? We are on HackerNews, not ProductShillNews, aren't we? I guess self hosting Wireguard is too boring to warrant any further discussion? The VPN equivalent of a Toyota Corolla.
Yeah. I don't think PG could even come close. Column-oriented is fundamentally different, and pairs well with all the SIMD acceleration ClickHouse is also doing. There's just no comparison. If a Postgres rewrite came close to that, it must've sacrificed something else.
But do the markets care about a Postgres in Rust? Probably not, or at least not right away. It is a long way towards commercial success.
I suspect rather than hire less people we will just produce more code changes.
Why? Towards what end? Code changes are output, not outcome. It also needs to be connected to someone willing to pay you hard cash. That is the hard part, a race to the bottom, and the reason I also believe there will be downwards pressure on salaries and even employment.
People still learn math, despite the calculator existing. Accounts still learn accounting, despite Excel and accounting software existing.
They do, but you need far fewer or none of the original workers whose full-time job this sort of stuff was.
Raw math does not matter, but what you do with it. Similarly, you could earn a (modest) living knowing nothing but raw HTML, JavaScript and a bit of browser tech not too long ago. That is no longer possible.
Programming and software engineering will be devalued. These occupations won't disappear overnight, but you will see compensation and growth stagnate until equilibrium is reached again. Currently, supply outstrips demand, and I do think it is structural, not just hype.
I'm certainly not creative enough, but I currently do not see demand picking up sufficiently; Gen Z is bearish on social media, VR was a bust, blockchain was a bust, software has already penetrated almost all walks of live and lines of work. There is no next big thing (Internet, ...) on the horizon, to unlock the next order of magnitude of demand. There is certainly more work to do still, but it very suddenly does not require the same headcount, but something like 5%-30% less. Lots of the remaining work will be around integrating LLMs into existing software, which does not sound exciting either.
Yes, my point was not related to null. For all I care you can have `&T` and `Option<&T>` in your language, but allow `&T` to be null. In Rust, that would be `Option<*const T>`. Is that useful? I don't know. But it still separates the two orthogonal concepts. Go conflates them, rolling them into one, permanently removing useful expressivity.
No, because RateLimiter is then copied on passing it around (pass by value).
That is problematic for two reasons: it might be a large type, so copying might be expensive. Second, more likely, it might violate invariants in your domain. For a rate limiter, this might mean accidentally copying around some internal state like a mutex, which then exists n times instead of 1 time, which can represent a problem (e.g. if you want to internally limit whole-app concurrency toward Redis).
This is the mess a language lands on when it conflates optionality (a semantic concept) with references/pointers (purely a machine concept). In Go, the requirement "need (non-optional) a reference to an object" is simply not expressible. This is a solved problem in other languages, for example `&T` vs. `Option<&T>` in Rust.
As a fan and believer of obscurity in support of security, I do not understand why
that step didn't add any security.
It is a decision that’s part of the entire process. A branch of many in the decision tree. Other branches are deciding which characters to type for the password; ASCII characters can be as little as 1 bit apart. Deciding between left and right is also 1 bit apart.
I think it boils down to what people commonly understand to be publicly knowable information versus understood-to-be-secret information.
One example: I self-host my password manager at pw.example.com/some-secret-path/. That extra path adds as much to security as a randomly picked username in HTTP Basic Auth: arguably none. Yet, it is as impossible for attackers to enumerate and find that path as it is with passwords.
The difference is that the path leaks easier. It’s not generally understood to be a secret. Yet I argue it helps security. (Example: leaking the domain name through certificate transparency logs AND even, say, user credentials means an attack is still unsuccessful; a strictly necessary piece of the puzzle is missing).
Just this month Google shipped what I understand as hard limits in AI Studio/Gemini/whatever it's called this week. I had existing billing alerts (best you could do before IIUC), but set these new hard limits up immediately. Feels good!
Interesting, though I disagree on basically all points...
No Silver Bullet
As an industry, we do not know how to measure productivity. AI coding also does not increase reliability with how things are going. Same with simplicity, it's the opposite; we're adding obscene complexity, in the name of shipping features (the latter of which is not productivity).
In some areas I can see how AI doubles "productivity" (whatever that means!), but I do not see a 10x on the horizon.
Kernighan's Law
Still holds! AI is amazing at debugging, but the vast majority of existing code is still human-written; so it'll have an easy time doing so, as indeed AI can be "twice as smart" as those human authors (in reality it's more like "twice as persistent/patient/knowledgeable/good at tool use/...").
Debugging fully AI-generated code with the same AI will fall into the same trap, subject to this law.
(As an aside, I do wonder how things will go once we're out of "use AI to understand human-generated content", to "use AI to understand AI-generated content"; it will probably work worse)
just ask AI to rewrite the code
This is a terrible idea, unless perhaps there is an existing, exhaustive test harness. I'm sure people will go for this option, but I am convinced it will usually be the wrong approach (as it is today).
Dijkstra on the foolishness of programming in natural language
So why are we not seeing repos of just natural language? Just raw prompt Markdown files? To generate computer code on-the-fly, perhaps even in any programming language we desire? And for the sake of it, assume LLMs could regenerate everything instantly at will.
For two reasons. The prompts would either need to raise to a level of precision as to be indistinguishable from a formal specification. And indeed, because complexity does become "exponentially harder"; inaccuracies inherent to human languages would compound. We need to persist results in formal languages still. It remains the ultimate arbiter. We're now just (much) better at generating large amounts of it.
Lehman’s Law
This reminds me of a recent article [0]. Let AI run loose without genuine effort to curtail complexity and (with current tools and models) the project will need to be thrown out before long. It is a self-defeating strategy.
I think of this as the Peter principle applied to AI: it will happily keep generating more and more output, until it's "promoted" past its competence. At which point an LLM + tooling can no longer make sense of its own prior outputs. Advancements such as longer context windows just inflate the numbers (more understanding, but also more generating, ...).
The question is, will the market care? If software today goes wrong in 3% of cases, and with wide-spread AI use it'll be, say, 7%, will people care? Or will we just keep chugging along, happy with all the new, more featureful, but more faulty software? After all, we know about the Peter principle, but it's unavoidable and we're just happy to keep on.
Jevons Paradox
My understanding is the exact opposite. We might well see a further proliferation of information technologies, into remaining sectors which have not yet been (economically) accessible.
uv supports it, https://docs.astral.sh/uv/reference/settings/#exclude-newer
This take was accurate about 2 years ago, up until perhaps one year ago. Current capabilities far exceed what you are outlining, for example using Claude Opus models in a harness such as Claude Code or OpenCode.
I use unbound (recursive resolver), and AdGuard Home as well (just forwards to unbound). Unbound could do ad-blocking itself as well, but it's more cumbersome than in AGH. So I use two tools for the time being.
The upside is there's no single entity receiving all your queries. The downside is there's no encryption (IIRC root servers do not support it), so your ISP sees your queries (but they don't receive them).
I have the same question, I am confused by the premise of this article. Now you're recording everything twice?
Agree. I am note sure I understand the premise of the article. You're now recording encountered errors twice, which can look like
cancel(fmt.Errorf(
"order %s: payment failed: %w", orderID, err,
))
return fmt.Errorf("order %s: payment failed: %w, orderID, err)
Perhaps I'm not seeing the problem being solved, but bog-standard `return err` with "lazy" context cancellation (in a top-level `defer cancel()`), or eager (in a leaf I/O goroutine) seems to carry similar functionality. Stacking both with ~identical information seems redundant.
Feldera speak from lived experience when they say 100+ column tables are common in their customer base. They speak from lived experience when they say there's no correlation in their customer base.
Feldera provides a service. They did not design these schemas. Their customers did, and probably over such long time periods that those schemas cannot be referred to as designed anymore -- they just happened.
IIUC Feldera works in OLAP primarily, where I have no trouble believing these schemas are common. At my $JOB they are, because it works well for the type of data we process. Some OLAP DBs might not even support JOINs.
Feldera folks are simply reporting on their experience, and people are saying they're... wrong?
I have been using nixos-rebuild with target host and it has been totally fine.
The only thing I have not solved is password-protected sudo on the target host. I deploy using a dedicated user, which has passwordless sudo set up to work. Seems like a necessary evil.
In thermodynamics, ultimately you need to input work to remove entropy from a system (e.g. by cooling surroundings). Humans do the same for software.
I am an avid user of LLMs but I have not seen them remove entropy, not even once. They only add. It’s all on the verge of tech debt and it takes substantial human effort to keep entropy increases in check. Anyone can add 100 lines, but it takes genuine skill to do it 10 (and I don’t mean code golf).
And to truly remove entropy (cut useless tests, cut useless features, DRY up, find genuine abstractions, talk to PM to avoid building more crap, …) you still need humans. LLM built systems eventually collapse under their own chaos.
I think your analogy is quite fitting!
Randomly fail or (increasingly) delay a random subset of all requests.
Interesting attack on my person!
The things I want to get done on my computer are so much richer than moving files back and forth in a terminal all day. Simple things should be simple. Tools should enable us. Moving files is a means to an end and these commands having so many sharp edges makes me unhappy indeed.
But yes, of course I am an IDE toddler and cannot even tie my shoes without help from an LLM, thanks for reminding me.
I have trouble gauging the effects of `*`, aka "will I cp/mv the dir or all contents of the dir"? Then shell expansion getting in the way, like expanding it all ("argument list too long"). I try to use rsync when possible, where intermittent `.` in paths are relevant, as well as trailing slashes... and don't forget `-a`! Then I forget if those same things apply to cp/mv. Then I cp one directory too deep and need to undo the mess -- no CTRL+Z!
The wide-spread use of `rm -rf some-ostensibly-empty-dir/` is also super dangerous, when `rm -r` or even `rmdir` would suffice, but you need to remember those exist. I find it strange there's no widely (where applicable) available `rm-to-trash`, which would be wildly safer and is table stakes for Desktop work otherwise.
Then there's `dd`...
I use terminals a lot, but a GUI approach (potentially with feedback pre-operation) plus undo/redo functionality for file system work is just so much easier to work with. As dumb as drag-and-drop with a mouse is, a single typo can't wreck your day.
I've had a color scheme plugin yanked from my IDE a while back, as it went malicious (Material Theme). It's just a bunch of hex codes, how is that even possible? Baffling and disappointing indeed.
git on the command line is the power tool. The VS Code plugin is the training wheels version.
I don't disagree with your underlying point, but git is perhaps the worst example. It is a horrible tool in several ways. Powerful yes, but it pays too large a price in UX.
I've only ever used git through the CLI as well, but having switched to jujutsu (also CLI) I am not going back. It is quite eye-opening how much simpler git should be, for the average user (I realize "average user" is doing some heavy-lifting here -- git covers an enormous number of diverse use cases).
What jujutsu-CLI is for me (version control UX that just works) might be VSCode's GUI git integration for other people, or magit, or GitButler, or whatever other GUI or TUI.
Who cares about training wheels? If the real deal is a unicycle with one pedal and no saddle, I will keep using training wheels, thank you very much.
All IDEs including VSCode have fuzzy-finding for files, and fuzzy-finding for symbols as well. Between these, I never find myself using the file tree, except when it's the best tool for the job ("what other files are in this directory?", file tree manipulation (which IDEs recognize you doing, adjusting imports for you!) etc.).
I actually notice how this pattern is very fast, but I lose a code base's mental map. Coworkers might take longer to open any individual file but have a much better idea of repo layout as a whole. That makes them effective otherwise.
No thank you...
One of the primary things shells are supposed to excel at is file system navigation and manipulation, but the experience is horrible. I can never get `cp`, `rsync`, `mv`, `find` right without looking them up, despite trying to learn. It's way too easy to mess up irreversibly and destructively.
One example is flattening a directory. You accidentally git-cloned one too deep, into `some/dir/dir/` and want to flatten contents to `some/dir/`, deleting the then-empty `some/dir/dir/`. Trivial and reversible in any file manager, needlessly difficult in a shell. I get it wrong all the time.
Similarly, iterating over a list of files (important yet trivial). `find` is arcane in its invocation, differs between Unix flavors, you will want `xargs` but `{}` substitution is also very error-prone. And don't forget `print0`! And don't you even dare `for f in *.pdf`, it will blow up in more ways than you can count. Also prepare to juggle quotes and escapes and pray you get that right. Further, bash defaults are insane (pray you don't forget `set -euo pipefail`).
How can we be failed by our tools so much, for these use cases? Why aren't these things trivial and safe?
just another kind of complex?
To some extent yes. If all you have is 2 GitHub Actions YAML files you are not going to reap massive benefits.
I'm a big fan of CUE myself. The benefits compound as you need to output more and more artifacts (= YAML config). Think of several k8s manifests, several GitHub Actions files, e.g. for building across several combinations of OSes, settings, etc.
CUE strikes a really nice balance between being primarily data description and not a Turing-complete language (e.g. cdk8s can get arbitrarily complex and abstract), reducing boilerplate (having you spell out the common bits once only, and each non-commit bit once only) and being type-safe (validation at build/export time, with native import of Go types, JSON schema and more).
They recently added an LSP which helps close the gap to other ecosystems. For example, cdk8s being TS means it naturally has fantastic IDE support, which CUE has been lacking in. CUE error messages can also be very verbose and unhelpful.
At work, we generate a couple thousand lines of k8s YAML from ~0.1x of that in CUE. The CUE is commented liberally, and validation imported from native k8s types, and sprinkled in where needed otherwise (e.g. we know for our application the FOO setting needs to be between 5 and 10). The generated YAML is clean, without any indentation and quoting worries. We also generate YAML-in-YAML, i.e. our application takes YAML config, which itself is in an outer k8s YAML ConfigMap. YAML-in-YAML is normally an enormous pain and easy to get wrong. In CUE it's just `yaml.Marshal`.
You get a lot of benefit for a comparatively simple mental model: all your CUE files form just one large document, and for export to YAML it's merged. Any conflicting values and any missing values fail the export. That's it. The mental model of e.g. cdk8s is massively more complex and has unbounded potential for abstraction footguns (being TypeScript). Not to mention CUE is Go and shipped as a single binary; the CUE v0.15.0 you use today will still compile and work 10 years from now.
You can start very simple, with CUE looking not unlike JSON, and add CUE-specific bits from there. You can always rip out the CUE and just keep the generated YAML, or replace CUE with e.g. cdk8s. It's not a one-way door.
The cherry on top are CUE scripts/tasks. In our case we use a CUE script to split the one-large-document (10s of thousands of lines) into separate files, according to some criteria. This is all defined in CUE as well, meaning I can write ~40 lines of CUE (this has a bit of a learning curve) instead of ~200 lines of cursed, buggy bash.