HN user

latch

13,734 karma

https://www.openmymind.net

Posts133
Comments2,245
View on HN
www.openmymind.net 2y ago

Performance of reading a file line by line in Zig

latch
4pts0
www.pondz.dev 3y ago

Show HN: Duckdb-proxy an HTTP proxy for DuckDB written in Zig

latch
4pts0
www.openmymind.net 4y ago

Elixir, a Little Beyond the Basics

latch
5pts2
www.brendangregg.com 5y ago

Poor Disk Performance

latch
192pts36
rbranson.medium.com 5y ago

Things I hate about PostgreSQL (2020)

latch
439pts272
www.financemagnates.com 5y ago

Etoro down for 17 hours due to MS SQL Server issue

latch
1pts0
www.cockroachlabs.com 6y ago

CockroachDB v20.1.0 Released

latch
2pts0
www.youtube.com 6y ago

An Amazon Scam: The Mofut Key Lock Box

latch
1pts0
www.openmymind.net 6y ago

Migrating to CockroachDB

latch
163pts112
www.quora.com 9y ago

John Carmack Answers: Why Was Doom Developed on a NeXT?

latch
2pts0
www.macleans.ca 10y ago

The dangers of our protein diet obsession

latch
2pts0
github.com 10y ago

Vim Plugin: Rainbow Parentheses Improved

latch
2pts0
teapi.io 11y ago

Show HN: Teapi – API as a Service

latch
3pts0
www.exploringbinary.com 11y ago

Ten Ways to Check If an Integer Is a Power of Two in C (2009)

latch
6pts0
openmymind.net 11y ago

The Little Go Book

latch
7pts0
openmymind.net 11y ago

Practical SOA / microservices – Hydration

latch
3pts0
gridland.doublespeakgames.com 11y ago

Gridland (from the creator of a Dark Room)

latch
5pts1
news.ycombinator.com 12y ago

Ask HN: Most useful travel accessories

latch
2pts1
techcrunch.com 12y ago

Nokia X2: Microsoft's first Android Phone

latch
2pts1
www.youtube.com 12y ago

Living with lag – an oculus rift experiment

latch
4pts1
www.nytimes.com 12y ago

Defying Japan, Rancher Saves Fukushima's Radioactive Cows

latch
1pts0
www.youtube.com 12y ago

Bill Gates Doom

latch
5pts0
modernfarmer.com 12y ago

The Post-GMO Economy

latch
2pts0
allthingsd.com 12y ago

Japan’s Rakuten Acquires Viki Video Site for $200 Million

latch
4pts0
openmymind.net 13y ago

How Meaningful Do You Find Github's Punchcard?

latch
1pts0
openmymind.net 13y ago

How Good Is Amazon's Route 53 Latency Based Routing?

latch
61pts22
openmymind.net 13y ago

Scaling Viki

latch
30pts2
news.ycombinator.com 13y ago

Ask HN: Advice on promoting my next free ebook

latch
3pts2
openmymind.net 13y ago

Things I Wish Someone Had Told Me About Go

latch
74pts44
openmymind.net 13y ago

Custom Redis Command: Performance of C vs Lua

latch
3pts0

Not sure how you get to that conclusion from the article when it ends with the conclusion from 5 health agencies that it's safe (and then more references from the scientific community that it's safe).

I'm not sure this is particularly telling. You can write a tiny program that generates a 4K image, and the image could be 1000x larger.

Or, if I write a short description "A couple walks hand-in-hand through a park at sunset. The wind rustles the orange leaves.", I don't think it would be surprising to anyone that an image or video of this would be relatively huge.

Zig 0.15 is pretty stable. The biggest issue I face daily are silent compiler errors (SIGBUS) for trivial things, e.g. a typo in an import path. I've yet to find exactly why this [only sometimes] causes such a crash, but they're a real pain to figure out over a large changeset. `zig ast-check` sometimes catches the error, else Claude's pretty good at spotting where I accidentally re-used a variable name (again, 90% of the time I do that, it's an easy error, but the other 10%, I get a message-less compiler crash). It sounds like the changes in the OP might be specifically addressing these types of issues.

Also, my .zig-cache is currently at 173GB, which causes some issues on the small Linux ARM VPS I test with.

As for upgrades. I upgraded lightpanda to 0.14 then 0.15 and it was fine. I think for lightpanda, the 0.16 changes might not be too bad, with the only potential issue coming from our use of libcurl and our small websocket server (for CDP connections). Those layers are relatively isolated / abstracted, so I'm hopeful.

As a library developer, I've given up following / tracking 0.16. For one, the change don't resonate with me, and for another, it's changing far too fast. I don't think anyone expects 0.16 support in a library right now. I've gotten PRs for my "dev" branches from a few brave souls and everyone seems happy with that arrangement.

It wasn't clear from the examples, and the gist doesn't have a `deinit` method, so what happens if an error needs to own data?

Here, sqlite.ErrorPayload.init saves 500 bytes of error message from sqlite

Who owns those 500 bytes and where are they being freed?

Claude code has those "thoughts" you say it never will. In plan mode, it isn't uncommon that it'll ask you: do you want to do this the quick and simple way, or would you prefer to "extract this code into a reusable component". It also will back out and say "Actually, this is getting messy, 'boss' what do you think?"

I could just be lucky that I work in a field with a thorough specification and numerous reference implementations.

I ... don't disagree with you. Thanks. It helps my understanding.

I know this is moving the goalpost, but it's still a shame that it [obviously] has to be a runtime error. Practically speaking, I still think it leaves lot of friction and edge cases. But what you say makes sense: it doesn't have to be unsafe.

Makes me curious why they asserted instead of erroring in the first place (and I don't think that's exclusive to the zstd implementation right now).

Author here. I see it both ways.

Blog posts are collaboration (1). I did get the sense that Andrew doesn't see it that way. (And for this post in particular, and writegate in general, I have been discussing it on the discord channel. I know that isn't an official channel).

My reasons for not engaging more directly doesn't have anything to do with my confidence / knowledge. They are personal. The linked issues, which I was aware of, are only tangentially related. And even if they specifically addressed my concerns, I don't see how writing about it is anything but useful.

But I also got the sense that more direct collaboration is welcome and could be appreciated.

(1) - I'm the author of The Little MongoDB Book, The Little Redis Book, The Little Go Book, etc... I've always felt that the appeal of my writing is that I'm an average programmer. I run into the same problems, and struggle to understand the same things that many programmers do. When I write, I'm able to write from that perspective.

No matter how inclusive a community you have, there'll always be some opinions and perspectives which get drowned out. It can be intimidating to say "I don't understand", or "it's too complicated" or, god forbid, "I think this is a bad design"; especially when the experts are saying the opposite. I'm old enough that I see looking the fool as both a learning and mentoring experience. If saying "io.Reader is too complicated" saves someone else the embarrassment of saying it, or the shame of feeling it, or gives them a reference to express their own thoughts, I'm a happy blogger.

Author here.

I finally got it working. I had to flush both the encrypted writer and then the stream writer. There was also some issues with reading. Streaming works, but it'll always return 0 on the first read because Writer.Fixed doesn't implement sendFile, and thus after the first call, it internally switches from streaming mode to reading mode (1) and then things magically work.

Currently trying to get compression re-enabled in my websocket library.

(1) https://github.com/ziglang/zig/blob/47a2f2ddae9cc47ff6df7a71...

A friend recently shared a newspaper article about his company. I hadn't seen him in a little while, and I told him how great he looked. Turns out, an AI-assisted image was used. It surprised me - I had hoped/expected more from that paper.

The only thing that can be surprising about real estate agents doing it would be if anyone was surprised.

I think having a thread pool on top of some evented IO isn't _that_ uncommon.

You might have a thread pool doing some very specific thing. You can do your own threadpool which wont use the Io interface. But if one of the tasks in the threadpool wanted to read a file, I guess you'd have to pass in the blocking Io implementation.

I don't disagree. It's like 13 years old. `GetWithoutPromote` was added in 2022, I assume someone asked for it, so I added it. That kind of stuff happens, especially when you stop building it for your own needs.

For the most part, you use a default config and use Get/Fetch/Set. Besides the excuse of its age, and not being seriously worked on for a long time (a decade?), I do think we both have a bias towards what's more familiar. What are the `ExpiryCalculator`, `Weigher`, etc... configuration options of Otter? (or `GetEntryQuietly`, `SetRefreshableAfter` ...)

Author of ccache here.

I've barely touched Go in over a decade, but if I did, I'd probably still use ccache if I didn't need cutting edge (because I think the API is simple), but not if I needed something at huge scale.

When I wrote ccache, there were two specific features that we wanted that weren't readily available:

- Javing both a key and a subkey, so that you can delete either by key or key+subkey (what ccache calls LayeredCache).

- Having items cached that other parts of the system also have a long-living reference to, so there's not much point in evicting them (what ccache calls Tracking and is just a separate ARC mechanism that overrides the eviction logic).

It also supports caching based on arbitrary item size (rather than just a count of items), but I don't remember if that was common back then.

I've always thought that this, and a few other smaller features, make it a little bloated. Each cached item carries a lot of information (1). I'm surprised that, in the linked benchmark, the memory usage isn't embarrassing.

I'm not sure that having a singl goroutine do a lot of the heavy-lifting, to minimize locks, is a great idea. It has a lot of drawbacks, and if I was to start over again, I'd really want to benchmark it to see if it's worth it (I suspect that, under heavy write loads, it might perform worse).

The one feature that I do like, that I think most LRU's should implement, is to have a [configurable] # of gets before an item is promoted. This not only reduces the need for locking, it also adds some frequency bias to evictions.

Fun Fact: My goto interview question was to implement a cache. It was always rewarding to see people make the leap from using a single data structure (a dictionary) to using two (dictionary + linked list) to achieve a goal. It's not a way most of us are trained to think of data structures, which I think is a shame.

(1) https://github.com/karlseguin/ccache/blob/master/item.go#L22

Author here. I agree this is a less-than captivating piece. I write a lot about Zig and wanted something I could reference from other pieces.

But, to answer your question directly: absolutely. In addition to writing a lot about it, I maintain some popular libraries and lurk in various communities. Let me assure you, beginner memory-related questions come up _all the time_. I'd break them down into three groups:

1 - Young developers who might have a bit of experience in JavaScript or python. Not sure how they're finding their way to Zig. Maybe from HN, maybe to do game development. I think some come to Zig specifically to learn this kind of stuff (I've always believed most programmers should know C. Learning Zig gets you the same fundamentals, and has a lot of QoL stuff).

2 - Hobbyist. Often python developers, often doing embedded stuff. Might be looking to write extensions in Zig (versus having to do it in C).

3 - Old programmers who have been using higher level languages for _decades_ and need a refresher. Hey, that's me!

There's definitely overhead with the GenericWriter, seeing as it uses the AnyWriter for every call except `write` (1)

    genericWriter        - 31987.66ns per iterations
    appendSlice          - 20112.35ns per iterations
    appendSliceOptimized - 12996.49ns per iterations

`appendSliceOptimized` is implemented using knowledge of the underlying writer, the way that say an interface implementation in Go would be able to. It's a big part of the reason that reading a file in Zig line-by-line can be so much slower than in other languages (2)

(1) https://gist.github.com/karlseguin/1d189f683797b0ee00cdb8186...

(2) https://github.com/ziglang/zig/issues/17985

A couple comments have mentioned that whatever issue my post raises, are minor compared to the simplicity of the language. But the current implementation of GenericWriter + AnyWriter (with some performance pitfalls) seems more, not less, complicated. Also, neither of these, nor anytype, lend themselves to _simple_ documentation, so that seems like another strike against the simplicity-argument.

As for anytype specifically, in simple cases where it's being used in a single function, you can quickly figure out what it needs.

But in non trivial cases, the parameter can be passed all over the place, including into different packages. For example `std.json.stringify`. Not only does its own usage of `out_stream: anytype` take more than a glance, it passes it into any custom `jsonStringify` function. So you don't just need to know what `std.json.stringify` needs, but also any what any custom serialization needs.

Their explanation for why Go performs badly didn't make any sense to me. I'm not sure if they don't understand how goroutines work, if I don't understand how goroutines work or if I just don't understand their explanation.

Also, in the end, they didn't use the JSON payload. It would have been interesting if they had just written a static string. I'm curious how much of this is really measuring JSON [de]serialization performance.

Finally, it's worth pointing out that WebSocket is a standard. It's possible that some of these implementations follow the standard better than others. For example, WebSocket requires that a text message be valid UTF8. Personally, I think that's a dumb requirement (and in my own websocket server implementation for Zig, I don't enforce this - if the application wants to, it can). But it's completely possible that some implementations enforce this and others don't, and that (along with every other check) could make a difference.