HN user

jincongho

10 karma
Posts1
Comments2
View on HN

Parquet file format writes its metadata including length info after all data, at the footer. It was counterintuitive when I first look at it, but smart thinking about it now. I haven't had to trade off for memory constraints, but being able to stream output is definitely easier!

Interesting point about the difference in escape characters, I stored length and the decoded value so it's ready for string view. But when I need them back as JSON string, I need to encode them again :)