Parsing millions of URLs per Second (2023)

https://onlinelibrary.wiley.com/doi/10.1002/spe.3296

by PaulHoule • 2 years ago

157 27 2 years ago

onlinelibrary.wiley.com

notamy• 2 years ago

The title seems to have a few words missing. Original title:

Parsing millions of URLs per second

HL33tibCe7• 2 years ago

HN’s stupid/arrogant automatic title rewriter strikes again

PaulHoule2• 2 years ago

Fixed

kristianp• 2 years ago

I've never noticed a title being rewritten automatically when posting an article. Are you sure that's really a thing?

Cicero22• 2 years ago

This sort of work is something I wouldn't be able to do, but I can't help but point out at least one potential issue with the paper. It's a lot easier to find problems than solutions I guess.

Are the benchmarks comparing node versions valid to conclude a real world performance increase?

one possible confounder is the version of V8.

https://github.com/nodejs/node/blob/v18.x/deps/v8/include/v8... https://github.com/nodejs/node/blob/v20.x/deps/v8/include/v8...

ideally, they would've patched Node 18.15 with their changes directly and test their patch against 18.15.

beached_whale• 2 years ago

Found the easier to read/download from Arxiv link

https://arxiv.org/abs/2311.10533

maartenscholl• 2 years ago

I had a lot of fun writing low latency parsers for various message standards C++. There are a lot of fun things you can do when you can take ownership of the read buffer and you can figure out how to parse in-situ (modifying the data in place as you move along)

Thorrez• 2 years ago

For example, the input string http://xn--6qqa088eba.xn--3ds443g/./a/../b/./c should be normalized to the string https://xn--xn6qqa088eba-l19f.xn--xn3ds-zu3b/b/c

Why would normalization change http:// to https:// ?

ramon156• 2 years ago

Because its 2024

marginalia_nu• 2 years ago

http:// is not a typo for https://. There's still a fairly large amount of web servers that do not talk https, and you simply cannot assume that they do. That will leave you with a lot of dead links. Besides, most that accept both will auto-renegotiate to https.

chrismorgan2• 2 years ago

There’s got to be some accidental mangling there. Somewhere. Because of that error, and still more because of the blatant error in the next sentence:

For example, given the base string http://example.org/foo/bar, the relative string http://example.com/ leads to the final URL http://example.org/example.com/.

That’s just… no. I do not believe I have ever encountered any software which would parse it in that way, and I refuse to believe such software ever existed. It would be <http://example.com/>.

But the PDF matches the HTML. I dunno, something weird is going on. Look at the hyperlinks there, too, “http://xn--ivg but not the rest of the URL that follows, and how the -- has been changed to –. Something went wrong somewhere in the editing or publication.

silvestrov• 2 years ago

My guess is that the html formatter changed the text "example.com" into "http://example.com" to make it a valid absolute URL.

youngtaff• 2 years ago

Lemire’s blog is well worth a read if you’re interested in this sort of thing https://lemire.me/blog/

grayhatter• 2 years ago

I wonder how much time was spent promoting this parser, vs time spent on writing it? I've seen a lot of spam for this one, and I'm not the only one.

https://daniel.haxx.se/blog/2023/11/21/url-parser-performanc...

yagiznizipli• 2 years ago

Almost a year of development, 3 months of writing paper. All of the benchmarks are public. Run it before sharing someone else’s blog post.

trung123f• 2 years ago

u has similar repo? tks

Parsing millions of URLs per Second (2023)

Related Stories