HN user

majke

5,473 karma
  Enthusiastic geek         https://idea.popcount.org
                            https://github.com/majek
Posts166
Comments620
View on HN
github.com 1y ago

Distributed Data Protocol (DDP) Specification – From Meteorjs

majke
2pts0
josephg.com 1y ago

I was wrong. CRDTs are the future – Joseph Gentle

majke
2pts1
blog.cloudflare.com 1y ago

The day my ping took countermeasures

majke
4pts1
www.science.org 1y ago

Derek Lowe: Revised and Extended: What's Happening Inside the NIH and NSF

majke
10pts1
blog.cloudflare.com 1y ago

Mptcp: Revolutionizing connectivity, one path at a time

majke
7pts4
sites.google.com 1y ago

Fast Kinodynamic Planning on the Constraint Manifold with Deep Neural Networks

majke
2pts0
ilya.grigorik.com 2y ago

Professional Aptitude vs. Tenure (2022)

majke
2pts0
unix.stackexchange.com 2y ago

Errno – ENOANO 55 No anode – what is it?

majke
1pts0
blog.cloudflare.com 3y ago

The day my ICMP ping took countermeasures

majke
3pts0
www.economist.com 3y ago

City of love and war: How a front-line city became Ukraine’s romantic capital

majke
2pts1
en.wikipedia.org 3y ago

Tech-noir: technology as a destructive and dystopian force

majke
2pts2
en.wikipedia.org 3y ago

Ralf Brown's Interrupt List

majke
136pts65
blog.cerowrt.org 3y ago

Features and flaws of speedtest.net, ookla and Cloudflare

majke
1pts0
blog.cloudflare.com 3y ago

When TCP sockets refuse to die

majke
4pts0
blog.cloudflare.com 4y ago

TCP Keepalives and TCP_USER_TIMEOUT socket option explained

majke
7pts0
www.youtube.com 4y ago

Linux Plumbers Conference – BPF Rough Edges

majke
2pts1
blog.cloudflare.com 5y ago

Conntrack turns a blind eye to dropped SYNs

majke
3pts0
blog.cloudflare.com 5y ago

Branch predictor: How many “if”s are too many?

majke
324pts105
programming.guide 5y ago

Robin Hood Hashing

majke
1pts0
blog.cloudflare.com 5y ago

How to get 100B Int ops per CPU: nearest neighbor in high dimensionality

majke
1pts0
idea.popcount.org 6y ago

BSD Socket Addressing

majke
2pts0
randsinrepose.com 6y ago

Managing 10x Engineers – The Wolf

majke
12pts1
carstein.github.io 6y ago

Build simple fuzzer – part 4

majke
1pts0
www.cs.technion.ac.il 6y ago

Iommu resistant DMA attacks – all modern IOMMU implementations are broken [pdf]

majke
2pts0
man7.org 6y ago

Linux 3.8 introduced unprivileged user namespaces [pdf]

majke
166pts51
zipcon.net 6y ago

Last Person on Earth Without Windows 95

majke
2pts0
lemire.me 6y ago

A Short History of Technology

majke
2pts0
software.intel.com 6y ago

History of one CPU instructions: Part 1. LDDQU/movdqu explained

majke
2pts0
idea.popcount.org 6y ago

Creating Network Sockets on Linux

majke
1pts0
idea.popcount.org 6y ago

Creating Network Sockets on Linux

majke
1pts0

If you want to know more about the cache invalidation in the whole pipeline of DNS requests, take a look at this

https://blog.cloudflare.com/tld-glue-sticks-around-too-long/

I hinted there how the NS chain of lookups works from . to your domain. The point is that we wanted to be able to move name servers around the ip addresss, but that wouldn't work for many domains. So - in some contexts moving IP's rapidly is possible, in some it's not. Fun.

sockmap has been through a lot of changes since then. Some done by CF (hi Jakub!) some by others, including bytedance and isovalent.

Recently there has been a push to figure out the backpressure situation, so it's worth checking again.

Marek

A lot of self-hosters running CF tunnels because they don't know better also contributes.

Are you saying CF documentation is better than Computer Science / Networking education resources? Why don't people know better? I thought the tunnels are mostly used to bypass NAT's.

Static blogs hiding behind bot protection

I'm not sure what is the proportion of the static vs dynamic sites, but I would argue that for wordpress CF is adding real value.

Lysk | lysk.ai | AI/Geo Engineer | Full-time | Warsaw / Paris

Lysk is a European defence startup, based between Paris and Warsaw. Our mission is to create modern software tailored for hostile environments. Our customers include European military and police units. We are in stealth but have already been recognised as one of France’s top 20 seed companies. Backed by Project A, Europe’s leading defence VC, and angels including the founders of Cloudflare, Quantum Systems, and ElevenLabs. We’re assembling a world-class team at the intersection of modern software and defence. We do not work on weapons systems or hardware products.

Currently we're looking for:

- Junior LLM engineer (gguf, on-prem models, llm validation and pipelines)

- Junior Voice AI engineer (we need customised STT models)

- Senior GEO-AI engineer (interpreation of speech-based geographical information)

I've had more success. Flipper taught me about sdr, and I was able to reverse quite a lot from my garage door pilot. Then I went on an adventure of cracking Keeloq cipher, and I haven't stopped since.

Not necessarily. In classic C we often build complex state machines to handle errors - especially when there are many things that need to be initialized (malloced) one after another and each might fail. Think the infamous "goto error".

I think defer{} can simplify these flows sometimes, so it can indeed be useful for good old style C.

Back in 2022 I published a doc on how the egress IPs work at Cloudflare:

https://blog.cloudflare.com/cloudflare-servers-dont-own-ips-...

In summary, the location at which an IP egresses Cloudflare network has nothing to do with the geo-ip mapping of that IP. In some cases the decision on where to egress is optimised for "location closest to the user", but this is also not always true.

And then there is the Internet. Often some country (say Iran) egresses from a totally different place (like Frankfurt) due to geopolitics and just location of cables.

Coordinator sees Node A has significantly fewer rows (logical count) than the cluster average. It flags Node A as "underutilized."

Ok, so you are dealing with a classic - you measure A, but what matters is B. For "load" balancing a decent metric is, well, response time (and jitter).

For data partitioning - I guess number of rows is not the right metric? Change it to number*avg_size or something?

If you can't measure the thing directly, then take a look at stuff like "PID controller". This can be approach as a typical controller loop problem, although in 99% doing PID for software systems is an overkill.

> faster - less context switches and copies

This is very much newbie way of thinking. How do you know? Did you profile it?

It turns out there is surprisingly little dumb zero-copy potential at CF. Most of the stuff is TLS, so stuff needs to go through userspace anyway (kTLS exists, but I failed to actually use it, and what about QUIC).

Most of the cpu is burned on dumb things, like application logic. Turns out data copying and encryption and compression are actually pretty fast. I'm not saying these areas aren't ripe for optimization - but the majority of the cost was historically in much more obvious areas.

This happened before my watch, but I always was rooting for Linux. Linux is winning on many aspects. Consider the featureset of iptables (CF uses loads of stuff, from "comment" to "tproxy"), bpf for metrics is a killer (ebpf_exporter), bpf for DDoS (XDP), Tcp fast open, UDP segmentation stuff, kTLS (arguably half-working). Then there is non-networking things like Docker, virtio ecosystem (vhost), seccomp, namespaces (net namespace for testing network apps is awesome). And the list goes on. Not to mention hiring is easier for Linux admins.

Falsehoods programmers think about addresses:

- parsing addresses is well defined (try parsing ::1%3)

- since 127.0.0.2 is on loopback, ::2 surely also would be

- interface number on Linux is unique

- unix domain socket names are zero-terminated (abstract are not)

- sin6_flowinfo matters (it doens;t unless you opt-in with setsockopt)

- sin6_scope_id matters (it doesn't unless on site-local range)

(I wonder if scope_id would work on ipv4-mapped-IPv6, but if I remember right I checked and it didn't)

- In ipv4, scope_id doesnt exist (true but it can be achieved by binding to interface)

and so on...

Years ago I tried to document all the quirks I knew about https://idea.popcount.org/2019-12-06-addressing/

In simplest scenarios software is not aware of the IP space. Like you bind to 0.0.0.0:443 and move on.

In more sophisticated configs adding / removing IP's or TLS certs requires restarting server, configuring applications. This gets out of hand quickly. Like what if your server has primary IP removed, because the IP space is recycled.

At CF all these things were just a row in database, and systems were created to project it down to http server config, network card setting, BGP configurations, etc. All this is fully automated.

So an action like "adding an IP block" is super simple. This is unique. AFAIK everyone else in the industry, back in 2012, was treating IP's and TLS more like hardware. Like a disk. You install it once and it stays there for the lifetime of the server.

I totally agree. Early days Cloudflare was a great example of this. We treated IP addresses as data, not as configuration. New subnet? INSERT INTO and we're done. Blocked IP? DELETE FROM, and tadam. This was a huge differentiator from other CDN's and allowed us extreme flexibility. The real magic and complexity was with automatic generating and managing HTTPS certs (days before SNI).

Most of the responses here are knee-jerk.

I liked the article. It's close to my adverntures with containers. I think the invention of docker is indeed mostly packaging: Dockerfile (functional) is pretty neat, docker hub (addressing a container) is awesome, and the ENTRYPOINT in Dockerfile is great, it distinguishes Docker from .deb.

But indeed, beyond Dockerfile things are bleak. Docker compose never rose to my expectations. To get serious things you need load blancer, storage, addressing, and these are beyond traditional containers scope.