Linux Raw Sockets

https://www.schoenitzer.de/blog/2018/Linux%20Raw%20Sockets.html

by nudin • 8 years ago

169 41 8 years ago

Once you create software that processes raw socket data, besides the endianness issues, the next thing you'll realize is the incredible breadth of RFCs that are covered for even "common" network traffic. Write your software to parse just one RFC, and a bunch of software using that protocol still won't work. Don't support the optional network layers that commercial network infrastructure splices into the packets, and you'll again miss a lot of traffic. Modern tcp/ip stacks are really complicated.

ChrisSD2• 8 years ago

I've found the IANA to be quite good at collating this information and producing best practices.

Personally I think the actual messaging protocols are usually relatively straightforward (all things considered) with sensible backwards compatibility. It's all the other stuff surrounding it that can get really complicated and break things.

sigjuice• 8 years ago

What are "optional network layers"? Do you have any examples? Thanks!

jzwinck• 8 years ago

VLAN is a popular one.

stagbeetle• 8 years ago

Some smart soul archived it: http://archive.is/nt67c

At this point, there should be a bot running that archives any post that starts getting traction. Or maybe have a HN serve a cached copy to offload the traffic? Who knows, it just seems like every blog post is killed the instant it reaches the front page.

johnramsden• 8 years ago

I'm not sure if you've ever visited https://lobste.rs, but it actually has this built-in - it caches every post. It's kind of like Hacker News except they've made some changes and added a bit more functionality.

striking• 8 years ago

I think something automatically does an archive.is of every front-paged HN post. Not sure who or what, though.

(site seems back up now)

nudin3• 8 years ago

Sorry, the server wasn't big enough to handle everyone coming from hn. But it's working again. BTW: The blog is also available via IPFS: /ipfs/QmdrFfK8yc6UkPK7aCcxAZzRM7K3ZP4JPfyjjbiHsHEDVH

adrianratnapala• 8 years ago

Didn't HTTP once have this cache thingy feature?

orf• 8 years ago

It did, I'll dig out my Firefox cache and send it to you?

deathanatos• 8 years ago

It does. But unless you have a CDN, caching is going to be per client. Since I've now visited the site, I have it cached, but that doesn't help the next unique visitor.

The site makes use of and respects cache headers, too. (And, I'll add, only makes 1¹ request aside from the main HTML, for some CSS; the site is refreshingly minimal, yet still looks nice.)

¹it also makes a request for MathJax, but that gets blocked b/c it's made over HTTP, and the site is HTTPS.

derefr• 8 years ago

Or maybe have a HN serve a cached copy to offload the traffic?

I've suggested this before; the usual problem people bring up in response is that it deprives these websites (which run on advertising) of their ad impressions.

Not honestly sure what percentage of Hacker News users are browsing without an ad-blocker, though...

FigBug• 8 years ago

If you have to deal with raw sockets, I highly recommend libtins: https://github.com/mfontanini/libtins Makes it very easy to support multiple platforms.

I recently had do support for a Digidesign Pro Control (https://medias.audiofanzine.com/images/normal/digidesign-pro...) that used it's own ethernet protocol. Libtins made it very easy and I didn't need to write my own driver or anything like that.

FraKtus• 8 years ago

"libtins depends on libpcap and OpenSSL, although the latter is not necessary if some features of the library are disabled."

Would you know something lighter to play with RAW sockets?

provost• 8 years ago

As mentioned.. Libtin’s OpenSSL dependency can be avoided if you can do without WPA2 support. Not sure about libpcap though

bb88• 8 years ago

I've used libpcap in the past, and it's been rather nice.

https://www.tcpdump.org/pcap.html

It abstracts all the hard work, gives you a handle that you can poll on, and then handles giving you a stream of packets to process.

encoderer• 8 years ago

This is great. Somewhat orthogonal, can anybody venture a guess on if we will ever be able to open raw sockets in AWS Lambda functions or if their continued restriction is necesary for their container security model?

jsjohnst• 8 years ago

What’s the use case?

jacbq6• 8 years ago

Other serverless platforms like CoherenceApi.com block raw sockets too.

I think there's probably too many security issues like ARP poisoning.

bthornbury• 8 years ago

Not to mention raw sockets bypass docker's default inter-container-communication blocking.

https://github.com/brthor/docker-layer2-icc

ibotty• 8 years ago

Aside: Can someone has any pointers on HIP? What it solves, who is behind it (is there a chance to have it supported in mainstream OSs).

bewo0012• 8 years ago

HIP stands for Host Identity Protocol. The idea is to separate a host's identity from its location. An IP address does both. It could be argued that DNS already achieves the ID/locator split: The DNS name of your host identifies it, the resolved IP address gives its current location.

nudin3• 8 years ago

Well DNS does not really split this because of the way TCP-connections are defined: The are the pair of IP and Ports of both sides. So TCP-connections are always between two locations not between two identities. That's the reason why connections get lost when changing from one network to another.

Also updating DNS entries is to slow to do it whenever you walk with your smartphone from one cell/wifi to another.

majewsky2• 8 years ago

Just in case the author is reading this: I can literally not read the article on my phone due to low contrast.

chrisseaton3• 8 years ago

The background is 99.4% white and the body text is 100% black. What more do you want?

squarefoot• 8 years ago

The problem might be due to antialiasing applied to the wrong font size. On my desktop the page is easy readable, but some letters have their vertical lines appear as two grey lines instead of a single black one or a mix of the two, in other words although it's 100% black text, the antialiasing renders some parts of it like 50% black to make it appear smoother. On a phone it might be even worse. Changing the font size can help.

Not an expert in anything about web publishing or similar stuff, so forgive if I used the wrong terms.

tssva• 8 years ago

Here are 2 screenshots from Chrome on Android displaying the article. The one in the left is how your article looks by default. The readable version on the right is how it appears when you accept Chrome's offer to show a "simplified version".

https://photos.app.goo.gl/ntvIcKSDTOfdkwAi1

nudin3• 8 years ago

Author here: the normal text is basically black on white. Do you mean the inline code parts? There contrast might be a bit to low, yes. I'm not happy in general with it's formatting and will change it in the future. (It's the default of the template I used) Thanks for the input!

chrisseaton3• 8 years ago

When you're picking colours you can use an online tool to test contrast against accessibility standards.

Your inline code parts still meet the double A level of the accessibility standard for contrast.

http://leaverou.github.io/contrast-ratio/#rgb%2882%2C89%2C93...

maskros• 8 years ago

It's not the color, it's the font that has too thin stems which can make it look grey depending on anti-aliasing and font hinting settings.

bebna• 8 years ago

It is worse on mobile firefox: https://imgur.com/a/WgS3d

majewsky2• 8 years ago

As others said, the problem appears to be with the font. A child comment of one of your sibling comments linked to a side-by-side comparison of how your site looks (left) vs. reader mode (which I forgot about when trying to read the article): https://photos.app.goo.gl/ntvIcKSDTOfdkwAi1

nemanjaboric• 8 years ago

Luckily it's easy enough to support IPv6: just replace AF_INET by AF_INET6 and it will work with both IPv4 and IPv6! So don't you dare to ever use AF_INET anymore without a good excuse

(emphasis mine)

AFAIK, on many systems (think FreeBSD) this is not true:

https://www.unix.com/man-page/FreeBSD/4/inet6/

By default, FreeBSD does not route IPv4 traffic to AF_INET6 sockets. The default behavior intentionally violates RFC2553 for security reasons. Listen to two sockets if you want to accept both IPv4 and IPv6 traffic. IPv4 traffic may be routed with certain per-socket/per-node configuration, however, it is not recommended to do so. Consult ip6(4) for details.

ChrisSD2• 8 years ago

I've consulted ip6(4) but I'm still unclear on exactly why they break with the rfc here. Do you know where they detail the security implications?

segmondy• 8 years ago

     >However, RFC2553 does not define the ordering constraint between calls to
     bind(2), nor how IPv4 TCP/UDP port	numbers	and IPv6 TCP/UDP port numbers
     relate to each other (should they be integrated or	separated).  Imple-
     mented behavior is	very different from kernel to kernel.  Therefore, it
     is	unwise to rely too much	upon the behavior of AF_INET6 wildcard bind
     sockets.  It is recommended to listen to two sockets, one for AF_INET and
     another for AF_INET6, when	you would like to accept both IPv4 and IPv6
     traffic.

     >It should also be noted that malicious parties can	take advantage of the
     complexity	presented above, and are able to bypass	access control,	if the
     target node routes	IPv4 traffic to	AF_INET6 socket.  Users	are advised to
     take care handling	connections from IPv4 mapped address to	AF_INET6 sock-
     ets.

hacknat• 8 years ago

Late to the convo, but I created a go library with no C dependencies that lets you play with raw sockets on Linux:

https://github.com/nathanjsweet/zsocket

lkrubner• 8 years ago

The article doesn't say why I would want to use raw sockets. The only place it comes close to giving a reason is at the end:

"This is the lowest we can get: this way ethernet frames are passed from the device driver without any changes to your application, including the full level 2 header. Likewise, when writing to the socket the user-supplied buffer hast to contain all the headers of layer 2 to 4. This is the deepest we can go in userspace – at this point we have full control of the complete ethernet frame. I hope you enjoyed our journey into the rabbit hole."

So, presumably, I'd prefer this to UDP in a situation where I need the complete ethernet frame. But when would that be? It would have been great if the mentioned a few scenarios where this is useful.

jandrese• 8 years ago

One use is if you need to replay captured traffic. Another is if you are doing protocol development or WAN acceleration.

Linux Raw Sockets

Related Stories