Network.framework: A Modern Alternative to Sockets (2018)

You can stream the video directly in VLC or Quicktime with https://devstreaming-cdn.apple.com/videos/wwdc/2018/715o2fzp.... Since its an m3u8, it includes multiple audio and subtitle tracks.

VLC: File -> Open Network Stream

Quicktime: File -> Open Location

layoutIfNeeded• 7 years ago

An API like this sorely needs async/await.

saagarjha3• 7 years ago

That would likely be a language overlay-level feature rather than API one.

merb• 7 years ago

well it's async. so async/await or a task system can be built ontop.

onefuncman2• 7 years ago

What kind, specifically?

async/await in C# is a flaming tire fire of hot garbage, it's more a virus than a help.

And why isn't an event driven API preferable?

kyberias• 7 years ago

Care to elaborate? I've been happily using async/await in C# without realizing how bad it is. It seems great, but maybe I don't just know enough. What should I know?

saagarjha3• 7 years ago

Documentation: https://developer.apple.com/documentation/network

exikyut• 7 years ago

Offtopic genuine question about self-improvement

This presenter shares a problem I find I have myself: I don't instinctively know how to make sure I'm getting enough air as I talk.

I've historically had a terrible time talking while walking (really hard), talking and standing (uncomfortable, like this guy), and standing and singing (gave up a very long time ago).

I'm usually okay, but on some days it's so bad I just don't have the oxygen and I find I'm predominantly yawns and hot air. Thing is, I don't know why, so while things have improved over the years I have no idea how to shove the needle over to where it needs to be.

Insight appreciated, thanks!

sergiomattei• 7 years ago

It's hard to explain through writing, but pause often to a) let the audience catch up and b) make a specific point much more memorable.

Brief pauses improve recall by the audience and make the talk much less boring. Strategic pauses are the key secret great speakers have: they utilize silence to their advantage.

Silence also adds another great opportunity to catch your breath. Dominate silence and you'll have the breathing part set.

idclip• 7 years ago

Ask yourself a few questions:m about your relationship to those you talk to, and how are you like as a story teller. How can you improve at telling stories, and do you ever practice reading your texts before you perform?

Your systems seem to be overloaded, and if you develop a storyteller (“come and listen to me” type of mentality) it would make you more at ease, and your natural skills would kick back in. You already know how to breathe, and its a matter of remembering by reducing your stress response.

Pigo• 7 years ago

I find myself gasping for air when I've done telephone interviews in the past. It's obviously a nerves thing, but it happens even though I'm aware of why it's happening. I think best while I'm pacing, so it doesn't help.

I think the problem is that I don't assume people will be patient while listening to me. I don't have a loud speaking voice, so I'm probably right a lot of the time. So I race to the finish of everything I say. But it's probably just that I don't do it that much. If I was interviewing everyday I'm sure I'd work the kinks out.

gotts• 7 years ago

It might be not that you're not getting enough air but you're getting too much by trying to control it artificially. What you described sounds like hyperventilation but I'm not a doctor and maybe you should find one.

isodude• 7 years ago

Three things come to mind. It seems weird though and I would recommend going to a doctor which could help you if there's anything physically wrong. Don't write it off, standing and singing should not be a problem.

Asthma?

There is a trick where you put your index finger on your fore head and thumb and middle finger on each side of the nose. Start by closing the lefy nostril and take a deep breath with the right one. Close the right nostril as well and hold, open the left nostril and breath out. Both inhale and exhale should be slow.

Do an easy task that need some thought and try to talk at the same time, try some different combinations. I can't talk if I do something that takes any form thought. It just could br hard and you need to practice, or you are just incapable and need to work around it instead.

carapace3• 7 years ago

There's breathing and then there's breathing.

Where are you breathing from, in your body? Are you taking small breaths just using your upper chest? Or do you breath from the diaphragm, filling the lower (and larger) parts of your lungs? Aka breath "from the belly". Lungs are sort-of bell-shaped: Image search of lungs: https://duckduckgo.com/?q=lungs&atb=v60-1&iax=images&ia=imag...

If you are hunching over and/or tensing up your torso that might be your problem right there?

rb808• 7 years ago

You shouldn't have to think about this. It could be your posture - I know when I relax and sit/stand correctly my first reaction is to have a big breath that feels great. Either a posture specialist (I had Alexander Technique coach) or maybe even a singing teacher.

nvrspyx• 7 years ago

My guess is you're breathing with your chest and not your diaphragm.

When you take a deep breath, do you notice your chest expand or your belly? If it's the former, try keeping your chest still and letting your diaphragm/stomach expand instead.

Otherwise, you might just be thinking about it too much. Try to focus solely on the talking and ignore the breathing.

loktarogar• 7 years ago

can anyone tl;dr? what is this, why is it better than sockets etc

js2• 7 years ago

From the introduction:

Network.framework is a modern alternative to sockets.

There are three primary areas in which it's very difficult to use sockets well. The first one is Connection Establishment. There's a whole host of reasons that establishing connections can be really difficult with sockets. For starters, sockets connect to addresses, so you have to, most of the time you have a host name, so you're going to have to resolve that host name to an address. When you do that, you often end up with way more than one address. You'll have some IPv4 addresses, some IPv6 addresses, and now you've got this challenge, which address should you try and connect to, in what order? How long do you wait before you try the next one? You can spend years trying to perfect this. I know because we have.

Once you get past the dual stack host problems, you run into a whole bunch of other issues.

There are some networks that use something called Proxy Automatic Configuration or PAC.

On these networks, there's a JavaScript that you get, and you have to pass a URL into the JavaScript, and the JavaScript runs and spits out an answer that says either you can go direct or you have to use this SOCKS proxy over here or that HTTP connect proxy over there.

And now your app has to support SOCKS proxies and HTTP connect proxies, and this can be really difficult to do well. And the most difficult thing is that you may not have one of these networks to test on, so you may get a bug report from one of your customers, and they may complain that it's not working well on their environment.

And you may want to add code to fix the problem, but that once you've got it in there, you really don't have a good way to test it. You have to end up building the whole environment to reproduce the same environment they have. It can be a real challenge. So connecting with sockets is really hard. The second thing that becomes challenges with sockets is data transfers. There's a lot of reasons that transferring data with sockets can be really difficult. The primary problem is the read and write model itself. If you're using blocking sockets, it pretty simple, but you're tying up a thread, and it's really not a great idea to be tying up a thread while you're waiting to read or write data. You can switch to nonblocking, but then you end up with a whole lot of other challenges that you run into.

When you're using nonblocking, you may tell the kernel I'd like 100 bytes, and the kernel will come back and say, I've got 10 bytes for you, why don't you come back later.

And now you have to build a state machine to keep track of how many bytes you read versus how many bytes you want to read. This can be a lot of work, and getting it to perform well can be a real challenge. On top of all of that, you really shouldn't be reading and writing to sockets directly because you should be using something like transport layer security or TLS. Sockets don't support TLS, so you're probably using some other library that is handling TLS for you and reading and writing to the sockets on your behalf, or you're writing the glue code between that library and sockets, and you have to figure out how to get all this to work with all the crazy connection logic that you put in ahead of time. There's a lot here that can be really difficult. Finally, mobility can be really challenges with sockets for a variety of reasons.

Today, we have these incredibly powerful devices in our pocket with multiple radios that may be on at the same time, and some of them are moving from network to network, and your application has to handle all these transitions well to provide a seamless experience to your customers. Sockets does nothing to help you with this.

You can use routing sockets, but it's really, really difficult. We think a transport API should do better. Fortunately, on our platform as an application developer you have a great API in URLSession. URLSession handles all of these problems for you.

It's really focused on HTTP, but it also has stream task that gives you raw access to TCP and TLS connections. You might think that URLSession is built on the same primitives that you would use yourself. But it turns out that's not the case. URLSession is built on top of something we call Network.framework. URLSession really focuses on all of the HTTP bits, and it offloads a lot of the transport functionality to Network.framework.

Network.framework is something we've been working on for a number of years, and in supporting URLSession, we've learned a lot, and we've taken a lot of those lessons to the IETF. A number of our engineers regularly participate in the IETF and meet with engineers from other companies, and they've been discussing a lot of what we've learned in the transport services working group.

And in those discussions, we've got some great feedback, and we've brought that back in and improved Network.framework based on that. We are really excited to announce this year that your applications can take advantage of this same library directly now. Now we know that one of the things people love about sockets is that it gives them very fine-grain control over just about everything, and they're really loathe to give that up. So as we developed Network.framework, we wanted to make sure that it did the right thing by default in the way that sockets doesn't, but it gave you all the knobs that sockets does. And it's kind of got this gradient, so the more knobs you turn, the more complex it becomes.

It gives you all the power you need, but you don't have to pay for the complexity unless you actually need some of it. Network.framework has incredibly smart connection establishment.

It handles the dual stack cases. It handles IPv6 only networks. It handles PAC. It handles proxies. It will help you connect on networks that are otherwise very difficult to deal with. It has an incredibly optimized data transfer path that lets you go way beyond the performance of anything you can do with sockets, and Tommy will cover that in a little bit. It has support for built-in security. It supports TLS and DTLS by default. It's really simple to use.

It has great support for mobility. It provides notifications about network changes that are relevant to the connections that your application is establishing. It's available on iOS, macOS, and tvOS as a CAPI with automatic reference counting, so it's easy to use from Objective C, and it has an incredible Swift API.

cocktailpeanuts• 7 years ago

he asked for a TLDR, i'm sure he can read too.

swiley2• 7 years ago

The dual stack thing is fixed with getaddrinfo. You’re probably speaking a protocol, I personally like to use regular languages with a reset byte (or octet or whatever.) at the beginning of every message. Since the data just goes into a state machine you can feed it partial messages and it will reconstruct them. This takes very little code, very little thought (just use Kleene’s algorithm) is extremely portable (unlike this which is exclusively apple) and very easy to mock and test.

Yes that still doesn’t take care of SOCKS but that (IMO) is heading into the “give the user an option to connect using pipes and netcat” territory (and that’s probably the best way to fix it.)

carapace3• 7 years ago

To me, that sounds like brutal epic turd-polishing.

Just use RINA:

The Recursive InterNetwork Architecture (RINA) is a new computer network architecture proposed as an alternative to the architecture of the currently mainstream Internet protocol suite. The RINA's fundamental principles are that computer networking is just Inter-Process Communication or IPC, and that layering should be done based on scope/scale, with a single recurring set of protocols, rather than function, with specialized protocols. The protocol instances in one layer interface with the protocol instances on higher and lower layers via new concepts and entities that effectively reify networking functions currently specific to protocols like BGP, OSPF and ARP. In this way, the RINA proposes to support features like mobility, multihoming and Quality of Service without the need for extra specialized protocols like RTP and UDP, as well as allow simplified network administration without the need for concepts like autonomous systems and NAT.

https://en.wikipedia.org/wiki/Recursive_InterNetwork_Archite...

joshbaptiste• 7 years ago

Prev (2018): https://news.ycombinator.com/item?id=17265288

bradhe• 7 years ago

Much of this actually seems a bit _more_ complicated than Berkeley sockets :\. The async features look easier to use...

dang• 7 years ago

A transcript is at https://asciiwwdc.com/2018/sessions/715. Should we change the URL above?

Edit: never mind; you can click to get a transcript via the submitted URL. I didn't see a way to link directly to it though.

jzl• 7 years ago

This is really cool. The telltale sign of a modern, robust phone app is its ability to gracefully handle constantly changing network connectivity. A number of big-name apps are still not very good at this. This framework will make it much easier to offload most of that logic to a well-designed API. I wonder if Android has something like this as well.

pedasmith2• 7 years ago

For a similar take on the problems, but with some different solutions, take a look at the UWP [StreamSocket](https://docs.microsoft.com/en-us/uwp/api/windows.networking....) (etc) classes.

Specifically, StreamSocket will happily go through proxies for you, will handle TLS / SSL (either at start up or after the connection is established), does IPv4 and IPv6 transparently. We also do service discovery using DNS-SD, although that's technically in a different class, as is the other network information. The SteamSocket class is designed to be very similar to the other socket-style classes like StreamWebSocket, so your code will (mostly) move from one to the other.

I've found from my own experience that the real key to a smooth network experience is the magic technology called "caring". It's easy to test if your code handles network disconnects; just run your code and yank the ethernet cable (or switch off Wi-Fi).

Source: am PM for the Network Developer Experience team at Microsoft and helped design the socket APIs.

nishotnew• 7 years ago

Is the DNS-SD API in windns.h under your purview? I have some gripes with it but don't know how to provide feedback.

pedasmith2• 7 years ago

No, but I know the team and will happily forward the feedback to them.

daniel-levin• 7 years ago

I write a lot of .NET, can confirm this is a great class

edit: any plans to add this (less Bluetooth support) to .NET standard?

Network.framework: A Modern Alternative to Sockets (2018)

Related Stories