Python Async IO Resources

by amarsahinovic • 12 years ago

33 6 12 years ago

nas• 12 years ago

I'm in the middle of doing an embedded control system project and I'm using Python 3.4 to do it. The embedded computer runs Debian Linux on an ARM processor. The Python program interfaces with the following: J1939 (CAN bus) network, dual RS232 serial ports, HTTP over TCP on wireless ethernet (for remote operator's panel). The HTTP server is Tornado. Most of the remote interface is done using Javascript.

I'm using Python 3 because I'm using the new asyncio library designed by Guido. So far, it is working out really well. There was some learning curve to get started but now it is very easy to add new modules to the system. I designed a very lightweight version of an internal message bus, similar to NASA's GMSEC. This allows separate modules to communicate without tight coupling between them.

The asyncio library doesn't have high-level support for socketcan so I had to use some lower level facilities. That took a little research and some digging through the asyncio source. The asynchronous design provides great responsiveness without heavy resource use (CPU or memory). I've done other similar systems using threads or polling but this Python 3 version is more elegant by far.

It would have been nicer if the migration path from Python 2 to 3 was smoother. However, after using it for serious work, Python 3 works great. I still wish 'print' was a keyword or there was some other debugging version of print (e.g. "dump var1, var1"). That's not a showstopper though.

ff_• 12 years ago

I discovered asyncio when I stumped on this [1] blog post (linked also at asyncio.org).

It was exactly what I needed for a personal project (a fast webscraping thing); before asyncio I went for the threads solution, but this thing performs one thousand times better.

I am really optimistic about this library, as the fact that is a core lib will make I/O with python A LOT easier.

[1] http://compiletoi.net/fast-scraping-in-python-with-asyncio.h...

kenrikm• 12 years ago

I've been using 3.4/Asyncio recently and coming from a background of C/Objective C it's hard to believe Python got by for so long without having something like Asyncio as a core package (yeah I know Gevent and Tornado have been around for awhile) It's still new"ish" (summer 2014) so there's not a lot of good documentation floating around yet, but it's easy enough to get the basics down if you've worked with Async stuff before.

lost-theory• 12 years ago

"asyncore" [1] has been in the stdlib since 1.5.2 (15 years ago!).

Twisted, gevent, tornado, etc. were newer projects and built to overcome the deficiencies and crudeness of asyncore. E.g.: https://mail.python.org/pipermail/python-ideas/2012-Septembe...

Asyncio is sort of a meta project to standardize and improve async in python, learning from all the work that's happened in the intervening years.

1: https://docs.python.org/3/library/asyncore.html

curiously• 12 years ago

I think Greenlets are fantastic but unfortunately worried of performance issues as I've read before. It has all the makings to being a node.js killer. Wish I could find that article explaining why greenlets are sometimes dangerous but I feel like the overall benefit of Greenlets outweigh the risks.

martius• 12 years ago

I'd like to know the risks, if you can find the article.

Python Async IO Resources

Related Stories