Fear and Loathing in Python: Building a Distributed Context System for Wool

https://gist.github.com/conradbzura/885a542ff0ccd548aa16fd05525a7a71

by bzurak • 22 days ago

2 1 22 days ago

gist.github.com

bzurak• 22 days ago

For those of you unfamiliar with Wool (which I'm assuming is most of you reading this), it's a distributed execution runtime for Python I've been working on that adds parallelism to Python's existing async primitives, allowing you to "await" CPU-bound logic without blocking the caller with a relatively simple service topology compared to some of the other distributed runtimes out there. Concerns like prioritization, queuing, and routing can be expressed in native Python, while Wool handles the inter-process/host transport.

This gist illustrates some of the trials and tribulations I went through while implementing a distributed context system for the runtime. Maybe some of you will find it (i.e., wool, the feature, or the lessons) useful, but in any case I enjoyed the process (I may be a masochist) and learned a lot.

Fear and Loathing in Python: Building a Distributed Context System for Wool

Related Stories