Growler: Asyncio Micro-Framework in Python

https://github.com/pyGrowler/Growler

by akubera • 10 years ago

116 30 10 years ago

github.com

tcgarvin2• 10 years ago

Does this work with uvloop?

sametmax• 10 years ago

Does it blend ?

hyperbovine• 10 years ago

*Will

akubera9• 10 years ago

Yes!

An initial test last night of a trivial growler app (i.e. one method that replies with a simple string) saw a median response time drop from ~200ms to ~80ms.

(using apachebenchmark, n=1000 c=100)

zeemonkee3• 10 years ago

How well would it work with, say, PostgreSQL? Would you need to use async wrappers like aiopg in your views?

xxpor• 10 years ago

Wait, p50 using the built in asyncio loop is 200ms?

That seems incredibly slow.

sandGorgon3• 10 years ago

do you have this in a github somewhere ? would love to check it out. Also would be nice to see this same thing with postgres DB access.

phantom_oracle• 10 years ago

How is this different to Twisted?

tcgarvin2• 10 years ago

It's asyncio native. Also I think syntactically it's different.

akhilcacharya• 10 years ago

Like Flask + Express, specifically.

akubera9• 10 years ago

As I understand, the asyncio module was inspired by Twisted.

Growler is, at its heart, a super minimal interface: passing (req, res) pairs through a pipeline of middleware (similar to node).

Since this is built with the standard library, all kinds of features, such as async DB access, will become available as the community builds them; and these may be added to your pipeline super cheap.

It's just an experiment.

avyfain• 10 years ago

Awesome work! I just watched your PyOhio talk[0], and seems really promising. I use Flask daily, and while performance has not been an issue yet, I know the key word in that phrase is yet.

Would love to play with the code. Could you add a CONTRIBUTING.md? I could try to figure out how to get started on my own, but having a few pointers would be really helpful.

I see there are no issues on GH. What are things you're hoping to add soon/are not implemented yet?

[0] https://www.youtube.com/watch?v=vs7XOn1TXVQ

akubera9• 10 years ago

Thanks!

The biggest missing piece is simple, well-tested middleware.

My focus has been on very general middleware; a kind of 'meta-middleware' that actually spec out a common behavior to minimize surprises and boilerplate. For example, my Renderer & RenderEngine classes[0] don't do anything on their own, but are intended to be used by other, more specific renderers (eg MakoRenderer, JinjaRenderer). I think some common functionality really need to be standardized and not done ad-hoc, as conflicts could arise. (more description on how Renderers work below)

The next one of these I'd like to solve is authentication: a standard interface for passwords/oauth/etc. I've kinda used http://passportjs.org before, might be a starting point; I don't know what a pythonic version would look like.

## Non-Middleware

- Figure out equivalent of JS EventEmitter interface. Currently I have no good way to do something like res.on('send-headers', cb) ~ (thinking res.events['send-headers'].add(cb)) with events as EventManager object. Maybe decorators here?

- ETAGs

- HTTP/2

- pytest-growler for automated fixtures

- I don't think an admin panel is necessary (can't beat django) but would like some CLI tools for site creation, testing, and deployment.

Renderer Explaination: (too many 'render' words)

The author creates a RenderEngine as the actual middleware: app.use(MakoRenderer("/path/to/view")). All this middleware does is attach a callable Renderer object to res at res.render if it doesn't exist, then add itself to res.render - this allows multiple RenderEngines in the same app (if the author desires) - and none of them will clobber one another. Upon calling the Renderer: res.render('foo', data_obj), the callable scans through the engines until one with 'foo' is found, then the rendered text is sent. It's non-(obvious|trivial) situations like these I want in Growler proper (some batteries included) - the rest can be extensions.

[0] https://github.com/pyGrowler/Growler/blob/dev/growler/middle...

akubera9• 10 years ago

Also pressing is the method for turning "route strings" (I call them "sinatra paths", I'm not sure where they actually originated) into a regex

/foo/{bar} or /foo/:bar => re.compile(r'/foo/(?P<bar>[^/]+)')

https://github.com/pyGrowler/Growler/blob/dev/growler/router...

sandGorgon3• 10 years ago

a quick request (and something similar was discussed here - https://news.ycombinator.com/item?id=11626454)

could you add a representative example to show how we can use your framework with Postgres ? For example, psycopg vs psycopg2 vs psycogreen vs psycopg2-cffi vs aiopg ... I dont even know which one will work properly using your framework and how (and maintain async characteristics).

Leynos• 10 years ago

(Comment from a bystander). You probably want aiopg in order to maintain the async characteristics and use the "friendly" asyncio syntax.

There is also peewee-async (http://peewee-async.readthedocs.io/en/latest/) if you want an ORM using asyncio. (I've been meaning to try this myself, but haven't had an opportunity).

sandGorgon3• 10 years ago

absolutely.. i am concerned about frameworks not fundamentally thinking about DB. With async, there is that additional problem of "this does not work in async mode". So I would hope the developers include database hookup as part of their unit test suite.

bullen3• 10 years ago

If you want this feature, but multithreaded and with hot-deploy + filesystem database clustered you can try: http://github.com/tinspin/rupy

andybak• 10 years ago

Slightly disingenuous not to point out that's a Java app. I suspect the choice of language is a factor for a fair few people, maybe even the majority.

bullen3• 10 years ago

Yep, languages are like religion. But you can't hotdeploy any other language properly.

gregn610• 10 years ago

Impressive stuff. But um, about that project name; ever watched "Bo Selecta" https://en.wikipedia.org/wiki/Bo%27_Selecta! ?

akubera9• 10 years ago

No. But if it's related to the urbandictionary definition...

I'm taking back the word!

cdnsteve• 10 years ago

Could you clarify this vs aiohttp (aiohttp.web)?

http://aiohttp.readthedocs.io/en/stable/

Here's a good example of use: http://igordavydenko.com/talks/lvivpy-4/#slide-1

akubera9• 10 years ago

I have not used aiohttp, but it looks like their application must be initialized with a list of middleware_factories. I do not see any logic in choosing middleware - just a straight shot.

Growler apps just add functions to an internal list, and these may be 'mounted' on particular end points/http requests.

Also, in aiohttp it is up to your endpoint to create the response object and return it. While in Growler it is created by the application (or rather the GrowlerHttpResponder) and passed around the middleware - getting enhancements (res.render('foo') is not standard) and having callbacks registered (to, say, log ALL headers sent - something you want done only at the time of sending headers).

I think it would be possible (or at least I would like to get to the point) where you could actually mount other apps from other frameworks into a Growler app. So if there is blog software you like written in aiohttp, there is special middleware to transform growler requests into a aiohttp requests.

  import growler
  import aiohttp.web
  aio_blog_app = aiohttp.web.Application()
  ...
  app = growler.App("main site")
  app.use(GrowlerAiohttpMiddleware(aio_blog_app), path='/blog')
  ...

THAT would be kinda cool.

Growler: Asyncio Micro-Framework in Python

Related Stories