News for hackers. Stuff that matters.
Re:Growler: Asyncio Micro-Framework in Python by avyfain on Thursday May 05, 2016 @01:05AM (#11634102)
Awesome work! I just watched your PyOhio talk[0], and seems really promising. I use Flask daily, and while performance has not been an issue yet, I know the key word in that phrase is yet.

Would love to play with the code. Could you add a CONTRIBUTING.md? I could try to figure out how to get started on my own, but having a few pointers would be really helpful.

I see there are no issues on GH. What are things you're hoping to add soon/are not implemented yet?

[0] https://www.youtube.com/watch?v=vs7XOn1TXVQ

Re:Growler: Asyncio Micro-Framework in Python by akubera on Thursday May 05, 2016 @02:14AM (#11634296)
Thanks!

The biggest missing piece is simple, well-tested middleware.

My focus has been on very general middleware; a kind of 'meta-middleware' that actually spec out a common behavior to minimize surprises and boilerplate. For example, my Renderer & RenderEngine classes[0] don't do anything on their own, but are intended to be used by other, more specific renderers (eg MakoRenderer, JinjaRenderer). I think some common functionality really need to be standardized and not done ad-hoc, as conflicts could arise. (more description on how Renderers work below)

The next one of these I'd like to solve is authentication: a standard interface for passwords/oauth/etc. I've kinda used http://passportjs.org before, might be a starting point; I don't know what a pythonic version would look like.

## Non-Middleware

- Figure out equivalent of JS EventEmitter interface. Currently I have no good way to do something like res.on('send-headers', cb) ~ (thinking res.events['send-headers'].add(cb)) with events as EventManager object. Maybe decorators here?

- ETAGs

- HTTP/2

- pytest-growler for automated fixtures

- I don't think an admin panel is necessary (can't beat django) but would like some CLI tools for site creation, testing, and deployment.

Renderer Explaination: (too many 'render' words)

The author creates a RenderEngine as the actual middleware: app.use(MakoRenderer("/path/to/view")). All this middleware does is attach a callable Renderer object to res at res.render if it doesn't exist, then add itself to res.render - this allows multiple RenderEngines in the same app (if the author desires) - and none of them will clobber one another. Upon calling the Renderer: res.render('foo', data_obj), the callable scans through the engines until one with 'foo' is found, then the rendered text is sent. It's non-(obvious|trivial) situations like these I want in Growler proper (some batteries included) - the rest can be extensions.

[0] https://github.com/pyGrowler/Growler/blob/dev/growler/middle...

Re:Growler: Asyncio Micro-Framework in Python by akubera on Thursday May 05, 2016 @09:20AM (#11635799)
Also pressing is the method for turning "route strings" (I call them "sinatra paths", I'm not sure where they actually originated) into a regex

/foo/{bar} or /foo/:bar => re.compile(r'/foo/(?P<bar>[^/]+)')

https://github.com/pyGrowler/Growler/blob/dev/growler/router...

Re:Growler: Asyncio Micro-Framework in Python by gregn610 on Thursday May 05, 2016 @07:32AM (#11635194)
Impressive stuff. But um, about that project name; ever watched "Bo Selecta" https://en.wikipedia.org/wiki/Bo%27_Selecta! ?
Re:Growler: Asyncio Micro-Framework in Python by akubera on Thursday May 05, 2016 @08:18AM (#11635394)
No. But if it's related to the urbandictionary definition...

I'm taking back the word!

Re:Growler: Asyncio Micro-Framework in Python by sandGorgon on Thursday May 05, 2016 @01:19AM (#11634152)
a quick request (and something similar was discussed here - https://news.ycombinator.com/item?id=11626454)

could you add a representative example to show how we can use your framework with Postgres ? For example, psycopg vs psycopg2 vs psycogreen vs psycopg2-cffi vs aiopg ... I dont even know which one will work properly using your framework and how (and maintain async characteristics).

Re:Growler: Asyncio Micro-Framework in Python by Leynos on Thursday May 05, 2016 @03:24AM (#11634512)
(Comment from a bystander). You probably want aiopg in order to maintain the async characteristics and use the "friendly" asyncio syntax.

There is also peewee-async (http://peewee-async.readthedocs.io/en/latest/) if you want an ORM using asyncio. (I've been meaning to try this myself, but haven't had an opportunity).

Re:Growler: Asyncio Micro-Framework in Python by sandGorgon on Thursday May 05, 2016 @04:27AM (#11634651)
absolutely.. i am concerned about frameworks not fundamentally thinking about DB. With async, there is that additional problem of "this does not work in async mode". So I would hope the developers include database hookup as part of their unit test suite.
Re:Growler: Asyncio Micro-Framework in Python by phantom_oracle on Wednesday May 04, 2016 @07:57PM (#11632941)
How is this different to Twisted?
Re:Growler: Asyncio Micro-Framework in Python by akubera on Wednesday May 04, 2016 @08:22PM (#11633060)
As I understand, the asyncio module was inspired by Twisted.

Growler is, at its heart, a super minimal interface: passing (req, res) pairs through a pipeline of middleware (similar to node).

Since this is built with the standard library, all kinds of features, such as async DB access, will become available as the community builds them; and these may be added to your pipeline super cheap.

It's just an experiment.

Re:Growler: Asyncio Micro-Framework in Python by tcgarvin on Wednesday May 04, 2016 @08:16PM (#11633020)
It's asyncio native. Also I think syntactically it's different.
Re:Growler: Asyncio Micro-Framework in Python by akhilcacharya on Wednesday May 04, 2016 @08:50PM (#11633181)
Like Flask + Express, specifically.
Re:Growler: Asyncio Micro-Framework in Python by cdnsteve on Thursday May 05, 2016 @08:38AM (#11635515)
Could you clarify this vs aiohttp (aiohttp.web)?

http://aiohttp.readthedocs.io/en/stable/

Here's a good example of use: http://igordavydenko.com/talks/lvivpy-4/#slide-1

Re:Growler: Asyncio Micro-Framework in Python by akubera on Thursday May 05, 2016 @09:37AM (#11635927)
I have not used aiohttp, but it looks like their application must be initialized with a list of middleware_factories. I do not see any logic in choosing middleware - just a straight shot.

Growler apps just add functions to an internal list, and these may be 'mounted' on particular end points/http requests.

Also, in aiohttp it is up to your endpoint to create the response object and return it. While in Growler it is created by the application (or rather the GrowlerHttpResponder) and passed around the middleware - getting enhancements (res.render('foo') is not standard) and having callbacks registered (to, say, log ALL headers sent - something you want done only at the time of sending headers).

I think it would be possible (or at least I would like to get to the point) where you could actually mount other apps from other frameworks into a Growler app. So if there is blog software you like written in aiohttp, there is special middleware to transform growler requests into a aiohttp requests.

  import growler
  import aiohttp.web
  aio_blog_app = aiohttp.web.Application()
  ...
  app = growler.App("main site")
  app.use(GrowlerAiohttpMiddleware(aio_blog_app), path='/blog')
  ...

THAT would be kinda cool.
Re:Growler: Asyncio Micro-Framework in Python by tcgarvin on Wednesday May 04, 2016 @06:04PM (#11632292)
Does this work with uvloop?
Re:Growler: Asyncio Micro-Framework in Python by akubera on Wednesday May 04, 2016 @06:28PM (#11632450)
Yes!

An initial test last night of a trivial growler app (i.e. one method that replies with a simple string) saw a median response time drop from ~200ms to ~80ms.

(using apachebenchmark, n=1000 c=100)

Re:Growler: Asyncio Micro-Framework in Python by xxpor on Wednesday May 04, 2016 @07:26PM (#11632797)
Wait, p50 using the built in asyncio loop is 200ms?

That seems incredibly slow.

Re:Growler: Asyncio Micro-Framework in Python by zeemonkee3 on Wednesday May 04, 2016 @06:58PM (#11632650)
How well would it work with, say, PostgreSQL? Would you need to use async wrappers like aiopg in your views?
Re:Growler: Asyncio Micro-Framework in Python by akubera on Wednesday May 04, 2016 @07:14PM (#11632745)
For optimal performance, yes use aiopg or equivalent (it looks like there is an asyncio sqlalchemy project which might be nice.)

Any middleware may be a coroutine object, so it's as easy as `await res.render("whatever")` in an async function

Re:Growler: Asyncio Micro-Framework in Python by ssahoo on Wednesday May 04, 2016 @07:08PM (#11632712)
Re:Growler: Asyncio Micro-Framework in Python by sandGorgon on Thursday May 05, 2016 @01:22AM (#11634164)
do you have this in a github somewhere ? would love to check it out. Also would be nice to see this same thing with postgres DB access.
Re:Growler: Asyncio Micro-Framework in Python by sametmax on Wednesday May 04, 2016 @06:25PM (#11632429)
Does it blend ?
Re:Growler: Asyncio Micro-Framework in Python by hyperbovine on Wednesday May 04, 2016 @06:50PM (#11632600)
*Will
Re:Growler: Asyncio Micro-Framework in Python by bullen on Thursday May 05, 2016 @04:34AM (#11634666)
If you want this feature, but multithreaded and with hot-deploy + filesystem database clustered you can try: http://github.com/tinspin/rupy
Re:Growler: Asyncio Micro-Framework in Python by andybak on Thursday May 05, 2016 @06:29AM (#11634987)
Slightly disingenuous not to point out that's a Java app. I suspect the choice of language is a factor for a fair few people, maybe even the majority.
Re:Growler: Asyncio Micro-Framework in Python by bullen on Thursday May 05, 2016 @07:05AM (#11635097)
Yep, languages are like religion. But you can't hotdeploy any other language properly.
Re:Growler: Asyncio Micro-Framework in Python by fleetfox on Thursday May 05, 2016 @07:20AM (#11635149)
I'm not sure what exactly categorizes as "hot deploy" but how about erlang, elixir?
Re:Growler: Asyncio Micro-Framework in Python by bullen on Thursday May 05, 2016 @10:52AM (#11636590)
Rupy can switch code on the fly, meaning you can have users hitting your server and they will run the old code one request and new code the next. No other system I know of works like this, hot-deploying on a remote machine over HTTP seamlessly. Maybe erlang can do it but then again I can't write a 3D game in that language so Java still has the upper hand for my requirements.
Re:Growler: Asyncio Micro-Framework in Python by akubera on Thursday May 05, 2016 @11:38AM (#11637047)
Off the top of my head, if you write a custom MiddlewareChain class that exposed its own management port, you could inspect and modify the state of the chain at any time, checkboxes to disable middleware, change endpoints, insert debug middleware, etc.

I like this... definitely worth thinking about. Thanks, all!

Re:Growler: Asyncio Micro-Framework in Python by akubera on Thursday May 05, 2016 @11:57AM (#11637196)
Also, if you're talking about specifically replacing a source files (i.e. modules), one could hack around with https://docs.python.org/3/library/importlib.html#importlib.r.... I think it would work with enough indirection/factory functions.