Re:New Python REST API and CLI micro-framework
by timothycrosley on Thursday February 18, 2016 @11:33PM
(#11131558)
Author here: I encourage anyone that is curious about the reason for this framework as opposed to X to read the project's architecture document https://github.com/timothycrosley/hug/blob/develop/ARCHITECT.... Some highlights: it allows some of the most concise Python code, produces automatic documentation, supports annotation based validation and argument transformation, automatically enables version management, and is way faster then Flask / Django at handling requests. Also, I'm free to answer any questions you may have :)
Re:New Python REST API and CLI micro-framework
by seabrookmx on Thursday February 18, 2016 @11:57PM
(#11131649)
> produces automatic documentation, supports annotation based validation and argument transformation
These aren't new, though the approach to them seems quite clean.
> automatically enables version management
There's syntax sugar in the decorators for supporting versioned api's, sure.. but you still need separate functions for each version. I think the wording oversells the feature a bit.
> and is way faster then Flask / Django at handling requests
In what configuration? Pretty much anything is faster than Flask's included development server. Is it faster than Flask configured with nginx over wsgi? If so, how much faster? The best way to convince people of this is to not only post benchmark results, but also commit the benchmarks themselves so people can run them.
Project looks neat but Flask is well though out, well supported, has amazing documentation, and supports both Python 2.x and 3.x.. which leaves little reason to switch (with the exception of the CLI interface.. which I can't deny is pretty darn cool).
Re:New Python REST API and CLI micro-framework
by timothycrosley on Friday February 19, 2016 @12:04AM
(#11131680)
You can see the performance difference bench-marked by a competing framework non the less, here: http://pycnic.nullism.com/ These are done running a standard uwsgi setup for all frameworks (even playing ground).
> These aren't new, though the approach to them seems quite clean.
This is really the crux of it, nothing is new. That is before Flask etc you could do everything you can do with Flask now, but the approach was bad and not clean. Required a lot of duplication of work. Flask helped fix that. You can do the hug enables through flask now, but the approach isn't clean, they require a lot of duplication of work. Hug fixes that.
The same reasons people started using Flask are the same reasons they should start using hug. It provides a cleaner, faster, and better base for the APIs of the future - just like Flask did when it came along.
Re:New Python REST API and CLI micro-framework
by ergo14 on Friday February 19, 2016 @10:28AM
(#11133965)
I would be interested to see the benchmarks against Pyramid, because it tries to be really minimal while at same time it is very friendly and flexible if you want to write REST API's with it.
Re:New Python REST API and CLI micro-framework
by Ciantic on Friday February 19, 2016 @04:43AM
(#11132571)
How well it understands types? Could I generate TypeScript interfaces for each API call?
I'm not that interested in RESTfullness, but rather easily navigable API through auto-completion. E.g. /users/save/ would become "Users.Save()" in JavaScript/TypeScript, with interfaces.
Re:New Python REST API and CLI micro-framework
by timothycrosley on Thursday February 18, 2016 @11:54PM
(#11131639)
There's a bunch of reasons, but they really come down to a focus on memory efficiency and knowledge of how certain things perform on Python (using hasattr instead of isinstance where applicable, using __slots__, only processing things when it's clear they need to be processed). And then for extra measure: Cython compilation. You can see a competing frameworks benchmarks showing the result of all this work here: http://pycnic.nullism.com/
Re:New Python REST API and CLI micro-framework
by timothycrosley on Thursday February 18, 2016 @11:25PM
(#11131530)
author here: This is way more then Flask/Bottle, I understand what would make you reach that conclusion, but if you look here https://github.com/timothycrosley/hug/blob/develop/ARCHITECT... You can see just how much it adds that really are core concepts that can't be done with simple decorators on-top of Flask.
Highlights:
- Automatic documentation
- Type annotation based validation and transformation of input fields
- Directives to inject more data (like request data) into existing functions in a safe and reusable way
- It's easily twice as fast at handling requests as Flask, making Flask a rather horrible base for any competitor
Re:New Python REST API and CLI micro-framework
by gsibble on Friday February 19, 2016 @11:05AM
(#11134294)
I would argue that it handles requests twice as fast because it contains about half as many features. And I've added automatic documentation to many of my flask projects. Not knocking on your excellent project, just suggesting you stop knocking on Flask, which is an excellent wsgi server.
Re:New Python REST API and CLI micro-framework
by timothycrosley on Friday February 19, 2016 @11:33AM
(#11134491)
Hug contains the features flask contains. I haven't and I don't think you will in this entire thread see a single example of someone actually pointing a feature missing in hug that is relevant for APIs (not HTML websites like Jinja)
Re:New Python REST API and CLI micro-framework
by kstrauser on Friday February 19, 2016 @11:31AM
(#11134467)
Likewise. Flask surely isn't the end-all, be-all of web frameworks, but I've added autodocs to it for a past employer that included full autodiscovery. It's not exactly trivial but it's far from difficult.
Re:New Python REST API and CLI micro-framework
by austinjp on Friday February 19, 2016 @12:19PM
(#11134812)
This looks very interesting, thanks for posting. I love the simplicity of adding the CLI, versioning, and web API.
@timothycrosley, if you're still monitoring this thread... :) ... a handful of questions:
Is HUG "production ready"? Could or should I use this to power a service with thousands of paying users? Is anyone currently doing this?
Is HUG likely to be around in a couple of years? I'm sure we all hope that it will be :) I guess I'm curious about how much time and resources you can dedicate to this.
Perhaps in summary I should just ask: why wouldn't developers just use Django, for example? While Django may be comparatively complex it also brings a degree of confidence, being "battle hardened" and having a recent cash injection from Mozilla. (I'm sure other frameworks may have similar boasts.)
I appreciate that these questions might be seen as cynical or snarky -- they're not intended to be, I'm genuinely curious. There's a huge amount of frameworks churn at the moment. It seems like a precarious area to be launching a product, and some confidence-boosting info regarding longevity and production-suitability would be greatly appreciated.
Re:New Python REST API and CLI micro-framework
by timothycrosley on Friday February 19, 2016 @12:48PM
(#11135063)
Hi @austinjp,
I would be more then happy to answer your questions!
1) Is HUG "production ready?": YES! Several large companies have emailed me or chatted with the community for support for their already in production APIs. Based on my communications for instance, I know for a fact that one API hosted with hug is leveraged by at least 10 fortune 100 companies.
2) I have several projects, that I continue to support like isort which are several years old. I don't abondon projects, and hug is already at the point where it isn't my project but a community project which has upwards of 16 contributors already.
The real reason to use hug instead of Django is because your code and therefore your liability and maintance cost will be drastically lower. In every example where I've ported or seen an API ported it's been upwards of 90% code reduction. In several cases even if hug just all the sudden went away, your cost for maintaining hug itself would still be smaller then using code developed on-top of Django. It's a major enough improvement not to be able to be ignored.
Re:New Python REST API and CLI micro-framework
by peletiah on Friday February 19, 2016 @01:00AM
(#11131899)
Hmm, does it offer anything over Pyramid/Cornice? Pyramid is also quick and easy to start (i.e. "Micro") and allows to extend a project to a full featureset when necessary (Auth, Routing, Model,...).
edit: Sorry I don't add anything more constructive to the discussion. Hope author does not feel discouraged, writing OSS and then getting asked "What's the point?" sure is disheartening.
Re:New Python REST API and CLI micro-framework
by Phemist on Friday February 19, 2016 @05:53AM
(#11132788)
Thanks! This looks great, will definitely try it out.
One question, what would be the recommended way to define complex data types? E.g.
def stuff(arg1: int, arg2: dict):
return 'Ok'
where arg2 would be a dictionary that should contain specific, predefined fields? This might also instead be a general question about Python type hinting
Re:New Python REST API and CLI micro-framework
by kstrauser on Friday February 19, 2016 @11:34AM
(#11134492)
May I make a suggestion? Please, PLEASE use a standardized Python modeling tool like Schematics, Marshmallow, etc. as your description language. Then your users can use the same models for their REST framework as for their ORM and everything else.
I'm begging: please do not reinvent this. Just don't.
Re:New Python REST API and CLI micro-framework
by timothycrosley on Friday February 19, 2016 @11:41AM
(#11134556)
kstrauser: Hug supports marshmallow! Right now, you can use Marshmallow types out of the box for any API call just set the Marshmallow type as the type after the annotation and it will work and validate as expected :)
Re:New Python REST API and CLI micro-framework
by mixmastamyk on Friday February 19, 2016 @01:24AM
(#11131958)
It isn't forbidden, just that it might leak information, doesn't perform well, and is unsightly. Py 3.6 should have the syntax to simplify, without locals/globals:
Re:New Python REST API and CLI micro-framework
by dagss on Friday February 19, 2016 @01:59AM
(#11132064)
As for leaking info, your Py3.6 example leaks exactly as much (i.e., what is referenced in the literal). In fact you could consider it syntactic sugar for the locals() (although faster I guess).
Re:New Python REST API and CLI micro-framework
by jonathan_s on Friday February 19, 2016 @04:40AM
(#11132555)
Yes, but it's unsafe for another reason. The new syntax can be verified statically by tools like Pylint. A call to locals() is hard to verify.
The locals() function itself could for instance have been replaced by something else. When using locals(), you won't know until execution time if a local variable, required for the interpolation is missing. Even worse, linter tools (pylint, pyflakes, jedi, ...) are now going to tell you that certain variables are not used, and people are going to remove it without thinking that somewhere a locals() call is going to use it. This is very bad. Actually, the effects of using locals() cannot be verified statically, even more because locals() is also a writable dict.
For f-strings, the name bindings are static, and editors are going to understand it while editing.
Re:New Python REST API and CLI micro-framework
by stdbrouw on Friday February 19, 2016 @05:23AM
(#11132703)
> This is very bad.
There's no need for hyperbole. The fact that `locals()` doesn't work well with linters is a bummer, but compared to a multitude of other code smells it's pretty mild and harmless.
Re:New Python REST API and CLI micro-framework
by squiguy7 on Thursday February 18, 2016 @09:30PM
(#11131123)
At a quick glance, this reminds me of bottle. The fact that it uses falcon as a basis for the engine is great too. I get the feeling that you could write up an API with this in a day and it would perform well.
Re:New Python REST API and CLI micro-framework
by NamTaf on Thursday February 18, 2016 @11:34PM
(#11131564)
I used web.py to do a similar process about 1.5 years ago. In that, I mapped the GET and POST elements to functions which then did whatever I needed them to do.
This makes it even easier and I see it having a good fit in hacking up PoC APIs. In my case, I was querying some databases and aggregating data for display in a web page as well as via XML/JSON in a RESTful interface. This would make the XML/JSON stuff very easy indeed.
Re:New Python REST API and CLI micro-framework
by wanda on Thursday February 18, 2016 @10:44PM
(#11131387)
I'm not the author, but I'm pretty sure GIFs start to inflate when you increase colour depth.
GIFs we see on social networks are typically captured from videos of real-life objects, places and people, so the source colour-depth is pretty high (to account for the necessary shades created by varying exposure to light in three dimensions) and thus even a poor-quality GIF is going to be quite big to represent the source media with any discernible accuracy.
Since that terminal sports very few colours, it can be saved with low colour-depth and thence remain very small, despite size and length.
Re:New Python REST API and CLI micro-framework
by RussianCow on Friday February 19, 2016 @11:52AM
(#11134627)
It wouldn't be useful in this particular case because you wouldn't actually get to see the browser window (I guess you could curl it or something). But this is a really neat tool, I'll definitely be using it. Thanks for sharing!
Re:New Python REST API and CLI micro-framework
by leetrout on Thursday February 18, 2016 @11:06PM
(#11131459)
This looks neat. I like the 404 handler showing a spec!
The title confused me as I expected something to help me write a CLI app... If anyone else was thinking that way I highly recommend looking at pyinvoke.org or Argh.
Re:New Python REST API and CLI micro-framework
by jdnier on Friday February 19, 2016 @11:09AM
(#11134317)
So how would I use Hug as a secure public API -- https, authentication, authorization? Would I read up on Falcon? Are there any examples you could point to of how that might be done?
Re:New Python REST API and CLI micro-framework
by timothycrosley on Friday February 19, 2016 @05:49PM
(#11137335)
That's not how things work when using any WSGI based framework, the SSL part should be handled by either uwsgi / gunicorn or nginx. Development servers (hug or Flask or any other) should basically never be used directly. Examples can, and should be done using HTTP to allow local testing.
Re:New Python REST API and CLI micro-framework
by timeu on Friday February 19, 2016 @04:22AM
(#11132493)
That looks really cool !
I actually started to use Falcon for a relatively simple REST only backend (for a py3 project).
This might make my life a bit easier.
Re:New Python REST API and CLI micro-framework
by timothycrosley on Friday February 19, 2016 @11:36AM
(#11134513)
They really aren't compatible concepts. It's sort of like the difference between werkzeug and Flask, those projects wouldn't and shouldn't ever be merged. Falcon is like a much faster werkzeug with a great maintainer, we've discussed the projects together and both agree that they both should remain separate and both are mutually beneficial.
These aren't new, though the approach to them seems quite clean.
> automatically enables version management
There's syntax sugar in the decorators for supporting versioned api's, sure.. but you still need separate functions for each version. I think the wording oversells the feature a bit.
> and is way faster then Flask / Django at handling requests
In what configuration? Pretty much anything is faster than Flask's included development server. Is it faster than Flask configured with nginx over wsgi? If so, how much faster? The best way to convince people of this is to not only post benchmark results, but also commit the benchmarks themselves so people can run them.
Project looks neat but Flask is well though out, well supported, has amazing documentation, and supports both Python 2.x and 3.x.. which leaves little reason to switch (with the exception of the CLI interface.. which I can't deny is pretty darn cool).
> These aren't new, though the approach to them seems quite clean.
This is really the crux of it, nothing is new. That is before Flask etc you could do everything you can do with Flask now, but the approach was bad and not clean. Required a lot of duplication of work. Flask helped fix that. You can do the hug enables through flask now, but the approach isn't clean, they require a lot of duplication of work. Hug fixes that.
The same reasons people started using Flask are the same reasons they should start using hug. It provides a cleaner, faster, and better base for the APIs of the future - just like Flask did when it came along.
I'm not that interested in RESTfullness, but rather easily navigable API through auto-completion. E.g. /users/save/ would become "Users.Save()" in JavaScript/TypeScript, with interfaces.
Would it be silly to use both Flask and Hug?
[1] https://github.com/mozilla-services/cliquet
PS: how did you create the gif with the code?
The API versioning bit is interesting, it would work well as a Flask decorator.
Highlights:
- Automatic documentation - Type annotation based validation and transformation of input fields - Directives to inject more data (like request data) into existing functions in a safe and reusable way - It's easily twice as fast at handling requests as Flask, making Flask a rather horrible base for any competitor
@timothycrosley, if you're still monitoring this thread... :) ... a handful of questions:
Is HUG "production ready"? Could or should I use this to power a service with thousands of paying users? Is anyone currently doing this?
Is HUG likely to be around in a couple of years? I'm sure we all hope that it will be :) I guess I'm curious about how much time and resources you can dedicate to this.
Perhaps in summary I should just ask: why wouldn't developers just use Django, for example? While Django may be comparatively complex it also brings a degree of confidence, being "battle hardened" and having a recent cash injection from Mozilla. (I'm sure other frameworks may have similar boasts.)
I appreciate that these questions might be seen as cynical or snarky -- they're not intended to be, I'm genuinely curious. There's a huge amount of frameworks churn at the moment. It seems like a precarious area to be launching a product, and some confidence-boosting info regarding longevity and production-suitability would be greatly appreciated.
I would be more then happy to answer your questions!
1) Is HUG "production ready?": YES! Several large companies have emailed me or chatted with the community for support for their already in production APIs. Based on my communications for instance, I know for a fact that one API hosted with hug is leveraged by at least 10 fortune 100 companies.
2) I have several projects, that I continue to support like isort which are several years old. I don't abondon projects, and hug is already at the point where it isn't my project but a community project which has upwards of 16 contributors already.
The real reason to use hug instead of Django is because your code and therefore your liability and maintance cost will be drastically lower. In every example where I've ported or seen an API ported it's been upwards of 90% code reduction. In several cases even if hug just all the sudden went away, your cost for maintaining hug itself would still be smaller then using code developed on-top of Django. It's a major enough improvement not to be able to be ignored.
Thanks!
~Timothy
edit: Sorry I don't add anything more constructive to the discussion. Hope author does not feel discouraged, writing OSS and then getting asked "What's the point?" sure is disheartening.
One question, what would be the recommended way to define complex data types? E.g.
where arg2 would be a dictionary that should contain specific, predefined fields? This might also instead be a general question about Python type hintingwe are currently working on a way of doing this which should be released to the development version this week and will be released with version 2.0
basically the modification would allow you to create a new type which has multiple predefined fields. like so:
After creating this type you could then use it as a type as normalI'm begging: please do not reinvent this. Just don't.
This is generally discouraged, I would change these in the readme:
The locals() function itself could for instance have been replaced by something else. When using locals(), you won't know until execution time if a local variable, required for the interpolation is missing. Even worse, linter tools (pylint, pyflakes, jedi, ...) are now going to tell you that certain variables are not used, and people are going to remove it without thinking that somewhere a locals() call is going to use it. This is very bad. Actually, the effects of using locals() cannot be verified statically, even more because locals() is also a writable dict.
For f-strings, the name bindings are static, and editors are going to understand it while editing.
There's no need for hyperbole. The fact that `locals()` doesn't work well with linters is a bummer, but compared to a multitude of other code smells it's pretty mild and harmless.
So could .format, or any|everything. Also, where do you think Python looks for {foo}? I'll give you a hint, it starts with "l" and ends with "ocals".
It makes me think about what other interfaces could be automagically created at the same time -- C? Simulink?
This makes it even easier and I see it having a good fit in hacking up PoC APIs. In my case, I was querying some databases and aggregating data for display in a web page as well as via XML/JSON in a RESTful interface. This would make the XML/JSON stuff very easy indeed.
GIFs we see on social networks are typically captured from videos of real-life objects, places and people, so the source colour-depth is pretty high (to account for the necessary shades created by varying exposure to light in three dimensions) and thus even a poor-quality GIF is going to be quite big to represent the source media with any discernible accuracy.
Since that terminal sports very few colours, it can be saved with low colour-depth and thence remain very small, despite size and length.
https://asciinema.org/
https://github.com/timothycrosley/isort
(Emacs integration works very well!)
The title confused me as I expected something to help me write a CLI app... If anyone else was thinking that way I highly recommend looking at pyinvoke.org or Argh.
The real question should be, why take the time to write Falcon when you can get the same performance and more code-reuse using hug?
Looks like a nice step forward. I am looking forward to trying this out!