Re:Pirates Use Flask, the Navy Uses Django
by dgilland on Wednesday April 22, 2015 @05:29PM
(#9423708)
Interested in knowing what backend setup you have that allows you to scale up to 27,000+ while using Flask:
1. How many application servers are you using?
2. How much RAM, CPU cores, etc. does each application server have?
3. How many database servers?
4. Which web server are you using?
5. Which WSGI server are you using?
6. What are the configuration settings for your WSGI server (mainly the ones related to number of processes, threads, etc.)
My motivation for asking was to get a sense of what it takes to serve higher traffic from a Flask API. I've yet to find a good resource that mentions what sort of web server/WSGI configuration is needed to serve high load applications that use Flask. Obviously, proper load testing is needed to really determine a reasonable setup, but I think having something ballparked from an actual production deployment could be helpful.
1 m3.xlarge ec2 instance running PostgreSQL with 4 vCPUs and 15GiB ram. Make sure you tune Postgres because defaults won't use all the resources available.
Re:Pirates Use Flask, the Navy Uses Django
by sirn on Wednesday April 22, 2015 @07:25PM
(#9424210)
>Who uses Flask?
>Reddit
I'm pretty sure Reddit is running Pylons.[1][2]
That being said, I really wish Pyramid[1] (formerly repoze.bfg) would get a little bit more recognition in the Python web community. I particularly like Pyramid for its supports for Python 3 (even 3.2, which means it works well with PyPy3), good documentation (although sometimes a bit overwhelming) and the fact that the framework grow very well along with the app size. You can swap out or extend nearly everything in Pyramid, from renderers (Mako, Jinja, anything) to session object down to request objects, and may be even more.
It's true that these features are not unique to Pyramid and dependency with Zope components scare people off, but after few years of using several frameworks (started with web.py, then Django, Pylons, Werkzeug although not exactly a framework, Flask, etc.) I've found Pyramid to have the right balance between extensibility and simplicity. It could be a single file simple application if you want, or a big guy framework with authentication policy built-in if you ever needed one.
Interestingly, I got a tie between Flask and Django in the worksheet.
Re:Pirates Use Flask, the Navy Uses Django
by jqm on Wednesday April 22, 2015 @10:10PM
(#9424641)
I recently built a moderate sized project with Bottle. The application won't get many simultaneous users as it's for a single small company. The single file, no dependency setup was nice. It's similar to Flask but simpler... (in fact I think it was the inspiration for Flask if I'm not mistaken). I was even able to use Jinja2. Kind of missed Flask-Login, but Bottle had something adequate built right in. Worth a look for a small project.
Re:Pirates Use Flask, the Navy Uses Django
by wtbob on Thursday April 23, 2015 @09:48AM
(#9426605)
I really hate this 'pirates are cool' meme. Pirates were vicious high seas bandits: they robbed, kidnapped, raped and murdered. Navies hunted them down (and still do today[1]).
That said, after using Django for the better part of a decade, I'm all about Flask.
Re:Pirates Use Flask, the Navy Uses Django
by robertfw on Wednesday April 22, 2015 @04:55PM
(#9423483)
Flask is much more suited to my ideals (simple libraries over monolithic frameworks) and I started my current project out on Flask - but I ended up going back to Django out of pragmatism. My backend is a fairly simple data store accessed over a REST api, and django-rest-framework solves that use case brilliantly. The recent addition of integrated migrations sealed the deal.
Re:Pirates Use Flask, the Navy Uses Django
by gregthompsonjr on Wednesday April 22, 2015 @04:18PM
(#9423259)
Nice. Flask bias definitely felt as o read. Not sure if you've had to deal with a big Flask code base. I'd choose Django over Flask almost all the time. But I state that without producing evidence of superiority of Django to Flask. So, I guess this post is essentially moot.
1. How many application servers are you using?
2. How much RAM, CPU cores, etc. does each application server have?
3. How many database servers?
4. Which web server are you using?
5. Which WSGI server are you using?
6. What are the configuration settings for your WSGI server (mainly the ones related to number of processes, threads, etc.)
My motivation for asking was to get a sense of what it takes to serve higher traffic from a Flask API. I've yet to find a good resource that mentions what sort of web server/WSGI configuration is needed to serve high load applications that use Flask. Obviously, proper load testing is needed to really determine a reasonable setup, but I think having something ballparked from an actual production deployment could be helpful.
1
> 2. How much RAM, CPU cores, etc. does each application server have?
Using an m3.large ec2 instance so 2 vCPUs and 7.5 GiB ram (https://aws.amazon.com/ec2/instance-types/#m3)
> 3. How many database servers?
1 m3.xlarge ec2 instance running PostgreSQL with 4 vCPUs and 15GiB ram. Make sure you tune Postgres because defaults won't use all the resources available.
> 4. Which web server are you using?
nginx (http://nginx.com/)
My main config: https://gist.github.com/alanhamlett/09c95d1be18aa50dd3fb
My site config: https://gist.github.com/alanhamlett/712ff3d39131f7fdbba2
> 5. Which WSGI server are you using?
uWSGI (https://uwsgi-docs.readthedocs.org/en/latest/)
> 6. What are the configuration settings for your WSGI server (mainly the ones related to number of processes, threads, etc.)
My uWSGI site ini file (at /etc/uwsgi/enabled/wakatime.ini): https://gist.github.com/alanhamlett/f495a240de3013300563
My uWSGI main init file (at /etc/init/uwsgi.conf): https://gist.github.com/alanhamlett/e57b6b6fdaf3124fe29d
>Reddit
I'm pretty sure Reddit is running Pylons.[1][2]
That being said, I really wish Pyramid[1] (formerly repoze.bfg) would get a little bit more recognition in the Python web community. I particularly like Pyramid for its supports for Python 3 (even 3.2, which means it works well with PyPy3), good documentation (although sometimes a bit overwhelming) and the fact that the framework grow very well along with the app size. You can swap out or extend nearly everything in Pyramid, from renderers (Mako, Jinja, anything) to session object down to request objects, and may be even more.
It's true that these features are not unique to Pyramid and dependency with Zope components scare people off, but after few years of using several frameworks (started with web.py, then Django, Pylons, Werkzeug although not exactly a framework, Flask, etc.) I've found Pyramid to have the right balance between extensibility and simplicity. It could be a single file simple application if you want, or a big guy framework with authentication policy built-in if you ever needed one.
Interestingly, I got a tie between Flask and Django in the worksheet.
[1]: http://www.pylonsproject.org/projects/pylons-framework/about
[2]: https://github.com/reddit/reddit/search?utf8=✓&q=pylons
[3]: http://www.pylonsproject.org/projects/pyramid/about
That said, after using Django for the better part of a decade, I'm all about Flask.
[1] http://www.navy.mil/ah_online/antipiracy/index.html
If using Flask, check out Alembic[1] for SQLAlchemy schema migrations.
[1] https://alembic.readthedocs.org/en/latest/