News for hackers. Stuff that matters.
Re:Why People Should Learn Python by K0nserv on Wednesday August 24, 2016 @05:59AM (#12350846)
I switched to writing python as my main backend/scripting language with my new job. Previously I was using Ruby for most of this stuff with some Go. I must say that so far Python has been a vastly inferior experience for me. The languages are fairly similar all though I prefer the more talkative/english style of ruby and the use of functions over list and dict comprehension.

The thing that really stands out to me though is the poor state of python tooling and the library ecosystem. Having used Rubygems and bundler pip feels like taking an enormous step back. It's much less expressive. It doesn't handle the difference between production only dependencies, regular dependencies and development only dependencies in a good way. It's difficult to differentiate between locked dependencies vs desired dependencies(the Gemfile/Gemfile.lock distinction). PyPi and especially using private PyPi registers is more complex than it is in Ruby. There seems to be fewer nice libraries and they seem to be spread across the web whereas Ruby centralises around GitHub. I also find that Python libraries have lacking or hard to find documentation in many cases.

I wonder how much of this is due to the community spending a lot of time on the unproductive Python2/Python3 debacle instead of moving the language and the ecosystem forward. It seems like Ruby and Python would be in more similar positions had it not been for the problems around the Python 2/Python 3 upgrade. Ruby had the risk of a similar problems in the 1.8 to 1.9 upgrade, but it seems to have worked out better than the Python 2/Python 3 upgrade.

Re:Why People Should Learn Python by danso on Wednesday August 24, 2016 @07:13AM (#12351135)
You bring up a good point. I'm in the same boat as you in terms of switching from Ruby to Python. Unlike you I've really enjoyed the switch. But also unlike you I've chosen to ignore (i.e. stick my head in the ground) the issues with the dev-tooling. Not that I ever had a strong understanding of how Rubygems worked, other than knowing why I needed to do `bundle exec rake [etc]`. But I've largely delegated the pain of Python package management to Anaconda. Also, I haven't had to deploy any production public-facing apps in Python.

I don't share your experience on the documentation of libraries though. I've generally had no problems working with the major third-party libraries (requests, python-dateutils, Flask).

And I also am reluctant to feel strongly against how the 2v3 chasm has been drawn out, though it's easy to say that as someone who didn't have to make the jump. The main difference from Ruby 1.8->1.9, AFAICT, is that the Ruby community has a near-monolithic devotion to Rails. When Rails left 1.8 behind, not making the jump would basically be career-suicide for a Ruby dev. The downside is that it shows how dependent Ruby is on Rails' momentum. Whereas in Python, a scientist (among several other use-cases) can live on 2.x for a very long time no matter what all moves to 3.x.

Re:Why People Should Learn Python by rtpg on Wednesday August 24, 2016 @08:57AM (#12351649)
python had the opposite problem where so many major libraries didn't take make the jump to Python 3, so even if you really wanted to use Python 3 it was extremely restricting to do so.
Re:Why People Should Learn Python by barrkel on Wednesday August 24, 2016 @07:33AM (#12351238)
I find chained enumerator / block style code much easier to write and understand than list / dict comprehensions too. It's probably the single biggest thing that I don't like about Python; lambdas are simply too awkward, but lambdas + monadic transformations are probably my primary mental abstractions for writing software these days.
Re:Why People Should Learn Python by empthought on Wednesday August 24, 2016 @08:52AM (#12351618)
With itertools and functools you should be able to program in that style just fine. You'll likely want to give your lambdas names with a "def", but you should be doing that anyway.
Re:Why People Should Learn Python by barrkel on Wednesday August 24, 2016 @09:55AM (#12351998)
Giving my lambdas names is exactly what I don't want to do - the syntactic overhead is what kills the style, makes it an exercise in boilerplate. It's like Java closures before Java 1.8 - anonymous inner classes implementing interfaces are technically just about enough, but in practice entirely deficient, because the style is entirely unidiomatic.

The way my brain works (at this point in my career, 25 years of programming) is just against the Python grain. I think in terms of data flow graphs. Python wants me to think in terms of procedures and loops.

Re:Why People Should Learn Python by CuriouslyC on Wednesday August 24, 2016 @10:55AM (#12352471)
There are very good reason for naming functions (and ideally dangling them from the class rather than making them inner functions) - this encourages you to separate the 'what' from the 'how' and tends to result in code that is self-documenting. When you inline all your functions the person reading the code is forced to build and maintain multiple abstractions at one time to follow execution flow. Additionally, you are conflating the 'what' and the 'how' which forces the reader to context shift. If you replace the function with a well-thought-out name, code becomes much easier to read.
Re:Why People Should Learn Python by rtpg on Wednesday August 24, 2016 @09:01AM (#12351667)
As a fellow FPer who does the monadic stuff in other languages, I think it's usually more trouble than it's worth in Python. Instead, trying to use more of the python-specific language constructs can help you get what you want.

Python is pretty imperative, and the language design ethos is pretty against designing around functional programming. But if you try to absorb some of the python-specific stuff like list comprehensions, generators, and context managers, you can write some tooling to help you get a DSL a little closer to what you're used to.

Re:Why People Should Learn Python by dood on Wednesday August 24, 2016 @08:50AM (#12351611)
Could you give an example of that?
Re:Why People Should Learn Python by K0nserv on Wednesday August 24, 2016 @10:02AM (#12352060)
Imagine you have a list of products with prices and you want to calculate the cost of all products that are of a certain type. In ruby you can write this as

products.select { |product| product.type == 'x' }.map(&:price).reduce(:+)

Re:Why People Should Learn Python by minikomi on Wednesday August 24, 2016 @10:11AM (#12352126)
And in Python you can write something like:

    sum([p.price for p in products if p.type == "x"])
Excuse untested code from phone, but I like it better than the ruby version.
Re:Why People Should Learn Python by lmm on Wednesday August 24, 2016 @10:32AM (#12352308)
I find the order of iteration in Python for comprehensions totally confusing every time. If you want to convert:

    for a in x: pass
into a comprehension, you write:

    [pass for a in x]
Simple, right? So to convert

    for a in x: for b in y: pass
I obviously write:

    [pass for b in y for a in x]
Bzzt. Nope, you have to write it in middle-endian order to get the same behaviour.
Re:Why People Should Learn Python by amyjess on Wednesday August 24, 2016 @11:30AM (#12352737)
FYI, you can leave off the square brackets for a small performance boost. That turns it from a list comprehension into a generator expression, which is lazily evaluated and doesn't create a list container (which would immediately be thrown out anyway). It's like the difference between range(n) and xrange(n).
Re:Why People Should Learn Python by minikomi on Thursday August 25, 2016 @12:33AM (#12356986)
Nice, thanks for the tip.
Re:Why People Should Learn Python by K0nserv on Wednesday August 24, 2016 @10:27AM (#12352261)
I guess it's a personal style choice, the Python code is more performant(less iterations over the data set), but I like the ruby version better.
Re:Why People Should Learn Python by ezrast on Wednesday August 24, 2016 @08:59PM (#12356293)
It's only one iteration if you throw a .lazy in there:

products.lazy.select { |product| product.type == 'x' }.map(&:price).reduce(:+)

Re:Why People Should Learn Python by barrkel on Wednesday August 24, 2016 @09:50AM (#12351963)
Think: bash shell pipes. Transforming input to output through successive filters and projections, applied in a concatenated fashion and developed interactively in a REPL. Model algorithms as transformations or compositions of the same, where possible, applied to lazy or eager iterators or consumers (the dual of an iterator, that acts as a sink for output) as required. Model data sources and sinks (like databases and network streams) as iterators and consumers. Program by building a data flow graph out of increasingly sophisticated (by composition) stateless widgets, rather than imperative loops etc.
Re:Why People Should Learn Python by kennell on Wednesday August 24, 2016 @09:02AM (#12351679)
I prefer Python over Ruby as a language, but you are correct

gem ist superior to pip. Take a look at "pip-tools" if you haven't already, it eases some of the pain.

For web-dev Rails is more stable, more modern and has better maintained library ecosystem than Django. Django is okay-ish but can't live up to Rails. Django still lacks some stuff that i feel it should bring with it out of the box.... dev/test/production setups, asset compilation etc.

On the other hand, if you don't need a full blown web framework and are looking for something more minimalistic, Flask is awesome.

Re:Why People Should Learn Python by Stubb on Wednesday August 24, 2016 @09:49AM (#12351945)
Amazing that Python 2 vs. Python 3 has still not yet been sorted out. I dipped my toe into Python a couple years ago when looking for a general-purpose scripting language and quickly gave up in frustration—Python 3 was by far the preferable flavor, but many libraries simply didn't work. I picked up Ruby and haven't looked back.
Re:Why People Should Learn Python by busterarm on Wednesday August 24, 2016 @10:35AM (#12352326)
Likewise...and when I have to work in Python it's a frustrating pain because the tooling is so much better on the Ruby side.

Heck, the tooling is better on the PHP side even...

Re:Why People Should Learn Python by lastofus on Wednesday August 24, 2016 @12:30PM (#12353212)
A lot has improved since a couple years ago. A majority of major libraries have both Python 2 and 3 support these days. I've since started all new projects in Python 3.
Re:Why People Should Learn Python by justifier on Wednesday August 24, 2016 @02:07PM (#12353983)
i started with python when 3 dropped in ~2008.917

ignorant of any prior ecosystem i just thought i was getting into the most current

i came for the prebaked arbitrary precision floating point arithmetic and now i use it for prototyping all of my projects, and all the other stuff that i would call general scripting

this 2to3 complaint made sense to me from an employed standpoint:i work as X and use 2.7's Y library; but you said general scripting

i wonder what libraries were you hoping to use but were unable, and for what general scripting purpose?

Re:Why People Should Learn Python by Stubb on Wednesday August 24, 2016 @03:12PM (#12354478)
This was back in late 2013. My general scripting had to do with automating wireless network simulations to explore parameter spaces and slicing & dicing results stored in MySQL or SQLite databases.

I recall several showstoppers in Python v3 including integration into Emacs, accessing a MySQL database, and basic visualization. The library system was also a mess. It didn't seem possible to develop on v3 and expect that the libraries you needed would be available. Or that anyone else would care since they were all using v2.7.

Re:Why People Should Learn Python by rtpg on Wednesday August 24, 2016 @08:56AM (#12351645)
I think a lot of this is due to these problems being solved through standardised processes in python.

For example, with dependencies, production dependencies and dev dependencies are defined through two requirements files: requirements.txt and requirements-dev.txt. requirements-test.txt for testing. No need for tooling, just pass those into pip.

Most python tooling is pretty limited in scope but works well. From my side, I've had more trouble with "turing complete tooling" that ruby has offered me. I have not had to use ruby a great deal though.

I think the difference in philosophy between the two communities is that python stuff will usually value ease of debugging/implementation over "ease of use". Most things in pyhton and its echosystem are straightforward. None of this "Library A adding methods to classes of Library B" stuff (not that it's impossible). Of course, you then end up with more boilerplate.

Py2/Py3 has definitely been a major pain.

Re:Why People Should Learn Python by K0nserv on Wednesday August 24, 2016 @09:17AM (#12351755)
Yeah the requirements-{group}.txt convention kind of works, but what happens if you want to save only your frozen test dependencies for example? If you are using a single virtual env you end up freeezing everything in that env not only the test dependencies and its transitive dependencies.
Re:Why People Should Learn Python by euroclydon on Wednesday August 24, 2016 @06:22AM (#12350948)
I wish Ruby had the math and scientific library support of Python.

Also, I've been using Jupyter lately to brush up on data structures and algorithms. It's been pretty sweet to write code in a web notebook and render output including digraphs beneath each block of code. Is there anything like that for Ruby?

Re:Why People Should Learn Python by _Wintermute on Wednesday August 24, 2016 @07:05AM (#12351103)
You can use other kernels in jupyter notebooks. I've never tried ruby, but you could try this: https://github.com/SciRuby/iruby
Re:Why People Should Learn Python by LyndsySimon on Wednesday August 24, 2016 @10:54AM (#12352466)
Not only does Jupyter have Ruby support, but you can go back and forth between languages in the same notebook while sharing a context.
Re:Why People Should Learn Python by K0nserv on Wednesday August 24, 2016 @06:31AM (#12350978)
Not to my knowledge and this is definitely an area where Python is a lot better than Ruby from my experience. My comment pertains more to web development than scientific computing.
Re:Why People Should Learn Python by _joel on Wednesday August 24, 2016 @06:48AM (#12351034)
You can run a Ruby kernel in Jupyter - https://github.com/SciRuby/iruby
Re:Why People Should Learn Python by kozikow on Wednesday August 24, 2016 @07:42AM (#12351275)
> Having used Rubygems and bundler pip feels like taking an enormous step back.

Personally I moved away from relying on language specific tools to do more than just "install library X" and manage my deployment and dependencies using docker. So far it works great for me.

> There seems to be fewer nice libraries

Depends on your domain. Ruby is very good at web development. Python is good at almost everything.

> Python libraries have lacking or hard to find documentation in many cases.

I didn't use Ruby, but JVM languages or Javascript don't come close to Python in the documentation coverage. Especially if you consider SO as a documentation, googling "how do I do X" in python usually gets you on the right track to the high quality solution. Javascript or Java SO post are more like a mine field in comparison.

Re:Why People Should Learn Python by zemanel on Wednesday August 24, 2016 @07:54AM (#12351332)
Did you take a look at pip-tools ? It allows dependency locking
Re:Why People Should Learn Python by StavrosK on Wednesday August 24, 2016 @09:00AM (#12351661)
What do you guys mean by dependency locking? What I do with pip is add a requirements.txt with things like:

Django~=1.10.0

And it keeps the env autoupdated with the latest patch revision but not the latest minor revision. pip-tools doesn't seem to do much related to that, no?

Re:Why People Should Learn Python by K0nserv on Wednesday August 24, 2016 @09:14AM (#12351740)
The problem with that is that you no longer have reproducible builds. If you checkout a year old commit and built deployment artifacts with it you want the resulting artifact the be the same as when it was created. This is what you get with Bundler and the Gemfile/Gemfile.lock split

You put

gem 'rails', '~> 4.2.7'

in your Gemfile and when you run `bundler install` the exact version of rails you ended up resolving with the above is locked in your Gemfile.lock. Thus if you have a version in Gemfile.lock it is the source of truth and you have reproducible builds. If you want to update rails given the pattern in your gemfile you just issue `bundle update rails` and it will update to the latest conforming version and update your Gemfile.lock

The fact that pip doesn't support this in a good way is a major limitation imo. You can have either reproducible builds or "auto upgrading", but not both. pip-tools solves this, but requires you to use a different tool than pip and by extension you have to teach and explain it to anyone working on your project.

Re:Why People Should Learn Python by kozikow on Wednesday August 24, 2016 @09:44AM (#12351914)
I version my docker images. Inside the Dockerfile I usually install the latest version (i.e. pip install without specifying version). By single version bump of my image version I can trigger update of everything.

If I check out an old commit I can look up docker image version in the source control, download exactly the docker image that was running at that time and start shell in this image. Inside the image I can run "pip freeze" to check exact version of each package that was running at that time. I can then update the old Dockerfile with those specific versions by adding the "==x.y.z" e.g. "pip install oauth2client==3.0.0" in the pip install to reproduce the old build.

Additional benefit is that such scheme works for system packages, or for any other language libraries I can think of.

Re:Why People Should Learn Python by K0nserv on Wednesday August 24, 2016 @09:55AM (#12352005)
Yeah that's what I mean by either reproducible builds(e.g locking each dependency to an exact version. Output of pip freeze) or what the GP is talking about which is specifying a version pattern.
Re:Why People Should Learn Python by antod on Wednesday August 24, 2016 @08:50PM (#12356258)
I always though the recommended way to handle that split was by using setup.py as the Gemfile equivalent with very loose versioning rules, and for the Gemfile.lock case to not edit requirements.txt but to only generate it from pip freeze (or whatever it was - I haven't used Python for a while).
Re:Why People Should Learn Python by K0nserv on Thursday August 25, 2016 @06:38AM (#12358016)
Yeah I've heard people say this as well, but from my understanding setup.py is more aimed at libraries than end user projects. I¨ve never seen anyone use setup.py for an end user project in any case.

Interestingly enough in Ruby it's the reverse, you use gemspec for gems which doesn't have the concept of locking. Instead you are supposed to specify semver conforming version patterns and then resolution of these happens when the gem is installed. Only end user projects use Gemfile.lock

Re:Why People Should Learn Python by StavrosK on Wednesday August 24, 2016 @10:56AM (#12352486)
> You can have either reproducible builds or "auto upgrading", but not both.

Ah, okay, this crystallized it for me, thank you. Other commenters were talking about one or the other as a limitation, which confused me because pip does do both (just not at the same time).

Yes, this is currently a limitation, and pip-tools seems to solve it. Hopefully pip will get a native solution soon.

Re:Why People Should Learn Python by curiousssly on Wednesday August 24, 2016 @06:18AM (#12350929)
"There seems to be fewer nice libraries and they seem to be spread across the web whereas Ruby centralises around GitHub. I also find that Python libraries have lacking or hard to find documentation in many cases."

Could you clarify a bit more on this with the real world examples ?

Re:Why People Should Learn Python by K0nserv on Wednesday August 24, 2016 @06:31AM (#12350976)
Well for the case of "nicer libraries" I suppose I mean high quality libraries. Also good to note that I am referring to libraries in the web development/backend sphere here. I have a hard time thinking of some real world examples of there being a larger quantity of quality libraries for ruby, it's more a feeling that I get.

Around documentation and usage I think it comes down to the fact that for Python libraries a google search usually takes you to the modules page on the PyPi registry, to a project website or a read the docs instance. I usually find that where ever I end up doesn't provide all the information I want, e.g what's the latest version, what are some typical use cases for this library. As a concrete example let's compare Flask and Sinatra.

For both Flask and Sinatra a google search gives me their respective websites as the top relevant result. In this case I find both websites don't really provide all the information I am after. When I navigate to their GitHub page however the difference is more stark. Sinatra uses GitHub flavoured markdown to render a nice guide with increasingly more complex use cases. Flask just has some non markdown text pointing me in a few different directions. I could probably find all the information I need to setup and use Sinatra from just the GitHub Readme whereas for Flask I'd have to go navigate their read the docs page where all the documentation and usage instructions are spread over multiple pages and sub pages. I guess it comes down to my preference for the more concise format of a long markdown README.md vs the experience with a read the docs like set of pages.

I recon this is a bit rambling, it's kinda hard to put into words what my feelings around this are and why I hold the opinions I do.

Re:Why People Should Learn Python by korijn on Wednesday August 24, 2016 @06:15AM (#12350921)
Check out conda sometime.
Re:Why People Should Learn Python by K0nserv on Wednesday August 24, 2016 @06:51AM (#12351043)
That looks interesting, I've also tried out pip-tools[0] which is really nice. However nice these tools are it seems to a bit at odds with the Zen of Python. Ideally there should be just one single good way to deal with packages. Ironically this is the case for Ruby, but not at all for Python. Ultimately you can always use some alternative method that solves it better, but you then have to take on the burden of convincing your co-contributors that it is a better idea than using pip and you also have to explain it to anyone that joins your project.

0: https://github.com/nvie/pip-tools

Re:Why People Should Learn Python by IndianAstronaut on Wednesday August 24, 2016 @04:09PM (#12354848)
It can conflict with any native Python installations. So if you are running any pre built Python libraries which point to any system wide Python, it can be a nightmare. Hue is one example I ran into.