I'm not sure if today that really matters though. I will admit I don't know much about Perl but Perl's death vs Python's life probably might have some influence from design choices but a lot of it just has to do with the libraries you have access to with Python. I know as a scientist, Python is the game in town, that has more to do with why I use it than design.
It was really only during the break-away adoption by the data science community (and other mass audiences) that started adding pressure to bloat the language. The adoption of 3rd party libraries like numpy, which are so critical to modern Python, also destroyed the community's "batteries included" philosophy.
Older Python felt a lot like using FreeBSD, where the whole system is cohesive and designed to work together, whereas modern Python feels a lot more like Linux – a bunch of disparate systems that try to work well together (in the best of times).
Both approaches have their strengths, and the language has certainly improved in some areas, but I prefer the older style.
In any case, it is fascinating to see how mainstream Python is now. Maybe this change in mindset was required for it to take over the world. We've come so far since Paul Graham cited the language as esoteric and its usage a high signal of competence (http://www.paulgraham.com/pypar.html).
You are right and your FreeBSD vs Linux analogy really hits the target.
Thanks to the advancement of hardare and the development of libraries like Numpy, Python has moved from it's scripting niche and became a global, general purpose language.
Was that an error? If we look at the "batteries included" philosophy and the poor state of Python's package management after all these years...I'd say yes.
Since quality of life tools and other soft things are hard to prove, I'll tell my anecdata story point. I sought out any package manager experience in python. I landed on Pipenv which uses pip. It failed to solve the tree. This led me to find poetry and the reason for existing which is exactly my experience. That was 2-3 years ago.
https://github.com/python-poetry/poetry#why
Combine this with asdf and it aligns with most other languages.
asdf + yarn/npm = javascript
asdf + poetry = python
asdf + bundler = ruby
asdf + cargo = rust
asdf + mix = elixir
asdf + shards = crystal
asdf + go.mod (others) = go
asdf + composer = php
What's wrong with just using pip and requirements.txt? There's no dev section. In addition, bumping deps is not the same. I have a blog post explaining semver updates to a python dev:
https://squarism.com/2021/09/10/sciencing-out-updates/
my strong assertion: Python and Go missed it from the start. That's why it is so confusing. There's no other choice in Rust but Cargo. Rust devs are never confused on how to add a package, semver it. The answer is always Cargo. It's in the tutorial. It's in the book. It's in the culture.
I think I've heard that pip might support the pyproject spec, poetry already does. If you want scripts like npm, you can have that too with "taskipy". You don't have to.
Syntax integration would be great. It's such a hassle to learn two syntaxes and two data/object models to work with numpy -- as inevitably your code converts from Python objects <-> numpy objects at some point (singletons!), and it makes learners' lives much more difficult than needed.
I wonder if there was no adoption into the language at that time as a result of fractured communities and lack of consensus.
I think Python 3 almost killed Python, too, and for the same reasons. The developers behind 2to3 saved the community, in my opinion, as did the folks who made the painful decision to publish a Python 2 end-of-life date sufficiently far enough into the future that people still using the old runtime had both a sense of urgency and plenty of time to complete their migrations. I finally made the leap to Python 3 around 2016. I'd like to claim that I jumped instead of being pushed, but the reality is that by then, people were developing new stuff I wanted badly on Python 3 exclusively.
It's sad because there was a lot I liked about Perl. I wish that community had better leadership because from my quick glance at the Wikipedia page, it looks like the Perl 7 project is making the same mistakes as Perl 6/Raku did. It's really too bad.
Frankly, I believe the terrible state of affairs on CPAN was part of the problem, but it was initiated by the "there's more than one way to do it" mentality. If I wanted to do something where it would be a good idea to have a library for that task, I would have to search many candidates, each of which did sixty to eighty percent of whatever, then see which were long-abandoned, did what they actually managed to write match what I needed, etc.
Certainly not. Perl was always the language that had all the libraries. Python only started to get a large ecosystem way after Perl was doomed.
> The number of independent primitive concepts has been minimized in order that the language be easy to describe, to learn, and to implement. On the other hand, these concepts have been applied “orthogonally” in order to maximize the expressive power of the language while trying to avoid deleterious superfluities
Viewing through this lens, the Python Zen's "obvious way" refers to concepts in the language, and not the wider ecosystem which includes libraries.
Sure, but tests in the submission (not talking about further discussions) are only about the language. And with so much new syntax added over the years I doubt that there can be only one obvious way for many problems.
"I doubt" is just my feeling. I have not make any efforts to demonstrate this by examples. And it wouldn't be easy for me because I am not very fluent with many of the newer features of the language.
Here's my summary of the tests:
2.4: generator comprehensions; the obvious way if that's what you need
2.5: if-else expressions; a bit too easily used when if/else statement is more appropriate, but there are times where it's a good fit.
2.7: set notation: {0} is the obvious improvement over set([0])
3.0: ... as a token - I don't really understand why Python changed here. I only use it in indexing. It's not obvious when I would use it at all.
3.1: multiple context managers; the obvious improvement over a multiple indented managers
3.3: yield from list; the obvious improvement over for x in lst: yield lst
3.5: @ operator added for matrix multiplier. Solves a special pain point. Obvious only for that case.
3.6: underscores in numbers. Generally better for larger numbers. Mostly obvious.
3.7: async - I've not done async in Python yet, so can't judge
3.8: x:=expr; the debate that broke van Rossum. Not going there.
3.9: allow more complicated @annotation expressions; more useful than the alternative if that's what you need
3.10: match; no experience with it. I suspect it's more obvious
3.11: more async that I can't judge.
So, there can and should be lots of ways to do things. But always one obvious way, a Happy Path that's not obscured at all or whatever.
> There should be one – and preferably only one – obvious way to do it.
> Although that way may not be obvious at first unless you're Dutch.
The first is that I think actually achieving a hard-mode form of that quote and maintaining it through the lifetime of the language is impossible in practice and theory. (Even "mostly" living up to the quote is an achievement.)
The second is that I am unsure about the obvious qualifier. Certainly I can recognize pythonic and non-pythonic constructs in python; but whether the difference is actually obvious or just a result of experience combining with community consensus I can't say.
The third is that I don't think the language is Pareto efficient between readability and most other variables. For instance, I've had code criticized as non-pythonic that was just as readable as the offered pythonic version, but had greater run-time efficiency. (To be fair, this is only an issue if you define the 'pythonic' way to be the 'one obvious way'.)
All the other things built on top that people use Python for are very interoperable because they use the same base data type.
>>> import this
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!If you think about it as "strong suggestions", then yeah it seems obvious that they're suggestions for how to approach ambiguous situations. However, that's also missing the point a bit.
It's often more useful to think about it as "thought exercises when trying to understand the design philosophy". Note that it deliberately contradicts itself.
Similarly, things like "flat is better than nested" and "sparse is better than dense" are completely false a lot of the time. There are plenty of cases where the opposite is true for practical reasons like memory use or access patterns. The point is not necessarily to make those as statements or suggestions. The point is to give you something to think about. They're more like koans than suggestions.
Think about why and where they're not true as much as why and where they are. And don't take it too seriously, either way! The whole thing of "import this" is tongue in cheek, after all.
It is from a different age, when intelligent people like Tim Peters still had influence and there was more of an academic atmosphere. These days it is about warming chairs, getting power, eliminating your enemies and speaking at conferences.
NaMeSpAcES ArE ONe HONkInG gReAt IdEa
eg instead of:
namespace.py
class C1:
class C2:
namespace/__init__.py
from .c1 import C1
from .c2 import C2
namespace/c1.py
class C1:
namespace/c2.py
class C2:I would prefer if the repr reported its primary public API name, not its internal implementation location.
Quite apart from implementation challenges (eg, needing to parse every file in package upfront to know what's in there), how else would you see this working? Are there other interpreted languages that manage this indirection in a more elegant way than having some central file that supplies the mapping information?
Ruby has its own problems, but I like that the modules are independent from the file that contains them. You can “reopen” a module and declare new classes or constants or whatever. You have freedom (and responsibility) to organize your files in a way that maps to module namespaces. The drawback is the `require “my_file”` doesn’t give you any hint about what you’re importing.
IO.pm <-- author: Larry Wall
IO/File.pm <-- author: Me
IO/Socket.pm <-- author: You
IO/Socket/INET.pm <-- author: Larry Wall
IO/Socket/INET/Daemon.pm <-- author: Me
>>> import numpy
>>> P = numpy.polynomial.polynomial.Polynomial
I've always thought this ran counter to "Explicit is better than implicit".
The NumPy developers think "practicality beats purity" is more important, and their main use-case is long-running programs where startup costs are slow, which I interpret as meaning their special cases is special enough to break the rules.
People who expect things to go fast want it all cached up front, for very obvious reasons.
Seems straightforward.
For one example, I had a command-line tool which needed to compute something related to hypergeometric distribution. (It's been a few years; I forget the details.)
This available in scipy, which need numpy. Most of my program's run-time was spent importing numpy. (The following timings are best-of-3.)
% time python -c pass
0.027u 0.011s 0:00.04 75.0% 0+0k 0+0io 0pf+0w
% time python -c "import numpy"
0.212u 0.059s 0:00.17 152.9% 0+0k 0+0io 14pf+0w
% time python -c "import scipy"
0.252u 0.077s 0:00.32 100.0% 0+0k 0+0io 14pf+0w
Numpy is not about making short-lived programs fast.
In any case, this specific choice of importing all submodules is not to cache things up "for obvious [performance] reasons", because there is no machine performance improvements.
Instead, it's to make the API easier/faster to use. When you're in a notebook and you need numpy.foo.bar() you can just use it, and not have to go up and use an import statement first.
However, what I'd say is that a short launch from rest is an even more obscure use case than the one numpy supports.
numpy has made the correct compromise IMO, optimizing for long lived programs and notebooks.
"import urllib" does not also import urllib.parse, so you can't do:
import urllib
urllib.parse.quote("A&W")
scikit-learn supports essentially the same use cases as NumPy and it doesn't import all its subpackages:
>>> import sklearn
>>> sklearn.linear_model
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'sklearn' has no attribute 'linear_model'
>>> import sklearn.linear_model
>>> sklearn.linear_model
<module 'sklearn.linear_model' from '[...]sklearn/linear_model/__init__.py'>
> The NumPy developers think "practicality beats purity" is more important, and their main use-case is long-running programs where startup costs are slow, which I interpret as meaning their special cases is special enough to break the rules.
Oh you want to print to stderr hmm?
def module_from_path( ctx, name, path ):
### thx to https://stackoverflow.com/a/50395128/7568091 ###
### thx to https://stackoverflow.com/a/67692/7568091 ###
import importlib
import importlib.util
spec = importlib.util.spec_from_file_location( name, path )
module = importlib.util.module_from_spec( spec )
sys.modules[ spec.name ] = module
spec.loader.exec_module( module )
return importlib.import_module( name )
It would be nice if that flaw could be fixed, as I like coding in python in general
count = 0
t1 = time.time()
for line in open("nucleosome.pdb"):
if line =~ m/(ATOM |HETATM)/:
count += 1
print count, "atoms in", time.time()-t1, "seconds"
None of it would work now. :)
>>> ATOM = HETATM = 1
>>> m = 2
>>> m/(ATOM |HETATM)/1
2.0
All the Perl libraries for doing certain things (db access, web stuff, process management) were "actually battle tested" and worked well even if they were utterly unreadable. Python looked a lot nicer but when you tried to do any real work with it everything seemed to leak memory or give you continuous paper cuts - and it was a lot slower.
I made the switch to Python eventually but it took a while.
That tallies with when I was using Perl (late 90s / early 00s). Python was very much on the up in that time but I didn't feel compelled to switch, then DayJob and personal projects took me away from such thing completely (DayJob was Windows/VB6/IIS/ASP based, home web stuff went PHP for a while and admin stuff to Bash).
https://en.wikipedia.org/wiki/There%27s_more_than_one_way_to...
(It's a joke due from this page: http://james-iry.blogspot.com/2009/05/brief-incomplete-and-m..., actual motto is "there should be one -- and preferably only one -- obvious way to do it")
In fact, I think this makes it so all print() calls by default write to stderr:
>>> import sys, functools, builtins
>>> real_print = print
>>> builtins.print = functools.partial(real_print, file=sys.stderr)
>>> print("Hello")
Hello import logging
logging.basicConfig()
logging.info('something happened')Python 2.7.15 Linux (logging is 60x slower):
[root@hbtest ~]# py -m timeit -s 'import logging' "logging.info('something happened')"
1000000 loops, best of 3: 1.31 usec per loop
[root@hbtest ~]# py -m timeit -s 'import sys; debug = False' "if debug: print >>sys.stderr, 'something happened'"
10000000 loops, best of 3: 0.0216 usec per loop
$ py -m timeit -s 'import logging' "logging.info('something happened')"
1000000 loops, best of 3: 0.925 usec per loop
[jim@mbp hbrel]$ py -m timeit -s 'import sys; debug = False' "if debug: print >>sys.stderr, 'something happened'"
100000000 loops, best of 3: 0.013 usec per loop
[root@hbtest ~]# python3 -m timeit -s 'import logging' "logging.info('something happened')"
1000000 loops, best of 3: 1.62 usec per loop
[root@hbtest ~]# python3 -m timeit -s 'import sys; debug = False' "if debug: print >>sys.stderr, 'something happened'"
100000000 loops, best of 3: 0.0163 usec per loop
Perl
print STDERR "foo"
print(file=sys.stderr, "foo")
warn("foo")
In Python you have to at least:
- remember not to forget to import sys - remember the file= syntax for the print statement - type about 10 extra characters
Bunch of annoying stuff when you are hurriedly debugging something...
print >>sys.stderr, "foo"Which requires politics, churn, adding non-essential features and covering up mistakes by the old boys.
I remember back then it was said the design philosphy was something like there should be one obvious way to code something correctly. In opposite to Perl where the philosohpy was that human thoughts can take weird ways and you should be able to put your thoughts to code directly. (Both characterizations from memory.)
Today I hear little from Perl. And with new syntax added to Python in every version I start to wonder how far away Python is drifting away from that original characterization above.