My take on the matter is that Python is gaining traction due to its market share of the Data Science/Machine Learning field, which is what's really gaining traction and growing quickly. Had Perl or Ruby been the premier language to use for DS/ML, they would be growing instead of Python.

Also many things about python that I don't like such as `len` being a function rather than a method on a list are very trivial when coming from a language like C which has hordes of tiny inconsistencies.

The whitespace thing in python is pretty gimmicky but does also hold a certain amount of pedantic appeal.

Take a look at the design rationale for the new __matmul__ magic method. It's not for the standard library, it's for the community.

But you can also see that Guido has been into types lately. He's been pushing the development of mypy and the new optional type system.

Rails is amazing and I personally love it. But it's not the only thing Ruby is good for.

I dislike the many inconsistencies in Python. You mentioned one. Others: sort vs sorted (destructive vs non destructive). Also: not being able to chain methods because many don't return a value. (Scheme, Elixir or even Ruby are imho much more elegant.)

But all in all Python is a very useful programming language with a rich eco system.

https://docs.python.org/2/faq/design.html#why-doesn-t-list-s...

Nevertheless I find this here much more elegant:

data.sort.tail.first # or this data.append(7).append(3)

More elegant than this:

sorted(data)[1:][0]

data.append(7) # and then

data.append(3)

I love method chaining in Elixir or Javascript. In Python I have to create temp variables here and there to capture / further process intermediate results.

Having said that: Python has it's strengths. I use it more than any other programming language.

    if (x=42) {}

> sorted(data)[1:][0]

I don't want to get into the weeds, but shouldn't that just be

    sorted(data)[1]

    first, second = sorted(data)[:2]

?

Well Perl was the premiere data science language for a while. Python made inroads and eventually took over. I think you're mostly right through, it's just a matter of other factors affecting the communities at the time causing Perl to not grow nearly as quick as Python, and Python getting a good numeric library. Had Perl not lost a lot of community in the early 2000's, it might have retained this area.

Well, Python is the scripting language of a ton of useful libraries which are relevant to a lot of very interesting fields.

R sits in a similar spot, but Python is a better, friendlier language, which means actual programmers are more apt to recommend it as an offhand solution to a simple problem someone who isn't seen as a programmer is having: "Oh, that's easy. Just copy and paste this Python code, and maybe hack at it until it does something." You can get scientists programming with recommendations like that.

https://www.scipy.org/scipylib/building/linux.html

Of course, the people who just install and run SciPy don't need to know that, but the fact Python has a nice C FFI means it can run literally decades' worth of software across a fair number of languages.

There are so many lovely and easy to use libraries in Python. I took a couple and tried to be inspired by them to write a C++ library, maybe for addition to boost.

However, if you are writing C++, people expect your library to be fast and memory efficient -- you can't go around sticking the entirety of a parsed file in a std::string. Also, people then start saying your code better interoperate with their templated classes well, and they want to replace the memory management entirely, and it goes on...

Of course, the resulting libraries are very fast and very clever, but they have lost the joy. They also take much longer to write, maintain and use.

Painting with broader brushstrokes with less concern for micro-efficiencies lets you block out a solution faster. Another part of the problem with C++ is that the components come with so many rough edges that need to be filed down that you can't help getting swamped in a certain level of detail, unless you start with a large toolbox or framework that lets you work back at a higher level again.

You ought to have the equivalent of Java's file stream, buffer and text decoder as three separate bits that you can compose, so that putting a whole file in a string is never even a temptation. But if you start out with a sparse toolbox, it's a slog.

Performance grants you some freedom.

Python encourages readable code, which enables smarter algorithms, which ultimately swamps all those little memory optimizations. Oh, and PyPy/Numba/Cython let you get those little optimizations, too.

I think the readable code thing is really really important, too. I can understand other peoples code quicker and learn more about the actual algorithm unlike in C++ where I wonder half of the time about syntactical non-sense and optimizations for certain compiler targets.

Then again, Python and C++ target very different groups and I like both languages for certain things.

I'd really like to know what they were doing and how they could do it so wrong that the C++ version took hours while the Python version was faster than a blink of the eye.

Similarly, naybe the C++ code was using an inefficient data structure for a large dataset compared to using something like Numpy.

The rewrite realized an approximation was reasonable in place of a more "correct" algorithm.

Note that I'm not saying Python was the cause, but the ease of reading and writing code may have helped the programmer come to this algorithm. Instead of stressing about bugs and deadlines. Algorithm choice seems correlated with language choice, in my experience.

Code rewrites are often more than a literal translation. They often eliminate old code, and allow to rethink the architecture. So it's really not a surprise that Python code would be faster here. A rewrite to C++ would probably be much faster still.

C++ libraries should allow people to squeeze every bit of performance juice out of them (e.g. custom allocator).

Python libraries should be intuitive to use.

---

Naturally, there are exceptions, like numpy.

You might use it to make a rope bridge across that performance gap, but how good your bridge is depends a lot on you - and if you just manage to hang yourself with it, well, that's on you as well.

With Python it may take longer to get across because you detour a mile over to a more solidly constructed bridge, but you still get there and for most cases you get there fast enough.

Sure you can. C++ is at best programmed like python. Just throw the working first version together - and once you hit a performance gap, you can optimize it.

Using a profiler.

Premature optimization is mostly pointless because for most non-trivial performance critical apps the bottleneck wont be where you expect it.

The faster you find the actual problematic hotspots with production content the better. The way to get there fast is to treat C++ like filthy python.

Don't trust anyone trying to goad you into premature optimization unlesa they understand the specific problem you are trying to solve, and only then with skepticism.

Why would anyone want to write code that would go into either of those? In the general sense, I mean.

Good production code is simple and is implemented preferably verbosely and understandably than cleverly.

Speaking of good software patterns, I'm not sure boost nor STL are great examples of good software architecture. Boost especially is very templatey-bloatey. The biggest merit of STL is that is exists.

"installing libraries can be fiddly."

Does anyone really need to do that? Install C++ specific libraries I mean. In 10+ years I've never developed C++ software that would not store all of it's dependencies part of the project hierarchy or sub-structure, except for the standard library.

The worst part of C++ is that is has these warts that subtly guide developers into copying all sorts of patterns for their own sake because they are seen "as the standard way to do it". Object orientation and templates should not be considered the basic building blocks of the archictecture of a C++ program, but rather escape valves when some usage pattern would end up more complex without them.

For example, when someone spends long production time creating an elegant template based solution for whatever, generally it is an indication that something is wrong somewhere. There are of course exceptions to this rule, but just saving some boilerplate is never a good reason to do it.

The STL is a library of common algorithms and data structures that are decoupled from one another, and the complexity of operations in the STL has been enshrined in to the C++ standard. These two achievements alone warrant granting it huge respect. Imho, it's the essence of great design and it should be the very first thing taught to programmers new to C++.

> copying all sorts of patterns for their own sake because they are seen "as the standard way to do it".

Yeah, Python programmers would never do anything like that. If they did they might all run around calling it being 'Pythonic' or something.

> Object orientation and templates should not be considered the basic building blocks of the archictecture of a C++ program

Just tools in the toolbox, and the amount of mileage you can get out of just these two features, along with other staples like RAII and function overloading is pretty astounding.

C++ pushes you to conflate optimization with the higher level meaning of what you're trying to implement. In Python you can write the higher level meaning in a more 'pure' way, which is fun of course. The really interesting question is if you can program at the higher level while still running fast, perhaps by specifying optimizations separately?

(BTW, for Python code that can be as fast at C++ see https://kratzert.github.io/2017/09/12/introduction-to-the-nu... - it doesn't sacrifice clarity, but is limited to a subset, for now).

Around 2009 I started questioning if I even enjoyed coding anymore. I was primarily using PHP and Java. Rather than quit the profession I tried Python and suddenly it was fun again. 8 years later, it still is. Python just spoke my language.

My career is based around using Python because it's the way I enjoy the work. Everything else, like the actual properties of Python relative to other languages, is irrelevant to me if using another language would make the work unenjoyable again. (Though if Python didn't measure up very well compared to other languages it wouldn't have become my go-to in the first place.)

Nowadays the panorama for good languages, thankfully, is much better. PHP is no longer that much of a dumpster fire with modern frameworks, and Java has decided to adopt better defaults and drop the XML madness. Not to mention all the good languages that have come up (Elixir, Clojure, Rust, etc).

That said, it seems foolish to ignore the clear data science use case. But that's also been a theme for at least five years now.

In some sense - there are no large projects. Only projects that have failed at being modular.

Maintainability involves keeping clear boundaries between functional units so that you can keep enough code in your head at any one time to reason about it.

If we assume that Python is capable of handling more code than you can keep in your head at any given time, then surely the problem is simply one of making sure your modules are nicely self-contained from each other.

Write function in module A that expects some complex data structure that comes in. In a static typed language, you put a type on the argument and thus guarantee, to some degree, the data in the object can be safely manipulated.

Over in module B we call that function. Because Python doesn't enforce types, I can pass in some type of data structure that might behave well in many situations, but in some that function is going to fail because I didn't properly construct the data structure.

How do other Python projects avoid this? Do they avoid large data structures of multiple fields?

Let's question some of the assumptions. Do you NEED the entire thing to be passed around? If your functions are small and only do one thing, can't they be fed the portion of the data they need and only manipulate that? If they are getting unnecessary data, soon someone will like the convenience and that piece of data will now become necessary.

I have seen Java projects with humongous "DAO" objects being passed across all apps. Even if Java had Haskell's type system, at some point it breaks down.

So, you do not enforce that on Python itself. Rather, you have to apply due diligence and ensure your data structures are clean, and functions do what they are supposed to do and no more.

> I can pass in some type of data structure that might behave well in many situations, but in some that function is going to fail because I didn't properly construct the data structure.

Then you construct the structure in a single place, and that should have some sanity checks and sane defaults.

Even if you are using a static typed language, there's no guarantee that what's inside the data structure even makes sense, only that the right type of structure is being passed on. That's helpful, but not as much as people would think.

And you write tests. Lots of tests. It is amazing the amount of Python code that gets written without proper testing. And Java, for that matter. Only Python programmers have less excuses, it's easier to replace what you need for testing.

You should always put validation in constructor so you are 100% sure that the object created is valid.

I guess I'm arguing both sides now. My point is that one needs to root out the weird cases somehow and that it's often hard to have "100%" validation of inputs. One's initial understanding of what is the valid and invalid state space is often incorrect.

Also, even without actual type checking (mypy), IDE support + type hinting (PEP 484 or reST et al) means you can find most issues before the interpreter/compiler is involved. PyCharm is fantastic at this.

https://github.com/jonathanslenders/pymux/blob/master/pymux/...

Here's one way. Those asserts will typically pick up on errors during spikes, development and tests and turn a head scratching problem into a straightforward issue.

A couple of times on a disastrously tech debt ridden project I used it to pick up obscure configuration errors in prod, too (shouldn't see it in prod on a relatively well written project tho).

Never been much of a fan of duck typing. It only really makes sense when you have objects that approximate built in types.

Take a random object with a .run() method, for example, and you don't have a fucking clue what it might be doing so there's no point interpreting that as a quack.

https://pypi.python.org/pypi/schema/0.6.6

Divide code into modules.

If you are worried about types, use object oriented programming to define the classes you want and then use the assert() instruction to check variables to comply with said classes/types.

A good IDE like PyCharm will take advantage of such assert() statements.

* Realistic integration tests that cover as many user stories as possible.

* Raise exceptions ASAP for any kind of invalid data or state (could be checking types, but could also be checking for file/directory existence). Could also be pip installing and using 'schema'.

* Do not build on poor quality modules and strive to decouple and uninvent poorly reinvented wheels.

* Just in general: loosely couple everything.

I don't see this as a static typing issue at all. Static typing only helps with a small part of the problem and it usually does that at some expense (usually more verbose & less flexible code).

I'm a little skeptical that super-strong type systems like haskell necessarily help all that much either. They clearly help eliminate classes of bugs, but the overhead is very high (the amount of everyday software written in haskell is seemingly small relative to its apparent popularity).

This is certainly a potential explanation. But I don't think it is self-evident that the trend of recent growth is entirely due to this.

Edit: The article supports the claim - I should have read it before commenting.

it's incredible how hard it is to get some people on board with this. I've gotten used to good project-local package management in other langauges, I can't imaging working with anything that depended on stuff happening to be available in a global namespace ever again.

E.g. is there something like 'cargo run' that just checks that the required python version is installed (I got the impression that this can be specified somewhere), installs deps if they're not installed yet and then runs the main module? Or something like 'npm install --save' that adds the latest version of a package you already know the name of as a dependency and installs it?

https://github.com/kennethreitz/pipenv

While virtualenv management isn't that difficult, Pipenv is VERY intuitive.

All in all, I found it neither much more or less fun to program with than e.g. Java, C# or TS. It might be a better fit for small projects than others, though.

Which parts did you find confusing? Was it the lack of a block-level scope, such as within for loops and if statements? I personally find the LEGB (Local, Enclosing, Global, Builtin) system to be pretty intuitive.

[0]: https://stackoverflow.com/a/23471004/3023252

Python is not perfect but the final code is simple to write and understand years later.

I wish this would die, it just scares off newbies. In my twenty years of Python I encountered conflicting reqs once or twice. It's easy to direct folks to the documentation in those circumstances rather than cramming it down every newbie's throat---YAGNI.

Now that pip can install with --user and containers are popular, the problem isn't even possible in production in many circumstances.

This discussion is a lot like version control. How many people made daily (hourly) duplicates of project folders before learning the benefits of version control?

You normally won't get a version control lecture during every programming language tutorial either.

However, some things can't be as easy as in Python, such as sympy, which does symbolic processing of in-python expressions (differentiation, integration etc).

I actually don't find python easy to use - every time I use it, I have to look up how to do for loops (what? range? zero-based? does it include the last one?), and the docs don't have links for arguments (even though they require specific types), because no static types.

But the libraries absolutely are easy to use. And because many are written in C (unlike most of Java's librries), you're arguably running C.

Then there is the GIL. I still don't know if multithreaded python apps are really MT or just faking it. And yes, I know you can do event driven stuff and that does look very interesting.

You can activate two different environments in two different terminals. Why wouldn't you be able to?

You can automatically activate venvs when you cd into a directory with virtualenvwrapper.

Rule of thumb: if a thread goes into C it's releasing the GIL. Not always, but often. Like reading a file, or a library like like numpy or Pillow.

One particularly big thing, that it doesn't have much of a distinct compilation phase. You'll write some script you want to run to do some one-off task that will take the next hour, only to find it crashed after 20 minutes when it hits some boneheaded syntax error you made.

    only to find it crashed after 20 minutes when it hits
    some boneheaded syntax error you made

They usually don't know better

They'd make a mess with every language...

A beautiful mess, don't get me wrong

There are better languages but there isn't enough "science cargo cult" around them

And honestly there aren't as many code snippets to copy from

My hopes are in luna[1]

p.s. distributing load or using multi cores is still shitty in python, the more dataset will grow in size, the more python will struggle

[1] http://www.luna-lang.org/