There are type checkers now.

Personally I find type systems a lot more annoying because they lead to more verbose code and more cognitive load. There are studies that show bugs being proportional to the number of lines of code, regardless of language. And that effect seems to be more significant than the difference among dynamic and static type systems.

Defensive programming can be achieved through various means. Type checking is one of them and often can't replace Unit and Integration testing. Validation of Inputs or other data is another issue.

A proper type system offers a level of safety and correctness that simply cannot be achieved without one, especially as the complexity of an application grows. Also, the supposed lack of cognitive overhead without a type system is actually a subtle form of technical debt where you move faster because you eschew thinking deeply about your data structures up front, but inevitably pay a bigger price in production when you have to debug data type errors at run-time. The debt has to be paid again by every new developer tasked with working on the project because they have to organically absorb an understanding of the application's data structures that could have been explicitly defined and enforced by the computer. Finally, the extent of "defensive programming" required to begin to make up for a type system actually adds a ton of extra code itself (e.g. tons of guard blocks testing brittle application values at run time). Not all type systems are created equally, but a code base without types is pretty much always worse than a comparable one with types (the exception being shell scripts and other relatively simple programs that aren't maintained by a team of developers)

> Defensive programming can be achieved through various means. Type checking is one of them and often can't replace Unit and Integration testing.

Of course they won’t replace unit or integration tests, but they will replace a lot of damn trivial tests you have to write to make sure you exercise all paths of the program, because there are no types to help you.

Also, at least when I think of defensive programming, I don’t think of how Haskell et al. allow me to more clearly express my intents in the type system. I think of the times where I write JS/TS and cannot rely on any data anywhere being in the format it says it’s in.

> Personally I find type systems a lot more annoying because they lead to more verbose code and more cognitive load.

Have you forgotten the cognitive load of "what does this return and what can I call or access on it?"

> There are studies that show bugs being proportional to the number of lines of code, regardless of language

But that value isn't the same across all languages, only the trend.

No, that was explicitly true across languages. That the number of bugs increases with code length within a particular language is a trivial fact.

I don’t think Haskell or OCaml are more verbose than Python.

Other traditional languages are adopting type inference, etc. We could be near an inflection point where we get statically type languages that are as convenient as Python.

> I don’t think Haskell or OCaml are more verbose than Python.

I love Haskell computing model, but holy hell, why is so difficult using things like State Monad or even a trivial thing like a global counter [1] ? Why something so simple should be so difficult? Don't say it's because functional programming, Scheme/Racket is not that difficult!

Haskell and OCaml has their own problems, I agreed with you that both languages are more expressive but the code density is not necessary a good thing neither.

[1]: https://github.com/scotty-web/scotty/blob/master/examples/gl...

A global counter is difficult because shared global mutable state should be difficult —- it can wreak havoc on your code. That said, if you’re really set on it it’s not particularly bad

    counter :: IORef Int
    counter = unsafePerformIO (newIORef 0)

    usesGlobalState :: IO ()
    usesGlobalState = do
      count <- readIORef counter
      putStrLn (“the counter was “ ++ show count)
      modifyIORef (+1) counter

It’s even easier in ocaml, where you’d use ref and not need to worry about the IO monad.

Anecdotally, I find it incredibly rare that such a thing is desirable, when you could instead create it in an IO monad and pass it through to functions that need it. But, it’s there when necessary. And making this kind of anti pattern hard is a strength of these languages, not a weakness.

Oh sure, but Haskell and OCaml are much less popular than Python, with consequences for community building, recruiting, reusing existing code bases etc.

The popular statically typed languages which are actually Python's competitors are for example C++, C#, Java and the like.

Sure as hell they are more verbose. Even Typescript was extremely intimidating to me when I started to try and understand Typescript code bases.

And I am NOT going to try and teach Haskell or Ocaml as a first programming language...

If one were to write perfectly type-annotated Python, and a hypothetical perfectly annotated standard library existed, what’s preventing a faster runtime being developed with performance similar to C#, for example? That will never happen, so maybe “what if” isn’t even worth asking

It's already being done: https://github.com/mypyc/mypyc

Yet the entire data science world is built on Python. I don't understand this blatant disregard of reality.

Python is ideally suited for data science and machine learning because 80% of the work tends to be data exploration, transformations and feature engineering and 19% of the work is iterative development of models. In such workflows the lack of type checking & simple REPL is a huge feature. Most of the underlying code for ML is anyway written in Fortran or C and Python becomes a very convenient front end glue

For the 1% work that is actually engineering the model's deployment in production, it's not uncommon to use faster languages with the exported model.

If what you're doing is writing a vanilla CRUD server that gets thousands or hundreds of thousands of hits a second, Python is probably a bad choice.

I’m not sure how many people will ever even work on CRUD apps of that scale.

Even a mere 1,000 req/sec adds up to 86.4M per day. That sounds like something at least as popular as Stack Overflow, for example. How many web apps are there, really, that have millions of users?

Or almost any advertisement tracker server.

Isn't that market dominated by a few, or at the max, no more than 10-15 companies? My point is just that are there really more than a few dozen or maybe a hundred companies that see more than 1000 req/sec (on web apps)?

Depends if you're talking maximum or average. Getting to the front page of HN could easily give you a burst of 1000 req/sec. I would agree there's very few CRUD apps that deal with 1000 req/sec on average.

On the other hands, there's various APIs and tracking type of services that probably do handle well in excess of that, being run by not-large companies none of us have ever heard of.

Uh why ? Thousands of hits per second will generally be IO or DB bound and will be done using clear code and a minimum of boilerplate.

I think Python is at a weird place regarding data science.

If you need easy-to-use tools, R libraries cover significantly more ground than Python. You don't have to worry about 2/3 compatibility of niche libraries. Plus the functional language forces the libraries to have similar API's. You also get to use tidyverse and ggplot2 in all of their majestic beauty. At most you have to write a "%>% do() %>%" to get around weird library API choices.

However if you are in the frontier, you could avoid so much awkward notation with an array-first language like Matlab or Julia. You have "^" instead of "np.linalg.matrix_power", "/","\" instead of "np.linalg.lstsq", the . operator and so much more life-saving tools.

R has so many problems itself! Attempting to run it at any sort of scale is basically impossible. Its dependency management is atrocious. You can't thread or multi-process effectively. The amount of overhead to even install R on a system is incredible, and amazingly its documentation is even worse than Python. We use R as our foundational Data science language and it has caused nothing but immense pain for everyone attempting to wrap the packages in scalable software.

None of the libraries that do the computations are written in python. None of the GUIs or matplotlib are written in python. If you want to build large projects or need performance, python is not the right language. I do like python a lot - for the things it's good at.

no, the trivial frontend is built in python; the real code is usually c++ or c.

Ignore that reality if you want to, but it is a fact.

Big complicated python projects are seldom pure python, they are usually a friendly python frontend to a serious application written in something else.

It seems in no way remarkable that someone wanting to build a serious backend type piece of functionality would pick another language that was, just for example, multithreaded.

The CPython interpreter itself is a C program. Acting like extension modules “aren’t Python” is highly disingenuous.

Oh please, go read the source code for tensorflow and then come back and we can have a real conversation.

I don't understand this logic. The users are learning Python, not C++ when they're trying to learn data science or implement a machine learning model. Should I say Tensor flow isn't written in C++ but CUDA or OpenCL? Any self respecting researcher is training their models on GPU or FPGA, not CPU.

The point is that Python is the entry point for large majority of data scientists currently and its absolutely disingenuous to try to dispute that reality.

> The users are learning Python, not C++

The key word here is "users". Python is fantastic for users. It allows users to get things done without having to worry about types or memory or the underlying hardware in any way.

I’d say it’s very disingenuous to claim that Python users don’t have to care about types. Many types are directly related to semantics, so of course they have to care about types!

I think Python made some really good choices with their types from UX that lead to you having to care more about your logic than the machine. Only one integer type, for example! For most users it’s perfect UX.

However, I think (possibly due to the early time when a Python was made) some of the decisions made practical but regrettable trade offs. Duck typing is ultimately just type inference with strong performance penalties and really weak static tooling, for example. _Many_ static type systems are still very far from the user’s actual domain, but the core techniques don’t have to be.

Not just data science: the entire machine learning and scientific computing world, if we’re honest. Need to do huge numerical linear algebra operations very fast with great memory safety and ease of use? Python.

Technically, you'd use a C/C++ library with a Python wrapper.

Yes, that is called Python.

No it isn't? The substantive portions of the software aren't written in Python. Python is a thin interface to the actual library. Users of the library who need to rapidly prototype can leverage the library using its Python frontend, but the authors of the library did not write the library in Python.

To program in CPython is to use thin wrappers around C-level implemented extension types and modules. For example, all major built-in container types.

If you program in Python for performance-critical applications, as I do, then you will frequently hand-write C-level extension modules, write extension modules in Cython, use JIT tools like numba.

All of this stuff “is Python.”

even for the crud apps so frequently referred to in this thread the equivalent 'real code' is the sql rdbms written in c or c++ no matter what the webapp language is. i don't advocate using python in million line monoliths despite it being my favorite lang in general at the moment but the volume of cognitive dissonance and false dichotomies in this thread is crushing.

Wrong: the data science world is built on R, which was designed from the ground up to be the freeware alternative to SAS and Mathematica.

Oddly in biology I’m seeing an uptick in R use as it’s easier to install and run tools in R for non programmers. The python 2/3 confusion (why doesn’t this google result work?), and various install packages (conda/pip...) seem to have made R more popular. This is despite biopython which is quite good. The newest single sell rna seq analysis tools are in R.

R is a strang thing to me, though it has its moments (graphing). I still think pandas which gives python R like data frames is one of the great tools out there.

pandas gives base R a run for its money, but I find the multi-indexing really confusing, and R tidyverse is just way way nicer than pandas.

The sentiment that R is easier than python to install/manage is common among some R users but I disagree. With R I'm constantly facing dependency hell problems -- one package wants an old version of R, while another needs the newest. Conda/venv solves this problem very nicely in python.

Recently lI've been using Julia more and really like it. One nonstandard case where I've found it really shines is parsing large bioinformatics data, such as pileup files. Python is just so slowww here, but neither do I want to write a C program to do the text parsing. Julia is perfect in this case.

Wrestling with borrow checker is enough punishment already. /s