But my main complaint about Python is trying to use someone else's software. Then it degenerates into a maze of twisty little passages of trying to install the correct set of dependencies, not knowing where those files are installed (which makes it difficult on a cluster, as you may need to do the install on every node separately). Then you try using pip and it fails because some element of what it is trying to install isn't compatible with your operating system for reason 1. So you try anaconda instead, and that fails in a different way, like it runs a C compiler on some file (!?) which fails because $REASON. I have numerous examples of Python software where I have followed the (detailed, precise) installation instructions, and it doesn't work.

Whereas with Java, you just dump the necessary jars into a directory, include them in the classpath, and it works.

Don't get me started on R.

The reason packaging is still a problem in Python are linked to the language:

- to get perfs, you need compiled extensions, which java doesn't need. So a Jar can be fast and portable, while in Python it's an "or" proposition. Wheels help, but they still are OS dependent, so you need a build per OS.

- PYZ arrived very late in Python, while WAR were a very early thing in Java. Few people even know you can distribute a whole Python program in a single zip file with dependencies using something like shiv (shiv.readthedocs.io). So few people do it, and it's a shame. We do it for 0bin though: https://github.com/Tygs/0bin/releases. And it doesn't solve the wheel problem, although nuitka does but it a higher prices to pay.

- packaging tooling have been a mess for 2 decades. Setuptools/distutils/setuptools2, easy_install/pip, and setup.cfg/setup.py/pyproject.toml. We failed as a community on this one. It's getting better, but it's a slow process.

- the Python community is at least half composed of amateurs. It's a good thing for the dynamism of the community, but it also means the software are not very polished. Hard packaging + amateurs = packages that are releases when they reach "just good enough" stage, not one inch more. It also means you'll get things like leaking stack traces, config files in src and the likes.

So yeah, compared to Java, Python packaging and distribution sucks.

But compared to rust all packaging and distribution for all languages do, so there is that.

or, even worse, scientists.

The previous crowd with love for monkeypatching everything and hate for abstraction other than dictionaries wasn't too pleasant, too.

Code style! Hah. First day on the job I asked about that and was told nobody cared and everybody had their own preferences.

... I'm very bitter today. I'm working on it.

Everything is jupyter notebooks, to the point where example documentation is delivered as notebooks and services run notebooks as it’s apparently easier to do that than to get people to write their code properly. “Oh but data exploration” they say, which is fine, but you shouldn’t be doing data exploration in your prod ML pipeline anyway, and you’re writing code, it’s really not that hard to follow some even rudimentary practices.

I'm forced to use Python at work, because data scientists demand it, and I can rant about your "good for data" language for at least 24 hours straight.

Python isn't great at anything specifically, but it's crap in a few specific places. No multithreading? Fine! But then Python core libraries are crap at IPC, when multiprocess is the preferred way of parallel computing!

I don't know where you read either of your propositions in any of my comment.

You've listed things that rely on full stack of libraries and deployment options.(except scripting) As someone who has the pleasure to work with Python - Python is horrible at a lot of these things, in spite of the beauty of the language itself.

Python is the best language in the world for small tools under 500 lines to solve glue and data munging problems.

There are two main exceptions to this rule. One is when an actual software engineer ends up working as a scientist and having to write papers (like me). Then, the paper is more likely to be about the software itself, and the citations of the paper are from people who have also used the same software. This creates an incentive to make the software high quality, in order to get those citations. The other exception is when an institute decides that the world needs a decent piece of software to fulfil a vital role, and actually funds its development.

Oh, I should probably mention - I just helped out with a programming course for undergraduate medical science students. Most of the time I was tearing my hair out about the terrible programming practices being taught by the module leader, who admittedly had absolutely no computer science qualifications whatsoever (and in fact didn't know the difference between a forward slash and a backslash).

I think a solution to this is to recognise software quality as a common benefit, and divert funding towards it, allowing real software engineers to have an actual meaningful career in university. Universities are usually all over common benefit stuff, so you would have thought this would be easy enough. The Research Software Engineer programme in the UK is a start.

Pyinstaller builds bundle the python binary with the project. It works without a Python installed on the system, is not portable and the resulting binary is fat. It also can fail in numerous way and be slow, but end users will find it the easier to use when they work.

Pyz files just contains python code. Shiv produce them by bundling your code and the dependencies, but you still need a Python installed on the system to run that. You pay the cost of zipping only on the first run though, after that, it's unzipped. And their are portable if you use no binary wheel. It's light and easy to get right.

In that sense, they look more like War files in Java.

Pyinstaller, like cx_freeze and nuikta are for end users.

Pyz are for devs.

> with Java, you just dump the necessary jars into a directory, include them in the classpath, and it works.

Ant, maven, or gradle, and then some jars, somewhere? What's the classpath and whats $JAVA_HOME for this project?

Oh and now I have to make it work in the IDE too, click through a billion menus and dialogs, sync the build system, sync the dependencies, configure runtimes and classpaths, try to rebuild for the 10th time, pray.

> Oh and now I have to make it work in the IDE too, click through a billion menus and dialogs,

You click "Project from existing sources" in IntellIJ and it will recognize both Maven and Gradle projects and just work?

Sync the dependencies? configure runtimes and classpaths? What? With Maven and Gradle? Your talking Ant here.

  - Install and set the right java version system-wide
  - Copy some config files for the project system-wide (granted, this is not java specific, but still)
  - Click in some IDE menu to enable some build settings
  - Click in some IDE menu to install some "Lombok" plugin
  - Click in some IDE menu to select the right jdk
  - Click in some IDE menu to edit the run configuration:
    - select main class
    - paste program args
    - paste env vars
    - select right “Use classpath of module:” option
  - Click in some IDE menu to fiddle with "Spring Loader path": modules, dependencies, reorder the dependencies (!)
  - Click in some IDE menu to configure getting the actual logs
  - Click maven-sync, rebuild-project multiple times, restart IDE until it suddenly works

I know you will say, this is all fault of whoever made this crap project, or my fault for not fixing it. However this was my experience, and it has been similar every time in my life that I had to touch some java project (although this last one is the worst).

In my experience (sample size of hundreds of projects), Maven projects are <<by far>>, on average, easier to build and manage than most other languages. The repo structure is standardized, the build cycle is standardized, the built packages are always in the same place, etc.

The Java installation is a 1-time think. It's also a quick google away. I see this complaint a lot, I never understand it. It's plain and simple bikeshedding: someone complaining about a 1-time task that takes 5 minutes in total.

The rest of the stuff is IDE specific and I have no idea why you'd have to configure "getting the actual logs"?

Treating the IDE like a separate problem is not realistic with java.

With e.g. Rust, I can use vim, vscode, whatever floats your boat.

> no idea why you'd have to configure "getting the actual logs"?

Some genius java logging library, I can only guess.

With Rust you code in VIM because there's nothing better. Improving Rust IDE's has been one of the teams big focuses for years. Everyone codes in random text editors because there's no amazing IDE.

You don't need to constantly "extract method", move to class, generate all these mountains of classes by rightclick. This says a lot about the language.

But the main point is that with rust it actually just works: there is one way to set global toolchain and per project/dir. You're not required to also click through a million menus and dialogs and copypaste stuff and sync and rebuild and restart to get the editor barely working.

In Rust, macros generate code on build. In Java, Annotations Processors do. I don't see a big difference between these approaches, besides Java's mechanism being older and crustier. Rust macros generate tons of code.

> if you have rust-analyze you get inline errors, code completion, jump to definition, autoimport, autoformat...

Last time I checked, Rust analyzer was in alpha/beta. Java has all this and it's battle-tested. Including language server for VScode and other lightweight editors, and several options for build plugins that auto format.

> You don't need to constantly "extract method", move to class, generate all these mountains of classes by rightclick.

Lombok, or one of the popular Builder libraries, gets rid of all this boilerplate. I don't generate anything using IDE. In my experience, most Java user don't. There's many mature libraries for geting around class ceremony.

> there is one way to set global toolchain and per project/dir. You're not required to also click through a million menus and dialogs and copypaste stuff and sync and rebuild and restart to get the editor barely working.

You are right. Rusts build system is simpler and better integrated. But Java has amazing build tools, maybe the best out there. A properly configured maven project will compile, test, and run with one command. And it will import into your IDE with a simple "import project, yes"

I have worked on a ton of Java projects over the years and never had to deal with most of what your describe.

True, but with IntelliJ 99% of what you wrote is auto-configured.

Eclipse needs a bit of hand-holding, but almost everything you described is due to project configuration.

> Some genius java logging library, I can only guess.

That's your problem, you're assuming that Java is dumb and its libraries are dumb. They're not. I'm reasonably sure that <<all>> mainstream Java logging libraries log to console by default.

The previous engineer must have been a marvel of a developer to manage to screw even that up.

What I'd say is: Java, especially older enterprise code, had a ton of over-engineered patterns. These are slowly fading away, except for Spring, and even Spring's simpler these days. Java also tends to have some low quality and a ton of average but unmotivated working devs on Java projects. So yeah, there's that ecosystem problem.

But for example, if you'd start a new project and you approach Java with an open mind, it's one of the most powerful and flexible techs available. And it's much more modern than the stereotypes imply.

It is not, that's what I'm saying. And I don't understand why, if IntelliJ is the de-facto IDE. I'm sure that a fresh project is a one-click import, but an average project that isn't fresh anymore, always have similar IDE-setup-magic problems. At least in my experience over the years.

I know plenty of people who don't use a formal IDE. This "I don't use an IDE" is a stupid flex.

It's not a flex, simply a matter of preference: I find using the mouse too irritating, others prefer it because it's simpler.

But that's not my gripe, my gripe is that setting up IntelliJ, the one defacto standard IDE to actually work is some voodoo ritual of a dozen steps and restarts and so on.

However I'd expect Lombok users to outnumber Lombok devs 100k to 1 :-)

I frequently find that the particular constellation of build tooling and libraries particular projects have requires a particular version of the jvm and all the environment variables set up correctly. In some other languages there is widely used tooling to manage the install and use of the appropriate platform version for the project.

If you're working with a variety of projects, fiddling with the installed jvm and java_home and friends is not a one time thing.

It is. There's really no excuse for that. If it's been similar every time then you've worked on crap every time.

Building a java project should be (and is on projects I work on) as easy as installing a jvm and running either maven or gradle to build it. End of story.

"Install some lombok plugin" is only necessary if you want the IDE to understand lombok, IntelliJ will just pop up a box saying "Want this?" and you click "Yes". I don't see what's so onerous about that.

The rest you describe is really down to poor developers.

So "no offense", but shove it.

Yeah that’s a crap project. My experience was always that mvn install / ./gradlew run will just work. On all three OSs. And if I happen upon a decade old random jar file, and run java -jar something.jar, it will just work in the great majority of time. I think no other platform has it this smooth.

The project could have been set up instead with the "Maven wrapper"(1) which provides "mvn.cmd/sh" scripts at the top level of the project to fetch and run the proper Maven. So after cloning the project, all you have to do is set a proper JAVA_HOME if it's not already set, and then do "./mvn clean package" at the top of the project to do a complete build. The IDE's will then set themselves up properly if you import the project as a Maven project, and will sync with changes to the Maven pom.xml whenever it changes (Intellij does at least). That's really hard to beat.

(1) https://github.com/takari/maven-wrapper (Being integrated into Maven itself soon).

We occasionally do remote debugging at work. It’s just a matter of copy/pasting the correct JVM incantation into the IDE.

Even if your project is so messy, then you should be able to just to version a working setup (maybe needing to figure out how to make the correct JDK version be used across different machines) and after that it'd become a question of just having to check out the project from the version control system and click a green button in IntelliJ.

This most likely wouldn't solve all of your issues, but at least would help make onboarding slightly less painful and slow.

The run configuration can be automatically created by Intellij by attempting to execute a runnable class. You only need to set program args and env vars if needed - and that would be true for ANY language or program.

The spring loader path appears to be specific for this project.

The run logs are independent of Java. Thats just getting program logs.

There is no need to maven-sync and rebuild-project multiple times, unless you are explicitly refactoring the project module/build structure. Once you have imported a maven project, you can simply click build.

Create new Java project in IDE Add no dependencies Use the built in IDE tooling to create a runnable JAR and tick the box to export the Ant build to a build.xml file.

Why would I want to complicate things by adding Maven/Gradle to the mix?

And I will also give you the secret trick of superninja senior developpers: be the one who manages the build process, so you will never EVER be fired.

Non-networked applications don't tend to need to be so paranoid about keeping up to date with the latest versions of dependencies.

My point is there are a wide variety of application types out there, not everything is a web application, not everything needs to use a complex build and dependency management system.

So just use Maven and save anyone coming after you a bunch of trouble when they have to extend your project ;-)

Dependency management is a mess in the Python world and my number 1 complaint. You have pip, pipenv, poetry, anaconda, to manage your 3rd parties. And if you want to avoid conflicts and have consistent runtime you need to create a virtual environment to install these libraries and redownload everything.

Otherwise libraries are installed user or system wide, in a directory that depends on the Python version that you use but they are not versioned there.

With Maven you don´t have the issue. Your 3rd parties are downloaded in a global cache directory (the Maven repo) and are all versioned. For the delivery you can either provide them all and have the runner set the CLASSPATH or have them packaged into a single jar.

I am not saying Python's packaging is any good, it sucks. But you can't realistically ignore Java's IDE shitshow.

All needed is having the correct version of Java, and maybe give maven a password so it can access private libraries. It's dead simple.

And more importantly: it's the same everywhere.

I haven't used Maven in a bunch of years.

But source code goes in src/main/java. Test code goes in src/test/java, resources (images, files, whatever) go in src/main/resources, test resources go in src/test/resources. There is a pom.xml file that contains dependencies, and if it doesn't, it points to a parent which <<does>> contain them.

There might be submodules and they all are folders containing another pom.xml. That pom.xml generally points to the main pom.xml.

Etc. There's a bit of tedium in there, but <<it's always the same>>!

The level of consistency is something most ecosystems dream about.

And you know what? That means that you can use your spare brain cells for actual creativity or hard work.

That is not my experience at all. I dont have to rebuild work project many times. Creating new scratch project for playing in is easy and fast. Dependencies are managed by maven and I rarely have to deal with that beyond added dependency tag.

Maven can build a nice fat executable jar, with just one XML block added to the pom.

I've opened a 5year unmaintained project with IntelliJ yesterday - click package and got a working executable.

I opened another project built with SBT... and guess what? No errors!

Let's try that with a Python project (LOL!)

That's an interesting take; my experience has been (at least the community) is quite proudly dogmatic on the One Right Way To Do It, for all values of "It".

I also don't like Python, but I think for different reasons and perhaps a bit less than you.

Never used R, but I understand it to be quite fiddly.

This hasn't been true lately. Python is growing ever more complex.

But, for people still working in "computer science-y" domains, I haven't seen the historical dogmatism lessen; in many cases people are digging in.

[0] https://github.com/opencpu/opencpu/tree/master/R

In other words R is wrong tool to use for building the software just from maintainability standpoint.

I've seen very complex R packages that have been around for 30+ years. They are actively maintained. Applying good design principles, unit testing, modularity - all that is very much doable in R.

And you will want to write software in R, especially if the purpose is to support the very activity of working with R packages and to expose the R statistical environment over HTTP. Case in point, the software I link to, in my comment above.

As a Java/Python/Go developer I can say that packaging is a solved problem as of 2021.

I like to point out Airflow as an example. It's still a complete nightmare to deal with and it has many steps for just packaging the application.

https://github.com/apache/airflow

I have less trouble packaging and running a 3 year unmaintained Maven Java application, than actively maintained Airflow. And let's not forget that I can target Java 8, without having to install Java 8 on my machine. (Which only adds to Python's complexity)

Like you mentioned, wheel files are not nearly as customizable as JARs. You can easily pick & choose the dependencies to include in JARs (fat JARs) and change namespaces of dependencies via shading to prevent collisions.

I've generally found it easier to add JARs to Spark clusters compared to attaching wheel files to PySpark clusters.

It's my guess that you are using a lot of Python libraries that are wrappers around C or C++ libraries which have dodgy cross-platform support, because I can't see another way this is happening. If so, it's hardly fair to blame Python for that...

Java is far worse on this front. Let's take a look at JEE or Spring - The entire experience is akin to summoning a demon with waving reciting correct incantations in correct places while having no idea about what even goes on in the background. There are MessageChannels, MessageProducers, ClientFactories that interact in mysterious ways and initiated in also very mysterious ways. And then there's EJB's and their containers which are just blackboxes of automagic.

Or how about Hibernate?

It now has 2 dependencies. It is a 1000 times easier to use than hibernate, despite being one of the worse ORM in Python.

That the problem with old Java libs like Swing or Hibernate: they have been designed for the 90' and don't leverage the potential of what a modern Java version can do.

There are many modern java libraries that are very lean and mean out there and package into small-sized redistributable assemblies.

You know the joke: a dev has a problem, uses Java, and now has a ProblemFactory :)

Not to say Java is not a great language for big projects, it certainly is well equipped for that, especially because it has superb tooling available on the market.

But I've done plenty of big projects in Python, some ended badly, some ended great. Not to mentions dropbox, youtube and instagram are testaments to Python can be used in large projects.

So I don't think Python is unsuitable for big sized projects. You certainly do have to worry about different problems than in Java, especially for projects that started without type hints. In my experience though, you reach less architecture dead ends with Python because the language is so flexible that a bad decision early on can easily be transformed or worked around even very late in the project without much fuzz.

So I don't use "size of the project" in my consideration to not user Python/Go/Rust/etc over Java.

I LOL’d at this joke (again) because it’s true, but if we’re honest, it’s also unfair. ProblemFactory is a bad idiom, rather than anything to do with Java The Language. You could (and IMO, probably should) just use a new Problem().

Java The Language was very stable for a very long time, so IMO the hype and fads got built on top. First we have a constructor - no, now we need a factory, but how to construct a factory? I know, let’s use another factory! All in the same unmodified language.

Likewise, how do we configure an object? First we have methods, but languages are cooler, so let’s build a DSL, no wait let’s use a fluent API so it just looks like a DSL!

I one had a dev replace a constructor call with a fluent builder, so now in the single use case where the object was ever created, we got five LOC where previously we had one.

His code was idiomatic, it was bad engineering, but it wasn’t Java the language.

Java is a pretty great language, but some of the crap people stuck on top of it was (and still is) a nightmare for readability and reusability. I use Go these days, and prefer their more basic approach to engineering, but there is much I miss from Java-the-language.

The Factory pattern’s pervasive use in Java was in large part a fairly direct consequence of JavaBeans requiring nullary constructors, so that the normal constructor work for a Bean had to be done elsewhere, i.e., by a Factory. Now, Beans was just a core part of the first-party enterprise framework that was central to the ecosystem and not strictly part of the language, but its not completely unrelated, either.

But my memory is that pervasive use of Factories came along many years later, and seemed to be driven by the perceived needs of TDD, for example to provide instances of mock classes in unit tests. And I thought the factory pattern itself came from the GoF, but I might be misremembering.

That said, I’m not trying to contradict you. It’s likely that there was more than one driver for the pervasive adoption of factories. In any case it was a bit of a nightmare for a while, hopefully things are improving there now.

It did, and GoF certainly influenced OOP outside of Java, too. But EJB created a need into which the Factory pattern was really the only solution. (GoF precedes EJB and even Java.)

But certainly its not the only thing; Java hit popularity right around the time GoF was most influential; OOP languages whose ecosystems developed earlier (e.g., C++) or later (e.g., C#, though GoF-heavy code isn’t unheard of there) wouldn’t be as influenced by it, all things being equal. Java just had the perfect combination of language features, early specs like JEE (which included Beans), and timing to be naturally inclined for maximum naive GoFism.

Stuff like "everything is a class".

I know the excuse that they are just meant to be used to wrap the return value of a function that might be null, rather than as a guard against NPEs, but that just means we’ve got a crappy, verbose Optional class ... and NPEs.

It was the most half assed attempt to fix the billion dollar mistake [0] anyone could have come up with. It needed syntax and compiler support, not a stupid wrapper class.

[0] https://en.m.wikipedia.org/wiki/Tony_Hoare

I don’t really get your other arguments. I think the problem has been solved better in other languages including Swift, Kotlin and Rust, and Java could have learned from them, but didn’t. I can’t see why direct language support couldn’t have been added to solve this “billion dollar problem”. Even the Elvis operator would have helped.

I'm absolutely convinced that's true for some of them. I was never 100% convinced it was true for all of them.

Java also had the unfortunate destiny of becoming popular at the time where "OOP all the thing" was a mantra in tech circles. Also design pattern were all the rage.

Those things have a huge inertia, and while Java is not a much more flexible language, the image stuck, to the point that new comers still implement singletons in the wild.

Java was a product of “OOP all the things”, it wouldn’t have existed without it.

It was very much a marketed language and a language designed to be sold to top heavy corporations. This fueled a lot of the crappy design decisions, the hype and the culture.

“OOP all the things” predated (and led to) Java, it wasn’t a product of Java (C++, Objective-C, Object Pascal and, well, a bunch of other things from the late 1980s and early 1990s were where it got started.) Java was about a decade into it.

I know not everyone will agree with me, and that’s fine, but I always found Java to be far more pragmatic than, for example, C++, which well and truely predated Java.

I think that applies to literally every language. Bad programmers write bad code, regardless of whether it's Go, Java, Python, Bash, or Assembly.

Same goes for Python. Reading the hacky code of a geographer for his SIG system is rarely a fun day.

I think this joke is probably about as dated as the versions of java it was made about.

Singleton is still a design pattern used in the wild.

Why bother setting things up explicitly in the way you want when you can declare things as injected, other things as singletons, and then have no real idea why it all went horribly wrong?

Inertia is huge in our industry, and the HN bubble doesn't gives a good picture of it.

Behind corporate firewalls I now see things done the way they were 10 years ago: packaging, string handling, concurrency...

I assume it's the curse of being popular and old.

And java has been popular for longer than Python, so inertia is even stronger. It doesn't matter if it has lambda since V8 if people still believe the best practice is to pass fat objects around.

(Poe's Law disclaimer: yes, that was light sarcasm.)

It's a funny joke, but it maligns a harmless pattern identical in FP to a function returning a function.

While FP coders would brag about how awesome high-order functions are, we just keep mocking our own style for some reason. Factories are very powerful when used right.

And me too.

Scaling a Python project complexity is not harder than scaling Java, the things you have to be careful about are just different.

In comparison one of my personal projects is in Rust and while that’s currently maybe only a thousand lines it’s already significantly easier to deal with.

Mypy is excellent. It has two-way type inference like Haskell. Good usage means catching almost all type errors before running.

So is PyCharm, which is IntelliJ for Python.

Fwiw, I’m normally on a beefy desktop so I might just be brute forcing this issue.

I think the main problem is that Gradle doesn't have the same support as Java for ABI compatible changes, so invalidation of modules are more common.

https://blog.jetbrains.com/kotlin/2020/09/the-dark-secrets-o...

I think you're right - ABI compatible changes are likely the culprit.

How do you mean? What IDEs are you referring to? Kotlin is fully (and very well) supported in IntelliJ Idea - unsurprisingly so, given that it's JetBrains own invention...