I work on a reasonably large Django project (1M+ LOC) with approximately 100 developers and mostly have a reasonable time debugging. We use mypy, lots of linters (some custom flake8 plugins), and a fairly strict layered architecture[0] that's enforced with tooling[1].

Without the tooling it's a nightmare. My previous Django project with approx 500k LOC had linting and some typing and that was a mess.

[0] https://github.com/octoenergy/conventions/blob/master/patter... [1] https://github.com/seddonym/import-linter

Is there any chance of making Octopus's vendor.bundle.js less than 15MB? It takes time to load on any device, and the size makes the site unusable on mobile.

There are some efforts going into the frontend to reduce these sizes. There's a bit of a transition between frameworks happening.

What’s the difference between mypy and python built in type hint?

MyPy provides static analysis that uses the type hints to actually, well, do things with it. Type hints are the specification, but MyPy|PyLance|Pyright|Pyre use those hints for and type checking.

200+ developers at this point

The typing issues are being solved through mypy: http://mypy-lang.org/

A large majority of packages are still missing typing hints, but the ecosystem is moving in the right direction on that issue.

What’s the difference between mypy and python built in type hint?

mypy, as well as things like pylance, enforce the type hints, the interpreter ignores them during execution.

My advice - go with Kotlin. You get the JVM, all of the Java ecosystem, type checking (of course), and a sane, modern language.

Why not just java? I thought it caught up to kotlin in the past few years

The gap has closed somewhat for sure, but there's still quite a few nice features in Kotlin that Java doesn't have. https://gist.github.com/makosblade/8fb1f577d81f02025784093d2... does a good job at identifying most. I'd add the semantics around lambdas are much nicer better in Kotlin.

Hardly. Despite the introduction of raw string literals Java still can't handle a regex without having to double-escape metacharacters.

How can you even say this?

Regex is very specific. It has added records, streams, virtual threads (preview I think) working with native better, pattern matching...

Java is like C++. Everyone talks bad about it but at the end of the day it is one of the most Getting things done language.

I am not sure why I should use Kotlin. Llokd nice! But for sure docs and support are not even close to Java.

Are you seriously comparing Java to C++?

One excels at helping you shoot yourself in the foot. The other gets things done…

With the passionate hatred both languages inspire, I'm not sure which is which in your statement.

One could argue C++ gets even more done, since it is appropriate for use in more contexts, like firmwares/drivers, native SDKs, games, etc. Java's main use cases have strong competition in languages like C#, Go, C++, Python, etc.

which is which lol

I see you have not used C++. Let me tell you that with all its quirks, it can do way more than any language low-level wise and go back to high level.

Of course some people just rant about it. But did you guys use it a span of time long enough to really know how good or bad it is actually?

Because that's what the cool kids use.

After using Kotlin in Spring projects for a couple of years, I rewrote most of it to Java 17 and stick to Kotlin strictly in Android work. It's not that much different anymore with all the recent additions to Java, and as Java progresses (and the main JVM implementation which is first and foremost being developed for Java) and diverges in other directions in some very important areas (value types, Loom), Kotlin starts to feel more and more like a second-class citizen. I mean, that's what some old-timers were preaching right here on HN from the very start.

Things like Hibernate require some pretty ugly (IMHO) hacks to work properly (you need something like three compiler plugins, and spread keywords around like there's no tomorrow).

The compiler was also very slow compared to Java. The same amount of code written in the same style took 2× times to build. This may have improved.

Probably the only thing I miss are nullability annotations (after getting used to them in C#, not Kotlin).

Edit: after re-reading what I wrote, the tone feels way too critical. Kotlin is a nice language, and certainly easier & more fun to both read and write (although it's trivial to write unreadable mess by going too far with its features). It just doesn't go far enough (for me) to warrant using a separate language with all that entails.

You raise an interesting point: Maybe Kotlin was the push Java core devs needed to increase their rate of innovation ("release cadence"). I saw similar with Netscape vs IE, Chrome vs IE, Clang vs GCC, Boost vs C++ foundation library, .NET vs Java, Java+.NET vs C++, Node vs Deno. Kotlin can continue to experiment with new and interesting language features. Those that prove most popular can be adopted by Java. Also, as a secondary effect, since Jetbrains wrote Kotlin and had the best IDE, they had a head-start to build the community. That also helped to sell more IDE licenses. (I have nothing against this 1980s spy movie "master plan": They are great company with many great products.)

Kotlin remains a way better language IMO.

Java has adopted maybe 20% of the things that make Kotlin better, but due to backwards compatibility it'll never catch up.

Personally I think it'd almost never make sense to adopt Java over Kotlin for any greenfield JVM project, unless the devs you hire are truly so low-skill they can't pick it up.

> Java has adopted maybe 20% of the things that make Kotlin better, but due to backwards compatibility it'll never catch up.

Kotlin supports neither structural pattern matching nor guarded patterns as they are available in both Java 18 and Scala. It's Kotlin that needs catching up nowadays.

> nowadays

It's an abstraction atop Java. Of course it plays the catch-up game. It inherits new features from Java. They can't have it instantly.

What's the point of Kotlin, aside absence of semicolons and the 'val' keyword, if it plays the catch up game with the rest of the features becoming available first in Java?

> They can't have it instantly.

Why not if Scala did it way before pattern matching was introduced into Java?

Off the top of my head:

Null-safety.

Read-only interface by default - `MutableList` vs `List`.

if-expression, they just look nicer than `a ? b : c`.

---

You're right that Kotlin is not an "abstraction atop Java (the language)". It's meant to be a better Java that feels familar to Java. With that rationale in mind, it's reasonable that they left some features from FP languages out of Kotlin.

As I have said in another reply, it was a great decision to not follow what Scala did with pattern matching.

In Java 18, this is the pattern matching you're talking about.

`if (o instanceof String s) {`

Not much benefits over Kotlin's smart casts.

Guarded patterns with this version of pattern matching should be translatable to adding another "and" condition. The Kotlin compiler should have no problem reasoning with that.

---

I find positional destructuring in Scala (which works similarly in Java 19) a bad design.

https://news.ycombinator.com/item?id=31399737

Kotlin seems like a pretty good choice. To me, it looks essentially the same as F#, just on the JVM rather than CLR.

Could you elaborate? To me these are (idiomatically, syntactically) two very different languages.

It seems like I only took the most surface of looks at it. And maybe mixing it up with scala a bit.

That's what I get for not reading about it more carefully.

Maybe with Arrow but by itself Kotlin is still mainly object oriented.

Kotlin may have many advantages over Python as a language (subjective), but the Java ecosystem is not an attraction over the Python ecosystem to me (both have flaws, but the Java world has a lot of bloated stuff)

JPMorgan's Athena platform is 35M lines of Python. Probably not too relevant here though as it's all Python2 code...

Do you know if BAML Quartz is similar (size and Python 2)?

Any number of resumes on LinkedIn with “Python 2 to 3 migration” will say yes.

Do they maintain all parts of their software stack? Third-party libraries are leaving Python 2 behind (or rather, have largely done so).

Yeah, been on multiple, and for me it felt like a nightmare (maybe I'm just not smart enough) I'll use python for small tools, depending on the planned deployment. But as a general rule, everything north of 2 months developer time (being it just me, or multiple devs) gets a language with static types, at least for the non-glue main components. Glad that JS got TS types for that. My approach has served me well so far.

Never tried mypy though, I must say. Maybe that saves the day.

I find mypy to be underwhelming in comparison to Typescript, but maybe it's just the ecosystem. Most JS libraries nowadays are either written in Typescript or have type definitions, but some big Python libraries don't (or the types are not complete).

I recently started using mypy and my experience so far is that it's a bit of a pain to get it to play nicely with numpy.

For typed loading data, my benchmarks show 3.11 is actually slower than 3.10.

https://ltworf.github.io/typedload/performance.html

So typed python runs slower on 3.11?

Not just typed, but when accessing types at runtime.

But I didn't investigate on which change exactly is making it slower.

~500k loc python project here. Detest it. For being such an old and established language, the ecosystem/community is very immature. Most stuff is unstable, there are thousands of ways of doing anything, no one has settled on which tooling to use so all projects are a miss-match of what was in vogue the month it got started. Typing is almost useless, like Typescript in the early days where you really cannot trust it not blowing up runtime even though you think you have typed everything. Especially since most libraries do stuff with (*args, *kwargs) and the type checker has to give up, or decorations not propagating types correctly, or Django ORM lying about types. You think you're safe, but secretly everything is just typed as Any. No safety enforced between modules, so every piece of code you write can be considered part of a modules public API as people will import and use it. Really lacking stdlib making fugly list comprehensions with mutations the standard way of solving things that no one will understand in a years time. Dependencies are a hell, everything break all the time, and "works on my computer" could lead to days of trying to figure out the setup on a coworkers computer where it doesn't work.

I don’t disagree with the “early TypeScript” comparison, but what’s the issue with args, *kwargs?

The problem is that you lose all help from tooling/IDEs. Like in Celery, the definition is "shared_task(*args, *kwargs)". This gives you no indication of what parameters you actually can use. Opening up the code doesn't help, as it's many layers down. The decorated function ends up untyped, but with some new methods on it that again are untyped. But like originalfunction.delay(...) should have the params of the original decorated function. But no, all that is lost. Just pray that the docs are correct.*

While it's of course not ideal, stub files can help with this issue. For example you can get stubs for Celery that make both `shared_task` and `delay` properly typed: https://github.com/sbdchd/celery-types

Only since 3.10, though, before that it was actually impossible to type the decorators correctly.

> Only since 3.10, though, before that it was actually impossible to type the decorators correctly.

There were type gymnastics you could do using overloads to reach any arbitrary level of coverage, but it was ugly and always short of fully general.

ParamSpec from 3.10 is adding some improvements to type hinting decorators and wrapper functions that just forward args and kwargs.

https://peps.python.org/pep-0612/

I agree with you that using Python for very large projects might not be the best choice. I love programming in Common Lisp, and there are similar issues as Python.

For huge projects, I still think that Java is a good choice, and although I have only professionally worked on one Haskell project (medium size), I think that Haskell might be good if a team is in place who can use it. A new friend of mine in town is enthusiastic about OCaml, and after a few evenings of studying, I wish that about 8 years ago when I started Haskell I had chosen OCaml for a production typed language.

For Python: I really like Python for deep learning, reinforcement learning, quick and small semantic web apps, etc. The common thread here is that I am not writing much Python myself, instead I am exploiting large well tested libraries.

> I wish that about 8 years ago when I started Haskell I had chosen OCaml for a production typed language.

Do you mind writing a bit more about why? I have been a curious bystander in OCaml land but some of the differences with Haskell, like the lack of type classes, have pushed me toward the latter.

What are the strengths of OCaml when compared with Haskell?

I think the issues you encountered may be due to the specific libraries. Lots of the pre-typing libraries haven’t adopted static typing, like Django and Celery and then when your project is 95% Django and Celery you’re SOL.

I’m not even sure it’s possible to have Django typed without reworking the ORM, I’m thinking about reverse relations, .annotate(), etc.

Yes, there are type stubs for these libraries but they’re either forced to be more strict, preventing use of dynamism, or opt for being less strict but allowing you to use all the library features, at the cost of safety.

I think in the end, new libraries built with static typing in mind, like Pydantic, FastAPI, and Edgedb, are the answer.

> Yes, there are type stubs for these libraries but they’re either forced to be more strict, preventing use of dynamism, or opt for being less strict but allowing you to use all the library features, at the cost of safety.

There are type stubs for Django that somewhat avoid these compromises: https://github.com/typeddjango/django-stubs

To be able to do this they have to use a Mypy plugin though. And even then it's still far from perfect.

And Pycharm and django-stubs can't work together for some reason.

I never really understood this distinction between large and small projects.

Why not invest into good boundaries and turn your large project into a group of small projects?

500k LOC project should have plenty of natural boundaries.

I recently worked at ~200k LOC Django project: the code was far from perfect, and yet I had no trouble onboarding new team members and making them productive.

Sure, things like strong typing do make the monolithic ball of mud more maintainable. But how about not building big ball of mud in the first place?

A better language would enforce those boundaries. In python you can't, not without making it a completely new app. And when you first is in the situation, it's virtually impossible to separate compared to for instance java.

It's always easy to say "just be better and more diligent programmers", but that doesn't work. If the language promote spagetti, spagetti will be written.

> It's always easy to say "just be better and more diligent programmers", but that doesn't work. If the language promote spagetti, spagetti will be written.

Oh, I completely agree and I would never say that.

But at the same time, Java promotes complexity and overengineering. I've seen 10+ nested classes for something that was a 5-line function in Python.

The big difference for me is when I talk to Python engineer they agree that their spaghetti sucks. They want to evolve out of it, they just haven't found a way yet.

It is much harder to convince Java folks that their class hierarchies are useless.

Fixing Python spaghetti is way easier than fixing Java folks mindset.

> But at the same time, Java promotes complexity and overengineering

Nothing in Java inherently does that. It's actually improved quite dramatically since Java 8, with many features like records, pattern matching, lambdas, SAMs, etc.

I feel very similar but have struggled to set aside enough time to find a better replacement. For work I often build one-off scripts, web scrapers/automaters, data tools, and backend web apps/APIs. While I don't disagree with your comments about the ecosystem, I find myself very dependent on it to do the aforementioned work (playwright + beautifulsoup, peewee/native sqlite3 lib, numpy + scikit, Flask/Django) and is probably the main reason I've continued using it. Does anyone have recommendations for some directions I could research? Go and/or Rust seem to be clear contenders but I'm not sure the ecosystem has equivalents or at least mature-enough equivalents for the libraries I use. Very open to learning about other languages too but simply am out of the loop. Something with a great type system and some reasonable flexibility would be amazing (eg I like that I can mix classes and functions in modules easily in Python compared to say old-school Java where everything is a class). I'm also not looking for a language that's primarily functional at this time, too much to learn right now on top of a new language, but it's on my long term to-do list.

Kotlin ticks a lot of your boxes. Can mix classes and functions. Very easy to write functional code, but also easy to not do it when needed.

I currently work at a startup that uses Python for everything. Since we're data science oriented (and Python is the only programming language the data scientists kinda-sorta know). And the CTO fears having to hire for more than one backend language (even though we have more people with prior Java experience than Python, and all of the backend API developers hate working with Python).

We get by, but it's pretty awful. People will give you all sorts of arguments for why loose typing and an NPM-like sloppy ecosystem are advantages for Python. And for the domains where Python really shines (i.e. devops, and manual data-wrangling), maybe these are advantageous. But what they DON'T tell you is that best practices for large-scale backend development call for bolting-on so many extra things, and accepting so many constraints, that it wipes out the purported advantages.

You'll need to use at least one linter, if not two, and a style checker to enforce the use of type hints. This gets you to something that is maybe 50-75% as effective as a Java compiler, but will never be any better than that, and will never have anywhere near the same level of IDE integration.

If you have a lot of dependencies (and you will), then you'll need a proper build system like Poetry. Which puts you right back in that Maven world, that escaping from was supposed to be one of the advantages. And the dependency ecosystem is SO... FREAKING... BAD. Unlike Java, where most of the major libraries are professional in nature with corporate financial backing, most Python libraries are "pure" community and written by unpaid volunteers or hobbyists. Many of whom have NO understanding of semantic versioning, so you find yourself having to pin ALL of your dependencies to specific fixed versions so the spider web doesn't tangle up. Upgrading anything is a nightmare.

You'll be told that it's easier to hire for. But then you'll find that the candidate pool is largely ops or QA/QE people looking to transition into engineering, boot camp grads, and other people with no professional experience in backend API development or even Python itself. Most capable candidates that you hire will take the job for other reasons (e.g. wanting some exposure to data science or ML), and will complain constantly about having to work with Python for general backend services.

You can get by, but as the project or company grows, you'll probably find yourself either re-writing Python portions, or at least deprecating them as legacy and eventually migrating to some other new greenfield thing. I can't imagine any sane reason to migrate a Java or .NET project in the opposite direction.

I've seen all-Python shops before and as the codebase grows they tend to regret it. The one that came to mind migrated to Java. I am one of the former devops guys myself and initially focused on Python to get into fulltime software, so I understood your comment completely.

I have years under my belt now, and what I ended up preferring is C# and TypeScript. I would only build a product on one of those two. This random person's opinion is that your company would probably be best served by using Python for the data team and TypeScript for front and backend.

I've been on large projects written in Python, it's no good. (I say this as a Python fan!)

For scripts and small projects or rapid prototyping Python is pretty much king. (Although Red/Rebol is a contender. Check out their GUI examples!)

For medium-sized projects (100K LoC, 1-10 devs) you might use Go or you could experiment with the usual suspects: Lisp or Nim or Haskell or OCaml or whatever you want.

For a large project (millions of LoC, 100's of devs) I would use Ada or Java unless the problem was very Erlang-shaped (in which case you would use Erlang (or Elixir.))

For front-end work I'll personally never use anything but Elm for the foreseeable future. Based on economics. Compared to Elm the entire JS ecosystem is a incredibly massive boondoggle, a total waste. (I'm cranky this morning and I'm practically trolling here. Apologies to those who are not entertained. I mean it though: in my considered opinion Elm mocks JS, brutally.)

Could you elaborate on this more?

What exactly happens between 20k and 100k LOC (or 2 devs and 10 devs) that makes Go more suitable than Python?

What exactly happens between 100k and 1kk LOC (or 100 devs and 500 devs) that makes Java more appropriate than Go?

How do you even find this boundary when same apps in Java and Python might have 2-3x LOC count difference?

I've seen this mantra repeated for decades (technologies X/Y/Z for small/medium/large projects) and could never make sense of it. For example Angular was always sold as a technology for "large" projects (as opposed to React). Where is Angular now? All those "large" projects are now legacy that everyone despises. And if you have 100+ developers on a "large" Angular project I bet you're are feeling fucked now.

Since everything breaks at scale, I'm actually into the opposite mantra: only the tech stacks that are great at small scale are capable of being great at large scale. Note: capable, not guaranteed.

I always prefer layering complexity on top of a simple technology, instead of praying that complexity inside of a complex technology will perfectly match my needs.

> Could you elaborate on this more?

> What exactly happens between 20k and 100k LOC (or 2 devs and 10 devs) that makes Go more suitable than Python?

> What exactly happens between 100k and 1kk LOC (or 100 devs and 500 devs) that makes Java more appropriate than Go?

> How do you even find this boundary when same apps in Java and Python might have 2-3x LOC count difference?

I've been thinking about those questions for roughly thirty years, and I still don't have exact answers. (If I did I'd be famous and hopefully rich, but that's another story, eh?) Speaking in broad generalities, the main thing seems to be something like the number of separate concepts you have to keep in mind to make (non-breaking) changes to the system. The more that the system you're using can do for you, the more suitable it is for larger-scale projects.

> I've seen this mantra repeated for decades (technologies X/Y/Z for small/medium/large projects) and could never make sense of it.

I gotta say, it's not a "mantra", it's experience. Have you not worked with various languages on various sized projects?

(In re: Angular, to me that was an obvious shitshow from day one. I would never hire anyone who admitted to ever thinking Angular was a good idea. Same thing in re: Heroku, while I'm at it. Lots of people do lots of foolish things when computers get in the mix.)

> I'm actually into the opposite mantra: only the tech stacks that are great at small scale are capable of being great at large scale. Note: capable, not guaranteed.

I don't see how that follows? (I also don't see how that's "opposite" to the other "mantra".)