News for hackers. Stuff that matters.
Re:I use Attrs instead of Pydantic by ThePhysicist on Thursday August 26, 2021 @04:23AM (#28312056)
Recently there's a strong tendency in the Python ecosystem to wrap perfectly fine idioms with even higher abstractions. Not sure if I like this, in the end it leads to a lot of cargo culting and people writing code they don't fully understand because it contains a lot of metaclass and runtime magic. But I understand that a lot of developers find this attractive, and type annotations really provide a good channel for realizing such functionality, so maybe I'm just being grumpy here.
Re:I use Attrs instead of Pydantic by jampekka on Thursday August 26, 2021 @06:23AM (#28312927)
One aspect may be that people just aren't that satisfied with some of the design decisions of Python. In this case the (perceived) issue is probably unnecessary verbosity in declaring classes, which I agree with. Due to the lack of macros changes are made through decorators and other metaprogramming magic, which brings about some collateral issues.

I find it preferable that such changes are possible outside the central committee bureaucracy. You really don't want to end up like C++ w.r.t language evolution.

Re:I use Attrs instead of Pydantic by ospider on Thursday August 26, 2021 @04:41AM (#28312189)
For myself, better code completion is enough to lure me into the world of fully type-annotated python. I don't understand what you mean by saying "cargo cult", please enlighten me.
Re:I use Attrs instead of Pydantic by jampekka on Thursday August 26, 2021 @04:47AM (#28312254)
Cargo cult probably refers to doing something based on superficial assesment without understanding what is actually done. Originally refers to problems in science. https://calteches.library.caltech.edu/51/2/CargoCult.htm
Re:I use Attrs instead of Pydantic by nooorofe on Thursday August 26, 2021 @04:37PM (#28320368)
> fully type-annotated python

When I do that, I ask myself: "how is it different from writing Java code?". I don't have a clear answer. It becomes almost as verbose as Java. Java has better type validation (doesn't compile on mistake); it is faster (at least some payback for verbosity); real type system has additional advantages, for example using types as part of method signature.

Re:I use Attrs instead of Pydantic by mister_mister on Thursday August 26, 2021 @06:08AM (#28312806)
I agree, Python is/was about simplicity. That is why for validation I use (Maat)[https://pypi.org/project/Maat/] It's just a dictionary to describe the data, and it's easy to expand.
Re:I use Attrs instead of Pydantic by techhazard on Thursday August 26, 2021 @06:13AM (#28312849)
I also prefer my python to stay simple.

Maat looks really interesting for this use-case.

Re:I use Attrs instead of Pydantic by janto on Thursday August 26, 2021 @06:42AM (#28313063)
Yes, this higher abstractions tendency is clearly away from "Flat is better than nested". Creating good abstractions is so difficult (and important) that it should not be done lightly by everyone.

The recent proliferation of type annotations is also trending towards the static typing mindset, which is also cargo culting on a massive scale.

Re:I use Attrs instead of Pydantic by kortex on Thursday August 26, 2021 @09:02AM (#28314376)
Static typing in python isn't "cargo culting." In fact I'd go even farther in the opposite direction, and say that if you are using python for production software engineering, and aren't typing most (>50%) of your function signatures, you are wasting everybody's time, your teams', code reviewers', and most of all your own time. Opinion, but empirical experience bears this out for me.

I'm not talking mypy --strict here, heck I've yet to achieve that myself. But if your are writing functions with obvious realizations of interfaces and not typing them, all you are doing is creating more mental strain for the next consumer (often future you) of that function down the road.

At $LASTCO I inherited some jupyter-notebook-copy-pasted datascience ML abomination of a pipeline. Dicts everywhere, mutation everywhere, zero docstrings, nary a type hint in sight. Took two exceptionally stressful months, with constant back and forth with the authors, to get it working in prod ("it works on my machine, I don't understand the problem"). $THISCO embraces type hints, functional style, etc. My stress level has actually normalized.

Typing `foo(bar: Optional[float])` takes what, 2s, more than `foo(bar)`? Asking "hey Steve, what does "bar" take in function "foo"? on slack is already more time and characters than just annotating it, times every dev that doesn't know and has to ask.

Re:I use Attrs instead of Pydantic by janto on Thursday August 26, 2021 @10:51AM (#28315700)
The problem with creating excessive types in production systems (which is what I want people to avoid and can happen if you go too far with Pydantic models + type annotations into pseudo-"static typing") is that you move outside the area that Python's core is already designed to generically cover in idiomatic ways. You end up wasting your own (and reviewers') time creating unnecessary code that future you will have to now maintain and extend to more generic cases.

I'm skeptical that the people that produce the kind of abomination you encountered would produce something less horrible if they went to town with static typing. I've seen too much Java to fall for that ;) I'm not averse to putting in some annotations and data validation chokepoints here and there.

I'd ask Steve why he called it `foo(bar)` rather than `set_width(width)`. Static typing fans shift the semantics of code onto a type system. I'd prefer it if they focused more on better naming than spending time on designing complicated Pydantic models.

Re:I use Attrs instead of Pydantic by maleldil on Thursday August 26, 2021 @09:07AM (#28314419)
Type hints just make everything so much easier. When you're writing a function, you _know_ that parameter is an int, or float, or something you can use a list comprehension on (iterable). Why not just say it right there in the function definition?

It's better than writing it in a docstring, because a type checker will tell you to change the type if you change how you use a variable.

Does everyone need to go all the way and type 100% of things and use heavily generic code to represent all possible cases? Well, that would be wonderful, but just sprinkling built-in types is already a massive improvement over no types at all.

Re:I use Attrs instead of Pydantic by janto on Thursday August 26, 2021 @11:08AM (#28315915)
A sprinkling of type annotations to help with ambiguity is nice. However, I don't recall the last time I spent ages figuring out what type I need to pass to a function if it simply wants a builtin type. It's usually the semantics or a library-unique type (ugh!) that I have to look up.
Re:I use Attrs instead of Pydantic by icedchai on Thursday August 26, 2021 @11:20AM (#28316071)
Definitely. This to be especially terrible with some libraries (example: sqlalchemy) which have a crap load of types. You're not really sure what's being returned, etc.
Re:I use Attrs instead of Pydantic by smohare on Thursday August 26, 2021 @08:31AM (#28314113)
I think the comment about types takes a rather narrow view of Python use cases. Sure, if you’re some lone sysadmin using Python as an alternative to bash to implement some questionable ETL pipeline comprising a bunch of random scripts running off a single machine somewhere then I agree. You probably are not the target audience for type annotations.

But even moderately complex libraries and applications are such a huge pain to develop, read, and maintain without some of the tooling that leverages type annotations. This is especially true in the ML / AI / Data Science world where a lot of the people implementing models have dubious coding practices.

Pure dynamic typing paradigms simply have not delivered on their promises over the last 30 years. There are definitely some areas where they make sense, but I doubt we will see a massive readoption until our tooling becomes sufficiently intelligent. Imagine, for example, a probabilistic type inference based on both the structural aspects of the code and previous runs over actual data.

Re:I use Attrs instead of Pydantic by janto on Thursday August 26, 2021 @10:52AM (#28315712)
People in the data science world with dubious coding practices should keep their data in commonly used types such as Pandas tables. The last thing we should want from them is encoding their data with ill-conceived class/type systems. Not sure if that is what you mean.

I'd say Python's large scale adoption is exactly a good example of dynamic typing delivering on its promises.