News for hackers. Stuff that matters.
Re:Python Type Hints – *args and **kwargs by akasakahakada on Sunday August 27, 2023 @09:55AM (#37282759)
Although these two comes in handly, people have been using them wrong. Often in scientific open source package, they slap *kwargs in function definition without documentation. How am I suppose to know what to pass in?

https://qiskit.org/ecosystem/aer/stubs/qiskit_aer.primitives...

Re:Python Type Hints – *args and **kwargs by Syntaf on Sunday August 27, 2023 @10:26AM (#37283034)
Especially when they don't even leave a doc string so you're forced to track down the packages documentation online just to interact with certain interfaces.

I work in a large python codebase, we have almost no usage of `*kwargs` beyond proxy methods because of the nature of how they obfuscate the real interface for other developers.

Re:Python Type Hints – *args and **kwargs by hqudsi on Sunday August 27, 2023 @10:53AM (#37283281)
When I was first starting out, a then senior engineer told me: "friends don't let other friends use kwargs".

That always stuck with me.

Re:Python Type Hints – *args and **kwargs by icedchai on Sunday August 27, 2023 @11:18AM (#37283491)
I once worked on a code base where we had *kwargs passed down 4 or 5 layers deep (not my idea.) It was a true joy.
Re:Python Type Hints – *args and **kwargs by akasakahakada on Sunday August 27, 2023 @11:29AM (#37283603)
This is literally me. It is a math program that can evaluate equations and generate code. 6 layers of heterogeneous data structure which the math operation being act on 1st layer has its effect down to 6th layer. Temporarily using *kwargs to make it works but still thinking what is the proper way to do it right.
Re:Python Type Hints – *args and **kwargs by crazydoggers on Sunday August 27, 2023 @12:13PM (#37284034)
Can you organize the data structures into classes or dataclasses?
Re:Python Type Hints – *args and **kwargs by akasakahakada on Sunday August 27, 2023 @12:34PM (#37284244)
Already doing this. The problem is there are 5 layers in between. Copy and paste the same docstring into all layers is doable but do not seem smart.
Re:Python Type Hints – *args and **kwargs by c32c33429009ed6 on Sunday August 27, 2023 @12:13PM (#37284035)
Out of interest, what sort of company/industry do you work in where you're able to work on this kind of thing?
Re:Python Type Hints – *args and **kwargs by nerdponx on Sunday August 27, 2023 @11:15AM (#37283469)
The worst is when someone puts **kwargs at the base of a class hierarchy, not only necessitating its use in subclasses (if you want to be strict about types) but also swallowing errors for no good reason. Fortunately I think this style is fading out as type hints become more popular.
Re:Python Type Hints – *args and **kwargs by toxik on Sunday August 27, 2023 @10:19AM (#37282965)
Sadly a problem with any wrapper function is that it nullifies this kind of information. Use functools.wraps.
Re:Python Type Hints – *args and **kwargs by akasakahakada on Sunday August 27, 2023 @10:23AM (#37283020)
My question is that can @warps wrap more than 1 function?

Maybe in some use case people need to merge 2 functions into 1, I don't know if it can handle this situation.

Re:Python Type Hints – *args and **kwargs by zbentley on Sunday August 27, 2023 @10:30AM (#37283072)
I'm not sure what it means to "merge two functions into one", can you elaborate?

If you are referring to a type signature for a function that passes through it's arguments to one of two inner functions, each of which has different signatures, such that the outer signature accepts the union of the two inner signatures, well ... you could achieve that with ParamSpecs or similar, but it would be pretty hard to read and indirected. Better, I'd say, to manually express appropriate typing.Union (|) annotations on the outer function, even if that is a little less DRY.

Re:Python Type Hints – *args and **kwargs by cbarrick on Sunday August 27, 2023 @11:04AM (#37283378)
> I'm not sure what it means to "merge two functions into one", can you elaborate?

I'm not OP, but I see this pattern often enough:

    def foo(**kwargs):
        pass

    der bar(**kwargs):
        pass

    def wrapper(**kwargs):
        foo(**kwargs)
        bar(**kwargs)
Re:Python Type Hints – *args and **kwargs by akasakahakada on Sunday August 27, 2023 @11:30AM (#37283610)
Yup, this exactly.
Re:Python Type Hints – *args and **kwargs by toxik on Monday August 28, 2023 @04:48AM (#37291429)
What would you have wraps() do for two functions? Concatenate the docstrings and union of the annotations? Perhaps you want only the latter, that seems like it could easily be its own decorator: `@functools.annotate_intersection(f1, f2, …)`
Re:Python Type Hints – *args and **kwargs by zbentley on Thursday August 31, 2023 @11:17AM (#37338524)
That makes sense. If you have two functions with identical type signatures, then you should be able to invoke @functools.wraps (or its underlying functools.update_wrapper) such that the annotations are propagated correctly. In this case, that might be as simple as "functools.wraps(assigned=('__annotations__'))".

The rest of what "wraps" does isn't really applicable to what you're trying to do, though: propagating __doc__, __name__ or __module__ etc. isn't really meaningful when combining two functions with different names, as toxik points out.

Coming at it from the other side, you can use typing.ParamSpecs to write a higher-order "wrapper()" that wraps 2 arbitrary functions with identical signatures, like this:

    S = typing.ParamSpec('S')
    R = typing.TypeVar('R')
    def wrapper(func1: typing.Callable[S, R], func2: typing.Callable[S, R]) -> typing.Callable[S, typing.Tuple[R, R]]:
        def inner(**kwargs):
            return func1(**kwargs), func2(**kwargs)
        return inner

You can additionally use typing.Concatenate to indicate additive modifications to those signatures if the two inner signatures aren't the same.

However, what I thought you meant originally is the common problem of deduplicating large/complex function type signatures so you don't have to write them out multiple times or risk drift. In the parent post, consider what would happen if the signatures of "wrapper", "foo", and "bar" were all be a) large and b) the same/very similar. Python's answers to that problem are much less good:

1. Duplicate them and write a unit test that ensures that the __annotations__ property of the functions that should have the same signatures remain in sync (or sub/supersets of each other, or whatever you'd prefer). This addresses drift, but requires a testing system and doesn't save the duplicate code.

2. A hack: use object constructors for functions. Write the in-common parts of the signature in the constructor of a parent class, and then put the body of "foo" and "bar" in the __init__ methods of child classes whose instances are not used/are useless, taking advantage of the superclass relationship to indicate to typecheckers that the parameters are all shared. For example, many typecheckers will do the right thing when handed code like this:

    class _Super:
        def __init__(self, arg1: SomeType, arg2: SomeOtherType, ...):
            self.arg1, self.arg2 = arg1, arg2
            
            
    class foo(_Super):  # Bizarre casing is intentional, this is not meant to be used as an instance
        def __init__(self, *args, **kwargs):
             super(*args, **kwargs)
             ...  # Business logic of old "foo()" method goes here, using object fields self.arg1 etc. instead of named variables.
             
    class bar(_Super):  # Bizarre casing is intentional, this is not meant to be used as an instance
        def __init__(self, *args, **kwargs):
             super(*args, **kwargs)
             ...  # Business logic of old "bar()" method goes here, using object fields self.arg1 etc. instead of named variables.
             
    class wrapper(_Super):
        def __init__(self, *args, **kwargs):
            foo(*args, **kwargs)
            bar(*args, **kwargs)
This does solve the duplication problem, and you can use dataclasses with __post_init__ methods for your "foo"/"bar" business logic to smooth out some of the boilerplate and weirdness there, but it remains a very bizarre coding style which substantially trades away readability/familiarity (and performance, if this is on a very hot path) in return for type-checker-friendliness.

3. Use a databag object (ideally an immutable slotted class, dataclass, typing.NamedTuple, [c]attrs, or something of that sort) to encompass all the data that would previously go in your argument signature, so that "wrapper", "foo", and "bar" all end up taking a single such object as their sole argument and accessing fields of that argument to do their work. This is probably (maybe? Lots of Scotsmen in this area...) the most traditionally Pythonic of these options, but is still a far cry from the convenience of something like functools.wraps.

Re:Python Type Hints – *args and **kwargs by franga2000 on Sunday August 27, 2023 @11:26AM (#37283572)
PyCharm usually figured this out if it's not too complex. I often wrap session.request() with some defaults/overrides and autocomplete usually shows me the base arguments as well.
Re:Python Type Hints – *args and **kwargs by tomn on Sunday August 27, 2023 @11:34AM (#37283638)
OT, but this is my number one peeve with code documentation: going to the effort to write a doc comment, taking up at least 6 lines, probably using some special syntax, cluttering up the code, but then adding no information that can't be derived from the signature.

If you're not going to document something (which I totally respect), at least don't make the code worse while doing it.

Re:Python Type Hints – *args and **kwargs by ddejohn on Monday August 28, 2023 @08:37PM (#37302119)
Even better when the docstring doesn't even match the signature.
Re:Python Type Hints – *args and **kwargs by dr_kiszonka on Sunday August 27, 2023 @01:09PM (#37284592)
I have been annoyed by this too! I like how seaborn handles it now in documentation: https://seaborn.pydata.org/generated/seaborn.barplot.html?hi...