The concept of 'type' is overloaded with too many meanings these days. It started out as a what-storage-to-allocate and which-instruction-to-generate (int v float) concept (FORTRAN I, variables starting with I..N are ints, others real). Deep class hierarchies muddied the waters by overlaying the idea of whether or not a class implements a particular property or method, and how to locate the property or method that you want.
Python shows us that types and protocols/interfaces are two very different thing. In Python, asking 'isinstance()' is usually a bad idea unless you are doing something very low level, low level as in next stop is a C extension. The missing predicate in Python is 'hasprotocol()'. Abstract base classes can be used to accomplish that: isinstance(foo, SomeABC), but I think the intent could be more transparent with different syntax.
I like the idea of a hasprotocol or supportsprotocol predicate, certainly over usage of isinstance, since one of the beauties of duck typing is that it works without requiring the type hierarchy. The tricky part will be in allowing you to define your own protocols, like "has these attributes/methods". An example would be the difference (or lack thereof) between classes and namedtuples.
* https://docs.python.org/3.4/library/collections.abc.html * https://hg.python.org/cpython/file/3.4/Lib/_collections_abc....
def __subclasshook__(cls, C):
if cls is Iterable:
if any("__iter__" in B.__dict__ for B in C.__mro__):
return True
return NotImplemented
Wow, hadn't seen this before. That's exactly what I mean. If I can define my own, e.g. Addable, which does a similar check for the presence of __add__ (and __radd__) and MyPy uses the subclasshook, then that'd be perfect.Interesting perspective. This would seem to be pretty well described by parametric polymorphism with constraints (Haskell-style polymorphism with typeclasses). In that context, too, we are often well served staying as generic as possible, and asking at runtime about the type of something is a bit of a code smell (and only possible where you have a Typeable constraint available).
# let f obj x y = (obj#m x)#n y;;
val f : < m : 'a -> < n : 'b -> 'c; .. >; .. > -> 'a -> 'b -> 'c = <fun>
Here I defined a function taking 3 arguments, called method m on obj, passing x as argument, then called method n on the result of that, passing y as argument.Note the inferred type signature for this: the first argument is of type <m : 'a -> <n : 'b -> 'c; ..>; ..> - i.e. any object that has a method named m (and possibly some other unspecified members - that's what ".." means), with said method m having a signature that allows us to pass some 'a, and returning another object of type <n : 'b -> 'c; ..> - i.e. something with a method named n that accepts 'b and returns 'c. 'a and 'b' are then used as types of x and y, respectively, and 'c is the the result of f.
So effectively, this is full-fledged duck typing in the vein of Python objects, but captured on type system level.
Or: Maybe haskelltyping
> mypy considers an int object to be valid whenever a float object is expected.
> Use Iterable for generic iterables (anything usable in `for`)
Thus you should be able to get a more broadly correct (though still not 100% if you have your own types implementing __add__ and __radd__) like this:
from typing import Iterator
def sum_and_stringify(nums: Iterator[float]) -> str:
return str(sum(nums))
It also looks like implementing an abstract base class SupportsSum (to check for __add__ and __radd__) may not take a whole lot of work, see `SupportsInt`: https://github.com/python/mypy/blob/d7f6ea617317103534d86686...http://mypy.readthedocs.io/en/latest/class_basics.html?highl...
Not sure if this is of interest to you, but the Nim[1] programming language offers a feature that allows you to do exactly what you described. It is called concepts and is documented here: http://nim-lang.org/docs/manual.html#generics-concepts
As a bonus, Nim is also quite Python-like.
# optionally: Number = Union[int, float, complex]
def sum_and_stringify(nums: Iterable[Number]) -> str:
return str(sum(nums))
also works. Except no one knows what the Number abstract base class is (or even the Iterable abstract base class for that matter)mypy will then tell you if you used your documented code wrong, according to your documentation
also sum excludes summing of strings if I remember correctly, so its any object that has an __add__ and __radd__ and isn't a string or subclass of string. which is just a bit insane to try and express on one line!
def sum(iterable):
r = 0
for v in iterable:
r += v
return v
it always starts with an integer, and since int.__add__(str) returns NotImplemented it tries str.__radd__(int) instead, which raises an AttributeError because str has no method __radd__, so no summing of strings and integers. >>> sum(['hello', ' world'], '')
TypeError: sum() can't sum strings [use ''.join(seq) instead]
But this is done explicitly against strings. They would _just work_ if not for this restriction, because based on usage they should work.On the point of documentation, this is potentially a fair point if you consider that you probably wrote this function for a particular usage and you know at design time that you're really only ever going to give it a list of ints (if you strip the types away, the parameter is still called nums which suggests that's what was really intended). In reality, the function is actually much more powerful than that, and if included in a library, many people could use it for many more things than just that.
I think all this says is that design and documentation considerations for functions in distributed libraries may be different than those for functions used locally. Something I certainly agree with, but not really a criticism of the parent point.
Correct and concise documentation is incredibly valuable.
For documentation to stay correct, it must be checked and updated as situations change. Types and assertions can both play this role. Tests can play this role.
Names also play this role - they are not mechanically checked, but "is my use here consistent with the name" is a check any programmer is applying as they work. This actually argues a bit against renaming refactor tools - if I am changing a name, I should be making sure that it squares with existing uses. From this perspective, checking of function parameter namess is weaker than checking of function names, since the names of the parameters typically do not appear at the use site - very different than the assignment of other variables.
Which is for practical purposes impossible.
Semantically, the presence of __add__ and __radd__ is a minimum, not sufficient.
Edit to add: To be clear, sure the result of calling this on any arbitrary sequence of addables might be semantically nonsense, but that's not the purpose of a type checker. It's only to verify type safety, i.e. you won't get runtime type errors if you give me at least this kind of thing.
A more constrained type throws away "the advantages of Python's duck-typing", but it also throws away disadvantages, and that can be precisely what you want.
We either disagree about the purpose of a type checker, or we disagree about what can constitute a "runtime type error". If a type checker can catch inconsistencies that would lead to my returning "semantic nonsense", at low enough cost, I absolutely want it to. Obviously it won't be able to catch all such inconsistencies in all cases.
sum_and_stringify :: (Show a, Monoid a, Foldable t) => t a -> String
sum_and_stringify = show . foldl mappend mempty
In this case there are additional rules (semantics, though not in the language) which we associate with monoids like associativity that don't apply to all "addables" (e.g. strings), that might make us feel more comfortable being so general. I think really it's just that we have more expressive language to describe our expectations. Though there's nothing stopping you from making String an instance of Monoid, or most any type for that matter, and producing nonsense. It's not a type error, so I say it's (in this case) not the job of the type system. But there are more expressive type systems that could let you guarantee these additional properties like associativity and turn nonsense into type errors, and I for one am all for that.In the case of Python, though, coming from no static type system or verification at all, I would only ask of this tool to turn runtime checks (type errors caused by arguments lacking __iter__ or __add__) into static checks. Anything on top of that would be fantastic, I agree with you on that, but I wouldn't ask for or expect it.
Finally, thanks for all your comments - I appreciate the discussion.
Ah, we probably do agree more than it seemed. "The MyPy team has done its job even if that's not possible/reasonable" is a statement I'd endorse - especially in the short-to-medium term.
Discussing the Haskell...
String concatenation actually does associate; in fact String does form a Monoid over concatenation, and has an instance visible from Prelude - specifically the list instance, because Haskell strings are just `[Char]`.
The only really "nonsense" thing about using a string here that I see is that we wind up with quotes and escaping in the result when we `show` the string. If the purpose of this is "gather results and format for logging", that might be fine.
Numeric types don't have an instance of Monoid, because there are two to pick. You can pick them with newtypes, but here that's a bit awkward (`sum_and_stringify . map Sum`? feels redundant), and then the Show will give you something like "Sum { getSum = 3 }", which is probably not what you want either...
Using Num rather than Monoid probably has better ergonomics, and may be closer to intent anyway. That is to say:
sum_and_stringify :: (Show a, Num a, Foldable t) => t a -> String
sum_and_stringify = show . foldl (+) 0
Of course, we have a `sum` defined (basically, perhaps literally) `foldl (+) 0` so why even pull this out into a function? `(show . sum)` is actually substantially shorter than `sum_and_stringify` and no less clear.I think the biggest issue here is that the example is so devoid of context (probably necessarily) that it's hard to say what good style would be. In actual code, a function with that name should probably have a very general signature - but we likely shouldn't be writing a function with that name. Why are we wanting to "sum and stringify" as one action in this way? Is that something that might change? Will the uses want to change together or are we just Huffman coding? With a domain-relevant name, we'd have a sense of what ways it makes sense to restrict the types.
> Finally, thanks for all your comments - I appreciate the discussion.
Likewise!
>Using Num rather than Monoid probably has better ergonomics, and may be closer to intent anyway. You're probably right on this point, in all honesty. I think I'm over-generalising the original intent.
>With a domain-relevant name, we'd have a sense of what ways it makes sense to restrict the types Yes, I think this is the classic helper function vs library function quandary. If you wrote it to help yourself right now, it's often good to be very specific with the name and signature. That way if later you need to add more functionality you can without changing the name or restricting the signature. If you wrote it as a library function, it's good to be generic, to let as many people use it as possible. And in this case there should never be a need to change the function, it does what it does. If you need something else you just create something else.
It's so catchy an acronym, and so seemingly self-explanatory, a lot of people don't realize it was originally framed in terms of knowledge, not syntax. So if syntax happens to be the same in N places, but in each of those places it is the way it is for a different reason, consolidating may not be "DRYing up the code". On the other hand, if I need to say "there's a text field here" in my markup and also in my logic, that could be more DRY even though the syntax looks nothing alike.
Tying this back here, we find as a corollary: when factoring out code we can ask ourselves "what piece of knowledge does this represent?" If it is knowledge about the language (this is a good way to sort things) it probably goes in a library, and should probably be very general. If it is knowledge about your domain (this is how we escape HTML), it probably goes in a domain-specific library, and it should probably be somewhat less general. If it is knowledge about your application, it probably goes in your application or an application-specific library, and should probably be pretty specific.
Consider the first example,
The type here is way more specific than it needs to be. Not only will this work on any sequence type (anything that supports __iter__), the element type only needs to support __add__ and __radd__ (to work against the implicit seed value of sum).I really feel like in order to get the best of duck typing, you need a good syntax for describing types by usage, not by explicit name. Please let me know if I'm missing something, and this is actually more expressive than the documentation suggests.