News for hackers. Stuff that matters.
Re:Pyre: Fast Type Checking for Python by nebgnahz on Friday May 11, 2018 @12:50PM (#17048845)
So, there is a trend to build static type checker for popular managed dynamic languages?

Facebook also built Flow, a type checker for Javascript [1].

[1] https://flow.org/

Re:Pyre: Fast Type Checking for Python by sametmax on Friday May 11, 2018 @03:02PM (#17049992)
For Python it makes a lot of sense.

It's a language that let you start small, and progress a lot with the complexity of your code. You can be productive in Python in 3 days if you know another language. But you can still learn new Python useful things 10 years after you started.

The progress curve is very sane.

And in the same way, your project may start small, then you add docstrings, classes, modules, packaging, unittests, infrastructure... And at some point, you may want types.

I use types on maybe 10% of my code in Python. It's great it's not mandatory. And it's great it's here when I benefit from it.

Re:Pyre: Fast Type Checking for Python by weberc2 on Friday May 11, 2018 @06:49PM (#17051274)
I write Python in my day job and I _always_ start with types. I find they make even prototype code so much easier to reason about. It's just nice to be able to tell at a glance what a function returns, what args it takes, etc., and I don't have to master Sphinx's cryptic syntax (which I still haven't managed after using it for 5 years). And I experience this benefit even for code that I wrote yesterday without even running the type checker. And it's only going to get better with editor integration.

Without types, if I have the code `a = foo(x=1)` then I have to hunt down the source file for `foo()`, which likely just returns `bar(x)`, so I have to hunt down the source file for `bar()` to figure out what the hell its return type is, and so on and so forth. With types, I just look at the type signature for `foo()` and I'm good to go (and again, editor integration means that I don't even need to look up `foo()` at all!).

YMMV.

Re:Pyre: Fast Type Checking for Python by sametmax on Saturday May 12, 2018 @05:49AM (#17053278)
> I write Python in my day job and I _always_ start with types.

It depends of the project. If you write a lot of flask/django code, writing the types is not that worth it except for a few functions/methods.

> Without types, if I have the code `a = foo(x=1)` then I have to hunt down the source file for `foo()`

No, you just hover the function and get the help() out of it in most framework and libs. Again, if your code is mainly using a well define, documented and popular framework/lib/api, that's not a big deal. And it certainly doesn't require YOU to add types.

Or if you write a program that is contained in 1 to 5 files top. Not use for types.

Or if you are writing your program in jupyter.

And you most likely copy a snipet from the doc anyway. After all, if you see that the function you want to use return a AbstractTranscientVectorServiceFactory object, you can't do much with the information without the doc anyway.

But let's be real, most functions in Python are named pretty explicitly, and return things that you expect like "iterable of numbers" or "file like object yielding utf8".

Types are particularly useful in the cases if you are in a big project with a lot of custom code or in a domain either very complex or that you don't master very well. They are a good for of safety net and documentation at the same time.

But they come at a cost and it's a good thing to be able to choose.

Re:Pyre: Fast Type Checking for Python by netheril96 on Sunday May 13, 2018 @12:10PM (#17059622)
> No, you just hover the function and get the help() out of it in most framework and libs. Again, if your code is mainly using a well define, documented and popular framework/lib/api, that's not a big deal. And it certainly doesn't require YOU to add types.

If the help or comments tell you the types of arguments and return values, that is just static typing in the form of comments. Even more verbose than language level type annotations, yet much harder to parse for editors/IDEs/linters.

Re:Pyre: Fast Type Checking for Python by weberc2 on Saturday May 12, 2018 @08:36AM (#17053773)
> No, you just hover the function and get the help() out of it in most framework and libs.

There are no editors that can reliably deliver type information. The ones that come closest (PyCharm and YouCompleteMe) are otherwise very poor editors and resource hogs (the former) or nearly impossible to set up correctly (the latter). And neither is of any use when I'm reviewing someone's code on Github or debugging over ssh.

> Or if you write a program that is contained in 1 to 5 files top. Not use for types.

Types are definitely _useful_ for small programs, but you're right that they're not _necessary_ to the degree that they are for larger programs.

> And you most likely copy a snipet from the doc anyway.

This... is not the kind of programming I do. Although it may explain a lot about our difference of opinion on the topic.

> But let's be real, most functions in Python are named pretty explicitly, and return things that you expect like "iterable of numbers" or "file like object yielding utf8".

Ah yes, the get_values_as_iterable_of_numbers() function. :) Let's be real, virtually _no_ functions are named this way, and even if they were, the typing syntax is surely a better way of facilitating this information. More importantly, it still doesn't tell us anything about the types or number of the arguments. Lots of popular Python libraries (Pandas, Matplotlib) take ridiculous combinations of arguments (two strings and an int sometimes, but other times its an int and a float and a file handle, and if it's called on a Tuesday in April then you can omit all int args). If they had to appease a type checker, these libraries would certainly be much more usable.

> But they come at a cost and it's a good thing to be able to choose.

There are tradeoffs in many areas of programming, but typing is a clear win. Specifically, what is the cost you mention? The type checker won't let your type documentation grow stale? Or perhaps it makes it hard to write clumsy signatures like those in Pandas and Matplotlib?

Re:Pyre: Fast Type Checking for Python by neves on Saturday May 12, 2018 @05:22PM (#17056080)
I'll will also learn because the language is constantly evolving and there's always new toys to play:-)

I program Python for 24 years now. It is my preferred language, but just recently I could go back to develop in it. It is impressive how much new idioms there are to learn.

Re:Pyre: Fast Type Checking for Python by m0meni on Friday May 11, 2018 @12:57PM (#17048895)
Also they're rewriting messenger in basically OCaml (Reason[0]) i.e. a statically typed FP language.

[0]: https://reasonml.github.io/

Re:Pyre: Fast Type Checking for Python by dons on Friday May 11, 2018 @01:26PM (#17049189)
And "eradicate" for Java... http://fbinfer.com/docs/eradicate.html

we love types. They help us ship stuff faster.

Re:Pyre: Fast Type Checking for Python by laylomo2 on Friday May 11, 2018 @03:45PM (#17050317)
This should be upvoted higher. ReasonML is a fantastically beautiful language.
Re:Pyre: Fast Type Checking for Python by weberc2 on Friday May 11, 2018 @06:54PM (#17051300)
Yeah, it just needs to work on its tooling and library story. Last time I tried, the OCaml toolchain was such a big pain that the Reason community basically told me to give up and compile to Node. And then there are relatively few native-ReasonML libraries, so you have to figure out how to integrate with OCaml which usually means learning how to _read_ OCaml without clawing your eyes out (I'm only sort of joking).

I'm rooting for Reason, but it has a few nontrivial hills to climb before it's practical.

Re:Pyre: Fast Type Checking for Python by Rotareti on Saturday May 12, 2018 @08:52PM (#17056842)
I find it already very practical for web frontend work. Editor support is great if you use VSCode.
Re:Pyre: Fast Type Checking for Python by weberc2 on Sunday May 13, 2018 @12:27PM (#17059700)
That's encouraging to hear. Unfortunately I don't do much frontend web development, but hopefully if the frontend web-dev story catches on then the backend will follow.
Re:Pyre: Fast Type Checking for Python by lacksconfidence on Friday May 11, 2018 @12:53PM (#17048862)
They also transformed php into a very similar language (hacklang) and built a type checker for that.
Re:Pyre: Fast Type Checking for Python by GordonS on Friday May 11, 2018 @12:58PM (#17048911)
It does beg the question: why not use a statically typed language in the first place?
Re:Pyre: Fast Type Checking for Python by bastawhiz on Friday May 11, 2018 @01:07PM (#17049003)
If the question is "why choose Python" or "why choose JavaScript", then I think you're missing the point. Python is simple and beautiful and highly productive to work in. JavaScript is ostensibly the only language that you can use in the browser without a compilation step (to JavaScript). Just because typing is added separately doesn't negate all of the other benefits of these languages. Hell, if all programmers cared about was type safety we'd all be writing Rust for every project.
Re:Pyre: Fast Type Checking for Python by drb91 on Friday May 11, 2018 @02:09PM (#17049559)
> If the question is "why choose Python" or "why choose JavaScript", then I think you're missing the point. Python is simple and beautiful and highly productive to work in.

Well, it depends on the problem! If you’re working with a lot of data structures, or with bytes and serialization, compiled system languages with static typing are going to be productivity boosts over python. Languages aren’t everything, but there are definitely poor language/problem space fits.

Re:Pyre: Fast Type Checking for Python by sametmax on Friday May 11, 2018 @03:06PM (#17050013)
> If you’re working with a lot of data structures, or with bytes and serialization, compiled system languages with static typing are going to be productivity boosts over python.

Python has fantastic, expressive and productive libraries for those kind of problems. Unless you have some performance issue, I can't see how you are going to be faster than that.

Re:Pyre: Fast Type Checking for Python by drb91 on Monday May 14, 2018 @10:53PM (#17071281)
Python does address those needs with librares. In other languages it’s not necessary because they fit the mold out of the box
Re:Pyre: Fast Type Checking for Python by owaislone on Friday May 11, 2018 @01:15PM (#17049081)
I think languages like Go are changing that. Most of the statically types languages have been very verbose and often more lower level than people like. If you are building a new web service, python, ruby or JS are very attractive as they help to get something out of the door very quickly. I think Go hits the right spot here between verbosity and safety/expresivness, and a lot of new startups are choosing Go for things they would have built in Python or Node. I think this will only improve as more dynamic languages start adopting static typing and most static languages start adopting things like type inference, etc.
Re:Pyre: Fast Type Checking for Python by Captain_Usher on Friday May 11, 2018 @05:43PM (#17050983)
>I think Go hits the right spot

You make an excellent point here that I don't see articulated very often. The Landscape of Popular Languages, let's say C/C++/Java/Python/Ruby/JavaScript, has a big gap in the middle. You have good choices between:

1. lower level "systems" oriented languages with static types, usually compiled, like C++. You get lots of flexibility and direct access to primitives. Static types help wrangle big codebases. Generally suited to large projects.

2. higher level "scripting" oriented languages with dynamic types, usually interpreted, like Python. Writing code for most tasks is easier. You give up some stuff you'd want for projects like operating systems or databases. Most projects aren't operating systems or databases, so that's usually a good tradeoff. Generally suited to small projects.

The problem is that lots of projects are medium-ish. You set out to build your web service backend or whatever, it would be a pain in the ass to write in C++, so you use Python. Getting it working is quick and easy. A few months later it's big, complex piece of software and working on it in Python is a pain in the ass. You can't win. What you really wanted was a language that's "easy to write" like Python, but with static or optional types, maybe better thread handling, and at this point the interpreter isn't doing much for you so it might as well be compiled. There are tons of cases where you just want a "better C" or "Python but faster and with static types", and for the longest time the Landscape of Popular Languages just had a giant hole there.

We needed that space filled and Go delivered. I'm usually very critical of Go, but I can't hate on it for being the wrong kind of language. It's definitely the right kind of language for these "goldilocks" problems that aren't too high or too low level, too big or too small. Part of being a good programmer is understanding that languages are tools, and you need to pick one that fits your problem. Go deserves all the success and praise it's gotten for being a language that fits actual problems.

Re:Pyre: Fast Type Checking for Python by fithisux on Friday May 11, 2018 @02:48PM (#17049872)
I agree.
Re:Pyre: Fast Type Checking for Python by electrum on Friday May 11, 2018 @01:25PM (#17049170)
These gradual type checking frameworks allow migrating massive code bases incrementally. Doing a complete big-bang rewrite into a different language isn't feasible (for the obvious software engineering and business reasons).
Re:Pyre: Fast Type Checking for Python by growtofill on Friday May 11, 2018 @08:57PM (#17051851)
So you can introduce a type system in a module, but you can’t write new version of that module in a different language? To me these seem comparable in scope.
Re:Pyre: Fast Type Checking for Python by netheril96 on Friday May 11, 2018 @11:55PM (#17052403)
They are in no way comparable in scope. Introducing a type system gradually doesn't break any compatibility, within the module or outside the module. It doesn't require you to understand what the legacy code is doing. Rewriting in a different language also means different libraries, different glue code, different interfaces, different everything.
Re:Pyre: Fast Type Checking for Python by alangpierce on Saturday May 12, 2018 @12:10AM (#17052435)
I've been adding TypeScript types to a bunch of JavaScript code recently, and it really is much safer and faster than trying to move to a different language. As long as you are only adding types, you don't have any chance of introducing a bug, since the types just compile away, and plenty of real-world code (especially untyped code) tends to have subtle details that you might overlook if you're reading it for the first time.

Another thing to keep in mind is that type annotations can be done incrementally. You can't take 1000 Python files and port them to Go one by one, but you can add types a little bit at a time with a completely usable codebase at every intermediate step.

Re:Pyre: Fast Type Checking for Python by unethical_ban on Friday May 11, 2018 @01:10PM (#17049036)
The rigor and concepts and generics and interfaces required to properly use staticly typed languages at scale are not always necessary. It's nice for it to be a "warning mode" when you want it, instead of having to design and build that from the beginning.
Re:Pyre: Fast Type Checking for Python by paulmd on Friday May 11, 2018 @01:32PM (#17049246)
I guess it comes down to a difference of philosophy. Never in my life have I ever felt compelled to shove an object of the completely wrong type into a function and see what happens. In the overwhelming majority of cases, the answer is going to be "it crashes", so there is no reason to do this.

The point of type-checking is that the function needs to work on a duck, then there is no point to ever pass it anything except a duck. In fact,the compiler should not even let you pass it something that's not a duck, because it's so pointless. That's literally the start and end of static typing, and if that's "rigorous" then yes, the whole point of static typing is to introduce this very basic level of rigor into your codebase. Because it's not going to work regardless of whether it passes the compiler.

Pretending interfaces do not exist does not actually make them go away. There is an interface there whether you explicitly enumerate it or not... even duck typing will fail if you try to call duck functions on something that is not a duck. Dynamic typing is not magic, it's the equivalent of passing everything around as Object or String in a static language. And that's an anti-pattern.

http://wiki.c2.com/?StringlyTyped

If you just want something to compile, you can pass in null-values of the appropriate type.

Now: there is a valid complaint that Java in particular really embraces the architecture-astronaut philosophy where everything is an overly-abstracted AbstractSingletonProxyFactoryBean (a convenient superclass for FactoryBean types that produce singleton-scoped proxy objects!). But usually it's fairly simple to wall that badness off from your actual business logic.

Re:Pyre: Fast Type Checking for Python by unethical_ban on Friday May 11, 2018 @03:55PM (#17050380)
>The point of type-checking is that the function needs to work on a duck, then there is no point to ever pass it anything except a duck.

If I know I'm only going to pass ducks, and the software is quick and dirty and non-critical enough that it doesn't matter if I accidentally pass it a goose, then I don't want to have a general contractor standing next to me developing saying "make sure that's a duck!"

Re:Pyre: Fast Type Checking for Python by paulmd on Saturday May 12, 2018 @02:05AM (#17052701)
> If I know I'm only going to pass ducks, and the software is quick and dirty and non-critical enough that it doesn't matter if I accidentally pass it a goose, then I don't want to have a general contractor standing next to me developing saying "make sure that's a duck!"

There is no situation that is so "quick and dirty and non-critical" that you would want an invalid function invocation that could never, ever possibly succeed.

Again, if you just want a dummy call while you're refactoring so that things compile, pass in nulls in those parameters. But there is literally no reason to ever pass totally invalid but real data to a function.

I'm literally at a loss for any situation where you would ever pass an apple to a function that expects a duck, and think that's not just a valid construct, but a positive one that a language should encourage. It just boggles the mind how stupid that is.

Guess people just hate nulls that much, even as a parameter to a dummy stub, that they're willing to give away type safety.

If you really, really, really want to do a construct like that in a typed language, you can always just do "myFunc((Duck) myApple)" to pass the apple as an instance of type Duck, but if a duck is not an apple then that's guaranteed to fail at runtime, just like in the dynamically typed language (because dynamic typing is not magic).

Re:Pyre: Fast Type Checking for Python by unethical_ban on Friday May 18, 2018 @04:00PM (#17104528)
Let me put it this way.

If am writing something quickly, and it's a small project or test, and I might be testing out whether I want to pass a single value or a list in my workflow, I don't want the rails on constantly. I don't want to guarantee safety by ctrl+H-ing my "str" to "Iterable" constantly. Move quickly, break things, get the product out. At least, when the product is a quick and dirty tool and the options are "spend 30 minutes to get it built" or "don't build it".

Re:Pyre: Fast Type Checking for Python by jhall1468 on Friday May 11, 2018 @01:41PM (#17049322)
> The point of type checking is that it has to be rigorous, it doesn't do anything for you if it doesn't check types.

I don't always need to check types. I don't need to know that isReady returns a bool. It clearly returns a bool. This idea that type checking has to be absolute is because in statically typed languages it does, not because that's a forgone conclusion.

Re:Pyre: Fast Type Checking for Python by merlincorey on Friday May 11, 2018 @02:52PM (#17049904)
> I don't always need to check types. I don't need to know that isReady returns a bool. It clearly returns a bool.

I'm sorry but you're mistaken, friend. I checked the signature of that function and it actually returns an enumeration of ready states:

    enum class Ready { Insert, Process, Clean };
    const Ready isReady();
Static typing is extremely useful for knowing what things return - names are not.
Re:Pyre: Fast Type Checking for Python by jhall1468 on Friday May 11, 2018 @06:48PM (#17051268)
> Static typing is extremely useful for knowing what things return - names are not.

Only if you completely ignore convention (that is prefixes return bools) then, sure, it's not useful. But if you just want to break conventions, then all of this is moot because you are guaranteed to do stupid stuff outside of that.

Re:Pyre: Fast Type Checking for Python by elbear on Saturday May 12, 2018 @02:25AM (#17052753)
Even if you don't want to break conventions, you might do it by accident. If you don't have a compiler (or a static checker of some sort), there's no one there to keep you consistent.
Re:Pyre: Fast Type Checking for Python by Too on Saturday May 12, 2018 @12:50AM (#17052529)
You are assuming that your usage of isReady will be flawless every time. What if you call a function updateUser("username", isReady()) but it turned out you remembered the order of arguments wrongly. Then the type checker will tell you as you can't pass a bool to a string argument and vice versa. This type of error is super common in %-string formatting.

It won't help if all the arguments have the same type unfortunally but at least it catches something. Just like unit tests, they won't catch everything so the more checks we have and the earlier we run them the less risk there is something slips through to production.

Re:Pyre: Fast Type Checking for Python by m0meni on Friday May 11, 2018 @12:59PM (#17048923)
With flow, you just gradually add typings and fix the resulting type errors. It's way easier to slowly integrate types file by file rather than rewriting everything.

Moreover I think that at the time there were no statically typed languages that targeted web.

Re:Pyre: Fast Type Checking for Python by camdenreslink on Friday May 11, 2018 @01:02PM (#17048948)
Java has targeted the web since forever. Ditto for C#.
Re:Pyre: Fast Type Checking for Python by m0meni on Friday May 11, 2018 @01:05PM (#17048974)
By target web, I meant compile to js with good interop. I.e. no languages like elm, purescript, reason, ghcjs, kotlin, etc.
Re:Pyre: Fast Type Checking for Python by zenhack on Friday May 11, 2018 @09:12PM (#17051914)
Reason/ocaml-bucklescript's js interop is actually very good -- it's one of the nicest FFIs I've ever seen.
Re:Pyre: Fast Type Checking for Python by sidlls on Friday May 11, 2018 @01:34PM (#17049272)
There are many reasons. Some positive ones are: unnecessary complexity for the initial project, prototyping, exploratory analysis for machine learning/data science, etc. Some negative ones are: fear/ignorance about complexity, incorrect approach to testing (e.g. preference for many fine-grained unittests that effectively act as pseudo-compilers and type checkers under the mistaken belief the code is being tested), incorrect understanding of the tooling/operational support required, etc.
Re:Pyre: Fast Type Checking for Python by p3llin0r3 on Friday May 11, 2018 @01:36PM (#17049281)
Dynamic languages tend to be more popular. Examples:

- ruby

- python

- Javascript

I was on a quest for a good, well supported, statically typed scripting language for quite a while, and in the end I defaulted to python.

Re:Pyre: Fast Type Checking for Python by umanwizard on Saturday May 12, 2018 @12:53AM (#17052544)
C, C++, and Java are incredibly popular too.
Re:Pyre: Fast Type Checking for Python by p3llin0r3 on Saturday May 12, 2018 @12:43PM (#17054836)
True, but all of those languages are very heavy-handed and have ancient and arcane build-tools and package management ( If they have it at all )

For me, I was looking for a language that is appropriate for both scripting and slightly larger web application projects.

Scripting / Web application development is NOT a great use case for C/++, and I feel Java is a bit heavy handed to be used for a scripting language.

I tried out Golang but did not end up liking the language.

Re:Pyre: Fast Type Checking for Python by weberc2 on Friday May 11, 2018 @10:46PM (#17052229)
Sometimes you find yourself with a legacy codebase. Other times you find yourself with a team that bucks hard at the thought of a statically typed language. Politics is hard.
Re:Pyre: Fast Type Checking for Python by sidcool on Friday May 11, 2018 @01:45PM (#17049354)
How's people's experience with Flow?
Re:Pyre: Fast Type Checking for Python by ggregoire on Friday May 11, 2018 @02:31PM (#17049720)
I'm using Flow but I'd recommend TypeScript. I love Flow but TypeScript just seems to be more seriously maintained and less buggy. I've been waiting some bug fixes for 2 years (like all the object destructing and spread issues[1]), but they prioritize the Facebook private internal roadmap before anything else (like they improve the performances in almost every patch, but it's useful only for FB and its millions of lines of code). So yeah, the support is kind of inexistent and it's frustrating. When you find a bug, you have to rewrite your code in another way to workaround…

On the other hand, I don't agree with some of the comments here. For example, Flow is not terminal only at all, I never use the terminal to run Flow. The editor integration is totally fine, especially in Atom and VSCode.

[1] https://github.com/facebook/flow/issues/2405

Re:Pyre: Fast Type Checking for Python by m0meni on Friday May 11, 2018 @01:51PM (#17049403)
JS with flow is way better than JS without flow. It prevents a whole class of errors, and you can basically opt-in and opt-out of the type system at any time if you're having trouble with it too. I wrote a blog post about it a while back: https://www.aria.ai/blog/posts/why-use-flow.html

That being said, while I started out using Flow, TypeScript just has way more community adoption and better tooling. So at this point, I usually recommend TS over Flow to most people. But using either of them is way better than writing just regular JS code.

Re:Pyre: Fast Type Checking for Python by kevinsd on Friday May 11, 2018 @10:14PM (#17052126)
Github stats on number of issues

# Totoal #

Flow: 2081 open, 2748 closed

Typescript: 2541 open, 14421 closed

# Between 4/11 and 5/11 #

Flow: 88 closed, 93 new

Typescript: 448 closed, 173 new

Note: We are using flow but we are not happy that Flow is not up to speed with answering questions or addressing concerns.

Re:Pyre: Fast Type Checking for Python by Epskampie on Friday May 11, 2018 @02:13PM (#17049585)
Typescript has way better editor support, and is faster on large projects. Flow is mostly terminal only, and required a really annoying syntax in it's config file to exclude all node modules.

That being said, Flow is better then no typechecks, and it was half a year ago that I looked, stuff might have improved.

Re:Pyre: Fast Type Checking for Python by xab9 on Friday May 11, 2018 @02:24PM (#17049665)
TS > flow.
Re:Pyre: Fast Type Checking for Python by wvenable on Friday May 11, 2018 @03:02PM (#17049993)
PHP has static type checking now, as well.