News for hackers. Stuff that matters.
Re:Dynamic type systems are not inherently more open by svat on Sunday January 19, 2020 @10:04AM (#22091768)
Six notable things I took away from this post:

- Structural typing, i.e. instead of "you eagerly write a schema for the whole universe", just limit to what you need (basically, encode only the same kinds of assumptions you would make in a dynamically-typed language).

- It’s easy to discover the assumptions of the Haskell program [...] In the dynamically-typed program, we’d have to audit every code path — Left implicit in many debates about these topics are the "weights" that one attaches to these things, the importance and frequency of attempting such activities. Surely they vary, depending on everything from the application (Is it "code once and throw it away", or does it need to be maintained when the original programmers have left?) and down to the individual programmer's approach to life (What is the cost of an error: how bad would it be to have a bug? Etc).

- "This is a case of improper data modeling, but the static type system is not at fault—it has simply been misused." — To me this shows that bugs can exist in either the data modeling or the code: in a dynamically-typed language the two tend to coincide (with much less of the former) and in a statically-typed language they tend to be separate. (Is this good or bad? On the one hand you can think about them separately, on the other hand you have to look for bugs in two places / switch between two modes of thought, but then again maybe you need to do that anyway?)

- Structural versus nominal typing, where the latter involves giving a name to each new type ("If you wish to take a subselection of a struct’s fields, you must define an entirely new struct; doing this often creates an explosion of awkward boilerplate").

- "consider Python classes, which are quite nominal despite being dynamic, and TypeScript interfaces, which are structural despite being static." — This is highly illuminating, and just this bit (and the next) elaborated with some examples would make for a useful blog post on its own.

- "If you are interested in exploring static type systems with strong support for structural typing, I would recommend taking a look at any of TypeScript, Flow, PureScript, Elm, OCaml, or Reason [...] What I would not recommend for this purpose is Haskell, which [is] aggressively nominal"

For what it's worth, my opinion is that posts like this, on hotly debated topics, would do well to start with concrete examples and be written in the mode of conveying interesting information/ideas (of which there are a lot here) rather than being phrased as an argument for some position, which seems to elicit different sorts of responses — already most of the HN comments here are about static versus dynamic type systems in general, rather than about any specific ideas advanced by this post.

Re:Dynamic type systems are not inherently more open by svat on Tuesday January 21, 2020 @06:25AM (#22105960)
After giving it enough time and rereading, I feel I have a better understanding. The crux IMO is here:

> The above JavaScript code makes all the same assumptions our Haskell code does: it assumes event payloads are JSON objects with an event_type field, and it assumes signup payloads include data.user.name and data.user.email fields.

What the post points out is that these exact same assumptions, and the same behaviour of what to do with an unknown event type, can be encoded into the Haskell types and code. Similarly, in the case of “val = pickle.load(f)” in Python, the moment we try to do anything with “val” we'll inevitably make some assumptions about it, and those assumptions are its type. (For example, if all we assume that `val` has a `.foo()` which returns a string, then that type can be expressed in a static type system, even in Java.) And for proxying unknown data, just don't specify more in your type than you actually need.

So the post repeatedly emphasizes that static typing does not mean “classifying the world” or “pinning down the structure of every value in a system” (and conversely with dynamic typing we never can process truly unknown data: whatever we assume, that's the type). Overall, the key thesis (“static type systems are not fundamentally worse than dynamic type systems at processing data with an open or partially-known structure”) seems (to me) convincingly demonstrated, and it seems everyone who understands the argument can get on board with her conclusion:

> There are many patterns in dynamically-typed languages that are genuinely difficult to translate into a statically-typed context, and I think discussions of those patterns can be productive. The purpose of this blog post is to clarify why one particular discussion is not productive

----

However, I think the word “inherently” or “fundamentally” is doing a lot of work here, to the extent that if the words “are inherently” were to be replaced with “tend to be”, then a good argument could be made afresh. The post admits to much the same thing, around “These two styles facilitate very different flavors of programming”:

> many dynamically typed languages idiomatically reuse simple data structures like hashmaps to represent what in statically-typed languages are often represented by bespoke datatypes (usually defined as classes or structs). [...] A JavaScript or Clojure program may represent a record as a hashmap from string or symbol keys to values, written using object or hash literals and manipulated using ordinary functions from the standard library that manipulate keys and values in a generic way.

and points out that this has many advantages (“the practical, pragmatic advantage of a more structural approach to data modeling”) to the way things are done in Haskell, and even more so in “all mainstream, statically-typed OOP languages”. These advantages are coming to “modern statically-typed languages” (see list).

So does this defeat the point of the whole post, if ultimately the approach usually taken in dynamic languages has advantages over the approach usually taken in ("typical") statically-typed languages? No, it's an matter of clarity — it's worth it to be clear when the issue is structural-versus-nominal-typing, not static-versus-dynamic-typing. So all this:

> may give programmers from a dynamically-typed background who have historically found statically-typed languages much more frustrating to work with a better understanding of the real reason they feel that way

Re:Dynamic type systems are not inherently more open by svat on Tuesday January 21, 2020 @07:43PM (#22113415)
Now for the flip side. IMO the post also contains "enough rope" to hang static typing with. The picture that emerges is that programming in a statically typed language, as the author conceives it, consists of separate phases of "data modelling" (making assumptions about the external world, and at each module/function level about the inputs and outputs), and writing code. Often the former is the hard part to get clear, and this explains the phenomenon familiar to programmers in a statically typed language like Haskell, that once you get the types exactly right, the code basically writes itself: the types guide you towards the right code, like gravity.

But unanswered are questions like:

1. Should they be separate? In a dynamically typed language the two are blended, you write and run code (with representative inputs) to discover and refine your assumptions. You don't have to switch between two modes of thought. In a statically typed language, bugs in your assumptions will be pointed out by the compiler, and bugs in your logic will be pointed out by the program's execution.

2. Is it always worth being explicit about your assumptions? One can often get useful work done even with unclear, undefined, or even inconsistent assumptions.

3. Fine, the same set of assumptions can be encoded in both kinds of languages, but how different is the experience of arriving at the right assumptions?

Look at these quotes from the post, and think about the words "only", "just", "simply":

> static type systems only make already-present assumptions explicit

> We just have to be explicit about ignoring it.

> A static type system doesn’t require you eagerly write a schema for the whole universe, it simply requires you to be up front about the things you need.

Being explicit or "up front" in this way surely has a cost. At the same time, not being explicit also has a cost (of bugs). What the trade-off? (One does not always operate in environments where bugs have a high cost. For example, even code that does the right thing for 90% of users, but something catastrophic for 10% of them--like lose all their data--may be ok, depending on the importance of the "data" (previous scores in your game?), what expectations you've made clear, etc.)

In fact, too-enthusiastic data modeling can also lead to its own kinds of bugs:

> Suppose we’re consuming an API that returns user IDs, and suppose those IDs happen to be UUIDs. A straightforward interpretation of “parse, don’t validate” might suggest we represent user IDs in our Haskell API client using a UUID type [..] this representation is overstepping our bounds. [...] This is a case of improper data modeling, but the static type system is not at fault—it has simply been misused.

The claimed benefits are also all similar, and may not often be wanted:

> easy to discover the assumptions of the Haskell program just by looking at the type definitions [...] In the dynamically-typed program, we’d have to audit every code path

> If we want to ensure it [UserId field inside SignupPayload type] isn’t actually needed [..] we need only delete that record field; if the code typechecks, we can be confident...

> ensure application logic doesn’t accidentally assume too much

> the type system helped us here: it caught the fact that we’re assuming the payload is a JSON object, not some other JSON value, and it made us handle the non-object cases explicitly.

> The runtime representation of a UserId is really just a string, but the type system does not allow you to accidentally use it like it’s a string

> the parsing style of programming has helped us out, since if we didn’t “parse” the JSON value into an object by matching on the Object case explicitly, our code would not compile, and if we left off the fallthrough case, we’d get a warning about inexhaustive patterns.

Whether this is really a help is probably the big question. Incidentally, while I was rereading and thinking about this, came across a related paragraph in an unrelated article:

> requires covering 100% of the conditions, not just 99%. Edge cases must have applicable code, even when the programmer doesn’t yet know what the happy path should do. During early development, these edge cases can often be addressed by causing the program to crash, and then rigorous error handling can be added at a later point. This is a different workflow than in languages such as Ruby, where developers often try out code in a REPL and then move that to a prototype without considering error cases at all.

(From https://stackoverflow.blog/2020/01/20/what-is-rust-and-why-i... )

Ultimately, there's selection bias:

> programmers in statically-typed languages are perfectly happy to supply their assumptions up front

-- of course: programmers who are happy to supply their assumptions up front tend to be happier with statically-typed languages.

Re:Dynamic type systems are not inherently more open by sethev on Sunday January 19, 2020 @02:08PM (#22093231)
I thought the article was good and addressed a real point of confusion, as evidenced by the two included comments (from Reddit and HN). You can consume arbitrary data using a program written in a statically typed or dynamically typed language. Whether it's decoupled from changes in the data depends on how the code is written and the data model, which have nothing to do with static vs dynamic typing.

To me the stronger argument is that the boundary between programs is dynamically typed (interpreted and checked at runtime). This is true in the statically typed example as well - the JSON is interpreted and checked at runtime, not at compile time. There's nothing your compiler can prove in advance about what's in the JSON that you'll receive at runtime.

If systems that extend beyond a single program require dynamic typing, doesn't it make sense to invest more in ways to do dynamic typing better?

Re:Dynamic type systems are not inherently more open by nickbauman on Sunday January 19, 2020 @02:45PM (#22093521)
I think it's a question of efficacy and productivity. People routinely make mistakes using type systems. While static typing tends to be really good at solving problems it itself introduces (change a type and, wow! My IDE knows where that type is everywhere and can change it for me! Nevermind that only the producer and final receiver should care, it's everywhere now...)

There hasn't been a lot of study on this topic* but what little there is shows that 3% of errors found can be mitigated with type systems, where they do not exist, fixing these classes of errors takes less time than it took to use the type system.

* https://www.infoq.com/presentations/dynamic-static-typing/

Re:Dynamic type systems are not inherently more open by frankpf on Sunday January 19, 2020 @03:14PM (#22093710)
> There hasn't been a lot of study on this topic* but what little there is shows that 3% of errors found can be mitigated with type systems, where they do not exist, fixing these classes of errors takes less time than it took to use the type system.

It seems to me like you're cherry picking evidence. Copy-pasting from a previous comment I made in another thread (https://news.ycombinator.com/item?id=19530274):

There is plenty of evidence showing that modern type systems reduce bugs considerably.

In Airbnb, they found out that 38% (!) of bugs could have been prevented by using TypeScript[1].

Another scientific study discovered that TypeScript and Flow could prevent about 15% of bugs in committed code [2]. And these aren't even measuring reduction of bugs in non-committed code!

Stripe is also writing their own type checker for Ruby and engineers have reported an increase in productivity[3].

[1]: https://news.ycombinator.com/item?id=19131272

[2]: https://blog.acolyer.org/2017/09/19/to-type-or-not-to-type-q....

[3]: https://sorbet.run/talks/StrangeLoop2018/#/

Re:Dynamic type systems are not inherently more open by nickbauman on Sunday January 19, 2020 @04:30PM (#22094245)
Well what I should have said is there has been a lot of studies on the efficacy of type systems, but much has been shown to be flawed. I haven't looked at your citations mostly because of the use of typescript which I have personal experience with and I know it's not helping. I'm not going to debate this with you, though; too little good papers on the topic so too much heat and too little light. I've spent most of my career using static typed languages in hindsight I've found most of the static typing not helpful for successful projects.
Re:Dynamic type systems are not inherently more open by mokus on Sunday January 19, 2020 @02:58PM (#22093584)
> If systems that extend beyond a single program require dynamic typing, doesn't it make sense to invest more in ways to do dynamic typing better?

Isn’t dynamic typing more or less a default state of not knowing anything at compile time about the values your data will take?

One could equally well ask “given that the boundaries of our systems are necessarily characterized by unpredictability, doesn’t it make sense to invest in more ways to isolate that unpredictability better (e.g. by use of static analysis)?”

Re:Dynamic type systems are not inherently more open by kazinator on Monday January 20, 2020 @02:02AM (#22096903)
No! The "default state" for not knowing anything at compile time about the values your data will take is typelessness, exemplified by machine-oriented languages like BCPL and most assembly languages. In this state, the machine knows nothing about the types of your values at any time. Each operation that you apply to a value just assumes that it has the right type for that operation.

Programs in such languages must be statically type checked --- by the coder.

Sometimes (usually?) those languages define the effects of type punning, so type mismatches are not necessarily errors. The programmer must know which are intentional (to be analyzed for correctness from a type-punning perspective) and which are bugs. This could come from comments or naming conventions.

Dynamic typing is a huge increment over this situation; what is surprising is how early the original Lisp people figured it out, while also inventing useful abstractions like symbols and whatnot.

Re:Dynamic type systems are not inherently more open by erik_seaberg on Sunday January 19, 2020 @03:54PM (#22093994)
> the boundary between programs is dynamically typed

Many interfaces have declared, enforced static types. I don't have to write any code to handle

  SELECT birthdate FROM employees WHERE id = ?
returning "fish" because the database would never let it happen.
Re:Dynamic type systems are not inherently more open by Izkata on Sunday January 19, 2020 @09:13PM (#22095697)
So you say, but sqlite has no problem with you storing "fish" in an integer column.
Re:Dynamic type systems are not inherently more open by sethev on Sunday January 19, 2020 @04:18PM (#22094156)
You know that but your compiler doesn't (generally speaking).
Re:Dynamic type systems are not inherently more open by thu2111 on Monday January 20, 2020 @08:24AM (#22098599)
Hence ORMs. It's possible to do much better than ORMs though. The object/relational type system mismatch is a property of how technology evolved, it's not fundamental.
Re:Dynamic type systems are not inherently more open by kazinator on Monday January 20, 2020 @12:32AM (#22096562)
> You can consume arbitrary data using a program written in a statically typed or dynamically typed language.

Really? Okay, below is something that meets the definition of "arbitrary data". Can some static program process it and evoke its full meaning without the programmer having to develop an ad-hoc dynamic typing system?

  (defun defset-expander (env macform name params newval setform)
    (with-gensyms (getter setter args gpf-pairs gpr-pairs ext-pairs
                   pgens rgens egens all-pairs agens nvsym)
      (let* ((ap (analyze-params params))
             (exp-params (car ap))
             (total-syms (cadr ap))
             (fp (new fun-param-parser form macform syntax exp-params))
             (fixpars (append fp.req fp.(opt-syms)))
             (restpar (if (symbol-package fp.rest) fp.rest))
             (extsyms [keep-if symbol-package
                               (diff total-syms (cons restpar fixpars))])
             (xsetform ^^(alet ((,',nvsym ,,newval))
                           ,,(expand ^(symacrolet ((,newval ',nvsym))
                                        ,setform)
                                     env))))
        ^(defplace (,name . ,args) body
           (,getter ,setter
             (tree-bind ,params ,args
               (let* ((,gpf-pairs (mapcar (op (fun list) (gensym)) (list ,*fixpars)))
                      (,gpr-pairs (if ',restpar
                                    (if (consp ,restpar)
                                      (mapcar (op (fun list) (gensym)) ,restpar)
                                      (list (list (gensym) ,restpar)))))
                      (,ext-pairs (mapcar (op (fun list) (gensym)) (list ,*extsyms)))
                      (,pgens (mapcar (fun car) ,gpf-pairs))
                      (,rgens (mapcar (fun car) ,gpr-pairs))
                      (,egens (mapcar (fun car) ,ext-pairs))
                      (,all-pairs (append ,gpf-pairs ,gpr-pairs ,ext-pairs))
                      (,agens (collect-each ((a ,args))
                               (let ((p (pos a ,all-pairs (fun eq) (fun cadr))))
                                 (if p
                                   (car (del [,all-pairs p]))
                                   a)))))
                 ^(alet (,*,gpf-pairs ,*,gpr-pairs ,*,ext-pairs)
                    ,(expand ^(symacrolet (,*(zip ',fixpars
                                                  (mapcar (ret ^',@1) ,pgens))
                                           ,*(zip ',extsyms
                                                  (mapcar (ret ^',@1) ,egens))
                                           ,*(if ,gpr-pairs
                                               (if (consp ,restpar)
                                                 ^((,',restpar ',,rgens))
                                                 ^((,',restpar ',(car ,rgens))))))
                                (macrolet ((,,getter () ^(,',',name ,',*,agens))
                                           (,,setter (,',newval)
                                              ,',xsetform))
                                  ,body))
                             ,env)))))))))
Re:Dynamic type systems are not inherently more open by yakshaving_jgt on Monday January 20, 2020 @02:54AM (#22097181)
Yes. A program written in a statically-typed programming language can parse the lisp you have written above.
Re:Dynamic type systems are not inherently more open by kazinator on Monday January 20, 2020 @11:17AM (#22099843)
I believe that such system cannot fully process the data, as is written in that data's specification, without providing the type system that is in that specification.

The requirement for dynamic typing can either come from the language, or it can be "Greenspunned", or some combination of the two.

For instance, in that data we can construct objects, and we can write expressions which interrogate an object's type.

Thus, that is an example of input data whose processing requires dynamic typing.

Re:Dynamic type systems are not inherently more open by yakshaving_jgt on Monday January 20, 2020 @11:55AM (#22100144)
I'm quite sure it's not. Statically-typed programs can recursively parse data, and they can also monadically (sequentially) parse data. The same is true of dynamically-typed programs.
Re:Dynamic type systems are not inherently more open by kazinator on Monday January 20, 2020 @01:26PM (#22100936)
Likewise academically speaking, statically typed programs can simulate Universal Turing Machines, QED.

If the detailed requirements describing the inputs require dynamic typing, then the statically typed program has to whip up a facsimile of that dynamic typing. It's not an implementation detail; it shows up in visible, testable system behaviors.

And then, there you are, with the possibility of errors that are not caught at your compile time.

Re:Dynamic type systems are not inherently more open by yakshaving_jgt on Monday January 20, 2020 @01:56PM (#22101203)
I don’t get it. Why are you concerned with compile time?

The debate that the article is addressing is the supposed difference of capabilities between programs written with static vs dynamic types, at runtime.

Re:Dynamic type systems are not inherently more open by kazinator on Monday January 20, 2020 @03:18PM (#22101787)
The situation is not written entirely with static types if the run-time processing requires the expression of dynamic typing. That's based on some actual dynamic typing from the language, or else some ad hoc dynamic types cobbed together in the program itself.
Re:Dynamic type systems are not inherently more open by sethev on Monday January 20, 2020 @05:10PM (#22102498)
You picked a strange example since so many Lisps were implemented in statically typed languages.

Here's one in C (less than 200 lines): https://carld.github.io/2017/06/20/lisp-in-less-than-200-lin...

Here's one in Haskell: https://www.defmacro.org/ramblings/lisp-in-haskell.html

Re:Dynamic type systems are not inherently more open by kazinator on Monday January 20, 2020 @05:29PM (#22102606)
That first one looks dynamically typed through a List * type, and tags put into pointers. That algebraic type in Haskell also looks dynamic: an Expr can be any one of four things.
Re:Dynamic type systems are not inherently more open by sethev on Monday January 20, 2020 @06:32PM (#22102961)
This supports the case the article was making. Clearly the type system in Haskell doesn't prevent it from handling arbitrary data - it's hard to imagine a program more open to change in the input data than a Lisp interpreter.

You seem to have a set of rules that you think code in a statically typed language has to follow in addition to the type constraints in order for you to consider it sufficiently 'typed'.

Re:Dynamic type systems are not inherently more open by kazinator on Monday January 20, 2020 @07:59PM (#22103399)
Well, just one rule: if we are looking at type tags at run-time, it's dynamic typing! Whether that be things put into C pointers, or integer fields discriminating unions, or algebraic sum types with pattern matching.

If someone writes Haskell applications by defining an Expr type that is a sum of several dozen things, and then makes al their function arguments and return values of type Expr, is that still bona fide static typing?