This is the best argument I've seen for dynamic types beyond the prototyping phase: https://lispcast.com/clojure-and-types/

Not sure if I'm 100% won over, but it makes some good points, particularly the section about JSON. Using TypeScript at work, one of the most painful (yet quite common) things is dealing with foreign JSON from an API, which is sometimes just... wrong. And then all our carefully-typed code falls to pieces.

The argument is that in a language like Clojure you can skip pretending to make guarantees about messy external data, and trade that for simple and elegant traversal code. Probably this is why dynamic languages tend to gravitate towards "glue code" use cases while static types gravitate towards insulated systems with minimal surface area to the outside world (compilers, anyone?).

Any bit of JSON either resolves to a valid object of some type in your data model, or is erroneous. "Simple and elegant traversal code" often makes assumptions about the data it operates over; when handed JSON that violates these assumptions, the result is undefined. If you use the "parse, don't validate" pattern (https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...), by walking the complete data structure early on and ensuring via the type system that the assumptions hold, or failing, your JSON handling code is much clearer and more robust.

Static types are an unmitigated win, if you know how to use them.

Makes me think a bit of Python's "ask for forgiveness not permission".

I have a few problems with "validate then do something if its valid". On top of what that article mentions, there are concurrency issues (e.g. checking to see if a file exists, then if it does, open it... oops, race condition), and general ergonomics (if I always have to do something before I do something else, there should be a function call that effectively does both).

Wouldn't work if you were implementing a library like Pandas when the data can be almost any tabular structure that you need to clean and wrangle into shape before you can be sure what types you're dealing with.

Type systems give you a way to specify what you know at design time. If "has any tabular structure" is all you know then that is what you specify.

I don't think "but my data is dynamic" is a good argument for or against dynamic typing. It's an entirely separate issue.

> but my data is dynamic

In my experience, most dynamically types code makes assumptions about structure that may not hold later (better have tests!) and when it breaks some kind of validation is needed anyway to select between the correct version of the different dynamic options. In static languages this is simply typically done sooner, since the type system requires it to parse the data. (Or it’s in variants that are accessed without checking that the accesses are valid, sigh)

The type system depends on the kind of data you want to keep around in your columns, which could be a mix of types, and it could be all strings and NaNs, for example. This is common with DataFrames. There's a huge number of possible combinations of data you could be reading in that you need to clean up, reshape and then analyze. And you might be making decisions on how to do that in the middle of your wrangling. It's just like what you can put in Excel or a CSV.

I know. I've been doing this sort of thing for most of my adult life.

You have to be sure, like always. You can't write the code dealing with the data you have no clue about.

The type reflects invariants, constraints given for you data. The less you know, the more general type your data would have. Dynamic typing gives no advantages here safe for providing a super-general ANY type.

In case of statically typed language you would need to parse your data and construct an object representing that data just like in case of dynamically typed one. But the more you know about your data in advance, the better, especially when you are dealing with protocols and communication.

> But the more you know about your data in advance, the better, especially when you are dealing with protocols and communication.

The point of a library like Pandas is that you will be cleaning and reshaping your data until it's in a format that can be properly analyzed. And that data can be very dirty and very large.

But if you prefer to clean it up in Excel or with some bash scripts, or custom written functions in PL of choice, then go for it. You still have to deal with the cleaning and wrangling. The point of mentioning Pandas is that you don't always have a guarantee on properly structured data where you can predefine the types and read it in. And you don't always have control over how that data comes to you.

All data has SOME structural metainformation that can be captured in a type. If you are importing a CSV file, you know -- at a minimum -- that it's a CSV and can be parsed as a set of tuples of variants (or else there's something really wrong with the input and an exception continuation should be invoked to handle it). From there you can do analysis on the data to further refine its type.

But ALL data is typed, and encoding its type early in the code that processes it makes your assumptions clear and compiler-enforceable.

You're right that a typed approach to handling arbitrary input data is not right for a library like Pandas. It's a fundamentally different approach than the one Pandas takes, but it's no less valid and brings with it certain advantages.

For the library, you are right, it has to support any tabular structure — but that’s what even ricks are for, to encode that there are parts that you don’t know until use.

When using the library, though, there is some level of assumed structure (otherwise your code wouldn’t work since it would be accessing fields that don’t exist. Sure data is messy in real life, so you encode that messy-ness into your structure as variants or optional that you then check, and ideally are forced to check so that you can’t silently let errors slip in).

This works in statically typed languages right now using generics, templates, types that aren’t inferred until the library is used, or whatever the specific language has to deal with this.

I don't understand your JSON example.

If your Clojure code expects an API response's json.{id,username} to exist, and to not exist is a hard violation of your assumptions and thus your code, then how is that any different from formalizing it into:

    struct UserResponse {
        id: Int
        username: String
    }

?

(Damn, rest of the comment is pretty long but let me try to both sympathize and present a solution that I do like).

What I will say is that most statically-typed languages I've used do get annoying when there's no 1:1 mapping between your struct and the JSON body. I've dealt with some hairy JSON, like APIs that accidentally re-encode nested JSON strings so they become like triple escaped. Or values that can have like 5 different shapes so you really just want to try 5 different decoders until one is successful. (These cases are trivial in my upcoming code example)

For example, Rust's serde and Swift's Codeable are both pretty annoying if you want to apply arbitrary transformations between JSON response -> your canonical representation. Serde, for example, has you implementing the whole trait + visitor if you need to do anything the cute decorator DSL doesn't support. Or you implement a bunch of envelope structs even though you just want to pull out the value at "obj.a.b.c". Or, what happens when you can't simply use a codec from https://app.quicktype.io/?

JSON decoder combinators (e.g. in Elm) make JSON pleasant again while retaining static typing. https://package.elm-lang.org/packages/elm/json/latest/Json-D...

For example,

    data Role = Member | Admin

    data User {
        id: Int
        username: String
        role: Role
    }

    -- API example body: { id: "12", role: "a", info: { uname: "foo" } }
    decodeUser : D.Decoder<User>
    decodeUser =
        User
            -- Crappy API gives us either an int or a string
            (D.field "id" (D.oneOf [ D.int, D.string |> D.map parseInt ]))
            -- Pull out value from nested object
            (D.fieldAt [ "info", "uname" ] D.string)
            -- Arbitrarily transform value and create our own error
            (D.field "role" D.string
                (D.andThen (\value -> 
                    case value of 
                        "m" -> Ok Role.Member
                        "a" -> Ok Role.Admin
                        unknown -> Err ("Unexpected role " ++ unknown))))

That almost compiles despite shooting from the hip directly into this textarea, but the point is that it's a pleasant way to decode JSON while allowing arbitrary transformation, and I think it's the ultimate middleground between static-typing and the sort of on-the-fly decision making you get with dynamic typing.

Your example isn’t tied to static typing, it is simply but behind-the-scenes checking types at runtime. You can do that in any good dynamically typed language (e.g Common Lisp) and hide it in a similar way.

Closure has definite problems that other Lisps don’t due to a combination of leaky JVM abstractions (incomprehensible stack traces, JAVA pollution), its semantics not really being Lisp semantics and its premature optimization into one paradigm that doesn’t really fit the language or most problems very well.

Yes but once the json has been validated once at runtime, you know it's safe for the rest of your program to use.

Whereas in a dynamic language (even an immutable one), your data could be mutated during program runtime and no longer conform to the type you expected. Yes you could re-validate it everywhere, but that requires you to exercise every code path when testing at runtime to find bugs.

I think the general argument that "You can't static type everything (like your API dependencies), so there's no point in doing it all" isn't a very solid one.

Your data can't be mutated if you're using immutable data structures. Which is what Clojure defaults to, in fact.

I didn't mean literally mutating the same structure in place - but a new version of your data structure could be returned from a function which changes some of the keys/values in unsafe ways.

Sure, my example isn't tied to static-typing but static-typing is especially helpful any time you're workin with combinators/transformations at a distance. But also, statically-typed languages like the two I brought up tend to be in most need of a nice validation/codec/transformation library because you can't simply procrastinate establishing your domain models like you can in dynamically-typed languages. And this is the sort of pain that leads people to say that dynamically-typed languages are more accommodating here.

Though I will have to defend Clojure from your last paragraph. I don't think there has been a lisp that has done so much for lisps than Clojure. For example, until I used Clojure, other lisps were just like any other dynamically-typed, mutation-heavy language. The idea of just shuttling around data and transformations on that data is something that fundamentally changed my approach to writing software.

I can certainly come up with worse things to say about Clojure. I kinda grew out of dynamic-typing, for one. And this is never more obvious than if you've ever read a large or complex Clojure codebase while dying to just know what the underlying Java types are. And you're so desperate that you get excited when you see a single ^ChannelHandler type-hint.

> But I would take any functional language with types over it

Which is what? Scala? Haskell? OCaml? F#? Typed Racket? Idris?

I think Clojure really hits the sweet-spot between stupidly boring, "pragmatic" PLs and idealistic, novelty, academic ones.

Haskell is awesome, but realistically it is really difficult to quickly train someone to the point of them being able to write production-ready Haskell code.

Scala has its own warts (besides, Kotlin seems to be slowly eating its pie);

OCaml still struggling to get any recognition in the industry, despite all attempts from a few good players.

I disagree with the notion that Clojure is not suited for large projects, but I agree that Spec needs to be improved (and it is being actively worked on). And I don't think it is not well thought out. Rich Hickey is famous for slowly, carefully, patiently designing things. Stability and predictability of Clojure and its standard library is phenomenal. For every single language (even those I have never worked with) I can probably name a thing or two off my head that involved breaking changes. Not for Clojure.

> Which is what? Scala? Haskell? OCaml? F#? Typed Racket? Idris?

We have one 100k loc project in Clojure and one 500k in OCaml and I would choose OCaml over Clojure any time any day. I really like that things don't brake accidentally when somebody changes code, and the amount of tests you need to write are orders of magnitude smaller.

> OCaml still struggling to get any recognition in the industry

So does Clojure. Both are extremely niche below the radar languages, so it's not a dichotomy really.

There is a dichotomy of choosing OCaml or Java/C#, but Clojure would be on OCaml's side of things here.

Today, if I'd need jvm, I'll simply take modern Java.

> despite all attempts from a few good players

The few players we have (JS, Facebook, LexFi, Citrix, unis) are doing well enough producing enormous amount of high quality libraries. At least in the domains of system programming OCaml is tightly packed with all sorts of batteries.

> Today, if I'd need jvm, I'll simply take modern Java.

100% Kotlin. Doesn't even have to be Clojure or ABCL.

> Haskell is awesome, but realistically it is really difficult to quickly train someone to the point of them being able to write production-ready Haskell code.

A former colleague once said that many companies using Haskell don't really want to spend money training existing developers to learn Haskell; instead they want to hire programmers self-motivated enough to learn Haskell on their own.

> Which is what? Scala? Haskell? OCaml? F#? Typed Racket? Idris?

If I want JVM interop, I'd use Scala. Or yeah, as you mention, Kotlin isn't so bad either.

For a fun project, I'd use Idris, Haskell, or Rust (of course, many people may not consider Rust a functional language)

On the frontend, Elm or Purescript.

> I disagree with the notion that Clojure is not suited for large projects

True, large projects can be maintainable or unmaintainable in just about any language. But there's a reason dynamic languages like javascript, python, ruby, and php are seeing such heavy investment into gradual typing: it's helpful to have easy-to-digest information about the shape of data going in and out of functions. Almost nothing about spec is immediately self-evident -- it just gives you more code to read and reason about.

>>Which is what? Scala? Haskell? OCaml? F#? Typed Racket? Idris?

These days Java works just fine. Why use anything else, that neither has dev mind share nor the library ecosystem remotely the same as Java.

When I was last at a Java shop, we we're pushing multiple-thousand line PRs every week, a good 40% of those lines were simply boilerplate generated by IntelliJ, and it would have been worse had we not fully instrumented our codebase with Lombok, and all this was for a pretty mature product that wasn't seeing any radical changes from day to day. At the clojure shops I've worked at since, we're building more and delivering product faster, and we're doing it on the JVM with less than a third of the lines of code. A thousand lines of clojure is a significant refactor or a big feature. A thousand lines of java is wednesday.

For me, clojure provides a simple-to-use API, high-level functional abstractions, an instantaneous feedback loop, access to a battle-tested JVM ecosystem, and more than fast enough performance. This is its killer combination, in my opinion. The immutable-by-default data structures are also an underappreciated boon for writing multi-threaded code without breaking a sweat.

> When I was last at a Java shop, we we're pushing multiple-thousand line PRs every week, a good 40% of those lines were simply boilerplate generated by IntelliJ

This actually one of the things I like about Kotlin - many of it features are similar in effect to pre-generated boilerplate - but it lives as part of the run-time, not in your code.

On the flip side, a new version of Kotlin could potentially introduce/reveal bugs - but at least you don't have to deal with four different generations of outdated boilerplate when diving into old code...

The library ecosystem is accessible from the many compile-to-JVM languages.

Half tongue-in-cheek, half serious: not having Java's dev mindshare is actually a benefit.

I don't understand how anyone can stomach Java, especially when Kotlin exists which has the same ecosystem as and better tooling than Java.

Unfortunately companies choose Java for conservative reasons which pretty much rules out anything like Kotlin making a dent in Java's mindshare. Most Java shops won't even upgrade beyond Java 8 so switching to Kotlin is a non-starter outside the Hacker News echo chamber.

A few years ago I started a personal project of implementing a lot of common ML algorithms in both Scala and Clojure simultaneously. I expected Clojure to be a much better fit for this sort of thing but I was surprised to find that as my codebase grew in both languages the Scala code was much, much easier to understand and maintain. There were a lot of time consuming bugs in my Clojure code that were prevented by Scala’s strong typing. I was able to refactor with a lot more confidence in Scala.

So as much as I like some things in Clojure I’d pick a statically typed language for most jobs now.

I'm honestly curious how Clojure + spec would fare in that comparison. Is that something you looked into?

The reason I'm asking is that while my background the typical webdev stuff, I loved TypeScript. And now while much of my work is backend, it's similarly dynamically typed by default because I'm using Elixir. I've dabbled in Elixir's 'spec' and quite enjoyed some of the benefits Dialyzer/Dialixir offered me, but not enough to be able to tell how beneficial it is in my projects.

More than once I've run into issues that I wouldn't have had if I had used specs, but the fact that it's not a default in the community gives me pause. It makes me wonder how much advantages 'gradual typing' actually give me over just being a more thoughtful developer.

It’s an interesting question. I’m not sure if the spec stuff existed at that time. Like you say I think it makes a big difference if it’s built into the language. Typescript is a lot more useful now that it’s become so mainstream, for example.

I would pick Clojure over any statically typed FP language simply because it's far easier to train news devs for it. My team has been using it for close to a decade now, and we've only hired a single person who already knew Clojure during that time. All the other devs came from mainstream languages, and we were able to get them productive within a couple of weeks on average. There's simply no way I could teach a language like Haskell in a couple of weeks and get somebody reasonably productive with it.

Monolithic design is not a good way to structure projects IMO. Any large project can, and should, be broken down into small isolated components that can be reasoned about independently. Not only does this reduce overall mental overhead of understanding the code, but it also makes it much easier to reuse. Static typing appears to encourage coupling via the types due to them having to be expressed globally. So, it naturally steers the design towards building a monolith.

I also completely disagree that the REPL is some kind of a crutch. It's a strictly superior workflow because it allows you to put the application in a particular state, and then develop functionality against the state you've already built up. Having to restart the application, and rebuild that state every time you make a change is simply not a productive way to spend your time.

Finally, the claim that Clojure hasn't found a niche is pretty weird seeing how it's used all over the place by companies big and small. It's certainly found a very nice niche in space of web development and companies like Apple, Atlassian, and Walmart are getting pretty good mileage from it.

If you are the type of person that fiddles with Clojure and other functional languages (OCaml? Haskell? Scala?) I'm pretty sure you will change your mind on and off, in different seasons of your career :-)

I used to think only languages with good algebraic data types (mostly ML based) where worth my time... but types can really become an straight jacket in large programs and become tedious to use. Clojure code, when reasonably well structured, is not a bad solution for a lot of varied problems, and is beyond great for personal projects! Low ceremony, fast feedback loop, excellent performance, and a huge library of ready made quality libraries.

Quality libraries built with a statically typed object -oriented language designed with mutability in mind. See the puroblem?

Honestly I don't think it is a problem. There's no perfect software and neither type systems nor immutability guarantee correctness. Clojure offers best of both worlds by minimizing and managing mutability.

Different strokes for different folks I guess. I would take it over Haskell and Scala any day. Haven't tried F# or OCaml so can't comment.

> its dynamic-ness makes it not suited to large projects IMO

I'm really curious about this. I work on large Clojure projects, and it works quite well. We have low defects, and quick pace of change. Maybe we're just lucky, touch wood.

> The currently-accepted attempt at a non-type-system-type-system 'spec' is not well thought out

If you thought of it as a type system, than it's normal you think it's not well thought out. It isn't trying to be a type system at all. It's a contract system. The goal is validation, generative testing, formal spec, parsing, etc. Not type checking. Even given a strong static type system I would still use it.

> The repl is fine but can be a crutch -- see startup time

Heard about this from some people, but dunno, it never affected me. I start my REPL at most once a day, often less then that. So waiting a few seconds doesn't bother me.

> Clojure still hasn't really found a niche

That's true. That said, it is great at everything. While it didn't find itself a popular sub-domain to dominate, I now choose it for everything. There's barely anything it can't handle well. I reach for it for everything nowadays. That's actually something I really like about it, it has huge reach.

I went pretty much all-in for Elixir/Erlang, but I do have to admit Clojure has always been a close second, and I envy the amount of interop it has.

There's various initiatives in the Elixir ecosystem, and tools such as LiveView make my life a ton easier, but man do I wish I could use Elixir as flexibly as Clojure!

> The currently-accepted attempt at a non-type-system-type-system 'spec' is not well thought out.

This is quite possibly true (I certainly have my complaints with it), but I'd love to hear more about what you're getting at when you say this.

When the canonical example of an fdef takes significantly more code than the function itself, I think you have to re-evaluate your approach: https://clojure.org/guides/spec#_specing_functions

That seems an unfair criticism, most of that fdef is about the relationship between the args/return as opposed to just the types.

Perhaps. I was trying to find something pithy to give as an example. It's difficult to express all the things about spec that feel off to me. I'll try to enumerate some:

- verbosity (I think schema does this better https://github.com/plumatic/schema)

- because it tries to not be about validating types of inputs and outputs, and rather treats everything as predicates, it tends to live in a grey area between type validation and unit-testing, and I think hinders both

- it only helps at run-time

- over-abundance of macro-usage

I suspect what you consider a 'gray area', spec considers the whole point. What is the specification for this function? It has a signature and a return value and some behaviour in between. Spec verifies all of that together because it doesn't make sense to do it separately.

Also I'm not sure what you mean about run-time, you can and should use spec as part of your development workflow.

Other two points are totally fair though, and the macros are part of the reason it never got out of alpha before a rewrite.

Fair. I find that I naturally tend to think about the general properties of argument/return values and not just the types (or perhaps equivalently expand the meaning of "type" to encompass such usage) so I like the design, but I haven't yet used spec in anger and so might yet change my mind.

Hm the /amount/ of syntax isn't something I think about I guess, this being a Lisp.

I think they've taken the approach of making the system thorough and capable and this being a Lisp, we users can skin it with any level of shorthand that we please.

Datomic (the database from Cognitect) has a similar approach with its schema, which is quite verbose, but many folks that I know tend to simply write a function which spits out the schema, since it's 'just data'.

I tend to use Ghostwheel [1], but there's also spec-tools [2] and others.

[1]: https://github.com/gnl/ghostwheel

[2]: https://github.com/metosin/spec-tools

spec.alpha is implemented as a bunch of macros that make it difficult to work with spec forms as if they were "just data". This is one of the issues that spec-alpha2 tries to address [1].

Sure, you can import libraries to try to work around it but that's hardly ideal.

[1] https://github.com/clojure/spec-alpha2/wiki/Differences-from...

Schema is a pretty nice alternative: https://github.com/plumatic/schema

> - its dynamic-ness makes it not suited to large projects IMO.

Clojure can be typed https://github.com/clojure/core.typed/

> - The currently-accepted attempt at a non-type-system-type-system 'spec' is not well thought out.

Spec is not a substitute for a type system. It is much more loose, since you are just dealing with implicit predicates but at the same time it is more powerful than what mainstream type systems give you in terms of validity and expressiveness.

> Clojure still hasn't really found a niche

To me Clojures sweet spot was dynamically typed, functional programming with an extremely large ecosystem of tools (due to Java interop).

That’s not a niche, that’s a summary of the language.

Tools designed for a statically typed object-oriented language where mutable state is encouraged. I don't see the value for a language like Clojure.

Having to maintain more than 42k LOC of Clojure code, couldn't agree more. I would also add that i haven't see a community with such fanatism towards a language and his creator. So i always avoid to interact with the community.

I don't know what you're talking about. I've personally worked in several language ecosystems and interacted with members of their communities. Clojure people are one of the most humble, reasonable, friendliest people in the entire CS universe. Many of them have years of experience working in different languages, they are not fanatics, most of the time their answer to the question "is that the right way?" would be "it depends."

While devs in other programming language ecosystems either debate endlessly or rush to add more "features" to their language, Clojurists quietly make things that simply work.

If you follow the trends and dig a bit more, you realize that Lispers carefully filter out good ideas and put them into use, they know how to tell the difference between a brilliant idea and a fad.

And there's no fanaticism and worshiping of language creator. Only respect and gratitude.

Looking at spec through the lens of a type system is weird because types solve two problems where spec solves one problem

So I'm getting the most leverage out of types when they solve: - Static analysis - Specification

Clojure's static analysis tools do use type analysis since the underlying primitives are typed, but a good static analysis does type analysis and other checks so types aren't the be all end all there

Personally I love types that track to primitives, the problem with types is when you let developers use them to express their Person class or whatever, your Person class is not as fundamental as that language's int class, for example, the semantics are completely different between your primitive types but not your Person class vs your Address class

Anyway what I wanted to say is I have yet to meet a type system where I'm happy with its specification capabilities without moving into slot based programming or so much syntax for the types it becomes overhead to read them, i.e. you can write complete programs in TypeScript's Type system

Spec is the competitor for type specification here, it is much nicer you can translate it into API, GraphQL, Database schema etc but it's still getting work for Spec 2, but you can generate data examples out of it, use it for generative testing, use it for parsing, validation etc It's good if you have that specification problem

The static analysis problem is getting more and more attention through clj-kondo and inbuilt analysis via Cursive and other editors, they'll stop you from doing things like (inc "hello world") because you can't increment a string, I think this feature is actually why people love types so much the fast feedback for silly mistakes is soo good and we have that too

What we didn't do was complect those two things together, it also doesn't mar our thinking, I don't think there's an end game where a sufficiently smart type system where it eliminates all programming problems, the next breakthroughs will be in data, not types see the whole area of deep learning, but we're so in love with types we create languages where their USP in the type system

When you get heavy into types you start seeing the world through the lens of types and that's super dangerous for the same reasons that trying to view the entire world as SQL tables is not a maintainable solution, the problem isn't reality it's your lense that you're viewing the world with

Also please don't read this as types are bad that's not what I'm saying, or tables are bad, letting them consume your thinking means you start thinking about problems in terms of typed or not typed boolean thinking instead of nuanced, since nearly all languages are "typed" at runtime and static analysis time anyway

Clojure Spec is what C++ 20 concepts is evolving to! The idea that there are no truly fixed types but concepts with constraints and requirements.

This is around 5/10 years ahead of its time in the general PL community.

Interesting perspective on spec since I find it very well designed, very consistent, and very useful. What isn’t well thought out about it?