News for hackers. Stuff that matters.
Re:A Year in Clojure by yogthos on Sunday October 07, 2018 @03:17PM (#18162167)
That is pure and utter FUD I'm afraid. My team has been working with Clojure for over 8 years now. We have tons of code in production, and we don't have any problems maintaining these projects. Furthermore, there's absolutely no empirical evidence to support the notion that static typing results in statistically significant reduction in defects, speed of development, or ease of maintenance. Considering how long both disciplines have been around, and the sheer amount of software written in each, that's quite the elephant in the room. I also have no idea who the people migrating from Clojure to OCaml, Haskell, or F# are. I don't think I've seen a single story about a company doing that.

My experience is that dynamic typing is problematic in imperative/OO languages. One problem is that the data is mutable, and you pass things around by reference. Even if you knew the shape of the data originally, there's no way to tell whether it's been changed elsewhere via side effects. The other problem is that OO encourages proliferation of types in your code. Keeping track of that quickly gets out of hand.

What I find to be of highest importance is the ability to reason about parts of the application in isolation, and types don't provide much help in that regard. When you have shared mutable state, it becomes impossible to track it in your head as application size grows. Knowing the types of the data does not reduce the complexity of understanding how different parts of the application affect its overall state.

Immutability plays a far bigger role than types in addressing this problem. Immutability as the default makes it natural to structure applications using independent components. This indirectly helps with the problem of tracking types in large applications as well. You don't need to track types across your entire application, and you're able to do local reasoning within the scope of each component. Meanwhile, you make bigger components by composing smaller ones together, and you only need to know the types at the level of composition which is the public API for the components.

[REPL driven development](http://blog.jayfields.com/2014/01/repl-driven-development.ht...) also plays a big role in the workflow. Any code I write, I evaluate in the REPL straight from the editor. The REPL has the full application state, so I have access to things like database connections, queues, etc. I can even connect to the REPL in production. So, say I'm writing a function to get some data from the database, I'll write the code, and run it to see exactly the shape of the data that I have. Then I might write a function to transform it, and so on. At each step I know exactly what my data is and what my code is doing.

Where I typically care about having a formalism is at component boundaries. Spec provides a much better way to do that than types. The main reason being that it focuses on ensuring semantic correctness. For example, consider a sort function. The types can tell me that I passed in a collection of a particular type and I got a collection of the same type back. However, what I really want to know is that the collection contains the same elements, and that they're in order. This is difficult to express using most type systems out there, while trivial to do using Spec.

Re:A Year in Clojure by dwohnitmok on Sunday October 07, 2018 @08:23PM (#18163356)
Our team (not at liberty to say which company unfortunately) moved away from Clojure back to Scala, or more accurately we built a couple of production services in Clojure, sunsetted them and haven't used Clojure since; the most vocal Clojure enthusiast moved on to another project and the rest of us were kind of meh on using it in the future after about a year of experience and went back to Scala (although certain things were quite fun and it was a fantastic learning experience and I would highly recommend at least getting familiar with REPL-based programming).

It wasn't horrendous (and it's not like Scala is absolutely amazing for us), though maintenance was something of a pain point. We might use Clojure again, especially with some new blood who are Clojure enthusiasts.

These days I'm less and less certain of what proactive role a language plays in team success. That is to say I think the arrow of causation is usually pointed in the wrong direction. A well-functioning, excellent team will, given sufficient latitude, converge on a language that suits a team well. This language is often different for each team. Introducing a "good" language is unlikely to save a team.

Re:A Year in Clojure by yogthos on Sunday October 07, 2018 @08:55PM (#18163468)
I'm in complete agreement here. Ultimately you want to use the language that the team enjoys working in. Different people have differing experience, aesthetics, and pain points. You want to pick the language that most devs on the team prefer using.
Re:A Year in Clojure by taylorwood on Sunday October 07, 2018 @04:10PM (#18162385)
>What I find to be of highest importance is the ability to reason about parts of the application in isolation

This is a huge factor I'd like to emphasize and write about in a follow up post, having worked on many Clojure projects at different companies now... Another Year in Clojure :)

I've maintained some large Clojure codebases I didn't write, and the easy ones shared this quality. The more painful ones were reminiscent of large OOP codebases and honestly I would've preferred they'd been written in Java.

Re:A Year in Clojure by platz on Sunday October 07, 2018 @04:03PM (#18162359)
OP was mistaken to claim 'mass migrations' of any kind - that was clearly made up.

alternatively, I'm always receptive to reports about experiences.

I'm very skeptical about demands to produce "empirical evidence" for or against any sociological programming practice, because it tends to be a method to place the burden of proof on the opposing side, while reserving the default position for own own side - which just turns out to be a way to normalize one's own beliefs.

Re:A Year in Clojure by yogthos on Sunday October 07, 2018 @04:43PM (#18162508)
The only reason I talk about empirical evidence is because I constantly see sweeping claims regarding efficacy of static typing. If somebody makes broad claims of one approach being strictly superior to the other, then the burden of proof is indeed on them to support said claims. The null hypothesis is that both approaches are effective, and it's in line with all the currently available research on the topic [1].

Furthermore, fixating on static typing as the one defining feature of a language amounts to concept space reduction. In practice, there are many complex factors that contribute to overall quality of the language that have to be considered holistically.

[1] https://danluu.com/empirical-pl/

Re:A Year in Clojure by uxcolumbo on Sunday October 07, 2018 @05:11PM (#18162638)
> I can even connect to the REPL in production

Programming is not my day job, so please correct me if I'm wrong - but every project I was involved in so far - having access to a production DB is a big no no.

Or does the project you refered to not contain any sensitive data, so having direct access to the production system is deemed 'safe'?

Re:A Year in Clojure by Jach on Sunday October 07, 2018 @11:35PM (#18163962)
Different companies do it differently, but I'm not aware of any that make it absolutely impossible to get at production data (that hasn't been encrypted with a key they don't have) and program instances, just some make one (reasonably) jump through many more hoops than others. It's very much a double-edged sword so access control, auditing, etc. needs to be done carefully. (Privacy By Design.) The value can be immense -- e.g. debugging and hotloading a patch on a spacecraft 100 million miles away (http://www.flownet.com/gat/jpl-lisp.html)
Re:A Year in Clojure by yogthos on Sunday October 07, 2018 @08:59PM (#18163480)
My team both develops and supports the applications that we build in production. When issues are reported, it's often necessary to check the database to resolve it. Being able to check what data the application is pulling during requests is very useful for troubleshooting.
Re:A Year in Clojure by physicsyogi on Sunday October 07, 2018 @04:37PM (#18162490)
The only example of a Clojure team migrating to Haskell is the Duckling team, a little while after Facebook acquired them. Among other things, they wanted a language with better support within Facebook.

https://medium.com/wit-ai/open-sourcing-our-new-duckling-47f...

Re:A Year in Clojure by yogthos on Sunday October 07, 2018 @04:49PM (#18162538)
Incidentally, that's a perfectly valid reason to migrate, but it's not the one the parent suggests. :)
Re:A Year in Clojure by jb1991 on Sunday October 07, 2018 @05:57PM (#18162844)
> The other problem is that OO encourages proliferation of types in your code. Keeping track of that quickly gets out of hand.

I'm curious how Clojure avoids this. In my large Clojure projects, there aren't "types" in the sense of static typing, but the shapes of data still exist, the types are there, but they are in my head or in other forms than that which a compiler can check. I still have to remember that a map is a certain "type" of data with certain keys/values that are also of certain types, and a large project might have many of these "types", even if they are not explicit. That is equally hard to maintain for me -- harder, really, since it's too easy to make mistakes.

Re:A Year in Clojure by yogthos on Sunday October 07, 2018 @06:29PM (#18162961)
The main difference I find is that with Clojure approach you have a common set of data structures that you operate on using common functions from the standard library. I find that it's usually easy to tell what type of data you're working with by seeing how these functions are used. When I see functions like map, filter, iterate, and so on, I have context for what the intent of the code is, and this helps me know what type of data I'm working with. At the same time I'm mostly working in a local context thanks to immutability, so I don't have to worry that a piece of data might get changed from under you via a side effect.

I also find that you learn patterns for structuring code that make it easier to track the types. For example, I avoid renaming keys or restructuring data except at the edges of the application. Adding Spec or Schema at component boundaries is also helpful, as you can see what the shape of the data was at the API level and work your way through the helpers from there.

There's also a big benefit of working with data as opposed to objects. Each class is its own ad hoc DSL with custom behaviors. Knowing what one class does tells you nothing about what another does. This approach doesn't scale well for large applications where you have thousands of classes with each with its own behaviors.

Re:A Year in Clojure by jb1991 on Sunday October 07, 2018 @06:38PM (#18162998)
I think I would definitely benefit from blog posts or whole books on Clojure style specifically, rather than the many constant re-introductions to its syntax and data structures (like the one in this article). Your thoughts have come closer to explaining a maintainable approach to the language than most of what is out there. In my experience, despite working for years professionally with Clojure, I've never successfully felt like I wasn't fighting the language because of the constant and unexpected nils and other odd wranglings of data that were not intended and not easy to track down at runtime. The flexibility of the dynamic typing and the many APIs for manipulating data can make it easy to restructure your values in unintended ways, and this to me is a meaningful problem unique to the language. I don't have the problem in a Java app, for example, of a Foo type accidentally becoming a Bar type or even some other "type" that was never defined but now has a unique data shape because I made a typo somewhere or used the wrong library function, etc.
Re:A Year in Clojure by Scarbutt on Sunday October 07, 2018 @08:09PM (#18163314)
Regarding nil, its considered bad practice to use it as values in maps.
Re:A Year in Clojure by hellofunk on Sunday October 07, 2018 @08:20PM (#18163347)
Right, which is why I said “unexpected nils”.