It seems pretty easy to control for this, just write the functional version first, and if it is as big free your results obviously sidestep this criticism.

Not really. Even then you'll get criticism that the teams were more skilled in one language, or that they didn't use the best practices of the other language. Also, even if you could manage to normalize for all these variables. You're still only going to be comparing the cost to create the project. You're never going to get someone to maintain the two projects side-by-side for years and compare the total maintenance costs. That is where the biggest benefit of languages like Haskell comes.

The fact of the matter is there never will be absolute proof of these questions. The naysayers will always have straws to grasp at.

Doing a rewrite in a different language and justifying language choice by a successful outcome of the rewrite is grasping at straws to justify the language change.

Every time I've been involved in a rewrite, including ones using the same language before/after, the outcome has been good. The act of doing a full rewrite is where the benefit comes from, it's hard to separate that from a language switch.

It wasn't a rewrite. It was two separate web apps. Type safety categorically prevents classes of bugs that I've seen in real Javascript apps on multiple occasions. Haskell frontend apps that I worked on both before and after the JS app never suffered from that class of bug.

On a political level, it may often be easier to justify a rewrite which switches languages to take advantage of bullet point features (promotional claims like 10x faster, totally secure, no more bugs, etc.) than it is to simply ask for time to do a rewrite, which will make many average managers start fantasizing about firing you. If you write a blog post afterward, that seems to make the company look good.

I'm not sure what you're getting at. He's saying Haskell's type safety makes impossible the bug that was in his JavaScript project. As far as I can tell, it's impossible to write a type safe system in pure JavaScript.

edit: To clarify this thought a bit, obviously you can transpile code in a type safe language to JavaScript. At the abstraction level of the code written, the project is type safe. The generated JavaScript itself is not; however this isn't a problem if the transpiler is correct. The same principle applies to Haskell compiling to non-typesafe machine code.

If Haskell is better than Python, then writing the first version in Haskell and the second version in Python should yield two versions that are closer in quality than if you did the reverse order. Assuming you can measure quality, this is a totally valid hypothesis to test!

Additional hypothesis: the result of this test depends strongly on which one you are more familiar with and which one you prefer. If I hate writing Cobol, I'm going to spend that part of the test pining for the other language and not using whatever the up-to-date idioms are for making Cobol manageable, but just fighting Cobol until I can switch to the language I really wanted to use.

So, anyone doing this should consider trying to get a sample of people who are more or less indifferent between the options, so they don't sabotage the test

> That's all well and good except for the fact that your standards are impossible to reach.

Right, yes. Perhaps this entire exercise is a complete waste of time. Perhaps we should be investing in skilled people who understand their problem domain and then just trust them to do the best they can, rather than trying to find silver bullets inside programming languages.

If it was a complete waste of time, we might as well write our web apps in machine language. There's never been a side-by-side study comparing machine language to modern languages that meets the scientific bar put forth. So even though you can't definitively prove causation, the history of programming languages has demonstrated that multiple substantial advances have absolutely happened. There's no reason to think there can't be more.

How about let's do both. Both are valid, especially when there are silver bullets like type safety available. Hire the best domain experts and teach them best tools.