Ok, although I would be wary of using a numerical comparison for anything except catching obvious errors.

I should have asked this before, but "data science" is a pretty broad term - who are you hoping to target with this? I'm guessing for pretty junior positions but want to clarify.

Oh, and one other question, do you /can you enforce the 60 min time? [edit: never mind, I answered this experimentally, you do cut it off at 60 min]

Agreed. Data Science is a very broad term. The challenge is designed for Data Scientists. We are trying to target all experience levels as of now through a screening/take-home test that should take about 60-90mins at a stretch. Do you think the timed challenge should be different for senior vs junior data scientists?

What skills would you consider important for senior vs junior Data Scientist?

I've added some top level comments.

For what it's worth, I think junior and senior DS roles should have fairly different evaluations & interviews.

I think at best this is fizzbuzz for DS, which is not inherently wrong. It's nice to know a software developer can write a loop and a data scientist can use a JN, so for weeding out people who have no practical experience with a given tool set, it could make sense.

The question then is how do you algorithmically (or even just consistently) distinguish a great data scientist from one who can accurately model answers to a question that was badly thought out?

Plus as pointed out before, the length of a take home could reduce applications from the most qualified candidates.

Wonder if this should be even shorter and more quiz/fun like so it intrigues rather than annoying more senior applicants, and still wondering the best way to identify the data scientists who ask better questions.