Re:Show HN: Real world (Jupyter notebook embed) way to asses...
by ska on Thursday August 29, 2019 @02:13PM
(#20832125)
That's useful information, thanks. I'll add some thoughts on the rubric.
[edit]. Initial thoughts:
- "data wrangling" scoring difficult given this task - more weight to "rationale", that's more important the "performance", here.
- not enough focus on communication capabilities
- really need something on validation
- "proficiency" measure you use is pretty much impossible to accurately evaluate from your example question
- way too much weight to modeling section overall
Re:Show HN: Real world (Jupyter notebook embed) way to asses...
by shikharja on Thursday August 29, 2019 @07:04PM
(#20834555)
By 3rd bullet, you mean Data Validation?
Re:Show HN: Real world (Jupyter notebook embed) way to asses...
by ska on Friday August 30, 2019 @07:58PM
(#20843810)
both data validation and model validation are important.
Of course more abstractly you want to validate the entire process - but I was referring to those two, as it is hard to see how to address the latter in a format like this.
This meant we couldn't automatically score/rank a candidate's solution. We do provide them with an evaluation metric in the problem description (Mean Absolute Error). Here is scoring rubric we provide to the interviewers when they review the submission - https://d.pr/i/hNYY0u
Would love to hear more opinions on our scoring rubric