Re:Machine Learning for Everyday Tasks
by nacc on Tuesday November 22, 2016 @04:04PM
(#13017633)
I might have missed something, but if they only have 2 features, simply plotting the data out will make any trend very clear.
[ Parent
| on HN ]
Re:Machine Learning for Everyday Tasks
by sergey-obukhov on Tuesday November 22, 2016 @04:23PM
(#13017797)
Re-pasting my comment here where it belongs as a reply:
It could be a great idea - to plot the data on 2 axes (x - html length or html size, y - processing time, if I understand this correctly). It's simple and elegant. I'll try that. It could be though that the chart will get messy with all this data points. One of the reasons I like the percentiles approach is that it makes it clear that there is a trade-off between message processing time and the number / percentage of messages we can process.
Re:Machine Learning for Everyday Tasks
by minimaxir on Tuesday November 22, 2016 @05:36PM
(#13018300)
> It's simple and elegant.
The OP likely made that comment because plotting the data is often done before the fancy machine learning as a part of the exploratory data analysis. (especially with a low number of variables!)
Re:Machine Learning for Everyday Tasks
by gmfawcett on Tuesday November 22, 2016 @11:24PM
(#13019996)
IanCal's comment about setting the alpha is a good one. Another easy option is to make a heatmap of 2D bin counts. Since you're using R already, you can use ggplot for this:
Set a low alpha to the points on the chart of there are loads.
I'm reminded of Anscombe's Quartet as an example of how a little visualization can reveal things that might be hidden behind summary statistics.
Re:Machine Learning for Everyday Tasks
by prashnts on Tuesday November 22, 2016 @07:54PM
(#13019042)
For higher dimensions, a RadViz plot with lower alpha is really useful too.
[0] http://docs.orange.biolab.si/2/widgets/rst/visualize/radviz....
Re:Machine Learning for Everyday Tasks
by kehrlann on Tuesday November 22, 2016 @04:14PM
(#13017712)
Agreed. Also, they don't discuss how they chose their features... It seems the problem was already solved before even applying ML to it. Maybe the example is too naive ?
Re:Machine Learning for Everyday Tasks
by sergey-obukhov on Tuesday November 22, 2016 @04:22PM
(#13017786)
Html size and html tags count was a natural choice. If it didn't work out the next step would be to try something else. You're right that it's a very naive example and that in a way it was solved before any ml was applied. The surprising part for me was that both features turned out to be interchangeable i.e. any of them could be used. I would expect html tags count to be much more accurate / reliable, etc. Another interesting part for me was the threshold. It's somewhat clear that it should be somewhere between 20 and 1 sec probably but where exactly?
The slowest part of any parser is the lexer - guessing whatever processing they do on the parse tree structure is insignificant by comparison.
Re:Machine Learning for Everyday Tasks
by stevetrewick on Tuesday November 22, 2016 @04:40PM
(#13017912)
Would the person or persons down voting all the author of TFA's comments mind explaining why? Seems a bit off.