News for hackers. Stuff that matters.
Re:Visualizing Large-Scale and High-Dimensional Data by vladimirralev on Sunday May 08, 2016 @09:21PM (#11656964)
I wonder why they are positioning this as visualisation technique rather than dimensionality reduction which would be a much bigger deal. Is there something to suggest the technique doesn't work well for reduction in arbitrary dimensions?
Re:Visualizing Large-Scale and High-Dimensional Data by eximius on Sunday May 08, 2016 @09:50PM (#11657079)
Disclaimer: Havent read it but have read summaries here.

It sounds like it is very fast but not very rigorous. This lets you get a feel for the data but it doesn't give you the same guarentees other dimensionality reductions do.

Re:Visualizing Large-Scale and High-Dimensional Data by cs702 on Sunday May 08, 2016 @10:36PM (#11657262)
That's my sense from skimming the paper.
Re:Visualizing Large-Scale and High-Dimensional Data by emn13 on Monday May 09, 2016 @06:36AM (#11658745)
What kind of guarantees are you thinking of?
Re:Visualizing Large-Scale and High-Dimensional Data by visarga on Monday May 09, 2016 @12:15AM (#11657580)
I read a paper [1] about visualizing the Deep Q-Network used for playing Atari games. They map to 2D the game state and then colorize it by the Q scores. As a result they could visualize the strategy and observe how it is hierarchically organized in clusters containing sub-clusters. They could show regions associated with initial and termination rules. This method can be used to map out a game strategy and to focus on specific regions.

[1] Graying the black box: Understanding DQNs - https://arxiv.org/pdf/1602.02658.pdf

Re:Visualizing Large-Scale and High-Dimensional Data by cbsmith on Wednesday May 11, 2016 @09:41AM (#11675134)
The dimensionality reduction problem space rarely tries to reduce down to only two or three dimensions. The fact that in the end you are being reduced to so few dimensions allows for a lot of potential errors to not meaningfully impact the result.
Re:Visualizing Large-Scale and High-Dimensional Data by st553 on Monday May 09, 2016 @11:21AM (#11660481)
Anyone know if its possible or makes sense to use something like t-sne for dimensionality reduction? If so, could the reduced data set be used to build a classifier?