HyperTools: A Python toolbox for gaining insights into high-dimensional data

Re:HyperTools: A Python toolbox for gaining insights into hi... by legel on Saturday April 28, 2018 @06:13PM (#16949681)

Looks neat and wonderful. It’s always impossible to compress all structure from high dimensions to 2D or 3D via something like PCA or t-SNE, and the focus on “geometric” insight is also encouraging. Cool/appropriate name too, nice choice of well-supported dependencies, and minimalist design are all appealing. This could become a go-to toolkit for early stage exploration, while it’s also pretty and smart enough to wow a coworker / investor. Hope it continues to really develop conceptually and practically!

[ Parent | on HN ]

Re:HyperTools: A Python toolbox for gaining insights into hi... by SubiculumCode on Saturday April 28, 2018 @06:22PM (#16949725)

Do you happen to know of a good graphing package for high dimensional data in R?

[ Parent | on HN ]

Re:HyperTools: A Python toolbox for gaining insights into hi... by hawktheslayer on Saturday April 28, 2018 @11:34PM (#16950579)

I find the best way for my feeble mind to understand more than 3 or 4 dimensions is a parallel coordinate graph where you can view the relationships between two of the dimensions at a time. You can do this interactively in R using plotly:

https://plot.ly/r/parallel-coordinates-plot/

[ Parent | on HN ]

Re:HyperTools: A Python toolbox for gaining insights into hi... by wodenokoto on Sunday April 29, 2018 @04:27AM (#16951184)

As far as I can tell, HyperTools are a bunch of convenience functions for doing something along the lines of the following dummy R code

    df %>% dim_reduction_function() %>%
       ggplot() + geom_*()

So in a sense, I think you should be looking for a dimensionality reduction library, rather than a plotting library.

[ Parent | on HN ]

Re:HyperTools: A Python toolbox for gaining insights into hi... by SubiculumCode on Sunday April 29, 2018 @04:34AM (#16951204)

so it seems, looking at their gallery, and I'd rather just write the lines calling pca, or whatever.

[ Parent | on HN ]

Re:HyperTools: A Python toolbox for gaining insights into hi... by hadley on Sunday April 29, 2018 @12:55PM (#16952790)

You should take a look at tourrr which implements a bunch of "grand tour" algorithms in R. These take your on a smooth tour of random projections of your data, visualised in various ways (including into 3d if you have some red-blue 3d glasses!)

[ Parent | on HN ]

Re:HyperTools: A Python toolbox for gaining insights into hi... by SubiculumCode on Sunday April 29, 2018 @02:00PM (#16953099)

Interesting thanks

[ Parent | on HN ]

Re:HyperTools: A Python toolbox for gaining insights into hi... by Blahah on Saturday April 28, 2018 @06:31PM (#16949772)

Good visualisation of high dimensional data is almost all about the mental model and mapping of dimensions, barely ever about the library. ggplot2 is excellent for displaying high dimensional data in R if you can reason well about your own data. If you can't, no library is going to save you.

[ Parent | on HN ]

Re:HyperTools: A Python toolbox for gaining insights into hi... by SubiculumCode on Saturday April 28, 2018 @06:34PM (#16949787)

Well I already use ggplot. I'm not sure ggplot does 3D much.... About the reasoning, well who knows.

[ Parent | on HN ]

Re:HyperTools: A Python toolbox for gaining insights into hi... by Blahah on Saturday April 28, 2018 @06:36PM (#16949799)

If you present an example of high-dimensional data I'd be happy to help you reason about how to visualise it (although this is perhaps better suited to StackExchange).

edit: if you feel you need 3D, you've already failed to represent the data clearly

[ Parent | on HN ]

Re:HyperTools: A Python toolbox for gaining insights into hi... by p1esk on Saturday April 28, 2018 @10:18PM (#16950425)

if you feel you need 3D, you've already failed to represent the data clearly

Why?

[ Parent | on HN ]

Re:HyperTools: A Python toolbox for gaining insights into hi... by nhlism on Sunday April 29, 2018 @03:23AM (#16951047)

I took a visualization course and to paraphrase the prof. you have a 3d object projected onto a 2d (display) plane and reasoning about depth gives additional mental load, not to mention occlusion. You have to interact with the model a lot to get a view you want. The course was based on old papers about UX with a dash of cognitive science, so take it with a grain of salt.

[ Parent | on HN ]

Re:HyperTools: A Python toolbox for gaining insights into hi... by Blahah on Sunday April 29, 2018 @09:37AM (#16951941)

Yup, this. There's almost always a way to represent the data that allows clearer interpretation.

[ Parent | on HN ]

Re:HyperTools: A Python toolbox for gaining insights into hi... by SubiculumCode on Sunday April 29, 2018 @02:05AM (#16950878)

He is umm. very opinionated. Its kind of like saying why need two dimensions, surely a 1D vector is good enough..

[ Parent | on HN ]