HyperTools: A Python toolbox for gaining insights into high-dimensional data

http://hypertools.readthedocs.io/en/latest/#

by schuetze • 8 years ago

174 23 8 years ago

hypertools.readthedocs.io

legel• 8 years ago

Looks neat and wonderful. It’s always impossible to compress all structure from high dimensions to 2D or 3D via something like PCA or t-SNE, and the focus on “geometric” insight is also encouraging. Cool/appropriate name too, nice choice of well-supported dependencies, and minimalist design are all appealing. This could become a go-to toolkit for early stage exploration, while it’s also pretty and smart enough to wow a coworker / investor. Hope it continues to really develop conceptually and practically!

SubiculumCode5• 8 years ago

Do you happen to know of a good graphing package for high dimensional data in R?

Blahah6• 8 years ago

Good visualisation of high dimensional data is almost all about the mental model and mapping of dimensions, barely ever about the library. ggplot2 is excellent for displaying high dimensional data in R if you can reason well about your own data. If you can't, no library is going to save you.

hawktheslayer• 8 years ago

I find the best way for my feeble mind to understand more than 3 or 4 dimensions is a parallel coordinate graph where you can view the relationships between two of the dimensions at a time. You can do this interactively in R using plotly:

https://plot.ly/r/parallel-coordinates-plot/

wodenokoto• 8 years ago

As far as I can tell, HyperTools are a bunch of convenience functions for doing something along the lines of the following dummy R code

    df %>% dim_reduction_function() %>%
       ggplot() + geom_*()

So in a sense, I think you should be looking for a dimensionality reduction library, rather than a plotting library.

hadley• 8 years ago

You should take a look at tourrr which implements a bunch of "grand tour" algorithms in R. These take your on a smooth tour of random projections of your data, visualised in various ways (including into 3d if you have some red-blue 3d glasses!)

Myrmornis• 8 years ago

The first example

http://hypertools.readthedocs.io/en/latest/auto_examples/plo...

seems a little disappointing -- it seems to exemplify more than solve the problems of representing 3D data on a 2D screen. (Is it interactively rotatable or something? I did run it locally in a notebook and it didn't appear to be.)

Blahah6• 8 years ago

Indeed, all the examples appear to be showing datasets that would benefit from good data visualisation and in some cases dimension reduction, but all of them are terribly represented. I don't think this necessarily a problem with the library, but the examples certainly do not exemplify good visualisation of high-dimensional data.

daveguy• 8 years ago

When you say it doesn't exemplify good visualization of high-dimensional data do you have alternate examples in mind? I would like to be able to see examples of good visualization of high dimensional data.

Blahah6• 8 years ago

I can't see anything in the tutorials or examples that makes this a good library for visualisation of high-dimensional data. It seems somewhat equivalent to base graphics in R, which is much less useful for visualising high-dimensional data than anything based on Wilkinson's grammar of graphics, like ggplot2 in R.

sanxiyn• 8 years ago

Quote of the day from HyperTools README https://github.com/ContextLab/hypertools

"To deal with hyper-planes in a 14 dimensional space, visualize a 3D space and say "fourteen" very loudly. Everyone does it." -- Geoff Hinton

kgwgk• 8 years ago

Don’t be silly. One can visualize the N-dimensional case and then it is just a matter of setting N=14.

bagrow• 8 years ago

Would this be useful for word vectors/embeddings?

gaius• 8 years ago

EdX has a good course on this (tho' taught in R) https://www.edx.org/course/high-dimensional-data-analysis-ha...

HyperTools: A Python toolbox for gaining insights into high-dimensional data

Related Stories