An Introduction to Hierarchical Modeling

http://mfviz.com/hierarchical-models/

by andyxor • 5 years ago

112 17 5 years ago

mfviz.com

plouffy2• 5 years ago

Very cool animation.

antipaul2• 5 years ago

How might this be combined with a training/testing data splitting paradigm, or even using an ML model?

causalmodels• 5 years ago

Probabilistic machine learning is still machine learning. If you're referring to one of the popular tensor frameworks, pretty much all of them have probabilistic features or add-ons [1][2][3]. There are also frameworks developed specifically for this kind of work [4][5].

This [6] is a more in depth look at hierarchical / multi-level modeling. The prediction section [7] specifically goes over cross validation and inference.

[1] https://www.tensorflow.org/probability/examples/Multilevel_M... [2] https://pyro.ai/examples/forecasting_iii.html [3] http://edwardlib.org/ [4] https://mc-stan.org/ [5] https://docs.pymc.io/ [6] https://docs.pymc.io/notebooks/multilevel_modeling.html [7] https://docs.pymc.io/notebooks/multilevel_modeling.html#Pred...

RA_Fisher• 5 years ago

ML are algos not models (estimators to be precise). This model could be fit by many of them. However, fitting the model directly with the linear model will be advantageous for computational reasons.

abeppu• 5 years ago

Though the visualization is cool here, I think this is actually not a good example. The final "hierarchical" model presented doesn't say anything about a relation between the coefficients for the different departments, so one can view this as really just having separate models, one for each department.

To me, the point of a hierarchical model is that one can assume relationships between these parameters (e.g. suppose that they are from the same distribution). Then when one department has very few data points, some of the information in your beliefs about the parameters for that department come from what you learned in other departments. E.g. the base salary is likely to be similar to base salaries of other departments.

jaygray0919• 5 years ago

Agree. The example is not a Class hierarchy - it is a group with different coefficients. But the concept is useful, and the related visualization (for Class comparisons) would be useful.

antipaul2• 5 years ago

What if you simply add department and department*salary interaction terms to a standard regression model?

wodenokoto2• 5 years ago

Then for any given data point, salary for all other departments will be zero (dep1salery = dep2salary = ... = 0) so data for each department will become separate and you will in effect have i number of different models, although they all share the same bias term / intersect.

plouffy2• 5 years ago

You could argue that a hierarchical model is already multiple models. If each department has a dummy variable then they wouldn't share the same intersect. The only difference is how you interpret that intercept. The department intercept would now not be the salary of an individual with 0 experience, but the difference between the salary of an individual with 0 experience in a department compared to the all other employees with 0 experience.

aseerdbnarng• 5 years ago

You'll run out of data pretty quickly.

https://statmodeling.stat.columbia.edu/2018/03/15/need-16-ti...

alisonkisk3• 5 years ago

Hierarchial model has the same problem, since it splits up the data into categories.

alisonkisk3• 5 years ago

I think it gives you the same answer without having the intuitive picture of your kernel that people like for toy problems.

ZephyrBlu2• 5 years ago

I love the visualization and animation, but I think it should be specified that this is Linear Hierarchical Modelling.

I was expecting something bayesian-related given that bayesian statistics lends itself to hierarchical models quite nicely.

gegtik• 5 years ago

I don't quite understand what's hierarchical about this example. seems like a few different mutually exclusive segments that can then be independently analysed?

ZephyrBlu2• 5 years ago

It's hierarchical because of the nature of the data, not because of the model itself.

alisonkisk3• 5 years ago

Is this a narrow special case of an SVM?

Why not use a SVM?

An Introduction to Hierarchical Modeling

Related Stories