JHonaker

I physically cringed at both the quote and the surrounding section. The idea of this project is cool, but the amount of LLM glazing is bizarre.

capable of breaking SBCL's jaw

What exactly do you mean by this? Speed? Portability? Ease of use?

Even better!

Totally get that. I was mostly just long-windedly complaining that the one problem I have with it seems to be exacerbated by, not fixed, by this. I was also hoping someone would say “oh it’s actually way easier than you think, see (amazing link).”

I really do think it’s a good idea to explore! Sometimes I feel crazy because I’m the only one in my department that prefers to just write SQL to deal with our DBs instead of fiddling with a python/R connector that always has its own quirks.

I applaud the project, and I completely agree that the concepts maps nicely to SQL. The R equivalent of a WITH data prep block followed by the VISUALIZE is pretty much how all my plotting code is structured.

However, I don't see what the benefits of this are (other than having a simple DSL, but that creates the yet another DSL problme) over ggplot2. What do I gain by using this over ggplot2 in R?

The only problem, and the only reason I ever leave ggplot2 for visualizations, is how difficult it is to do anything "non-standard" that hasn't already had a geom created in the ggplot ecosystem. When you want to do something "different" it's way easier to drop into the primitive drawing operations of whatever you're using than it is to try to write the ggplot-friendly adapter.

Even wrapping common "partial specificiations" as a function (which should "just work" imo) is difficult depending on whether you're trying to wrap something that composes with the rest of the spec via `+` or via pipe (`|>`, the operator formerly known as `%>%`)

Most of the memories I have of my grandmother are post-serious Parkinson's progression. She was able to live a very long life, but it seriously affected her. Good luck, hoping for the best for you.

Fuck Parkinson's.

I’m not sure what your professional experience is in, but as a counterpoint, I’ve never been in a situation where I hadn’t wished for a system I’m working with to already be in a Bayesian framework. Having said that, I only occasionally am building things from scratch instead of modifying existing systems, so I’m not always lucky enough to be able to work with them.

The pain points around getting a sampler/model pairing working in a reasonable timeframe is definitely a valid complaint. In my experience, inference methods in Bayesian stats are much less forgiving of poorly specified models (or said another way, don’t let you get away with ignoring important structural components of the phenomena of interest). A poorly performing model (in terms of sampler speed/mixing) is often a sign of a problem with the geometry of the parameter space. Frustratingly this can sometimes be a result of conceptually equivalent, but computationally different parameterizations (e.g. centered vs non-centered multi level effects).

The struggles are worth it IMO because it is helpful feedback that helps guide design, and the ease with which I can compute meaningful uncertainty bounds on pretty much any quantity of interest is invaluable.

I shorted it and it crashed the page. I feel like that was appropriate. :D

My go to for teaching statistics is Statistical Rethinking. It’s basically a course in how to actually thing about modeling: what you’re really looking for is analyzing a hypothesis, and a model may be consistent with a number of hypotheses, figuring out what hypotheses any given model implies is the hard/fun part, and this book teaches you that. The only drawback is that it’s not free. (Although there are excellent lectures by the author available for free on YouTube. These are worth watching even if you don’t get the book.)

I also recommend Gelman’s (one of the authors of the linked book) Regression and Other Stories as a more approachable text for this content.

Think Bayes and Bayesian Methods for Hackers are introductory books from a beginner coming from a programming background.

If you want something more from the ML world that heavily emphasizes the benefits of probabilistic (Bayesian) methods, I highly recommend Kevin Murphy’s Probabilistic Machine Learning. I have only read the first edition before he split it into two volumes and expanded it, but I’ve only heard good things about the new volumes too.

Any time you start conditioning on something, i.e. selecting subsets of data to analyze. You can fool yourself quite often if you do something seemingly innocuous like select "everyone with at least one X" and compare expectations to what's true unconditionally (meaning not conditioning on anything, not "in all cases") with conditional computations.

This is beautiful whether you interpret it genuinely or as satire.

Agreed wholeheartedly. I have argued with the VP of our department about this paper quite a few times.

I feel like Breiman sets up a strawman that I've never encountered when I work with my colleagues that are trained in the statistics community. That doesn't mean it didn't exist 25 years ago when he wrote it. I concede that we are sometimes willing to make simplifying assumptions in order to state something particular, but it's almost like we've been culturally conditioned to steep everything we say with every caveat possible.

Whereas I am constantly having to point out the poor feedback we've had about some of the XGBoost models despite the fact that they're clearly the most "predictive" when evaluated naively.

I'm so slow it takes too much time for me to not think that it's a waste of time because I could have done something more meaningful instead.

That's the funny thing about the idea of meaningful things. It is solely determined by what you think is meaningful. Personally, just sitting and making something is an extremely meaningful activity to me.

Sometimes even eye to brain to hand latency is too long...

Very neat, but audio doesn't work on desktop either. (Linux, Firefox) I had an old-school generated voice say "Chinese letter Chinese letter Chinese letter." It was definitely amusing, but I don't think that's what you were going for.

Wow. I thought you were being glib, but the average comment length is noticeably higher in the linked discussion. While length isn’t necessarily a valid proxy for meaningful conversation, this was definitely an eye-opening contrast to the current thread.

Yes but your library still legally obtained those copies in the first place.

Ah, I did not realize that the `realNVP` was a link! Thanks.

Thanks! I've read the first one before. I'll take a look at the other two!

I'm very excited by the work being put in to make Bayesian inference more manageable. It's in a spot that feels very similar to deep learning circa mid-2010s when Caffe, Torch, and hand-written gradients were the options. We can do it, but doing anything more complicated than common model structures like hierarchical Gaussian linear models requires dropping out of the nice places and into the guts.

I've had a lot of success with Numpyro (a JAX library), and used quite a lot of tools that are simpler interfaces to Stan. I've also had to write quite a few model-specific things from scratch by hand (more for sequential Monte Carlo than MCMC). I'm very excited for a world where PPLs become scalable and easier to use /customize.

I think there is a good chance that normalizing flow-based variational inference will displace MCMC as the go-to method for Bayesian posterior inference as soon as everyone gets access to good GPUs.

Wow. This is incredibly surprising. I'm only tangentially aware of normalizing flows, but apparently I need to look at the intersection of them and Bayesian statistics now! Any sources from anyone would be most appreciated!

There's no reason you still can't!

I've been looking for such a walkthrough for months. I can't find anyone actually showing them working and building something. There are plenty of, "look at this code this thing spit out that successfully builds," but I've not been able to find anything I would consider truly showing someone building something novel.

If anyone has a YT video or a detailed article, please show me!

I am not the author of the book!

As far as the wiki: It's not hosted. It's just on my local machine. You basically just drop the minified JS library in a tiddler and then create a custom widget type and a ViewTemplate to automatically display the widget when the content-type is the correct one.

I used http://whatfettle.com/2008/05/TiddlyProcessing/ as the basis (although it's old) and the main docs on creating a custom widget (https://tiddlywiki.com/dev/#Javascript%20Widget%20Tutorial)

I love TiddlyWiki. I've been developing a highly non-linear document for some conceptual documentation with it at work.

If you want to see what you can really do with TW, or want to learn how, check out Grok TiddlyWiki (https://groktiddlywiki.com/read/)

I built a personal chess knowledge base with it too. I added a content-type that let me just paste in a plain-text representation of the game and it would display the game with an interactive board.

Ah, I see. I read the article, but not the links. I even searched for eigenvalue/eigenvector to see if I was crazy.

The first linked article is very good. I always enjoy Ian’s work.

Did you post this on the wrong article?

The fact that I can go months without updating a rolling distro and things very rarely break is incredible. Tumbleweed is rock solid. (Personally, I've never experienced a break, but I know one just happened for a lot of people. Granted, I believe it was Nvidia or Mesa drivers at fault.)

Yea, the more I learn about things, the more I realize that everything is defined relative to something else. Even measurements are defined relative to some standard (that beautifully manicured kilogram ball, the speed of light in a vacuum, your ruler's hand width or foot length, etc.) That's both very satisfying and extremely frustrating.

Programming languages can't escape it either; see the tautologies at the top of your favorite programming language (metaclasses, metaobject protocols, etc).

That was a great article.

If I'd read this when I was first learning measure theory, I would have had a much easier time. In fact, it took me an embarrassingly long time to realize that sigma algebras were just the "nice sets and subsets" of things that we can extend measures from finite additivity to countable additivity.

I used to think mathematical objects were somehow "inherent". I was always amazed that people had discovered and proved so many interesting things about them. Once I realized they were often just defined to be the thing that has the property we want to prove something about, it got a lot less mysterious.

Note, I'm not saying we just stop there, or that this is somehow bad. The next obvious step taken by mathematicians is to start removing bits of the objects they study, and try to figure out what's still provable until we get to categories, logic, and start arguing about things like the axiom of choice.

Markov Chain Monte Carlo for program synthesis isn't exactly novel. The most immediate reference I thought of is Josh Tenenbaum's [1].

There's also a lot of demos in WebPPL (web probabilistic programming language)[2] like [3] for the synthesis of 3D space-ships. I highly recommend their associated books on The Design and Implementation of Probabilistic Programming Languages [4] and Probabilistic Models of Cognition [5].

I also highly recommend taking a look at the publications of the MIT Probabilistic Computing Project [6].

[1] Human-level concept learning through probabilistic program induction. https://www.cs.cmu.edu/~rsalakhu/papers/LakeEtAl2015Science....

[2] http://webppl.org/

[3] https://dritchie.github.io/web-procmod/

[4] https://dippl.org/

[5] http://probmods.org/

[6] http://probcomp.csail.mit.edu/