dccsillag

I'm sorry, but what exactly is the problem with the code? I've been staring at it for quite a while now and still don't see what is counterintuitive about it.

Take enough moments and you'll be able to differentiate any distributions.

I think you vastly overestimate the capacity of Python typing.

Typing still sucks big time. Same for perf, unless you are working with numerics that fit with something like NumPy or JAX.

No, it's generally not valid -- it will depend on the specifics of the test (especially if the test is valid only asymptotically). You need some method that supports sequential inference. Nowadays your best bet is probably some sort of anytime-valid method from the e-value literature https://en.wikipedia.org/wiki/E-values https://projecteuclid.org/journals/statistical-science/volum...

Hard disagree. There are a lot of implementation "details" that, if you want to do properly, are a lot of hard work and very much nontrivial. For example, do try to write a compiler with efficient incremental compilation, and especially one that does so while also having optimization passes. And that's just one example, most things in compiler implementations actually turn out to be fairly complex. And lots of features that modern languages support e.g. more powerful typesystems, trait/typeclass systems, etc. are also very very tricky.

While designing a language is by no means trivial, it generally really occupies just a very small fraction of the language/compiler developer's time. And, in most cases, the two things (language design + implementation details) have to walk hand-in-hand, since small changes to the language design can vastly improve the implementation end.

I've never seen this done in a research setting. Not sure about how much of a standard practice it is.

It has `catch_unwind` [1], but that still retains the panicking runtime, so not sufficient in the context of the post.

[1] https://doc.rust-lang.org/std/panic/fn.catch_unwind.html

Yeah, I'm fairly sure that there is such a flag/toplevel attribute... and if there isn't, there should be one.

It also feels like most of the pains on avoiding panics centers around allocations which, though a bit unfortunate, makes sense; it was an intentional design choice to make allocations panic instead of return Results, because most users of the language would probably crash on allocation fails anyways and it would introduce a lot of clutter. There was some effort some while ago on having better fallible allocations, but I'm not sure what happened over there.

Can we please rename this submission? This is excessively grandiose, way over the top......

If I'm not mistaken, the treatment of forward declarations proposed in the article actually breaks the C standard, which would be a rather pressing concern. As far as I am aware, that is the reason why things are the way they are right now in C land.

(In the past, there were more legitimate concerns on the ease of implementation. Nowadays, as the article points out, they are pretty moot, other than having to keep backwards-compatibility.)

I'm also rather bothered that on the bit on const execution in the article, there was no discussion on how to deal with functions that may not terminate or take rather long to execute. Especially considering the unit tests motivation, this seems like a rather blaring omission.

If thresholding of P-values is the issue, E-values -- a recent, much more elegant, easier to work with, and more robust alternative to P-values -- solve this.

https://arxiv.org/abs/2312.08040 https://arxiv.org/abs/2205.00901 https://arxiv.org/abs/2210.01948 https://arxiv.org/abs/2410.23614

I've used Quiver a number of times now, and all of them were great experiences. Kudos to the authors!

(Side note) That's odd, lobste.rs seems to be down for me, and has been like that for a couple of months now -- I literally cannot reach the site.

Is that actually just me??

EDIT: just tried some things, very weird stuff: curl works fine. Firefox works fine. But my usual browser, Brave, does not, and complains that "This site can't be reached (ERR_INVALID_RESPONSE)". Very very very weird, anyone else going through this?

Well, yes, but that's an extremely low bar!!

I find it odd that all of the mathematicians cited at the end are actually pretty much CS people, working on proof assistants. Kinda renders that section pointless, IMO (though the comment by Isabelle's author was interesting).

IMO, whether something like this makes sense is a separate matter. Personally I always just think of division in terms of multiplicative inverses, so I don't see how defining division by zero helps other than perhaps making implementation easier in a proof assistant. But I've seen people say that there are some cases where having a/0 = 0 works out nicely. I'm curious to know what these cases are, though.

Theory-wise, I'm not convinced that the models have good approximation properties (the Kolmogorov-Arnold / Kolmogorov Superposition Theorem they base themselves on has quite a bit of nuance), and the optimization problem might be a bit tricky. I'm also can't see how to incorporate inductive biases other than the standard R^n / tabular regression one, and the existing attempts on this that I'm aware of are just band-aids (along the lines of feature engineering).

In practice, I've personally ran some benchmarks on a collection of datasets I had laying around. The results were generally abysmal, with the method only matching simple baselines in some few datasets.

Finally, the original paper is very weird, and reads more as a marketing piece. The theory, which is touted throughout the paper, is very weak, the actual algorithm is not sufficiently well explained there and the experiments are lacking. In particular, I find it telling that they do not include and even go out of their way to ignore important baselines such as boosted trees, which are the state-of-the-art solution to the problem that they intended to solve (and even work very well in occasions where they claim that both KANs and MLPs perform badly, e.g. in high dimensions).

Ah, Kolmogorov Arnold Networks. Perhaps the only model I have ever tried that managed to fairly often get AUCs below 0.5 in my tabular ML benchmarks. It even managed to get a frankly disturbing 0.33, where pretty much any other method (including linear regression, IIRC) would get >=0.99!

Yeah, I know the account you are talking about, it really is a bit over the top. It's a shame, I've met a bunch of people who mentioned that they were actually turned away from Conformal Prediction due to them.

But having said that, Conformal Prediction works as advertised for UQ as a wrapper on any point estimating model. If you've got the data for it - and in the ML setting you do - and you don't care about things like missing data imputation, error in inputs, non-iid spatio-temporal and hierarchical structures, mixtures of models, evidence decay, unbalanced data where small-data islands coexist big data - all the complicated situations where Bayesian methods just automatically work and other methods require elaborate workarounds, yup, use Conformal Prediction.

Many of these things can actually work really well with Conformal Prediction, but the algorithms require extensions (much like if you are doing Bayesian inference, you also need to update your model accordingly!). They generally end up being some form of reweighting to compensate for the distribution shifts (excluding the Online Conformal Prediction literature, which is another beast entirely). Also, worth noting that if you have iid data then Conformal Prediction is remarkably data-efficient; as little as 20 samples are enough for it to start working for 95% predictive intervals, and with 50 samples (and with almost surely unique conformity scores) it's going to match 95% coverage fairly tightly.

That's a particular form of Conformal Prediction, called Split Conformal Prediction. Incidentally, it's also one of the best ones (i.e., most extensible, strongest guarantees, easiest to implement, remarkably sample-efficient).

Making a calibration set is pretty easy, it's just a data split (just like the train/test split). The hardest part (which is still fairly easy) is creating a 'conformity score', which is a function that receives the input and a candidate output and scores how well this candidate output 'conforms' to the input. This is where an underlying ML model can come in handy: it can, itself, estimate this! Split Conformal Prediction then does a fairly simple quantile calculation on these scores (or some variant thereof) to then form the set prediction.

In a sense, you could use Bayesian NNs to produce a conformity score. But that doesn't seem to be much better than just using e.g. the model's logits for your conformity score. Theory-wise, Conformal Prediction methods have a number of favorable guarantees that Bayesian models (and especially Bayesian NNs) generally don't, and in practice we've seen that conditional on the model giving calibrated outputs (which is guaranteed for Conformal Prediction, but not for Bayesian NNs), Conformal Prediction predicted sets seem to be tighter than the Bayesian NN ones.

Bayesian Neural Networks just seem like a failed approach, unfortunately. For one, Bayesian inference and UQ fundamentally depends on the choice of the prior, but this is rarely discussed in the Bayesian NN literature and practice, and is further compounded by how fundamentally hard to interpret and choose these priors are (what is the intuition behind a NN's parameters?). Add to that the fact that the Bayesian inference is very much approximate, and you should see the trouble.

If you want UQ, 'frequentist nonparametric' approaches like Conformal Prediction and Calibration/Multi-Calibration methods seem to work quite well (especilly when combined with the standard ML machinery of taking a log-likelihood as your loss), and do not suffer from any of the issues above while also giving you formal guarantees of correctness. They are a strict improvement over Bayesian NNs, IMO.

And the potential outcomes framework (Neyman-Rubin) is even more general :)

Either way, Holland's 'Statistics and Causal Inference' paper (1986) is a nice read on the different frameworks for causality, especially in regards to Granger (&friends) versus do-calculus/Neyman-Rubin.

All know 20 year old latex documents will be compilable in 20 years.

Except it's not. For example, in 2021, the NeurIPS template compiled differently in different versions of TeXLive (https://tex.stackexchange.com/questions/598567/different-pdf...). To my knowledge this change of behaviour has not been addressed in TexLive.

Latex is horrible but makes beautiful documents and the relevant world either uses latex or Mircosoft Office. Why use anything else?

And why invent anything new when there are already existing alternatives? Why invent C++, Rust, Zig when there already was C? Why invent Clang when there already was GCC? Why invent Git when there was already CVS? And so on. (Maybe not the best examples, but that's what I could think of on the spot.)

The point is that there are aspects in which the current solutions (in this case, LaTeX) fails, and new systems can address them, since they tend to have much more liberty in the design space than the older system.

Because a library in a programming language would be madly inconvenient? I can't see something like this being pleasant to use:

  document(
    paragraph("blah blah blah"),
    math("sum_(i=1)^n i = (n (n+1))/2"),
    paragraph("foo bar")
  )