contravariant

There are a couple of references, most on some obscure detail. The ones that might be worth a try for an introduction are:

- Walters, P., An Introduction to Ergodic Theory, Graduate Texts in Mathematics, 79. Springer-Verlag, New York-Berlin, 1982

- Billingsley, P. – Ergodic Theory and Information, John Wiley and Sons, 1965.

You may need a refresher on measure theory, or just ignore that part I suppose

Not sure, I'm basing most of this on lecture notes I got at university.

I think looking into ergodic theory would help fill in the gap between completely random and completely rule based. In ergodic theory you have some kind of 'state' and a function way to 'advance' the state, if you partition the state space you also get a sequence of 'bytes' to send (simple example: take a number in [0,1] multiply by 10, send the first digit and continue with the remainder). The state needs a distribution, that needs to be preserved by advancing the state, but it's pretty close to running an algorithm otherwise.

You'll probably also want to look at uniquely ergodic systems. Those are interesting in the way that they behave 'random' for every input, one example is if you keep multiplying a number by anything other than a power of 10 and divide it by a power of 10 so it's between 0 and 1. Turns out you get the same distribution of results every time (eventually).

And while you can define an entropy rate without I think ergodic theory also helps inform why that is the correct generalisation. Your current article only considers the case where each digit is independently random (it's clear why, it's a lot simpler and all that information theory tends to talk about).

I think the explanation of entropy's blind spot is a bit off. It's not actually a problem for entropy if something is generated by a rule, you can calculate entropy for things like the continuous fractions for instance, with an easy rule to generate them for any particular number. Likewise for decimal expansions.

The real blind spot is that entropy is meaningless for a specific sequence, you can't really ask about the entropy of pi if you don't have a theory for how the numbers are generated. Sure if it is pick a uniformly random real number between 0 and 10 then both files have equivalent entropy, but sending pi is also vanishingly unlikely.

There's actually a more subtle way in which this is a blind spot, which takes a bit more machinery. You can define entropy for an ergodic system, which could be considered a kind of mathematical RNG. Now as it turns out this provides a way to generate something almost equivalent to a particular distribution except that this argument only holds for most starting points not all. A direct example would be how pi generates a perfectly fine random distribution of digits (we think) but something like 1/3 does not.

If compliance was the main issue we wouldn't have had to invent ways for computers to do something other than exactly what they were asked.

You'll recognise the good engineers as the ones that remove code.

Even that is overselling the effort. Last time I checked you could find IDs with a simple image search.

I just want them to tell me if they don't know.

It's the one question that AIs seem unable to answer correctly.

If you feel the need to hide how you got the answer then you know something is wrong.

To minimise the KL you just calculate the surprisal. The integral can be approximated by sampling over your training data. It's a direct expression of the information loss between your real data and your fitted probability distribution.

Calculating the JSD could be more difficult, the expression uses a mixture between the 'true' and 'fitted' distribution. You can still simulate this, but half the time you'd be fitting the model to itself, and I just don't see why that would be useful.

I think the JSD is most useful when you need an actual metric, but as long as you have a fitted and target distribution the KL divergence is a natural fit since you can interpret the result as information loss.

I wonder if using a hilbert curve (or perhaps the simpler z-curve) access order would help things. If ought to work well regardless of cache size.

Actually I think I recall some GPUs storing textures that way, but I'm not entirely sure.

I was expecting more like a minimal usage that makes sense for a butchery or something, not get rid of the people using email.

What's the LLM equivalent to email and a basic webpage?

That's just a baseless assumption. To use AI well you should do the things that allow you to use stuff well. You shouldn't just use it any way you can because you assume that 'not using it at all' is not the best option.

This is literally the same with every single technological development.

You've somehow confused using AI well with using it extensively.

Sometimes using something well involves not using it at all.

I'm also pretty sure 14 points font is a bit outdated at this point, 16 should probably be a minimum with current screens. It's not as if screens aren't wide enough to fit bigger text.

Oh that's annoying, seems to me there wouldn't have been an issue if you just merged B into A after merging A into main, or the other way around but that already works fine as you pointed out.

I mean if you've got a feature set to merge into dev, and it suddenly merges into main after someone merged dev into main then that's very annoying.

Huh interesting, my mental model is unable to see any difference between them.

I mean a branch is just jamming a flag into a commit with a polite note to move the flag along if you're working on it. You make a long trail, leave several flags and merge the whole thing back.

Of course leaving multiple waypoints only makes sense if merging the earlier parts makes any sense, and if the way you continue actually depends on the previous work.

If you can split it into several small changes made to a central branch it's a lot easier to merge things. Otherwise you risk making a new feature codependent on another even if there was no need to.

I think being connected is easy enough, being non-cyclic is trickier I suppose. If you do it badly the shape of the maze is going to depend on the order it's generated in. I imagine some people may have looked into it.

I'd probably go with something like the wave function collapse algorithm. It should be possible to make it generate trees with somewhat uniform probability.

Since it's AI and Microsoft I can believe that someone who doesn't know what they're doing would be given a mandate to promote AI under any means necessary at the cost of some other team's reputation.

But it's an insane move. If anything AI has made it more important than ever to know who authored something and then someone does this to promote AI.

The fact that they chose to publish the ad _under someone else's name_ is so tone deaf that I just can't understand what they were thinking.

Oh that's neat, though I might split this into two functions in most cases, no need to entangle opening the file and counting the words in a filelike object.

That's two neat tricks that I'm definitely adding to my bag of python trickery.

I dislike loading files into memory entirely, in fact I consider avoiding that one of the few interesting problems here (the other problem being the issue of counting words in a stream of bytes, without converting the whole thing to a string).

If you don't care about efficiency you can just do len(set(text.split())), but that's barely worth making a function for.

For reasons I never quite understood python has a collections.Counter for the purpose of counting things. It's a bit cleaner.

There's bound to be a way to turn a stream of bytes into a stream of unicode code points (at least I think that's what python is doing for strings). Though I'm explicitly not volunteering to write the code for it.

The phrase "when it is detected" makes me nervous.

I can't really think of a polite way to phrase this, but I'm not surprised throwaway mobile apps do benefit, while relatively mature python packages do not. That matches my estimation of how much programming skill you can reasonable extract from the current LLMs.

Really the one thing that conclusively has changed is that the 'ask it on stackoverflow' has become 'ask it an LLM'. Around 95% of the stackoverflow questions can be answered by an LLM with access to the documentation, not sure what will happen to the other 5%. I don't think stackoverflow will survive a 20-fold reduction in size, if only because their stance on not allowing repeat questions means that exponential growth was the main thing preventing them from becoming stale.

Note that a more complete model would multiply each term by P(track)_j — the common-mode detection-tracking-classification factor developed in the previous section — but the standard WTA formulation assumes perfect tracking.

I'm not sure that is a useful model, or more complete. I don't think you can assign interceptors to undetected missiles, so considering their effect on the value is rather pointless. It's effectively a sunk cost.

Multiplying with the probability also makes no sense from an optimisation point of view. Why would you assign lower value to a target about to be hit simply because you were unlikely to detect the missile?

The tracking probability only shows up in the meta game described at the end, where one side is trying to optimise their ability to hit valuable targets and the other is trying to optimise their ability to prevent that from happening.

The paper is just listing studies that happened. There might be a political agenda behind it, but identifying a study about cannabis use for PTSD and drawing conclusions about the results is what you would expect this review to do. Leaving studies out is much more suspicious.