HN user
Those other parties all declined to compete in this by-election. They want Farage to stand to the rule of law on his corruption, and not endorse this procedural manoeuvre.
The timing of this by-election is also such that, were there an actual race, Farage would likely get his propaganda on the front of the newspapers throughout the summer months while everyone else in politics is on holiday.
"Ethnic Chinese" is a term covering 1.45 billion people. I don't even know where one would find a definition for "ethnic English", but if you take the NHS' "ethnic British" which ridiculously excludes even Irish heritage, I can see how you'd come to 50%.
A reasonable comparison would be to look at "Ethnic European", covering roughly 1.2 billion people.
london's trust level and prosperity have become a shadow of their former selves
As someone who traveled the world a bit and lived for a decade in London, I'd beg to differ without further elaboration.
A datapoint that points to the contrary:
- Singapore pre-1965 was a low trust, racist society. The tensions were so bad that in 1964 there were racial riots.
- Today, after a lot of immigration, within 1-2 generations, Singapore became a high-trust society.
Similar stories in South Korea and Taiwan. In all 3 cases, the trust came recently, within 1-2 generations, and from different approaches of the state. They combine high institutional trust and the economic prosperity with medium-low interpersonal trust.
I reckon Paul only claims to have created the webpage titled 'Triple Dragon Fractal'.
I fact checked: Chomsky has never been on Epstein's island.
In the OECD data averaged here, I reckon this chemo therapy is counted as sick leave. On a first search, the metric seemed to be defined as "(compensated) absenteeism from work due to illness, days per employee per year"
I probably get there, but I have young kids.
But yes, these are averages between 1 person having 12 weeks and 12 people with 1 week. It's most likely power law distributed, so the average will feel weird.
There are people with cancer, severe car crashes and other horrible but temporary medical conditions in this average.
My portable CO2 monitor goes off. You can set a levels of warning on the aranet. It is a very quiet, non-disruptive alarm, but an alarm nonetheless.
I agree with OP. I don't always carry it along, but it has been a massive boost to my productivity.
The UK did not ban AC. Don't read the Telegraph, and if you do, don't trust it.
Source: I live there
Well if it was the data that was copied, nobody would be using LLMs. The data-generating process is the thing that would be copied.
That's why we do appreciate the nth artist making classical or techno music.
Right, and what happens at that limit is most exciting! A model that has a cross entropy at that limit for a data stream of text, produces a stream of text that is both theoretically and practically indistinguishable from the original stream.
And so if the datastream has been produced by something intelligent, the resulting model is indistinguishable from that intelligence. That is the whole compression idea behind artificial intelligence.
The limit is not a bug, it's a feature!
for LLMs, because in practice floating point inaccuracies (in particular after exponentiation) prevent values from being exactly equal.
In one thinking trace of 10k tokens, with fp16 or bf16 logits, I don't reckon a collision is rare? There are only 65k floating point numbers with that accuracy. And an agent can quickly rake up 100k tokens, so while not every token will have such a collision of equiprobable logits, it is not rare.
If RL was used to train the model, the model will have been trained on its own sequences. Those will have been generated with a temperature of 1.0. They must be, otherwise you would get a premature collapse or explosion of your entropy if the temperature was respectively lower or higher.
After that RL step, you want to stick to the RL distribution, and so keep a temperature of 1.0. Other temperatures will drive the model out-of-distribution.
That is why the sampling step for agents or thinking LLMs are usually kept at a temperature of 1.0.
Actually, Google's TPUs are also deterministic!
Look at the softmax function and take the limit as T->0. It becomes a dirac-delta function.
In pure math, it does not always do that. It becomes a dirac-delta comb with equal weight on every maximum. There can be more than 1 maximum. Setting the temperature to zero turns into greedy sampling, but greedy sampling is not necessarily deterministic as you can have multiple equally optimal options.
so in principle, setting temperature to 0 _should_ result in deterministic outputs
It is a common misconception, but it is not true even in principle. If I have 2 or more logits which are equal to the maximum of my logits, I will sample uniformly random from them with any temperature, even zero. Sampling from softmax([1, 0, 1]) is still stochastic at temperature 0, because the limit is to sample uniformly from the first or the last element.
Anyway: "GPUs don't do deterministic matrix multiplications" is the biggest source of randomness in LLMs. GPUs put the associativity of the sums in matrix multiplications in arbitrary order, and this has a huge impact on the logits coming out of the neural network.
A flower, a picture of the flower in print and the picture shown on a screen will all have different spectra, but look the same.
See the first minutes of this video, where he has a spectrum analyser: https://youtu.be/-DyrBDsKA5s?si=mRJPT2ecy6NqpB4N
Yes, people are using the programming language Lean for that, and there are a few less popular alternatives as well.
Fundamentally, there is a one-to-one correspondence between mathematical proofs and programming. Proofs are isomorphic to type checking.
https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspon...
Ha, thanks for sharing this!
I've been building Gemini live since before ChatGPT came out. I am so thrilled to see it actually helping people in the wild!
I mean, he looks uncomfortable, but you would be surprised at how much people can tolerate when there is no alternative. Here is an interview I found [0].
I think he looks better than every 90 year old. But he also mentions that it still is terminal.
[0] https://www.nytimes.com/video/opinion/100000010826256/what-d...
It's been used, along with every other divergence and distance you can think of.
In practice, which divergence you use doesn't seem to be very important. The KL is the one with the most theoretic foundation though, i.e. will work with infinite data. The important aspect seems to be that neural networks are Lipschitz bound, and that that is the most important constraint preventing collapse.
It's not just kidnapping though. You also need road safety, or some level of pedestrian safety.
By far the most dangerous thing for kids, is traffic. And in many places that is the delimiter of their freedom.
To get there, you often need to slow down your brain's electrical activity, moving from active Beta waves into Alpha or Theta waves through meditation or deep focus. This blurs the boundary between the inner "self" and the outer world.
Quantum Mysticism, a.k.a what if we write about spirituality as if we would report pop-science.
It's been a while since I came across one as flagrant as this article.
If you want to read more: the concept is called "the dual state" [1] after the eponymous book [0]
I agree that the phrase is somewhat contradictory, but it is the best way to describe what was going on. As long as you were within the confines of the normative state, you experienced a rule of law. But as soon as you stepped into the prerogative state, anything could happen. So a "rule of law", except that it didn't apply to everybody, but only to most people. And importantly, the existence of the prerogative state is mostly hidden when you're in the normative state (so unlike a king, which everybody knew was outside of the law)
Not sure if it matters, but that is at least not true for nazi-style fascism. In there, they had a very strong rule of law for most people. But, there was a dual, a parallel system where there was no law at all, it operated outside of the legal system. You could win a trial and be exhonorated, only to be taken away by the gestapo at the door of the courtroom.
It was important for the nazis to keep businesses running, and have most people continue their lives without noticing major changes. Most people would not come into contact with the second system, and barely knew it existed. But if you entered the second system, you often would not come out alive.
This way, they could transit into an authoritarian system without hurting the economy. They knew this and planned it, and it turned out to be correct.
Named tickets, like airplane seats?
Sorry, I only thought about this for 5 seconds, but there are markets where scalping doesn't cause issues. We could look at those.
A theory that skips the parameter space, and understands grokking theory, comes up with an unexplained update rule, which notably works on a per-parameter level by dropping the updates for most parameters.
I suspect there is going to be a lot of handwaving to actually go from eNTK to that new update rule.
I also doubt it helps in the non-grokking regime, given the focus of the theory, which is where all the practical applications I have ever heard from live.
Don't get me wrong, I did enjoy reading this essay. It's well written and reasonably argumented without going into details.
The Netherlands is also the #4 importer of food in the world. I reckon with these transport heavy countries it is very hard to estimate how self-dependent they actually are, versus how much is settlement lag on the transport portion.
https://wits.worldbank.org/CountryProfile/en/Country/WLD/Yea...
Source: I write transformers for a living.
There is a piece of knowledge you seem to be missing. Yes, a transformer will output a distribution over all possible tokens at a given step. And none of these are indeed zero, but always at least larger than epsilon.
However, we usually don't sample from that distribution at inference time!
The common approach (called nucleus sampling or also known as top-p sampling) will look at the largest probabilities that make up 95% of the probability mass. It will set all other probabilities to zero, renormalize, and then sample from the resulting probability distribution. There is another parameter `top-k`, and if k is 50, it means that you zero out any token that is not in the 50 most likely tokens.
In effect, it means that for any token that is sampled, there is usually really only a handful of candidates out of the thousands of tokens that can be selected.
So during sampling, most trajectories for the agent are literally impossible.
We, westerner, want our values to prevail.
This comes to the core of the issue, and is where I think the disagreement comes from. Many Westerners in fact do not want "Western" values to prevail.
Why? For me those values have led to outcomes so horrendously antithetical to _my_ values, that I would not wish them for the rest of the world. Even worse, this Western centrism has led to jingoist conclusions for at least 400 years.