saeranv

From the article: "The project’s binary distributions didn’t include the required MIT license notice for the llama.cpp code they were shipping. This isn’t a matter of open-source etiquette, the MIT license has exactly one major requirement: include the copyright notice. Ollama didn’t."

I'm always puzzled by how often I see this happen in the open source community. Can the llama.cpp take legal action against Ollama for this? What's the point of these licensing requirements if they aren't (or can't be) enforced meaningfully? Note: I'm not blaming the llama.cpp team for inaction here, I just want to see them get some justice!

I said before that the function shape changes as you're updating the parameters that govern the function but that's actually very misleading, (sorry), since the kernel parameters are only indirectly governing the function. What the parameters directly govern is the joint probability distribution P(f(x1), f(x2), ..., f(xn)). So the function f is implicitly defined by how likely the entire sequence of f values are.

So how does it change shape? Well this part is actually something I don't fully grasp myself yet. But I can sketch a crude bayesian interpretation, which is how I think of it. Not completely correct but works as a placeholder until I fully work out the math of updating the parameters.

Basically, from a bayesian perspective we can condition the joint distribution of function outputs as a likelihood conditioned on the kernel parameters theta: p(f(x1), f(x2), ... | theta).

Then we can derive the posterior distribution over theta p(theta | f(x1), f(x2), ...) like so:

p(theta | f(x1), f(x2), ...) ≈ p(f(x1), f(x2), ... | theta) p(theta).

So we fit the theta parameters based on how well it fits the observed data we feed our bayesian model.

FWIW, I recommend chapter 14 of Richard McElreath's Statistical Rethinking for a better introduction of GPs. This article kind of glosses over a lot of the intuition and introductory concepts that you need to really grok it.

Why is the 'true process' changing here? I understand our best guess or model is changing with new observations, but the true process should not be changing. If it actually is, then the formulation should be changed to isolate the parameters that is feeding back to it.

He's not saying the true process is changing, just the functions that are being sampled from the GP. The true process refers to the true, underlying function so it's deterministic if you have correctly identified all its inputs.

So is the shape of each function changing?

Yes, the function changes shape as you get more data because the parameters governing that function (that we define in the kernel) are updated with new observational data, so that over time it converges to the 'true' process/function we are trying to discover.

What is the 'distribution' over the functions doing? Is that also changing? Is the said 'distribution' just flat mean of these functions?

I think you're confused because the example given with cheese is really confusing when we're trying to understand the functions as arising from a multivariate distribution. So, I'll try to clarify that part. GPs are typically used to represent some function where the input is time or distance. This is why its called a 'process' - because the variables in a random process are indexed by space or time. So in this 1D example, in the X domain, [x1, x2, x3] represents something like fixed increments of increasing cheese. f(X) represents the gold amount. Now imagine gold can take any value from 0-100. Now plot all possible values of f(x1) on the x axis of a grid, f(x2) on the y-axis of the grid, and f(x3) on the z-axis of the grid. We have 100^3 points in this 3D grid. If we select one point, it's x,y,z coordinates correspond to the f(x1), f(x2) and f(x3) gold amounts. The dimension index, corresponds (typically) to something like time, or distance. In this example it's cheese.

In a GP, we're modeling the sampled f(X) point as if its from a 3D multivariate normal distribution. So sampling one point gives us the gold amount for cheese amount 1, 2, and 3. This is the 'function', and as we sample more points, we get more 'functions' that give us varying gold amounts for cheese amount 1, 2, and 3. And because it's a multivariate distribution, we can capture correlations between dimensions, so the amount of gold you get for cheese-1, should influence how much gold you get at cheese-2 because its close by. This relationship is defined by the covariance function of the gaussian.

GP(m(x), k(x, x')) What is 'x' here? (Sigh! We need to learn to define the variables before using.) I can infer that x' is not derivative of x.

x refers to some amount of gold, and k(x, x') just means that the kernel consumes any two values in our X vector (i.e. [x1, x3] or [x1, x2]).

"In the context of GPs, a kernel or covariance function k(x, x') = Cov(f(x), f(x')), encodes which function values should vary together." It does not seem the 'f' here is intended to be the specific 'f' introduced at the beginning of the article.

I believe it is the same f actually. He's saying the kernel function takes in two values of x (cheese), and outputs the covariance between their output gold amounts. This illustrates his previous point that the "closeness" between x values should be reflected in the gold amounts.

The plots now have y and x, and x1 and x2. How are these related?

y is gold. x is cheese. x1, x2 correspond to the first two x-values in the linear plot.

And with k(x, x') = Cov(f(x), f(x')), what is 'f' for the various kernel functions being plotted.

f(X) is the approximation of the "true" process we're trying to learn from observational data. The observations are tuples of cheese and gold amoutns, so f(x), f(x') is just the corresponding gold amount, we don't actually model that function explicitly. The gaussian distribution we are sampling from for functions just models correlations between our variables, so it represents the function implicitly.

Greg Brockman honestly sounds like a psychopath:

In 2017, Amodei hired Page Hedley, a former public-interest lawyer, to be OpenAI’s policy and ethics adviser. In an early PowerPoint presentation to executives, Hedley outlined how OpenAI might avert a “catastrophic” arms race—perhaps by building a coalition of A.I. labs that would eventually coördinate with an international body akin to NATO, to insure that the technology was deployed safely. As Hedley recalled it, Brockman didn’t understand how this would help the company beat its competitors. “No matter what I said,” Hedley told us, “Greg kept going back to ‘So how do we raise more money? How do we win?’ ” According to several interviews and contemporaneous records, Brockman offered a counterproposal: OpenAI could enrich itself by playing world powers—including China and Russia—against one another, perhaps by starting a bidding war among them. According to Hedley, the thinking seemed to be, It worked for nuclear weapons, why not for A.I.?

For me, it was when I found out Greg Brockman's MAGA donations. From wikipedia (https://en.wikipedia.org/wiki/Greg_Brockman#Personal_life):

Brockman and his wife were the biggest donors to Donald Trump's Super PAC, MAGA Inc., in 2025 with each of them donating US$12.5 million. Brockman and his wife also donated $50 million to Leading the Future, a super PAC dedicated to AI deregulation that he helped found with Andreessen Horowitz co-founders Marc Andreessen and Ben Horowitz.

This is awesome. How do you integrate morphology into the simulation? Does morphology effect movement (via area friction or mass impact on momentum) or metabolism (via area/volume ratio)?

Archive link: https://archive.ph/IX9bP

That presumes that languages didn't evolve independently across different communities. The fact that different ancient languages have completely different grammatical structures, for example, provides some evidence of this.

A simple counter example here is instinctual behaviour. A sea turtle is born, and with little to no guidance, experimentation, or exploration heads to the sea. That knowledge is embedded at birth.

I think the analogy of the brain as hardware devices ("neural processor", "I/0 devices", etc) is misleading. I think I understand the very strict mind-matter dualism you're alluding to here. But so far attempts at using actual computer hardware to reproduce human-like cognition has gotten nowhere close, despite consuming order of magnitude more energy and data.

The idea is that if you can produce an accurate probably distribution over the next bit/byte/token...

But how can you get credible probability distributions from the LLMs? My understanding is that the outputs specifically can't be interpreted as a probability distribution, even though superficially they resemble a PMF, due to the way the softmax function tends to predict close to 100% for the predicted token. You can still get an ordered list of most probable tokens (which I think beam search exploits), but they specifically aren't good representations of the output probability distribution since they don't model the variance well.

I think they are accounting for the entire context, they specifically write out:

> P(next_word|previous_words)

So the "next_word" is conditioned on "previous_words" (plural), which I took to mean the joint distribution of all previous words.

But, I think even that's too reductive. The transformer is specifically not a function acting as some incredibly high-dimensional lookup table of token conditional probabilities. It's learning a (relatively) small amount of parameters to compress those learned conditional probabilities into a radically lower-dimensional embedding.

Maybe you could describe this as a discriminative model of conditional probability, but at some point, we start describing that kind of information compression as semantic understanding, right?

I read through the whole LW post, and think there's enough troubling evidence here that she shouldn't be dismissed. It certainly shouldn't be flagged.

I initially was leaning to this being a high possibility of a delusion springing from a mentally unstable person, for all the reasons other commentators are mentioning. But, two things in particular struck me that changed my mind:

1. She apparently mentioned the abuse to her mother as a child.

2. She describes childhood behaviour consistent with someone who has experienced sexual abuse (i.e. thoughts of suicide, weird night behaviour like taking baths, body issues as she got older).

A small child doesn't have any incentive to make accusations, or to pretend to have been assaulted. If true, this should be taken seriously. Her mother is still alive, and there may be doctors, relatives or others that would be able to substantiate these points.

Finally, why has this post (and previous related posts) been repeatedly flagged? It's very troubling, I expect this from some HN users, but would have thought the HN moderators would have unflagged (or reposted) it upon consideration of the seriousness, importance of the subject matter, and undeniable relevance to the tech industry. At minimum, you would think someone would have unflagged them to avoid the appearance of bias and favorable treatment to the former YC president. At this point HN looks really sleazy.

I sure hope all the HN commentators confidently predicting this would never happen because of his large donations to the democrats see this.

Thanks for the correction, my understanding of electricity is shaky, and I was assuming everything connected to a wall outlet is AC-driven, unless there's a boxy inverter along the cord like a laptop, but you're correct, that's not neccessarily correct and monitors are DC-driven.

I disagree with other commentators here arguing that there's no benefit this monitor would bring over simply reducing screen brightness. But I also think the claim made by the article here is at best, miscommunicated: that the bounced light improves the monitor light quality itself. On this point, I think the criticism from HN is correct, there wouldn't be a meaningful difference between equivalent bounced light, and light from the monitor. There might be a possible benefit from the lack of light flickering from AC-driven electric lights, but that is only true if the space is daylit.

However, I think there is a quantifiable benefit from making the monitor light directly dependent on environmental light, which forces our perception to adjust to a more, low contrast, diffuse environmental context.

Part of the problem is that "brightness" in the context of monitors is different from how "brightness" is used in science of light, where it is defined as the subjective perception of light that changes relative to differences in light levels[1]. So you can see your way to the washroom in the middle of the night with no lights, but can't see your way back after you turned on the washroom light, because your subjective perception of light (brightness) has changed, even though objectively the amount of visible light (illuminance) has not changed.

Therefore, having a screen lit by the environment, would shift your perception of light to better see duller, low-light conditions, which is better for our eyes, since more uniform, diffuse light causes less strain than strong, directed light.

[1] https://en.wikipedia.org/wiki/Brightness

[2] https://en.wikipedia.org/wiki/Luminous_efficiency_function

That kind of innate skeptism is good science though.

Unfortunately, the HN crowd has an incredibly naive view of what journalism is. I think it's mainly a product of ignorance, typically conflating op-eds with the journalism, and false-equivalencies that lump the Washington Post into the same bucket as Fox News, and People's Daily.

I'm always surprised the degree to which this kind of anti-journalism rhetoric ignores the role of journalism holding powerful governments and corporations accountable.

To your credit, at least you got data to begin investigating your claim. I'm usually to lazy to do that...

Okay fair enough, but why look at female founders in terms total valuation percent, then? I think my point about the distortion from the (likely) exponential distribution of total valuation still stands, although in this case with regards to comparison between VCs.

Isn't it straightforward to just look at percent of female founders?

Careful, this seems to be falling for the prosecutor's fallacy where the probabilities aren't normalized to account for base rates of the hypothesis occuring. So in this case, the percent of successful valuations that have a female founders will likely be extremely low simply because the gender ratio is disproportionaly male, and being successful is extremely rare.

For example, let's say probability of being successful is due to random chance, and effects both genders equally. Then ratio of successful women to successful men would just reflect the ratio of women to men. Now imagine if the actual valuation of successful companies is exponentially weighted, so that the top 1 or 2 companies make up the bulk of the total valuation. Again, these two companies are more likely to be founded by men given the base gender rate, and now the percent of valuation attributed to men would completely dominate the valuation percent.

To get the real effectiveness of female founders, we need to account for the low probability of success and low probability of being female.

I think both can be achieved with Bayes theorem:

  p($_high|f) = p($_high) p(f|$_high) / Sum_i[p($_i) p(f|$_i)]
  p($_high|m) = p($_high) p(m|$_high) / Sum_i[p($_i) p(m|$_i)]

Note: p($_i) represents a sequence of valuation probabilities subdivided to reflect low valuation, moderate valuation etc. I think this should account for the exponential distribution of valuation, but someone correct me if I'm wrong.

Good point. So when we try and evaluate a person's "social radar", we should probably limit ourselves to cases where they are the minority dissenter and the person they dislike is given the opportunity, since it gives us the most controlled scenario in which we can evaluate their hypothesis.

Funny, I was thinking that the only disadvantage of the previous example is that you don't get any feedback from comments.

The ability to label your "posts" seems pretty powerful too. Then your blog can be navigated by your tagging system, which means it can also start to act as a personal wiki...

Off topic, but just wanted to say I love the way this guy has repurposed a github repo to be a blog. One of the most effective integration of aesthetics and functionality of minimalism I've seen.

I think this is the first time I've heard of a successful long-term career, and general life-fufillment from one of these outlier prodigies that are doing difficult university degrees in their early teens or younger. Terence Tao's another one, and in the past there's plenty, i.e. Mozart, John von Neumann etc.

But, I typically hear of the other extreme these days - child prodigies that struggle to fit into society and end up underemployed and neurotic. Maybe its just a selection bias (its not newsworthy to report on fufilled, grown-up child-prodigies doing their 10th postdoc in knot theory)...

This is such an interesting experiment, and extremely well written.

But it feels half-finished. I agree with his friend's thoughts, rerunning the experiment again, with his newfound knowledge seems like the logical next step. The results of that could validate his hypothesis, and seems like it would be a much more personally meaningful way for him to end his experiment, versus the current conclusion.

I have a couple of naive questions about this, that hopefully someone with more knowledge can answer:

1. If the central problem is the price of imports (i.e. natural gas) outpacing exports (service sector), is this problem a downstream effect of Brexit closing off valuable markets? This seems obvious to me, but I don't see any mention of Brexit as a causal factor in the BOE Chief Economists comments (perhaps I missed it), and so am wondering if I'm overestimating Brexit's impact on the UK economy.

2. If Brexit is a relevant, and significant factor, in theory, can post-Brexit trade negotiations increase the export sector to offset this?

3. Assuming the natural gas is primarily used for building energy, is it easy for the UK to transition to alternative thermal energy sources (i.e. switch to electric generated from non-gas sources, heat pumps)? I've always thought of the UK as a very efficient user of energy, like most European countries, due to it's high density and stricter efficiency standards relative to the US and Canada, and am surprised the natural gas price increase seems to be framed as a relatively permanent problem that the UK is powerless to stop.

To be fair, I suppose a huge challenge is the older building stock can't easily use lower capacity electric heating, or heat pumps due to the historical reliance of thermal mass over insulation, but there are established ways to fix this, and shouldn't be an issue for modern buildings.

A country's "number of people" isn't an independant parameter that you can just slide down so that your GDP/person increases.

If you reduce the "number of people" (i.e. by reducing immigration) you are typically reduce your GDP at a much higher rate, and thus will have a lower GDP/person overall. Your eating into your economy's ability to support the division of labour, specialization and other efficiencies that can only emerge at a large scale. Intuitively, think of it this way, do you think a village of 10 people, or a city of 2 million people will produce more GDP per capita?

Going back to the example of immigration, you're basically reducing a stream of young, educated workers who are consistently net creators of jobs, especially in Western countries with a very high ratio of aging senior citizens (Boomers) to young workers.

In architecture school, we were taught that drawing is a powerful tool for thinking and analysis, and were drilled with different techiniques to articulate our ideas quickly and concisely in an image. Our professors were heavily influenced by the Bauhaus school in this line of thinking (i.e. Paul Klee[1]).

[1] http://ing.univaq.it/continenza/Corso%20di%20Disegno%20dell%...

The animated illustrations of the "arthropod" biomechanics is fascinating. It sheds some light on why arthropods would be interesting enough to draw in their own community of enthusiasts.

Spearing: https://en.wikipedia.org/wiki/File:20220123_stomatopod_strik...

Smashing: https://en.wikipedia.org/wiki/File:20220123_stomatopod_strik...

In case they manage to find this thread: @Junnn1, do these biomechanic animations incorporate the dynamics, or maybe just the kinematics of the physical forms? Is there anywhere (i.e. blog) where you discuss the techniques you use to develop your animations?

Slightly off topic, but am I the only one that dislikes the GUI-based method for adding and reviewing anki cards? Anyone find a decent terminal-based plugin for anki? I've actively looked in the past, and surprisingly there are some partial solutions (can review on terminal but can't add), or hacks, but no complete terminal based SRS...