SkyBelow

How errors add depend upon how you treat them.

Two independent task? You have a 95% chance of each task being correct, so total success is .95^2 = 90.25% correct.

Having two AIs verify the same task with each being 95% correct? That's a 5% error per each, so 1-(.05^2) = 99.75% correct.

That is assuming fully independent error rates, which is the real issue. If it is pure hallucinations, then that is an okay assumption, but if there are issues in the underlying prompt/data causing the error rates, then it becomes a very dangerous assumption and I think more human involvement is recommended to better understand and resolve the underlying data problem.

The real interesting bit is that more AI's mean the 2 in the above equations gets bigger, so the error rate gets worse or better depending upon which equation is in play. So if you are in an in-between landscape where you aren't sure there is independence or not (or how much independence between errors), it also becomes unclear if more AI makes the error rate better or worse.

I see an AI as an advanced calculator, but I see even a simple calculator as a poison to education if not strongly controlled.

Number sense is such a nebulous concept, but it captures what I've seen people struggle with. A general sixth sense about where an equation should or shouldn't go, a feeling that random numbers don't appear quite random enough and might have a pattern, or even recognizing that having a pattern signifies some relationship that otherwise has no evidence.

That entirely depends upon the cost of preventing them. At no cost, sure.

But nothing is ever free. So what is the cost.

<insert a villain who determines destroying all life is the best way to stop all future harm>

There is also the consideration of if a crime being a crime is just. Consider crimes that use to be on the book that we now consider horrible things to have outlawed or even crimes that are still on the books but not enforced because we reject them.

For example, some places us to make it a crime for kids to play pinball. Is preventing kids from playing pinball, even if it came at no cost, an unquestionably good thing?

For a sufficiently bad crime, at sufficiently low costs, preventing it is good. But those two factors are both very big questions, directly challenging the notion of "unquestionably good".

Many developers are too focused on delivery value compared to maintenance cost.

The way this is worded feels like it leaves the blame on developers. Aren't these developers focused on exactly what they are being judged by? Shouldn't we say it is the management who is too focused on delivery value compared to maintenance cost? Is it the developer's job to guide management or the manager's job to request guidance, assuming such guidance is needed?

We need to make sure the responsibility to resolve this problem falls on those with the power to act on it, and in this, developers tend to be receiving far more responsibility to fix than power to fix.

Are court decisions universally recognized? If a court rules someone true in one area, couldn't it still be considered slander/libel in another area if another court rules it as not being true?

It is a bit like saying convicted murderer vs murderer. One is claiming the conviction for murder happened (which generally correlates to the murder happening but doesn't openly claim such) while the other directly claims the murder happens. If the conviction is later overturned, does the second claim open one up to more libability for a false claim?

I think it might be built on something more than deterministic systems. Some property that is a subset of deterministic, so all your argument still apply, but merely being deterministic is not good enough.

LLMs are what made me start considering this. Imagine a company using an LLM that was fully deterministic. All RNG was either removed or seeded in such a way that the same input (so many the seed counts as part of the input) gave the exact same output. Fully deterministic.

But such an LLM, with a slight drift in input, could still produce very different outputs. This isn't being non-deterministic, but more than the change in outputs does not naturally follow from the input. I'm thinking like how 2 double pendulums can (but not always do) greatly diverge given a very small change in their input.

So in light of that I've begun to call this new property non-chaotic. So Enterprise depends on non-chaotic systems, which are a subset of deterministic systems, and then wrangling the chaotic elements they cannot remove as much as possible.

The follow question I now have is if all LLMs are inherently chaotic, or if it is possible to have a non-chaotic LLM.

The parents job is to keep their kids out of trouble.

I challenge that anyone believes this, and for my evidence, I would submit all the age based laws that protect children regardless of what parents do.

We have already, long ago, decided that it is the government's job to protect children, at least in cases where parents fail to do an adequate job. That's why I don't see this ending any other way. The march to total domination by the side of the government might be slow, but they already won the war around a century ago (exact timeline for laws protecting children in place of parents is a very long topic and does differ country to country, I recall hearing some places still even let kids buy alcohol if they say it is for their parents to consume).

Never really like this because it feels some arbitrary as to what counts as making a claim. Human language is filled with implications, and those implications are filled with implied claims. Yet burden or proof seems mostly used when someone makes one of those implied claims explicit, say by explicitly rejecting one.

"We should do X."

"X is not possible."

There was an implied claim that X is possible and an explicit claim that X is not possible, but often it feels that in such a scenario, it is the second individual who is deemed having a burden of proof.

This is made all the worse that there is rarely a single implied claim, but more a group of related implied claims that are also weighted. Like some combination of "data centers don't destroy the environment" and "or maybe they do, but only in amounts of destruction that are tolerable compared to human behaviors and worth the outputs gained" and even "they modify the environment, but it doesn't count as destruction, just change". And even worse (yet again), different readers of the conversation will find themselves placing different weights on these different interpretations making it near impossible to agree on what the original claim is. Thus the first person who makes an explicit claim gets all the attention which ignores all the messy implied human communications preceding it.

But why does it do so inconsistently, and sometimes even forgetting to swap back to English when it comes time to do 'normal' output? It also seems recent, as when I was using deepseek even a week ago this was very rare compared to what I was seeing yesterday. I had to start including a line asking it to stay to English because I can only speak/read English.

and everyone else will poor and unemployed

How so? Everyone would still have their skills to provide goods and services and everyone would still have wants for other's goods and services, so an economy would still run. AI can shift the economy but it doesn't lock the entire population out of the economy. It can lock out any one group because everyone else gets the good/services of that group for cheaper from the AI, but if everyone else can't afford the AI, if the AI locks everyone out, then they trade between themselves instead. And that is the sort of 'worst case possible' outcome, not even what is likely to happen as the AI makes some things much cheaper.

I don't think that is the argument.

The argument is more "I want to do good thing X, but it will also cause bad thing Y." followed by "Wait, bad thing Y is going to happen anyways, so I might as well do good thing X so we get both X and Y instead of just X."

Viewed this way, the idea is that given the world will have bad thing Y regardless, the one impact of your choice is if good thing X exists or not, and it is better to create good thing X.

Where it becomes an issue is that there is no clear X or Y. There are many different but very related bad things, so if the one you would add is actually better or worse than what is already out there, or maybe it'll exist both ways but you make it more popular, and very subjective things to judge, so different people look at the same outcome and some agree that bad thing Y would have existed anyways and others say that no, this is a new bad thing Z that wouldn't have existed anyways.

From where I sit it seems reasonable for Anthropic to not want their product used to create malware

Yes, I think there is a PR component to this that is often left out of this discussions.

Is this self contradictory?

Making a claim of 3 star service, without providing 3 star service evidence, isn't any different than a claim like they don't clean the dishes enough. Sure, you didn't write out the exact words, but by submitting the review, you still submitted such a claim, and so it remains a question of if you provided evidence to prove it, which gets into how deep the legal system wants the evidence verified (is a picture enough, or do you need to provide evidence the picture is of the store at the date and time relevant to your review).

One of the major concerns with prototyping, at least in my experience and based on the general vibe I've felt from others over the years, is that clients will generally do some variation of "You have a working prototype? So how many days until it is prod ready? No more than a few weeks, right?" and that will be their expectation. AI makes prototyping easier, but makes this specific drawback much worse. Expectations are going to be misaligned, leading to many disappointments, and likely some number of developers taking undue negative outcomes. I'm not even sure if it will normalize to reasonable expectations, given that this tension never seemed resolved even before AI was in the mix.

Does the model need to offer new testable hypothesis if it provides a way of explaining existing results that current models can't?

If it is competing against another model that does both that and offers new testable hypothesis (which experiments match), the other model is the clear winner. But lacking that, if no other model explains all existing data, is new testability really necessary when it is the only model that currently explains all existing tests?

That said, aren't most of theoretical models only contenders for such, as in they haven't been expanded to actually explain all testing results, only that, as far as they have been expanded, there are no contradictions yet? So they need physicists to expand them, but if the model is wrong, the effort might largely be wasted, and we have some models that there is disdain for not because they contradict existing experiments, but because they have eaten too many careers without showing value in return?

Didn't the original authors end up leaking this before OpenAI fixed it? They gave them a chance, but then had to decide between staying fully silent or publishing the details despite malicious actors learning about it before it was fixed or leaving users in the dark. They chose it was better to warn users and inform malicious actors despite it not being fixed.

This vulnerability was responsibly disclosed to OpenAI. Despite multiple follow-ups, we received no communication beyond an automated reply to our initial disclosure. OpenAI's documentation fails to describe sensitive capabilities granted to the model (e.g., running privileged scripts) or risks of model manipulation via indirect prompt injection, instead focusing solely on functional limitations and data-handling concerns. As such, we are publishing our findings to enable informed decision-making regarding the risk surface.

That very last sentence was considered justification of putting this knowledge into the wild when OpenAI refused to fix it. So, if we consider it justified with a delay, then we are saying it is acceptable (it is "responsible") to give the information to malicious actors as long as you tried to warn the right party first.

Compare that to two alternatives. Alternative 1 is never disclosing it to the public until fixed. Saying it is never acceptable to let malicious actors know until it is no longer a concern, even though this will mean users are kept in the dark about the risk.

Alternative 2 is to reduce that timeline to 0. Say that users are immediately warned, despite the risks of making it known to bad actors.

So if we are saying the current delay is acceptable, but both a longer and a shorter delay are unacceptable, then why is that? What justifies the current delay, what makes that the responsible one, rather than a shorter or longer window?

I can't imagine something being "more responsible" for the public than privately notifying the owning party to give them time to fix the issue, before notifying the rest of the world (including malicious actors) about it.

What about ensure they have fixed it, and only considering it responsible to disclose it when fixed (alternative 1)? If it is never fixed, then the bug is never disclosed, because it is not acceptable to tell malicious actors how to exploit a vulnerability? Even evidence of use wouldn't be justified, as publishing this makes all malicious actors aware of it rather than just a subset of them.

And if you disagree and think some window is reasonable, then apply that argument to a slightly shorter window and repeat until either the argument hits some built in limit or reaches a window of 0.

responsibly disclosed

Isn't this a double plus good phrase? What makes this more responsible? Reasoning about first order effects of different disclosure models? But what if someone uses higher order reasoning and critical thinking to reach a conclusion that other disclosure models are better for the average user and the long term health of the industry, even if they are worse in any individual case. A difference in the security culture incentivized by different disclosure patterns. Why does this one win the name of responsible while other alternatives, which have never been proven to be worse, are automatically marked as irresponsible?

Reminds me a bit of the concept of identity theft, as a way to say that even though the bank (or other creditor) was the one who had money taken from them, it is actually the random person not involved in the transaction who is the victim and has to hold the debt until the issue is resolved.

Nothing of the level of rocket failure, but I've tracked down issues where you are never sure of the cause. You keep the doubt and let it drive you. You aren't as much sure of a theory, as you have the theory you most want to disprove and keep failing to do so. The more you fail to disprove a given theory while other people with their own personal 'targets' do end up disproving them, the more you can report that the theory is the reasonable conclusion. But you never given up the idea of looking to disprove it. Eventually others join you and work to disprove your theory. As the group continues to fail to disprove it, it becomes the officially stated cause unless someone can provide evidence otherwise.

Sometimes I'll have one that I'm stuck on for a month before finally disproving it, and it is an interesting feeling. There is some level of happiness I succeeded at my goal, but it is very bittersweet because it normally was my last working theory and now I'm simply lost until I can formulate a new one. Sometimes disappointment in myself that I might've missed some easy way to disprove it for so long, but other times the way to disprove it was sufficiently hard enough that I just accept it is what it is.

Depends upon the filtering system.

Consider indie games. If there are 10 of them and 5 are great, you don't need any filter. You look through 5 great and 5 not so great games and end up with 5 great ones.

Now go to a world where indie games explode. But only 1 in every 100 are great. There are now 100,000 games, but most qualify as very low quality. There are now 1,000 great games (and a few of these might be the perfect game you dreamt of), but if you don't have a filter and are buried under 10s of thousands of horrible games, things feel worse.

With a filter, you now live in a world where you can easily find most of those great games with only a few lower quality ones showing up. So as long as the filters that exist, whatever they might be, can handle it, more is better even if quality drops.

Unless the quality extremely fast, say my previous example of 100,000 games but only 1 in a million was a great game. I think this level of quality drop is extremely unlikely. Instead, I suspect the real problem is if the filters can keep up, because they depend upon human effort, so it is possible to hit a point where they are overwhelmed and stop functioning properly. That's when things get worse. As long as the filters hold, more building leads to better outcomes even with a drop in quality.

Apologies on the typo.

One other question I had but wasn't sure if it would leave my previous post too unfocused is "aren't we a bit too early to determine in our current government systems are really the most effective?" This is something that will be decided by political scientists far removed from the current societies who can see how our current societies evolve.

Isn't this still assuming we can even determine what is true or false?

Newtonian physics is false, but it works well enough we teach it in college. But our best models of physics are currently in disagreement, so can we even say they are true? Given the replication crisis, especially in social sciences, how many of peer reviewed findings can be called true? Even experimental results can be false (consider studies that found FTL neutrinos, which were rejected as an error in the experiment, and which was eventually confirmed but it took quite a lot of work and in a softer field than physics with a claim less absurd than FTL, would have likely long been accepted as a true finding).

Even in math, basic statements aren't really true or false, but more a question of "given these axioms, can we prove or disprove it" noting that we have different systems with different axioms. If we are talking basic sets, most people are using naive set theory which is inherently contradictory, which means that notions like true or false probably can't be considered well defined.

If it was so clearly ineffective, why does it get challenged more often and replaced? Existing corporations aren't likely to change, but new startups and work owned coops exist, so why don't they compete?

Maybe ranking it on a scale of best to worse is too simplistic a view, and there are reasons this develops. Maybe it is the best option when there is a good leader, thus such structures dominate, much as a government ran by philosopher kings are better. But this only lasts as long as a wise rule is in charge, and it reverts back to a norm, and eventually, due to pure time and chance, enough bad leaders come on board that slowly dismantle the giants, but this happens at a time scale we don't particularly notice due to how much inertia large corporations can have (before we even get into the less pleasant issues like regulatory capture).

I think it depends upon the effort involved.

Think the difference between AI saying "This paragraph seemed muddled and lacks a clear point. Consider rewriting it." vs "Here, I rewrote this paragraph to focus it more on bridging the previous and next paragraphs."

The problem with this as a metric is that it is loosely defined so it becomes quite easy for a person to twist it to justify almost any level of AI usage as "well, it is still more effort than <X>".

This is just the modern equivalent of "just google it".

I don't think that is always the case. Sometimes it is. Other times the social cues and later follow up makes it seem the person thought they were being really helpful, not sarcastic, by sending the response. Yet other times, the person acts as if it was their own response and not the AI's, almost akin to passing off the AI's work as their own.

This is most notable when the original question shows effort was put into it and it isn't a simple case.

Because being right 60% of the time with minimal work is still amazing, as long as one accounts for the failure rate correctly.

Say I want to look up some game from my childhood, which I barely remember any details for. Going to google and trying is likely going to be very difficult unless I happen to get lucky with some key element. But if an LLM can get it right even a minority of the time, it can lead to me quickly finding the game I'm looking for.

This does depend upon the ability to evaluate the answer, like checking against source or some other option where you know a good answer from bad. If you can't, then it does become much more dangerous. Perhaps part of the reason AI seem to empower experts more than novices in some domains?

Are you also disillusioned with professional sports, music, acting, and art?

Not the person you were asking, but I think we need to double down on disillusionment in these. I've spoken to too many kids who dreamed of careers in this well into high school, often at cost to other academic paths, when their performance already clearly showed they weren't going this route. Sadly, it is hard to be strong about correcting kids because it is seen as not believing in them and not encouraging them.

As disillusioned as one might become in academia, the path one is on to get there tends to better align with setting students up for a successful career outside of it compared to the ones you listed.

Isn't this just a false dichotomy in web comic form? Implying that there are only two options, full participant or complete withdrawal from society, as a means of suggesting that one cannot apply morals to how one chooses to engage with society.

But in reality, there are many many ways to engage in society, some more or less ethical/moral than others, and one is free to criticize individual choices.

Even if we consider something like social media, there is still a range of choices other than fully engaging in social media and rejecting all social media. There are attempts to use it responsible, limiting and curating use to less harmful versions while attempt to get most of the benefit, while still postulating that the overall effect of the average use case of social media is harming society.

It feels a lot like saying that, since it is impossible to live a perfectly ethical/moral life, ethics and morals can be completely ignored without regard for what options one does have available to them.

I believe in A, I don't take a strong position on B

But if A and B are opposed, then there is a question of why a strong position on A can be allowed with a weak position on B, if the reason for the strong position on A would also indicate a strong position against B.

The underlying argument being implied (but rarely ever directly stated) is to question if your reason for the strong position on A is really the reason you state, or if that is just the reason that sounds good but not the real reason for your belief.

In effect, that you don't apply the stated reason to B despite it fitting is the counter argument to why it doesn't actually support A.

If there is an inconsistency in arguments being applied, any formal discussion falls apart and people effectively take up positions simply because they like them, contradictions irrelevant. This generally isn't a good outcome for public discourse.

Sometimes there are two groups of people who have different opinions that don't interact, but given the extent they take up the same platform and don't seem to see each other, I'm not sure it is really a fallacy even then.

First, it becomes possible for people who have a double standard to hide behind this. One can try to track an individual's stance, but a lot of internet etiquette seems to be based on the idea of not looking up a person's history to see if they are being contradictory. (And while being hypocritical doesn't necessarily invalidate an argument, it can help to indicate when someone is arguing it bad faith and it is a waste of time as someone will simply use different axioms to reach otherwise contradictory conclusions when they favor each.)

Second, I think there is the ability to call out a group as being hypocritical, even when there are two sub groups. That one group supports A generally and another group supports B generally (and assuming that A + B is hypocritical), but they stop supporting it when it would bring them into conflict indicates a level of acceptance by the change in behavior. Each individual is too hard to measure this (maybe they are tired today, or distracted, or didn't even see it), but as a group, we can still measure the overall direction.

So if a website ends up being very vocally in support of two contradictory positions, I think there is still a valid argument to be made about contradicting opinions, and the goomba fallacy is itself a fallacy.

Edit: Removed example, might be too distracting to bring up an otherwise off topic issue as an example.

This is the same logic of 'not a booby trap' booby trap,s which sometimes do work out in the favor of the one setting them if they weren't too open about it. If your commit message is that you are talking about OpenClaw just to booby trap your repo, then I suspect it wouldn't fly, where as if you gave it some plausible deniability, a lawyer would be able to get any suit or charges dismissed.

This is all under the assumption we eventually live in a world where booby trapping repositories becomes a legal issue. On one hand that feels silly. On the other hand, we have had far less sensible cases make it to court and there is a small kernel of similarity which the legal system might latch onto.

What's the chance that it is market motivated? That the companies most likely to succeed are those willing to break the rules (this isn't to say that breaking the rules makes one likely to succeed, you have to break the right rules and not the wrong ones, and that distinction is often times unknown til after the fact).

This might mean that the companies that we see explode in popularity are those whose cultures are already biased in ways that don't consider negative outcomes, as the companies that did consider them already excluded themselves from exploding in the market (they might still be entirely successful startups, but at a vastly smaller scale of success).