zoogeny

It would be nice if there was a listing of exactly what was included somewhere.

I am very ambivalent on this post. On one hand, I agree that excessive defensiveness stinks up a code base. On the other, I am a huge fan of local reasoning. Especially in the world of LLMs, I don't want to rely on my, my teammate's or my LLM agent's ability to know every single code path that results in `Allow` begin called.

Of course, this really comes down to the type system and the fact that non-nullable pointers are missing.

The one definite thing I would say, swallowing the error and just trying to do a reasonable thing is the most wrong thing here. At the least, there ought to be an ERROR log, even if one was trying to be defensive against outright panics.

On my first run I aced all of the sections until the last and then I got some really weird ones (e.g. zenzizenzizenzic). When I ran it again I got 95% new words and I fumbled a couple in the earlier rounds but did a bit better on the final section.

No cheating, only one or two guesses both rounds in the final section. No idea how the scoring works.

I ran through it twice, first time 91 second time 90, score: 69,500. Midwit confirmed.

I'm not following your logic here. Are you basing your predictions on the predictions of CEOs?

If I understand you correctly you are saying, CEOs predicted X and they were wrong. Now CEOs are predicting Y, and since they were wrong about X you predict they are also wrong about Y.

But Y's success seems independent of other people's ability to predict X?

I believe that the statement `a.x = 100;` is invalid since Record types are immutable.

That is, the situation you are afraid of should be impossible.

I think I mostly got this, but just to test it, it would be like in Typescript where I might say:

    type Foo = { x: number; }
    type Bar = { x: number; y: number }
    type FooBar = Foo | Bar;
    function baz(x: FooBar) {
      if ('y' in x) {
        // compiler now knows x is a Bar
      }
    }

More generally than just Java, nullability is often a property not of a type but of a variable.

This is a tangent, but I'm not sure I follow this. Can you give an example to make this clear?

Stoicism in Ancient Rome was COMPLETELY about controlling your emotions,

This is so false it deserves comment. For example, the SEP for Stoicism [1]. You'll notice that the first entire two sections are Physical Theory and Logic. Ethics comes later and it isn't until 4.3 (after Telos and Virtue) that you get to indifference. From the intro:

"Stoic philosophy was, from Zeno onwards, conceived of as comprising three parts: physics (phusikê), logic (logikê), and ethics (êthikê)."

I think you are over-indexing on one work by one Stoic. There is much, much more to actual Stoicism than "controlling your emotions".

1. https://plato.stanford.edu/entries/stoicism/

Unless you break the hypothetical by adding some alternative option C

That is kind of the point, isn't it? That my hypothetical scenario isn't realistic.

Let's imagine two worlds. A world where individuals refuse the false dichotomy and search for option C. And the world where someone accepts the false dichotomy and justifies evil.

I would argue that anyone that advocates for the justification of evil is actually using motivated reasoning. It breaks my original premise "Company A founds itself on doing 0 harm to Area X". Clearly they didn't and their embracing of evil shows that their principles mean nothing.

As a moral test, ask yourself: If I said "you must kill 99 people otherwise I will kill 100", would you feel morally justified to kill those 99 people? If your answer is "yes", then you are manipulable by those who want you to commit evil on their behalf. They don't have to commit any murders, just convince you that you have no other choice.

I think the big lesson of this that hasn't yet been learned: it literally doesn't matter how moral Amodei or anyone else is at Anthropic. When push comes to shove, the US government can step in and take it away.

They will either play ball with the US government on the US government's terms or they will be replaced or destroyed.

It is a misconception for them to believe they can dictate the terms of this technology.

Credit where credit is due I suppose. I'm still concerned over the direction this is going but at least Anthropic is listening.

I think we are very disconnected on the topic of conversation here. Somehow you've confabulated my point with an attack on the prisoners dilemma or races to the bottom?

The person I was responding to made the point that if you want to minimize evil in the world, sometimes you have to add evil to a lesser degree. As in my example, if I do 9 points of evil but prevent 10 points of evil then according to OP I've added value to the world in the form of the 1 point of evil I have reduced.

I responded that this can lead to an escalation trap. This assumes that we would all prefer less evil in the world, right? So how do we get out of the escalation trap? Repeated application of the maxim "always do a bit less evil than the worst possible competitor" will not lead to a minimization of evil overall, only a creeping increase in the total amount of evil in the world.

How are you equating this to me arguing against the existence of races to the bottom?

We have 10,000+ years of human civilization at this point. There must be some other active ethical maxim operating other than "choose the lesser of two evils" to explain why there is so much cooperation amongst humans. Evidence is not on the side of the preeminence of races to the bottom.

You should investigate the repeated prisoners dilemma.

That's a bit like asking how the defendant in a legal case is an interested party.

Even if you think someone is guilty, it does make sense to allow them to at least submit their defense. And if they choose to use that time to advocate for their own promotion, let them.

Your logic doesn't hold up well to simple escalation logic.

Company A founds itself on doing 0 harm to Area X. Competitor B shows up and starts finding success doing 10 harm to Area X, so Company A makes a "moral" decision: If we do 9 harm to Area X, we are preventing 1 entire harm. Isn't that real value? then Company C shows up and starts finding success doing 100 harm to Area X, so Company A changes it's moral stance to "unless we do 99 harm to Area X ..."

I know an old lady who swallowed a fly kind of logic going on here.

Totally understand on managing context length. But in a sense, the work is just providing the library of primary texts (and given the public domain nature of these kinds of works, that seems both ethical and legal) and some mechanism to include them into the context as desired by the user.

As an example user story, maybe I want to get Plato's reaction to Buddha. It might be convenient to have a library of sutra's that I could grab extracts from in order to send to the instructed model for further reflection. That puts the context management into the users hands. From a UI perspective you would need a library interface, the ability to select extracts, some indication of context available vs. context used, etc.

Imagine you are in the shower one day and you come up with the sketch of a possible innovation on model architecture, however there are some fine details and tricky implementations that you need to do in order to test it out. So you fire up Claude Code and describe your idea and ask it to provide some reflection on the idea and work out some proof-of-concept code.

In this scenario, this is your idea. You aren't "training off of other closed frontier models" in a distillation sense. This is your insight, your idea, possibly gained from reading a lot of papers and built on your own experience.

How do you feel if the model refuses? Do you consider the scenario I described a violation of someone else's rights?

I accept your point, in the sense that I wouldn't suggest that they have any obligation to share their own research.

What seems to be different here, is that they are saying they won't let you use their tool to do your own research.

It is a subtle but important difference. They aren't saying "we have secret sauce we won't share", they seem to be saying "we will prevent the tool you are paying for from independently creating a competing idea".

It is very difficult to see this move as anything other than Anthropic pulling the ladder up behind itself. They can dress it up in "safety" all they want, I find it hard to interpret this in a charitable way.

This reminds me of how dark-pattern common wisdom in Web 1.0 website development was to ban external links. Then how social apps prevented the export of data and actively worked to nerf significant interoperability through APIs.

But this is a tool, not just a data moat. Like a knife that degrades your ability to create knives. Or like a text editor that prevents you from implementing a text editor.

This is an interesting project and in some ways similar to an idea I had. My idea was actually just to aggregate primary texts (whatever public domain versions are available) for a wide range of philosophical and spiritual work and provide an easy way to include it as context in straight-forward LLM calls.

I've skimmed this announcement, your github repo and your site and it isn't clear to me, are these custom models? Are they fine-tuned from some base model? e.g. do you have 30 separate models?

I have not done any calculation on the capex but I'm guessing that SpaceX has. We're also just guessing on what these "datacenters" will look like. It is silly to think of them in the same way we see the football-field-sized ultra secure facilities on land. They can be highly distributed in a way that just wouldn't make sense on land. Perhaps even incrementally built out in the same way that Starlink capacity was.

Regardless, if Google is spending just shy of 1 billion USD per month, that suggests that there is a pretty high ceiling on capex available.

Consider the PR of massive datacenters here on Earth. People complain about noise, water usage. It doesn't even matter if those concerns are valid, the PR is bad enough. That might attract other massive corps that want to outsource instead of deal with the headache of building local.

You realize that not long ago companies were exploring building nuclear power sites next to their data centers to handle the expected power needs?

I'm not saying it will work. I'm saying if it does, SpaceX will own the market for a good while.

Sure, but considering the size of the challenge it makes sense to figure out the parts that can be studied on the surface first. Challenges like procuring, challenges like setting up relationships with potential customers. You probably want to figure out everything you can so that when you move on to the hard part you aren't distracted by the rest.

Consider the alternative. SpaceX figures out how to build the datacenter in space thing but fails at the rest. That would be an expensive mistake.

If xAI is a datacenter REIT, it is a special kind that has a promise that no other datacenter provider could dream of: LEO datacenters. As far-fetched as that may sound, the biggest profit center for SpaceX in my understanding was Starlink. xAI already has extremely high-bandwidth connections from Earth to LEO available. Connecting that to solar powered orbital datacenters seems doable in realistic timeframes, especially once Starship comes online and gives them a significant boost in launch capacity.

If that ends up being viable and profitable, there is no realistic competition for decades. In this view, xAI earning a reputation as a reliable AI hyperscaler is just another tactic in that strategy.

That sounds like a good attitude.

I think that Kierkegaard was considering that this attitude has to be permanent. You can't praise yourself for having resigned yourself last year, last month or last week. You have to resign yourself each and every time the urge to "dwell on the past" arises. And that urge finds a person when they are at their weakest.

There is some subtlety in how one chooses to look at the past. For example, you say "I choose thankfulness" and you recognize "the life I have is wonderful, and full of joy and satisfaction" which sound healthy. However, this describes your attitude to the present while leaving out your attitude toward the past.

He specifically calls out when a person trying to move on from the past degrades the lost alternative. In the metaphor of the princess, perhaps the Knight forms a negative opinion about her, or about aristocracy in general. He may choose to lie to himself like the fox from Aesop's fables, who when it cannot reach the grapes decides they must be sour. He may get angry with the girl, angry with himself, or angry with the world.

For Kierkegaard the ethical life requires staying true to your desires, even in the face of depravation. One ought not deny that they still want the thing they can't have. They will not rely on self-deception or ignore their true desire. They will realize that these desires will resurface and they cannot spend their life running from them.

All of this is easy to say but hard to live. It ends up forming the basis of what Kierkegaard suggests is a true faith. Fear and Trembling goes on to describe a Knight of Faith that lives beyond the Knight of Infinite Resignation but that is another thing entirely.

I have an understanding/interpretation of the philosopher Kierkegaard's metaphor of the Knight of Infinite Resignation from his work Fear and Trembling [1] that relates to this.

He tells the story of a Knight that falls in love with a princess. In the olden days princesses were married off by their parents for political reasons. There is no way his love, even if it is returned, can ever be fulfilled.

So the Knight resigns himself and marries the butchers widow. After all, she is pretty enough, she has inherited a profitable business from her late husband. And she will be elevated socially by a marriage to a Knight, so she is very keen.

But the Knight has to resign himself constantly, like in the dead of night while lying in bed and dreaming about what might have been. He must avoid falling into resentment and maintain the strength of his will.

This is a central concept to Kierkegaard, started in Either/Or and continued in Fear and Trembling.

1. https://en.wikipedia.org/wiki/Fear_and_Trembling

Yes, right now all we have are vibes/feelings. My point was that one benefit of the hype and the "CEO psychosis" is that we'll find out together fast. Uber, and companies like it, have the money to take the kind of risk that accelerates learning.

And the first data point is in your favor, kind of. I mean, Uber engineers were sufficiently incentivized to use the tokens they were given. It isn't easy to determine what the exact motivation was. What might result from this latest round of CEO backtracking is either relief (don't have to pretend to use AI anymore) or frustration (upset at a useful tool being taken away).

There are two possible stories here. One, they forced everyone to use AI and didn't get enough benefit to justify the cost. Two, they gave the opportunity to their employees to use unlimited AI and those employees jumped at the chance with a vigor that management didn't expect.

All we really know is that value per token must have been low enough to cause this change.

This is a contrarian view and I am a biased AI-maximalist. But I actually think these kinds of results are genuinely important.

There is a lot of frustration and even anger over CEOs pushing AI onto employees and some schadenfreude when it goes wrong. But there is some element of "fail fast" happening here.

I am glad wealthy corporations are footing the bill by stretching this technology to its limit. The fact of the matter is, we don't know how effective the best-of-the-best models are at scale.

There is a feeling that once we figure out how to leverage these agents, we'll see explosive growth. It's just going to cost a lot of money figuring it out.

It seems that for now, handing over 100% of code writing to LLMs is going to be too expensive. Cost per token for equivalent code is too high.

My own anecdote related to this idea was on a playthrough of Skyrim.

I was between objectives and wandering through the map. I came across one of the ubiquitous caves which I decided to enter. I was attacked by some generic low-level bandits and I cleared the cave.

After dispatching the enemies I was looting through the cave and came across some letters. They detailed a tale of a family that came on hard times in a nearby town and were forced homeless by circumstance, how they were trying to rebuild their lives, etc. I looked around the cave and could tell the individuals mentioned in the letter were accounted for in the cave. I mean, they were generic bandit models but the designer had matched them to the narrative.

I thought about the situation. I was this extremely high level wizardy kind of build trekking though the wilderness and I came across an encampment. When I barged into their makeshift home they rightly were like "get out". And then I slaughtered them all with no reason and was now deciding if the clutter was worth packing and re-selling.

I more or less stopped playing Skyrim after that.

You have at least two options:

1. Argue from ignorance. Never try unions in any other programming languages and completely disallow their use in C# codebases that you participate in.

2. Try them out and adopt an informed opinion.

You may even choose to remain in ignorance until someone wastes their own time trying to convince you. But it isn't my job or desire to teach someone who won't put in the effort to learn for themselves.