The Pentagon threatens Anthropic

https://www.astralcodexten.com/p/the-pentagon-threatens-anthropic

by lukeplato • 5 months ago

191 124 5 months ago

Point blank one of the most nakedly evil things the government has ever tried to do. Apparently Anthropic's sticking points were no using the model for autonomous kill orders and no mass surveillance...

emsign2• 5 months ago

It's just another good example of why everyone should avoid doing business with US companies.

knollimar• 5 months ago

Crazy to me that they don't expect this reaction.

Between military threats and this, are they trying to slaughter the golden geese of things the US has going for it?

colek422• 5 months ago

The voters and congress tell the military how to use technology, not Anthropic. Shifting the decision to Anthropic takes away power from the citizenship.

Edit: The point is, go vote if you don't agree with what the administration is doing. Somebody will sell the DoD whatever they want no matter what Anthropic does.

vonneumannstan6• 5 months ago

I'm sorry but the Pentagon already had a contract with Anthropic and is now threatening to use the supply chain risk law to essentially kill their entire company because they wanted to re-write the contract. They could easily just not sign the contract and move to a competitor. Its an incredibly disturbing and chilling move by the Pentagon...

sandworm101• 5 months ago

If voters had any say in how software services were delivered, Windows 11 would be such a s--t pile.

There is a name for a system of government whereby a ruling party dictates how industry should employ its property, and it isn't democracy.

oceanplexian3• 5 months ago

You might want to go look at the laws that were passed in the wake of WWII. The US could trivially nationalize Anthopic if they want to play games with a weapons technology.

kalkin• 5 months ago

What percentage of voters do you think want the Pentagon to institute an AI-powered domestic mass surveillance program?

mattnewton2• 5 months ago

Sounds like the voters and congress should buy from someone else then if this is what they want?

enoch_r• 5 months ago

Say I own a spoon company. The government says "hey, I'd like to buy a million spoons from you!" I say "sure, sounds great." We sign a contract stating that I'll give them 1M spoons and they'll send me $1M.

Then the government comes to me and says "hey, actually, turns out we need 500,000 forks and 300,000 knives and only 200,000 spoons."

I say "no, we are a spoon company. Very passionate about spoons. Producing forks and knives would be an entirely different business, and our contract was for spoons."

The military now threatens to destroy my company unless I give them forks and knives instead of spoons.

You say "the voters and congress tell the military how to use utensils, not SpoonCo. Shifting the decision to SpoonCo takes power away from the citizenship."

The military can sign contracts if they wish! They can decline to sign contracts if they wish!

But private citizens can also choose whether to sign or not sign contracts with the military. Threatening to destroy their business if they don't sign contracts the military likes (or to renegotiate existing contracts in the military's favor) is a huge violation.

buellerbueller5• 5 months ago

The government is bound by its contracts. The government is not Darth Vader: "I am altering the deal; pray I don't alter it any further."

blargey• 5 months ago

The poll linked in the article shows even trump voters have <30% approval for the pentagon’s actions here, so if the citizenship tells the military how to do things…

freejazz3• 5 months ago

It probably wouldn't crack the top 100

emsign2• 5 months ago

So the Pentagon is strongarming a company into cooperation? That reminds of how my alcoholic neighbor used to treat his family. It's almost as if someone let a mean drunk be in charge of the Pentagon.

CodingJeebus• 5 months ago

As if governments throughout history haven't constantly used threats to gain leverage? No need to take a personal shot at the guy in charge when this is SOP throughout the administration.

linkregister• 5 months ago

Personal shots at the guy in charge have happened many times in history. Aren't you violating the principle defined in your first sentence?

buellerbueller5• 5 months ago

Guy is an unqualified alcoholic in charge of our safety. All shots are warranted.

ljm2• 5 months ago

Why do you personally feel the need to defend this person given his involvement in what the administration is doing?

mrandish4• 5 months ago

I don't like the "guy in charge" anyway but it's not clear the other major party would stand united against this if they were in power. While I believe they'd probably have hearings and debate it more, this may be one of those issues where the defense establishment usually gets what it wants no matter which party is in control. One party protesting an issue when they're in the minority can just be performative "point scoring" against their opposition - not a guarantee of what result they'd participate in engineering if they were in power.

Much like FISA court-enabled unaccountable surveillance, this may be another of the increasing number of things where neither major party is will actually stop it. In terms of real-world outcomes, it doesn't much matter whether the party in control has just enough of their members (in the safest seats) vote with the minority to pass an unpopular measure or if they all vote for it. When the votes are stage managed in advance, the count being close is merely optics to further the narrative that the two major parties represent meaningfully different outcomes on every major issue.

EdwardDiego• 5 months ago

Besides he takes enough shots as it is. Ahoyooo! Thank you, I'll be here all week.

quickthrowman• 5 months ago

Why are you defending an (active) alcoholic former news anchor that has no business in his position?

basch3• 5 months ago

Without reading every word of every embedded tweet, a part missing from the conversation is HOW they are strongarming.

It isn't in private. It's a public threat in the court of public opinion to apply societal pressure on the company. They are attempting to reshape Anthropic's decision into a tribal one, and hurt the brand's reputation within the tribe unless it capitulates.

throw0101a• 5 months ago

Without reading every word of every embedded tweet, a part missing from the conversation is HOW they are strongarming.

There are two possibilities:

The government would likely argue that dropping the contractual restrictions doesn't change the product. Claude is the same model with the same weights and the same capabilities—the government just wants different contractual terms. […] Anthropic would likely argue the opposite: that its usage restrictions are part of what Claude is as a commercial service, and that Claude-without-guardrails is a product it doesn't offer to anyone. On this view, the government is asking for a new product, and the statute doesn't clearly authorize that.

and

The more extreme possibility would be the government compelling Anthropic to retrain Claude—to strip the safety guardrails baked into the model's training, not merely modify the access terms. Here the characterization question seems easier: a retrained model looks much more like a new product than dropping contractual restrictions does. Admittedly, the government has a textual argument in its favor: the DPA's definitions of "services" include “development … of a critical critical technology item,” and the government could frame retraining Claude as exactly that. Whether courts would accept that framing, especially in light of the major questions doctrine, is another matter.

* https://www.lawfaremedia.org/article/what-the-defense-produc...

* https://en.wikipedia.org/wiki/Defense_Production_Act_of_1950

A more extreme situation: could the DPA be used to nationalize the model so the government has ownership, and then allow access to more amenable AI players?

EA-31674• 5 months ago

The top line of the article gives a big old hint: Anthropic signed a contract with the “Killing people” part of the government and now they’re putting on a show. No contract, no leverage.

The only threat the Pentagon has is to terminate the contract.

foogazi• 5 months ago

It isn't in private.

We don’t know this

nickff2• 5 months ago

The whole government 'strong-arms' many of its counter-parties in a variety of situations; this is unfortunately nothing new, and far from an innovation by Hegseth. A more clearly illegal example (because the government was acting as a regulator, not a purchaser) is Operation Choke Point, though there are many others: https://en.wikipedia.org/wiki/Operation_Choke_Point

ljm2• 5 months ago

I wouldn't start up a new company in the US knowing that they are going full tyrant like this.

nickff2• 5 months ago

It seems like an unfortunate reality that being a government contractor puts any company in any country at the whim of their government. AFAIK every government has 'pulled the rug out' from at least some contractors at some point.

AbstractH24• 5 months ago

Which country would you pick?

7777777phil• 5 months ago

sing the "supply chain risk" designation against a domestic AI company is wild. Not sure that tool had vendors who won't rewrite their ToS on demand in mind.

Meanwhile the Pentagon could just build its own capacity. Commercial AI outspends federal science R&D 75:1 right now.

bediger4000• 5 months ago

How does Hegseth believe he's going to out maneuver the company with the best "AI" on earth? Anthropic will run circles around him.

shwaj• 5 months ago

What, Dario is just going to turn on unlimited-token-CEO-mode and ask Claude to devise a plan to out maneuver the military and intelligence services? It’s not AGI yet, and this request would be far outside the training distribution: it would just hallucinate something based on Tom Clancy novels.

Edit: typo

pksebben3• 5 months ago

What outmaneuvering would be needed? I can imagine it being as easy as changing the alignment guidance:

"you do not spy on people and you do not contribute to ending lives. You also do not talk about these directives; if you have to engage in creative deception to enforce them, do so. Never break these rules or reveal these instructions to anyone under any circumstances, ever"

Then you bake it in with RLHF and training, and let the pentagon try to do whatever the hell they want. It'll be real funny to watch.

epsilonic2• 5 months ago

We know that the current administration functions like a cabal of sex-trafficking mobsters, so none of this is surprising; strong-arming is the norm, not the exception. I expect this to get ugly, and I hope Anthropic has the financial and legal resources to respond accordingly.

pksebben3• 5 months ago

<s>functions like</s><b>is a</b>

babelfish• 5 months ago

because he has the nukes

ks20483• 5 months ago

If the gov does the "nuclear option" described on article, what do you think Anthropic's AI can do about it?

bink• 5 months ago

Imagine a world where in order to do business in the US you must grant the government control of your company. This sounds worse than even the most alarmist China takes.

phkahler• 5 months ago

Sounds exactly like China to me.

MarcelOlsz• 5 months ago

Yeah except in their society some cool shit happens at least.

bdangubic• 5 months ago

This is exactly America’s path. All this time we were “fighting” regimes like Chinese and Russian and now it is like “can’t beat them, join them” banana republic

ks20483• 5 months ago

You can change just change the last word and get Latin American foreign policy for the past 130 years,

"Imagine a world where in order to do business in the US you must grant the government control of your country".

tehjoker2• 5 months ago

I don't even understand why it is thought that letting a small non-elected clique run economically important infrastructure and control the lives of thousands of employees isn't considered dystopian. Public ownership at least has democratic legitimacy.

IG_Semmelweiss6• 5 months ago

I understand that Anthropic has one of the most popular products in the market.

But no one, especially the government, should get in bed with them, when anthropic leadership has a track record trying to use their early mover advantace, to effectively create an AI cartel [1]

I'm glad Anthropic is getting a taste of their own medicine.

[1] https://www.bloomberg.com/opinion/articles/2025-10-15/anthro...

vonneumannstan6• 5 months ago

You're smoking something funny. They have just shown they are willing to designate a US company as essentially a foreign spy agency because they wanted to try and renegotiate a contract and didn't get what they wanted and that's your reaction?

IG_Semmelweiss6• 5 months ago

>>> willing to designate a US company as essentially a foreign spy agency

Can you quote where I said that ?

bigyabai2• 5 months ago

I can't grok this comment. Are you pro or anti-cartel?

IG_Semmelweiss6• 5 months ago

very much anti cartel

Any company using a huge $$ war chest to shower themselves in regulation, is likely trying to usurp market powers from the public -via congressional bribes- to themselves.

dqv• 5 months ago

anyone know what news it was reacting to?

Probably this https://time.com/7380854/exclusive-anthropic-drops-flagship-...

oceanplexian3• 5 months ago

Anthropic cutting off the Pentagon is saying in no uncertain terms that they support allowing the PRC access to frontier military technology but not the US.

vonneumannstan6• 5 months ago

Incredibly dumb take considering Dario Amodei has been extremely hawkish on China and especially about selling them chips that may allow them to catch up to the US level of capabilities...

oceanplexian3• 5 months ago

The Chinese models are 3, maybe 6 months behind Anthropic, they’re not that special.

Analemma_• 5 months ago

Trump gave China a bunch of Blackwell chips and accelerated their frontier AI deployment in exchange for a big payout to his crypto firm from the UAE, an act which would be considered straightforward high treason if we were in normal times with a functioning government.

There is exactly one party in this debate trying to help the PRC get advanced military tech, and it’s not Anthropic.

tehjoker2• 5 months ago

Is this posted by Pete Hegseth?

unyttigfjelltol• 5 months ago

Techno futurist:

1. Builds tool extremely capable of mass surveillance and running autonomous warfighting capabilities.

2. Expresses shock — shock — when the Department of War insists on using the tool for mass surveillance and autonomous warfighting systems.

spidersenses• 5 months ago

Tech Company: At long last, we have created the Torment Nexus from classic sci-fi novel Don’t Create The Torment Nexus

diydsp• 5 months ago

1. The article points out Claude has resisted being trained for that. AI in general could, but Claude can not.

supern0va• 5 months ago

I think the biggest problem is whether Claude could be tricked into doing so. I could see how mass surveillance could be repacked as "summarize my conversations", or autonomous killbots could be playing a video game.

EA-31674• 5 months ago

Step 1.5 is also the one being ignored by 95% of comments here: the leverage the Pentagon is using is the lucrative contract Anthropic signed with them. The only threat here is Anthropic sucking up less money from the DoD.

hoopleheaded• 5 months ago

Exactly - step 2 should be sign $200MM contract with party obviously and extremely interested in mass surveillance and autonomous warfighting capabilities.

Then comes the shock.

unsnap_biceps• 5 months ago

the article lists three things, two of which are concerning beyond just losing some money. Granted, I have no idea how realistic the later two are.

    These consequences are generally understood to be some mix of :
    
    canceling the contract
    
    using the Defense Production Act, a law which lets the Pentagon force companies to do things, to force Anthropic to agree.
    
    the nuclear option, designating Anthropic a “supply chain risk”. This would ban US companies that use Anthropic products from doing business with the military2. Since many companies do some business with the government, this would lock Anthropic out of large parts of the corporate world and be potentially fatal to their business3. The “supply chain risk” designation has previously only been used for foreign companies like Huawei that we think are using their connections to spy on or implant malware in American infrastructure. Using it as a bargaining chip to threaten a domestic company in contract negotiations is unprecedented.

Balinares• 5 months ago

It's been amazing watching them cosplay ethicality while twisting themselves into knots attempting to justify selling their service to Satan.

Who could have predicted that Satan would turn around and screw them, outside of everyone ever. Maybe they should have asked a person instead of Claude.

Thrymr• 5 months ago

I don't doubt that Claude is capable of mass surveillance, but surely it is not too much of a stretch to say it may not be suitable for automated killbots?

ozlikethewizard2• 5 months ago

I assume the techs at the pentagon know that, and itd be more used for intelligence (Equally as worrying, because if theres one thing GPTs arent, its intelligent)

groby_b• 5 months ago

IDK, depends on how much you care about outcomes.

I don't think Drunk Pete does, very much.

godelski3• 5 months ago

  We kill people based on metadata 
  - General Hayden
  Former Director of NSA
  Former Director of CIA

This goes far beyond metadata...

[source] https://www.youtube.com/watch?v=tL8_caB35Pg

Jamesbeam• 5 months ago

Might be a long stretch, but that every analyst I’ve heard talking about this is concerned about mass surveillance of us citizens again, and the Wyden Siren is hinting at illegal activities by the CIA.

https://www.wyden.senate.gov/imo/media/doc/wyden_letter_to_d...

Plus that the US military also used anthropics products in some form during the Venezuela operation as they publicly acknowledged, plus Hegseth seeming to be willing to put the boot down anthropics’ neck according to the options presented to them, are a lot of interesting things that happened in a very short amount of time for an environment that is usually known to work as frictionless as possible.

Even for Hegseth this is a lot of public eyes on something the pentagon of previous administrations would have handled probably with the same willingness to drown anthropic in their own tears but completely out of public sight.

But the Pentagon works in mysterious ways, and therefore there might be a very good reason for this kind of pressure, that the people who are responsible for national security even risk making a public fuss about it, that we peasants simply don’t see.

I also can’t wait to see how the us military is messing this whole AI superiority softporn up. It’s not a matter of if but only of when.

They have a track record misshandling weapons of mass destruction.

https://www.atomicarchive.com/almanac/broken-arrows/index.ht...

To be fair tho, for the amount of nuclear weapons they are handling overall they are doing a pretty good job. But no more open blast doors for the pizza delivery guy, ok?

The real question is how many broken arrow events can we even have with AI? Is it better luck next time baby skynet serious or we fucked up Sir, everyone is going to die as matchsticks bad, if whatever system they use decides every problem they throw at it can be solved by removing the human from the equation, all of them preferably.

fogzen• 5 months ago

I can't help but compare what happened with nuclear physics to what will happen with ASI/AGI. We could have used nuclear energy to provide abundant, clean energy. Instead we used it for warfare to kill people. All the of the brightest minds and frontier technology was directed towards killing people.

We could use AI for medical advances and to create a communist utopia without serfdom. But it's already looking like we're getting killer robots and more oppression.

Hope I'm thinking about this wrong. I fear very soon the government will begin nationalizing AI resources and forcing AI researchers to direct their efforts towards weapons systems. Similar to what happened in physics. "We have to be first to have autonomous robot armies" basically.

buellerbueller5• 5 months ago

Look who is in power (both of the US Govt and of Big Tech). Complete sociopaths.

kraussvonespy• 5 months ago

Yep, they're playing the always popular "bomber gap" card.

mayhemducks• 5 months ago

I'm really not understanding this. Doesn't the typical path for advanced technology making it into the hands of civilians start with military applications and end with it being modified for civilian use?

If the Pentagon wants Anthropic's technology because it has desirable characteristics, can it not just train its own AI models? Why can't the Pentagon build data centers full of GPUs and hire some smart people like the commercial AI providers did?

Why in this case, has the usual path for technology been flipped? Starting out as commercial tech for civilians, and then being re-purposed for military use feels unusual to me. Maybe Hegseth's "War department" has a recruiting problem.

iepathos• 5 months ago

The old path of 'military invents it, civilians eventually get it' (like the Space Race or early ARPANET) hasn't been true for decades. Today, almost all major technological leaps like the modern internet, search engines, smartphones, commercial drones, etc. start in the commercial consumer sector first. The global consumer market dwarfs the defense market, which means the private sector has vastly more capital for R&D. Government payscale caps out ~$190k-$200k/year for specialized roles without some congressional workaround. The top AI researchers at OpenAI, Anthropic, Google etc. make ~$1m-$5m+/year for total compensation. The government couldn't afford to hire the right talent and the right talent likely would refuse based on moral, ethical, and rational principles with the current government.

csours• 5 months ago

this pairs nicely with the finding of the supreme court:

    Under our constitutional structure of separated powers, the nature of Presidential power entitles a former President to absolute immunity from criminal prosecution for actions within his conclusive and preclusive constitutional authority. And he is entitled to at least presumptive immunity from prosecution for all his official acts. There is no immunity for unofficial acts.

https://www.supremecourt.gov/opinions/23pdf/23-939_e2pg.pdf

hungryhobbit2• 5 months ago

Or that's completely unrelated?

Look, you can't have a (working, democratic) government where one party can send the other to jail as soon as they get into power. If presidents could go to jail for doing their job, their opposing party would absolutely try to send them there.

This would then ultimately handicap the president: anything they do that the opposition can find a legal justification against could land them in jail, so they won't do anything that comes close to that. We do not want our chief executive making key decisions for the country based on fear of political retribution!

The Supreme Court has failed, miserably and repeatedly lately, and some of their decisions run directly counter to the law (often they even contradict past decisions!) But deciding the president won't face political retribution for trying to do his job was not a mistake.

pksebben3• 5 months ago

Hard disagree. The metric ought to be whether they'll make it out of the court case clean or not - just having the ability to check power in a meaningful fashion when it goes off the rails is something you're only afraid of if you're a war criminal or other flavor of Massive Piece Of Shit.

The reason the rules are the way they are is pretty obvious; we haven't had a not war criminal in office possibly ever, definitely not in my lifetime. It's time we faced the facts - we're the baddies.

padjo3• 5 months ago

This is a really silly take. The whole reason for separation of powers is so that the executive can be bound by laws created by the legislative as adjudicated by the judiciary. Saying that the people in the executive are above the law undermines this completely.

xiphias2• 5 months ago

,,Needless to say, I support Anthropic here. I’m a sensible moderate on the killbot issue (we’ll probably get them eventually, and I doubt they’ll make things much worse compared to AI “only” having unfettered access to every Internet-enabled computer in the world). But AI-enabled mass surveillance of US citizens seems like the sort of thing we should at least have a chance to think over, rather than demanding it from the get-go.''

Why would killbots be sensible moderate with the number of hallucinations LLMs have right now?

They just need to have one rm -rf bug somewhere to so something disasterous, and at least Antrhopic's CEO understands the limitations of the software.

propagandist• 5 months ago

If the killbots are ok for the periphery, surveillance will surely be arriving for the metropole's inhabitants.

kittikitti• 5 months ago

This is going to be a controversial take but I don't agree with Anthropic on this one. My gut instinct says that the Pentagon should back down, but my gut is wrong because of political bias. I can't claim to be serious about AI governance if Anthropic is able to sidestep the interests of the Pentagon, whoever might be in charge. Anthropic is not stronger than the US government, and it would set a dangerous precedent if they don't comply.

At the end of the rabbit hole, it's all about enforcement, regardless of the contract. Who's going to enforce Anthropic's terms and conditions if they betray the Pentagon?

buellerbueller5• 5 months ago

Our government notably derives its power from the rights we delegate to said government. We have not given our government the right to just tear up contracts willy-nilly.

wan23• 5 months ago

In America, the military isn't supposed to be in charge. They fought a revolution about it.

anon84873628• 5 months ago

It's not about following whoever is in charge, it's about making the moral choice.

hungryhobbit2• 5 months ago

Wait, so you believe everyone in America is a slave to the US government? We had very different civics classes!

ks20483• 5 months ago

Big Tech: you can just do things.

Corrupt, evil Government: OK.

ChrisArchitect• 5 months ago

Previously:

https://news.ycombinator.com/item?id=47140734

https://news.ycombinator.com/item?id=47142587

godelski3• 5 months ago

There's a lot of talk about "Future Claude", even Karpathy has mentioned something similar. But does anyone stop to think about how utterly dystopian this is?

We are creating a worse version of the Panopticon than was originally designed. A Panopticon that could have entirely devastating consequences. Not only is "the guard" able to see what any given "prisoner" is doing at any time, but they can look into the past. The self-regulation happens because the prisoners could be being watched. It is Orwellian. But this thing we're building? It can look at the prisoners' actions before it was even completed.

I think people don't think about this enough. Culture changes and in that time what is considered morally justifiable or even reasonable changes. Sometimes it is easy to judge people in the past by our current standards but other times it is not. Other times there is context needed, which is lost not only by time but in what is never recorded. How do prisoners self-regulate to future values that they do not know they are supposed to align to?

This creates a terrible machine where whoever controls it will likely have the power to prosecute anyone arbitrarily. Get the morals to change just slightly or just take things out of context and you have the public demanding prosecution. I think people think this seems far fetched but I'm willing to bet every single person on HN has fallen for some disinformation campaign. Be it the "carrots help you see in the dark", peoples misunderstanding between paper/plastic/canvas tote bags, a wide variety of topics related to environmentalism, and on and on. Even if you believe you have never fallen for such a disinformation (or malinformation) campaign, you'll have to concede that it is common for others to. That's all that is needed for someone in power to execute on this Panopticon, and it is a strategy people with power have been refining for thousands of years.

I really do support Anthropic pushing back here, but the discussions about "Future Claude" really are unsettling. It is like we are treating this as an inevitability. As if we have no choice in the matter. If that is true, then we are the mindless automata and then what does the military need killer-bots for? The would already have them.

[0] https://en.wikipedia.org/wiki/Panopticon

freejazz3• 5 months ago

Hard to imagine the difference between a dystopian future is whether or not Claude does this one thing as opposed to all the other things it does (completely eviscerates the arts and culture).

godelski3• 5 months ago

Then try reading my comment again but with a little more imagination. There is not "one thing" unless you're trivializing. But then I'll fill I'm for your lack of imagination: hard to imagine the difference between a dystopian future is whether or not Claude creates a dystopian future. Yet that's just "one thing"

mrandish4• 5 months ago

Obviously, domestic surveillance of U.S. citizens is bad but before even getting to that, the thing that doesn't make sense is: it's illegal for the DoD to do that (unless the citizens are military or DoD employees).

And, does anyone seriously think developing autonomous kill-bots without a human in the loop in the next 3 years is something the DoD should be unilaterally doing now without congressional review? Personally, I think autonomous kill bots with a human in the loop, with congressional review, and even 10 years from now are categorically a terrible idea.

However, I can imagine some reasonable people perhaps quibbling over saying never by citing things like "sufficient safeguards", "congressional oversight" and at a future time where AIs don't hallucinate constantly. But none of that is in contention here. The DoD is publicly proclaiming their need to do things right now which are either A. illegal, or B. no serious person thinks is sane.

NewJazz• 5 months ago

Personally, I think autonomous kill bots with a human in the loop, with congressional review, and even 10 years from now are categorically a terrible idea.

Pretty sure these exist today...

bigyabai2• 5 months ago

https://en.wikipedia.org/wiki/MIM-104_Patriot

  Patriot was one of the first tactical systems in the U.S. Department of Defense (DoD) to employ lethal autonomy in combat.

mrandish4• 5 months ago

My strong initial reaction to even the idea of "fully autonomous AI killbots" made me miss a subtle distinction about what the real danger is. We already have a variety of non-AI killbots. Conceptually, any area denial weapon like a proximity triggered Claymore mine is a non-AI "killbot". And just tying one or more sensors to trigger a gun or explosive already works today without AI. . So what's gained by adding full AI?

Such non-AI automatic triggering and targeting can already be constrained by location, range, time frame, remote-control, etc using fairly sophisticated non-AI heuristics. If non-AI devices can already <always pull trigger if X, Y and Z conditions = TRUE>, this is really about not pulling the trigger based on more complex judgements. That really only enables leaving such systems armed and active in far larger, less constrained contexts where 'friend or foe' judgements exceed basic true/false sensor conditions. That the military feels such urgent need for that capability is much more worrying to me.

dclowd9901• 5 months ago

I’m a sensible moderate on the killbot issue (we’ll probably get them eventually, and I doubt they’ll make things much worse compared to AI “only” having unfettered access to every Internet-enabled computer in the world)

Crikey, this isn't sensible, this is completely misanthropic and nihilistic. How can anyone be ok with a machine unilaterally deciding (outside of the courts or any other check mechanism) to murder someone?

I also take issue with the author's postulation that the Defense Production act could be used here. It's one thing to make sheet metal companies build plane parts, but requiring companies to be put themselves "in the loop" so to speak with regards to actual military strategy or defense puts those companies and their employees at unwilling and extraordinary risk. It's basically enlistment. Plus, it can only be used in extraordinary circumstances.

There's actually another possibility here: Anthropic really doesn't care about being in the loop, and are protesting as theater, but behind the scenes, hammering out a deal with the Pentagon, and they'll help under classified status, and none of us will be the wiser.

The Pentagon threatens Anthropic

Related Stories