How the U.S. military thinks about AI [audio]

48 50 7 years ago

lwneal• 7 years ago

The substantive part of this conversation comes about 35 minutes in, when Allen describes the DoD's five AI ethics principles: that any deployed AI systems must be "Responsible, Equitable, Traceable, Reliable, and Governable." [1].

I think "Responsible" raises some hard questions. Today, any weapons use by a robot might require a responsible human operator to first (remotely) pull the trigger. But what if an adversary allows their robots to fire without waiting for human permission? The first side to do so would gain a critical advantage, and we cannot have an autonomy gap.

Some have raised conerns that in the extreme case, this could lead to a world of machines anonymously killing without attribution. If you haven't, see Stuart Russell's compelling short film "Slaughterbots" [2].

[1] https://media.defense.gov/2019/Oct/31/2002204459/-1/-1/0/DIB...

[2] https://www.youtube.com/watch?v=9CO6M2HsoIA

dx87• 7 years ago

Autonomous killing could be something that needs to be added to international laws of war. The USA already follows restrictions that could give it an advantage in war, but it doesn't because of international agreements. For example, our military has to wear uniforms during war, and we have to follow proportional force that restricts what weapons can be used in response to an attack. Countries largely follow the rules because nobody wants a free-for-all war with no rules.

tehjoker• 7 years ago

The US follows rules to the extent they can get away with breaking them, no more no less. Witness the Hauge invasion act of the early 'oughts that sought to avoid accountability by the international criminal court by threatening to invade it if a US military person was ever put on the dock.

https://en.wikipedia.org/wiki/American_Service-Members%27_Pr...

catalogia2• 7 years ago

How would autonomous killing be precisely defined? Whenever a machine is designed to kill without an affirmative command from a human operator? Would that include landminds? Maybe it should...

Mirioron• 7 years ago

Countries largely follow the rules because nobody wants a free-for-all war with no rules.

And somehow these rules tend to put restrictions on things which are a force multiplier for smaller forces against larger ones or would make it very costly for a larger country to attack a smaller one. They protect the incumbents. It makes sense why big militaries would want rules like this - this makes it much less costly for big countries to war with smaller ones. Russia didn't want Ukraine to have nukes. A few decades later Russia invaded Ukraine.

It is okay for a US drone to strike a target and kill civilians. The civilians will just be defined as "enemy combatants" and the world moves on. However, if people from that same place somehow manage to kill the drone operator then that would be a terror attack.

DoingIsLearning4• 7 years ago

Traceability

A question for the AI Researchers and Engineers On the topic of _Traceability_ and as someone outside the field this is something that has puzzled me:

(Broad assumptions) Assuming a 'neural network' is a black box probabilistic model. How do we _prove_ in the context of _safety-critical_ systems that they will behave in a known determined and traceable way for all known cases?

I would be curious to understand the process/methods that people use in ADAS system or Medical Imaging (and now Defense applications as well) in order to provide 'verification' evidence for such a system? (Focusing more on verification/provability from a regulatory standpoint)

kuu• 7 years ago

There is work in progress for understanding the content of a neural network and avoid this "black box" effect. This is still far from reached but there are some advances. See [0] for example.

Also, remember that AI includes more than NN. You can use some other models, as a Linear Regression or a Random Forest which are perfectly explainable.

[0] https://christophm.github.io/interpretable-ml-book/neural-ne...

ganzuul3• 7 years ago

Various model compression schemes get rid of high-frequency noise in the model, combating over-fitting. This results in robustness against adversarial inputs.

Some of these schemes are motivated by rather robust theory. Gaussian Processes are a good starting point to learn more.

bumby3• 7 years ago

DARPA has acknowledged this problem and is working on it.[1]

See slide 11 for some of the technical approaches to making AI more interpretable.

[1]https://www.darpa.mil/attachments/XAIProgramUpdate.pdf

yorwba2• 7 years ago

But what if an adversary allows their robots to fire without waiting for human permission? The first side to do so would gain a critical advantage, and we cannot have an autonomy gap.

You're making the bold assumption that the robots will actually work, instead of wasting ammunition on all the wrong targets, including friendlies. I'm pretty sure the first side to allow fully autonomous weapons will do so before the technology is ready (humans are never patient enough to wait until the technology is ready) and end up regretting it.

speedplane• 7 years ago

You're making the bold assumption that the robots will actually work, instead of wasting ammunition

I'm pretty skeptical the U.S. military is concerned about wasted ammunition. They waste so much on training and preparation that anything wasted in combat is minimal.

I'm pretty sure the first side to allow fully autonomous weapons will do so before the technology is ready

It doesn't have to be autonomous to be deployed before it's ready. Iran managed to should down a civil aircraft based on human error. The U.S. has done the same decades earlier. Clearly there were either technology or human process problems, I suspect that autonomous military robots will follow this trend.

lordlic• 7 years ago

You're making the bold assumption that the robots will actually work, instead of wasting ammunition on all the wrong targets, including friendlies

I could easily imagine the US military deploying autonomous combat bots to some middle-eastern country unaccompanied by human troops. They would care approximately zero about wasted ammo, and we already know from our drone strike policy what the military thinks about accidentally killing non-combatants. As long as there are no Americans around to get killed by rogue bots, it doesn't really matter how much collateral damage they cause, within reason.

salty_biscuits• 7 years ago

Maybe that makes sense for some applications though, like a landmine?

arh68• 7 years ago

"Responsible" seems pretty easy: follow the rules of engagement.

What if a convoy passes a group of nuns? What if the nuns "fire without waiting"? Should we let the nuns have the "critical advantage" of first shot? Well, yes, it'd be absurd to do otherwise.

If some other country wants to develop auto kill bots that commit war crimes, I don't think we have an imperative to match them.

ethbro2• 7 years ago

That's not the problem parent is talking about. They're talking about an autonomy gap.

Tolerant rules of engagement are feasible because reactions are "similar" in timescale. Enemy fires on you, you fire back, etc.

But there are many developments that make that system less tenable.

Firstly, increased weapon lethality. When the first shot has a 90%+ kill probability (say, putting a Hellfire missile on a civilian vehicle), can you afford to let the enemy take the first shot? Or do your rules account for reality and back up a step? Maybe now, radar detection, firing preparations, or even entering a given airspace are sufficiently provocative to counterattack.

In literature and experience, this is casualty modeling moving from attrition (some constant, scaled by time) to volley (majority of casualties caused in bursts, at specific points in time).

Secondly, and this specific issue: reaction time between autonomous vs manual systems. If an opponent is using autonomous targeting and engagement, and you're using autonomous targeting and manual engagement, there may be some orders of magnitude difference in reaction time. Coupled with the first point, that may mean your entire force is wiped out before you have a chance to react.

killjoywashere• 7 years ago

I try to tell you guys about multi-million dollar opportunities to save lives available for the next 10 days and I can't keep it on the front page, even with dang's help (1). By comparison, I can't keep this softball podcast off the front page. Please, tell me you're more interested in saving lives than kibitzing.

(1) https://news.ycombinator.com/item?id=22050285

ackshually• 7 years ago

The sound design on this podcast is awful. Interesting podcast, but the music is way too loud and doesn't fade in or out at all. the mic of the host seems to occasionally tear as well.

How the U.S. military thinks about AI [audio]

Related Stories