A DARPA Perspective on Artificial Intelligence [pdf]

https://www.darpa.mil/attachments/AIFull.pdf

by nulluint • 4 years ago

186 54 4 years ago

darpa.mil

readhn• 4 years ago

cool stuff. thanks for sharing!

itsmefaz• 4 years ago

Nice

Animats• 4 years ago

"Contextual adaptation" is what they want, but that doesn't mean it's coming in the near future. However, this does mean that funding for research on it will be available.

As I've said for years, the big lack in AI is in the "common sense" and unstructured manipulation area. Nobody can build something with squirrel levels of manipulation and agility, even in simulation. Robot manipulation in unstructured situations is still very poor. The people trying to simulate C. elegans at the neuron level can't get that to work, despite a full wiring diagram and years of effort.

Something very low level is not understood. There's a Nobel Prize waiting for whomever figures that out.

hans1729• 4 years ago

Nobody can build something with squirrel levels of manipulation and agility, even in simulation.

Isn’t that what alpha go or the StarCraft league are? Organic strategies in well-defined contexts (action options of the squirrel at Tn)? “Squirrel” is a nice reference frame.

TaylorAlexander• 4 years ago

Manipulation in the sense of being able to physically pick things up and manipulate them. Which is a difficult problem. StarCraft is moving a mouse pointer and clicking.

davidmanheim2• 4 years ago

No - those systems simulate "control" at the level of giving commands, not actual motor control. So practically, current AI can win at Go and play at superhuman levels, but still cannot drive a car as well as most adults.

digitcatphd• 4 years ago

Well said

jcims2• 4 years ago

This is one of my favorite little videos to illustrate the dichotomy - https://i.imgur.com/o7lI08Y.gifv

For all of our mastery of information processing, our ability to fabricate and motivate fine differentiated structures is really quite incredibly poor relative to biology. The little beetle is basically alien technology relative to the toy. But add in the fact that it can feed itself, repair itself and reproduce itself with a system that can scale 3-5 orders of magnitude up and down based on need is pretty mindblowing.

The mRNA vaccines might be one of the first times where we've leveraged biology to directly fabricate a discrete part with desired properties. Certainly in raw tonnage of output I can't think of anything that comes close. I feel like this is only going to advance with time, to the point where we may be growing robots instead of milling and printing them.

jtbayly• 4 years ago

Why doesn't that play in Safari?

jonplackett• 4 years ago

I find the comparison with AI and insects really interesting to illustrate just how far off the mark we still are.

You have a room sized super computer that trains for zillions of hours and can’t even pick out pedestrians with especially good precision.

Then you have a bee with a minuscule brain that can do complex pattern recognition to find flowers, complex manipulation to get nectar and pollen out of them, object avoidance, swarming behaviour, and fancy communicative dances (both performing and interpreting), plus all the other stuff bees do to get by in the world.

Something is up there for sure.

jd115• 4 years ago

The low-level thing that isn't understood is that intelligence is not progressive. You can not build up to it. Sure, you can forever continue to build better and better approximations of certain aspects of intelligence and pretend you're making progress, but you are not. Of course, there is great utilitarian value in those approximations, so we MUST continue to do this.

Still, my position is that NO progress has ever been made in the area of AI, and no progress will be made any time soon.

I'll take it a step further (in case I don't get enough downvotes for what I've written so far). I maintain that you CAN NOT build intelligence. You can only TAP INTO it. So the very direction in which all of our AI efforts are headed is a dead-end.

grapescheesee• 4 years ago

I agree. We need to stop this over use, actually the use of AI as some grounded in science matter of fact. What we have is ML, and thus a system for algorithmic refinement and definition.

leto_ii• 4 years ago

I maintain that you CAN NOT build intelligence. You can only TAP INTO it.

I wouldn't go exactly this far, but I would say that whatever process might exist to create artificial intelligence, it might be closer to gardening than to engineering.

My vague feeling is that there might be some sort of (non-supernatural) "mysterious" component to intelligence that we won't be able to engineer and that might just emerge under the right circumstances.

In that case we would just have to "grow" AI, without being completely sure that our effort will work.

rndphs• 4 years ago

I'm sure people said the same thing about motive power before the steam engine. You cannot build it, you can only tap into it.

rbobby• 4 years ago

Nobody can build something with squirrel levels of manipulation and agility, even in simulation.

While not squirrel level, this impressed the heck out of me: https://ashish-kmr.github.io/rma-legged-robots/

yobbo• 4 years ago

"Squirrel level of manipulation" seems more a problem of building tiny physical robots and giving them appropriate interfaces for AI to control, which is an unreasonably difficult benchmark.

A reasonable first step would be autonomous small cheap drones, but I can't say what their missions would be. AlphaStar and Openai Five are mentioned elsewhere here, and these demonstrate that the problem isn't unapproachable.

There probably isn't enough confidence yet to arm autonomous drones, or there isn't a meaningful tactical purpose.

jzer0cool• 4 years ago

I see papers being written by scholars and professors. Are reputable ways to publish papers that can get reviewed without being in school?

Ziggy_Zaggy• 4 years ago

Very thought-provoking.

mooreds• 4 years ago

Nice high level overview of the three waves of AI (two have happened, the "Contextual adaptation" wave is yet to occur). Includes examples and successes and failures. ("Young man holding a baseball bat", indeed :) ).

"Systems construct contextual explanatory models for classes of real world phenomena" is the next goal. That is, understanding + being able to describe the reasoning for the understanding.

No technical depth, really, but lots of words to google if you want to learn more.

davidmanheim2• 4 years ago

People may want to see the appendix to our RAND report on areas related to DARPA's focus, for more depth on the relevant uses of automation, ML, and AI -https://www.rand.org/pubs/research_reports/RR2489.html

jll29• 4 years ago

It's good to see a main funding agency's perspective.

As a researcher, I like their non-hype way of defining AI as "programmed ability", which is accurate and realistic -- also puts AI further apart from real intelligence, which means unanticipated activities.

I would like to know more what they see as "abstracting", from their perspective.

We haven't got much further in our scientific understanding of intelligence - if you bought a psychology text book today and ten years ago there wouldn't be much of a breakthrough change detectable in terms of modeling cognition. And as impressive as some computer science AI models perform certain tasks, I haven't been taken by surprise by them asking me a question out of the blue, which is one of my personal litmus tests for intelligence.

dataking• 4 years ago

As a researcher, I like their non-hype way of defining AI as "programmed ability", which is accurate and realistic

This is the first point of advice in the Heilmeier Catechism

https://www.darpa.mil/work-with-us/heilmeier-catechism

hiddencost2• 4 years ago

I wonder who is trying to get what funded? "Second wave" is going gangbusters, despite Gary Marcus' every-six-months rant; the review of statistical learning is reasonable given a barely technical audience, but the summary of "third wave" seems designed to extract large amounts of funding from people who aren't up to date on the state of the field.

gibsonf12• 4 years ago

Theranos got a great deal of funding too, and the truth about how none of the "Second Wave" ML self driving car technology comes even remotely close to safe self driving will be coming out probably later this year. The issue is that ML has no conceptual and causal understanding of anything. For confirmation, I've been carefully watching the countless "self driving" startups in San Francisco driving around, and I have almost never seen a driver in those cars not actively steering it.

akomtu• 4 years ago

None of those self-driving MLs have a virtual world model in their "head", right? They just react to the latest video frame. If so, it's not even a fish level intelligence, it's more like a house plant.

AlotOfReading2• 4 years ago

You can't always tell whether the car is operating autonomously even from the back seat, let alone another vehicle. A lot of operators keep their hands very close to the wheel and follow its motions to minimize response time when they need to take over.

I've worked at companies whose vehicles you've probably seen and have never heard anything like what you're suggesting is "the truth". Can you provide more details on how current solutions don't remotely approach reasonable safety levels?

hiddencost2• 4 years ago

The issue is that ML has no conceptual and causal understanding of anything

Practitioners work with much more tightly defined objectives and methods for improving their systems than "conceptual and causal understanding of anything". From the perspective of folks in the industry we've been making remarkable progress, well beyond what we could have expected. We're saturating major benchmarks, sometimes in one to two years. Back in the day, during the "AI Winter", it was often 10 years before a new breakthrough happened.

redytedy• 4 years ago

Cruise is actively running a driverless robotaxi service in SF, so your assertion is at least partly wrong.

alins• 4 years ago

FYI the presentation is from December 2016.

sigmoid10• 4 years ago

I was gonna say this seems really outdated. 2016 was ages ago in the AI world. We've had Openai Five, AlphaStar, MuZero, BERT, GTP and tons of other groundbreaking stuff since then. This presentation tells you nothing about the current state of the field.

gone35• 4 years ago

(2016?)

Also:

First wave: Handcrafted knowledge
Second wave: Statistical learning
Third wave: Contextual adaptation

I understood clearly enough the first two, but the slides become increasingly ambiguous and fuzzy towards the end; and it seems to me they are mixing up a bunch of not self-evidently related desiderata.

It is not immediate, for instance, that small "generative" models that are easy to interpret necessarily lead to better "abstraction" (whatever that means). And whatever this all has to do with "contextual adaptation" is to me anyone's guess.

Highly alarming (but sadly, from experience, unsurprising) to see such fuzzy position document from such an important funding agency for AI.

Khelouiati2• 4 years ago

2016 ??????

ochicial• 4 years ago

Here is the presentation in form of a blog post:

https://machinelearning.technicacuriosa.com/2017/03/19/a-dar...

gwern5• 4 years ago

This submission should have a (2016)/(2017) on it. I looked at the image caption example and immediately went 'wait, that's wrong, current 2020/2021 captioners would get that right', but of course, for 2017 that's plausible.

Khelouiati2• 4 years ago

2016 ?????

abel_• 4 years ago

The most pressing dangers of AI most researchers see:

- error rate too high

- you can trick a classifier with noise

- it's racist sometimes

Actual dangers of AI:

- stop problem

- infeasibility of sandboxing

- difficulty of aligning black boxes with human values

shpat• 4 years ago

Here’s the presentation in video form with a bit more context & detail.

https://youtu.be/-O01G3tSYpU

gwern5• 4 years ago

BTW, the Tay example in the slides is bogus. Zero AI involved. You should ask what's cut off when you see a tweet reply like that. This is what's cut off: https://twitter.com/daviottenheimer/status/71288991553350041...

Tay had a 'repeat after me' feature (an ancient feature of any IRC chatbot to just echo or print a given string). That's all this is. A troll account issued the command 'Repeat after me' and then tweeted '@TayandYou HITLER DID NOTHING WRONG!', and the Tay daemon dutifully repeated it back to the troll in that thread.

mach1ne• 4 years ago

How much of the controversy surrounding Tay is a result of this specific command?

gwern5• 4 years ago

Possibly all of it. As best as I can tell, there may have been a few milquetoast rudenesses, of the usual sort for language models, but the actual quotes everyone cites with detailed statements about the Holocaust or Hitler seem to have all been repeat-after-mes then ripped out of context. It's hard to say given how most of the relevant material has been deleted, and what survives is the usual endless echo chamber of miscitation and simplification and 'everyone knows' which you rapidly become familiar with if you ever try to factcheck anything down to the original sources. (This is why lots of people still 'know' Cambridge Analytica swung the election, or they 'know' the Twitter facecropping algorithm was hugely biased, or that 'Amazon's HR software would only hire you if you played lacrosse', or 'this guy was falsely arrested because face recognition picked him' etc.) If you look at the very earliest reporting, they mostly say it was repeat-after-me functionality, hedging a bit (because who can prove every inflammatory Tay statement was a repeat-after-me?), and then that rapidly gets dropped in favor of narratives about Tay 'learning'. (Even though if you look at the chatbot code MS released later, it's not obvious at all how exactly Tay would 'learn' in the day or so it had before shutdown.)

Anyway, long story short, the Tay incident is either entirely or mostly bogus in the way people want to use it (as an AI safety parable). The real story, of an `echo` gone wrong, is vastly less interesting, and is about as important as typing '8008' into your calculator and showing it to your teacher.

hackernewds• 4 years ago

This is quite a tall claim. Are there any other supporting arguments? Missing an edge case abuse could be a vulnerability for any good AI still

A DARPA Perspective on Artificial Intelligence [pdf]

Related Stories