Toyota Research claims breakthrough in teaching robots new behaviors

https://www.tri.global/news/toyota-research-institute-unveils-breakthrough-teaching-robots-new-behaviors

by geox • 3 years ago

403 239 3 years ago

treespace8• 3 years ago

This looks impressive. Much more than even the Boston Dynamics demonstrations.

Flipping a pancake is extremely difficult because each pancake is different. I know that these videos must be cherry-picked but to be able to train a Robot to do this just by demonstrating feels like a massive leap.

irtefa2• 3 years ago

Yes! In layman's terms: is the most efficient way to train these robots by showing them billions of videos of how it's done?

GlenTheMachine3• 3 years ago

Almost certainly not. Because the sense of touch is an important part of the problem and that data isn’t present in videos.

sashank_1509• 3 years ago

Flipping a pancake was done in 2010. What looks impressive for humans is easy for robots and vice versa: https://youtu.be/W_gxLKSsSIE?si=HDyNXe1Ys_eFXiVU Another case in point: robot juggling was done in 1990s and to date we do not have a robot that can open any door reliably like a human. Kind of like Moravecs Paradox

LostLocalMan2• 3 years ago

To be fair it is far more complex for a robot to grip a spatula and use that spatula on a griddle than to use dynamic motion to flip a pancake in a pan.

dclowd99014• 3 years ago

And here I thought manual labor jobs were safe for a very long time. I really hope people at the policy level are thinking about what it looks like to have a world of people that don’t have any work to do.

teeray• 3 years ago

Our research in robotics is aimed at amplifying people rather than replacing them

Noble, but the reality is that once the genie is out of the bottle it will be used by many MBAs to replace people.

reidjs• 3 years ago

Controversial, but we can look at this as a good thing. If your entire job is simple enough for a robot to do that job shouldn’t be filled by a human. However, the human should receive a decent standard of living, regardless of their employment, which may be politically impossible.

endisneigh2• 3 years ago

However, the human should receive a decent standard of living, regardless of their employment, which may be politically impossible.

Not gonna happen.

Alupis6• 3 years ago

which may be politically impossible

Try economically impossible for starters.

Do some napkin math on how much an individual needs to live each month, and then multiply that by how many people you need to support nation-wide. The dollar amounts are staggering, and make our current annual budget (all of it!) look like mere child's play.

We're talking hundreds of billions of dollars every month. It's simply not possible.

LostLocalMan2• 3 years ago

However, the human should receive a decent standard of living, regardless of their employment, which may be politically impossible

They'll get pie in the sky when they die

numpad03• 3 years ago

This rather sound like an inside facing expectations adjustment implying "don't fire assembly workers yet", with Toyota eagerness I'd imagine you'll be either fired on spot for blatantly lying or forced to make it happen by end of the quarter if you'd even suggest that.

motoboi• 3 years ago

It won't replace people, it will displace them.

tspike• 3 years ago

To be a bit flippant, the last thing our species needs is amplification- we are plenty loud, thanks.

distortionfield• 3 years ago

This is FUS. Replacing people with machines is the story of the entire Industrial Revolution, and yet we haven’t ever had widespread unemployment as a result. Quite the opposite.

kristopolous• 3 years ago

This sounds like LoRA but for robotics, am I teaching this right?

irtefa2• 3 years ago

The prospect of robots able to manipulate liquids, cloth and other tricky materials shows how dexterous they're becoming. Major milestone. Exciting stuff!

reportingsjr• 3 years ago

This sounds similar to some of the work that google has done, such as PaLM-E: https://blog.research.google/2023/03/palm-e-embodied-multimo...

Very exciting times in robotics!

modeless4• 3 years ago

This looks way better than PaLM-E because the robots they're using are more capable and the tasks much more complex. And they're doing the behaviors at the same speed a human does them while puppeteering the robot. The PaLM-E demonstrations were all shown in sped-up videos because they are agonizingly slow in reality.

galaxytachyon• 3 years ago

Ok here it is. An LLM with physical senses. They mentioned sense of touch and I think that is a big deal. You can teach me all you want about a new color with all the text description available and it would be worse than just show me that new color.

In the same way, letting an AI actually touch and interact with the world would do wonders in grounding it and making sure it understand concepts beyond just the words or the bytes it was fed during training. GPT4 can already understand images and text, it should not be long until it takes care of videos and we can say AI has vision. This robot from toyota would have touch. We need hearing and smelling and then maybe we will get something resembling a true AGI.

nradov4• 3 years ago

There's no reason to expect that such advances will get us closer to a true AGI. I mean it's not impossible, but there's no coherent theory or technical roadmap. Just a lot of hope and hand waving.

I do think that this is an impressive accomplishment and will lead to valuable commercial products regardless of the AGI issues.

criddell3• 3 years ago

true AGI

What is that? Most humans have general intelligence, but do other apes? Do dogs? A quick google search suggests that the answer is yes.

If that’s the case, then this approach may indeed yield an AGI but maybe it’s not going to be much better than a golden retriever. Truly excellent at some things, intensely loyal (I hope), but goofy at most other things.

Or maybe just as dogs will never understand calculus, maybe our future AGI will understand things that we will not be able to. It seems to me that there’s a good chance that AGI (and intelligence in general) is a spectrum.

mxkopy• 3 years ago

"True AGI" is often used in a way that means "human-like intelligence, but faster, more consistent and of greater depth". In that case, knowing that embodied agents are the way forward is quite trivial. We've known for a long time that the development of a human brain is a function of its sensory inputs - why would this be any different for an artificial intelligence, especially if designed to mimic/interface with a human one?

xwdv3• 3 years ago

Why imitate human senses? AI should be able to reach out and touch radio waves, and interpret their meanings, much how we interpret the meaning behind gusts of wind.

TaylorAlexander2• 3 years ago

We can do both, but clearly the senses that mammals have are well adapted for existence on Earth.

JumpCrisscross8• 3 years ago

reach out and touch radio waves

At the point we're describing "touching" massless particles, we might as well say that's what our retinas do. In terms of novel senses, some kind of gravitometric sense would be neat. LIGO-on-a-chip and all.

consumer451• 3 years ago

Ok here it is. An LLM with physical senses.

See: Pieter Abbeel & Jitendra Malik

https://www.therobotbrains.ai/copy-of-jitendra-malik

1970-01-01• 3 years ago

Toyota hinting at AGI in 2 years time? Yes, I'm skeptical.

"Siri, make me a sandwich"

"You're out of bread, would you like me to bake some bread and then make your sandwich?"

isn't going to happen by 2025.

reaperman• 3 years ago

I can honestly see this by 2030. Just not reliable or safe enough for a consumer product.

There will be a gap between “can be done reliably in testing” (probably doable by 2030 even in a randomly chosen house that’s not part of the test set) and “safe enough for consumers, children, and pets to not get injured by the movements of a robot who is strong enough to lift and carry a 40lb bag of flour or 40lb laundry basket”.

That safety gap for “co-working” robots will be very difficult to close enough for the CPSC to be satisfied and also avoid expensive class-action or individual lawsuits.

anticensor• 3 years ago

You forgot the sudo: https://xkcd.com/149

moffkalast4• 3 years ago

RemindMe! 2 years

Ah wait, this isn't Reddit. Well anyhow we'll see I suppose. My money would be more on unfreezing and reheating bread instead, dough rising takes too long for practical sandwich creation.

hammock• 3 years ago

We already have robots that make sandwiches. And robots that make bread. And robots that order sandwiches and bread. And robots that deliver sandwiches and bread.

So maybe it’s not one robot, but 3-4 robots that interface with each other.

stuckinhell• 3 years ago

This seems like world changing technology, I can't believe these robots can learn complex motions just by watching a teacher.

modeless4• 3 years ago

They don't learn by watching. That would be another breakthrough entirely. They learn only by physically completing the task under the complete control of a puppeteer. Still very cool though.

micromacrofoot• 3 years ago

learning through a shared experience!

jsemrau• 3 years ago

My understanding is that the puppeteer records the same movement over and over again and this dataset of trajectories (pose, speed, acceleration) is then "diffused".

kelseyfrog• 3 years ago

Wake me up when a robot is the one teaching the other robots. The bottleneck in requiring human instructors makes this basically a no-go for me.

dpeckett• 3 years ago

In the linked TRI video they hint they are working on building generic dexterity models. If that proves possible, maybe oneshot/fewshot learning of dexterous behaviors isn't farfetched.

JoeAltmaier2• 3 years ago

Once a robot is trained, it's training dataset can be copied to other pretty quickly? So that's 'one robot teaching another'.

giardini• 3 years ago

I don't see any breakthrough here. I think it's simple marketing: "We're among the first using a variant of LLMs to enhance robotic behaviors."

The first pancake sequence starts with the pancake half off the baking surface and in danger of falling to the floor. So the device pushes the pancake back onto the baking surface. Good enuf if the pancake is "stiff" (already cooked and rigid). So hope you like your pancakes well-done!

And the pancake-flipper doesn't flip the pancake - it slides the spatula under the pancake and then, just like a robot, rotates the spatula until the pancake drops. In any case, fuggeddabout "eggs over easy".

danielbln2• 3 years ago

I don't see any innovation here. I think it's simple hype: "We're among the first to use a variant of these 'horseless carriages' to enhance mobility."

The first jaunt starts with the vehicle half off the pathway and in danger of colliding with a fence. So the operator manipulates the wheel to guide the contraption back onto the path. Fine enough if the path is "level" (already flattened and firm). So hope you enjoy your rides on well-paved roads!

And the operator doesn't truly control the vehicle - they merely grasp the steering device and then, just like a carriage driver, rotate the wheel until the vehicle alters its direction. In any case, fuggeddabout "tranquil and fragrant travels".

NalNezumi• 3 years ago

Cool to see Russ Tedrakes recent work! His online course Underactuated Robotics is a very good course to get a grasp on the complexities faced in robotics.

It's exciting to see someone with a bit more deeper knowledge than "Flex tape slap LLM on robotics" featured here, which is majority of Robot Learning work upvoted on HN.

There's more to it than just language learning to be solved before we can have proper embodied agents in the chaotic real world.

zwieback• 3 years ago

Dr. Ruth might have some ideas what other skills robots should learn.

colordrops4• 3 years ago

I'm pretty ignorant of state of the art robotics and had assumed for years that approaches like this were used, e.g. by Boston Dynamics. Surprising to see that it's a new thing.

jononomo5• 3 years ago

Has Boston Dynamics ever done anything other than produce a viral video approximately every 5 years? Mr Beast produces a viral video twice a month.

colordrops4• 3 years ago

Have you not observed the content of those viral videos? Who else is doing robotics at that level? Extremely complex, natural bio-mimicking movements I've not seen anywhere else.

lbussell• 3 years ago

If I had to guess, we’re less than 5 years away from seeing real-life C-3PO.

nradov4• 3 years ago

As long as you don't mind the extension cord.

modeless4• 3 years ago

I'll step over an extension cord all day long if it's powering a robot that does the laundry and the dishes and the house cleaning for me.

seanr88• 3 years ago

I would disagree. All of what we are seeing from this latest surge in AI is essentially jumped up predictive text. To get to C-3P0 there is a whole additional layer of Intelligence needed. C-3P0 can make plans and execute those plans. This latest wave cannot reason about the world, it does not know or understand the world it just assembles words (and here motions) in a way that we value. It is not planning anything.

modeless4• 3 years ago

That's the easy part. Making high level plans is trivial compared to the fine motor control and dexterity and sensing necessary to do things like turn a T-shirt inside out or install a fitted sheet or crack an egg or whatever. If you give me a robot with all the fine motor skills necessary for all the steps to cook a meal but no planning capability whatsoever, I'll have that robot cooking your dinner within a year.

moffkalast4• 3 years ago

I think you're giving C3PO too much credit for the bumbling idiot he usually is when on screen. Well aside from calculating the odds of successfully navigating an asteroid field, but I'm sure GPT 4 will let you know what that is just as easily, as well as translating any language into any other language which is supposedly 3PO's whole schtick.

Also:

can make plans and execute those plans

https://github.com/antony0596/auto-gpt

cannot reason about the world, it does not know or understand the world it just assembles words

They can reason a surprising amount given that they only work with text. With vision/actuation encoding there's potential for far more. Remember, it doesn't have to be smart or conscious as long as it gets the job done with cold hard statistics while just appearing as such. A submarine does not swim but crosses the ocean just the same.

epolanski3• 3 years ago

C3PO is a translator droid that is basically stupid at everything else (other than math or facts listing, as he's a robot).

So yes, I think he seems a reasonable target.

edgarvaldes2• 3 years ago

Considering that C-3PO waas verbally very competent, but somewhat clumsy, I agree. Even the "old" Boston Dynamic robots are more agile than 3PO.

darkerside• 3 years ago

Hmm, perhaps that clumsiness was a programmed affect designed to disarm those he interacted with. After all, as a translator and master of protocol, outshining his master in any way would be quite against his programming.

jononomo5• 3 years ago

If I had to guess, we're over 50 years away from seeing a real-life C-3PO -- hopefully before I die. I think self-driving cars are even further away than that, however.

rcarr2• 3 years ago

This is getting pretty close to how I think we get to the general purpose humanoid robot. This is how I see it playing out:

- You have your Boston Dynamics style humanoid robot at the job site, lets say it's a bricklayer for the purposes of this example.

- You have a human somewhere offsite in an open room with an omnidirectional treadmill floor, and cameras and depth sensors positioned all around the room. They're wearing a Hollywood style motion capture suit and have a VR headset on so they can see what the humanoid robot sees through their cameras.

- The human then acts as they would on site, walking up to the pile of bricks, picking them up, placing them etc. The robot moves in real time on the job site, mimicking whatever action the human performs. I don't know if you'll need props to do this properly or if the muscle memory from years on the job will be enough for the humans to get the motions right.

- You log all the data. You then have someone watch through the video stream, labelling each action that is being performed.

- You run it all through a machine learning algorithm, until you get to the point where you can just send the architectural plan to the robot and essentially say "Build this wall for me".

tshaddox• 3 years ago

And yet, as far as I know we don't even have mature implementations of this even for machines with much courser movement whose movement are controlled by a loose physical mapping to the human operator's movements. Things like excavators with dual joystick controls.

ramesh312• 3 years ago

This is precisely how Comma and Tesla do self driving.

Lemmi• 3 years ago

In short term perhaps but not long-term.

I believe they will send out a team to digitize the job site.

The team will then create a digital twin.

The architect will map everything into this twin.

The computer system will simulate the build steps.

Robots will be brought onto the job site and get a finetuned model (if necessary) and will build it automatically.

mortureb3• 3 years ago

No, long term we somehow get something magical like LLMs that swoop in and magically pull off these tasks.

0xbeefcab• 3 years ago

The first 3/4 points were all almost exactly from the 2008 movie Sleep Dealer to the point where I thought you were referencing that

rcarr2• 3 years ago

Never heard of it before, any good?

ramesh312• 3 years ago

Yes, it's called reinforcement learning.

pests• 3 years ago

I once read a novel or short story with this concept. The robots were piloted (on-site) for training and then let loose on their own. I forget the plot besides that though or who may have wrote it.

martin_henk• 3 years ago

Google kuffner

neom• 3 years ago

Website isn't loading for me, but found the video on the Toyota research youtube:

https://www.youtube.com/watch?v=w-CGSQAO5-Q

aniken• 3 years ago

Thanks. In the video around 2:40, he describes it as a “kindergarten for robots”, that’s an interesting way to think about it. I wonder if it would be possible to crowdsource the training of new tasks with a standard training tech protocol? That way you bid on the task you want and someone who solves it gets a bounty and everyone benefits? The point is there’s a long tail of tasks and a centralized lab probably can’t do them all.

BillSaysThis3• 3 years ago

This seems pretty cool. But I'm not clear how someone can be a (full-time) professor at MIT and also be a (full-time) vice president at TRI. I've seen this kind of two-job situation before but never understood how it's practical, unless the person works 70+ hours a week.

epolanski3• 3 years ago

You probably still 40 hours week or less, but you're so expert in those fields that your 10 hours of work cannot be replaced by somebody else working full time.

In software engineering terms, you would gladly pay a full good salary and give a good role to John Carmack to work on your projects 6/7 days per month anyway, because he's John Carmack.

BillSaysThis3• 3 years ago

I guess but that feels more applicable to an individual contributor role, albeit a very senior one. VPs I've known in my 30+ years in software are fully booked all day every day. A professor maybe gets to count his outside work as part of his research efforts but what about say class load, mentoring grad students, serving on committees, and the like?

anuvrat1• 3 years ago

For anyone interested, here[0] is the YouTube channel of Russ Tedrake, which has:

    - 6.4210 (2023) Robotics Manipulation
    - 6.8210 (2023) Underactuated Robotics

[0](https://www.youtube.com/@underactuated5171)

JoeAltmaier2• 3 years ago

Pretty fumble-fingered robots they were using. All you needed for industry up til now.

Now the can learn, quickly, perhaps a more dexterous robot with flexible digits etc will become the norm.

UberFly• 3 years ago

That shot of it spreading goop on bread was amusing. This video feels like an Onion production to me for some reason.

extr• 3 years ago

Yeah lol I had the same thought. I almost expected it to zoom out and be a guy with his hand up the robotic arm.

nico2• 3 years ago

I wonder if giving cars a sense of touch is what will ultimately be the key to enable full autonomous driving

Are there any cars out there that have something like a sense of touch and with it can sense the road or things that they crash into?

jpadkins• 3 years ago

I think cars already have a sense of touch with the road with traction control sensors.

For everything else isn't touch too late, especially at high speeds? the point is to avoid the crash.

serf• 3 years ago

there is a lot of work to be done with gauging tire traction.

it's (generally) done via comparison of RPM, but one can imagine some future tire that could realistically give feedback about how the weight is actually sitting on the tire, which portions are heating disproportionately, and which parts are experiencing friction anomalies versus the rest of the tire.

that's the direction it would have to kind of go towards to reach the dimensionality and resolution of human proprioception.

that kind of stuff is approached on F1 test rigs and cars. multiple IR laser therms and lidar to compare temperature across the tire at speed as well as judging deformation.

Kerb_• 3 years ago

Most cars are able feel when they crash into things. Usually that means it's too late to react to it.

https://wellsve.com/products/electrical-lighting-body-system....

nico2• 3 years ago

In production yes, but it’s like kids, you don’t want them to fall and hurt themselves, however that’s how they learn

Likewise, maybe self-driving cars need to get a few bumps and scratches to learn not to crash

numpad03• 3 years ago

Cars with electric power steering has steering force sensors, as well as accelerometers, though cars generally don't have central computers - they're networked collection of feature computers in a topology somewhat like a sugar molecule.

jonplackett• 3 years ago

OK, so those nice safe real world jobs? AI is after them too now…

fhd2• 3 years ago

Might just take a while for it to be economical for lots of jobs. The amount of humans is increasing, the amount of natural resources, different story.

valine3• 3 years ago

There’s no real dignity in work that can easily be done by a robot. A lot of these jobs make people miserable anyway, maybe we shouldn’t be fighting so hard to keep them.

GedHawk• 3 years ago

All Labor has Dignity

macintux• 3 years ago

We’re back to the recurring question: what economics model will work without workers?

jimbokun• 3 years ago

I don't think that's the issue.

The issue is training humans fast enough to stay one step ahead of the robots and the LLMs.

willio582• 3 years ago

"There’s no real dignity in working to make a living." FTFY

/s But seriously, while I think humans will always find some meaning in work, there will come a day when that work is no longer required. Or at the very least, work will look so different that it will be unrecognizable to us currently. I think "work" will look more and more like "art" in the year 4000 for example.

The idea of a person sacrificing some of their time to enrich the person at the top of a company already sounds bad, but we all accept it must happen so we can afford to live. But when robots and AI takes over labor in a significant way, what will be left for those humans that remain? And let's say we converge on something like a universal basic income to fix this, where will humans find meaning?

randomdata2• 3 years ago

A lot of these jobs make people miserable anyway

A lot of these jobs also make people happy, though. That is why the loss of manufacturing in particular was such a blow to Americans. People love manufacturing – to the point that having small-scale manufacturing facilities in one's garage so that one can keep on manufacturing things on the weekend is a dream of many.

JumpCrisscross8• 3 years ago

those nice safe real world jobs?

"American manufacturers use far fewer robots than their competitors, in particular in Taiwan, South Korea and China" [1]. And specialized manufacturing is in a permanent skills shortage. More automation may boost employment and wages for blue-collar workers. Particularly if such kit enables new entrants to challenge incumbents.

[1] https://www.wsj.com/economy/american-labors-real-problem-it-...

flangola72• 3 years ago

This will effect jobs that are not manufacturing. Anything that requires fine motor control in noon uniform environments

seydor• 3 years ago

people were hoping it would replace labor intensive jobs, instead ai seems to replace the white collars first

mr_toad• 3 years ago

People underestimated how much brainpower is involved in simple mechanical tasks, and they probably overestimated how much is required for things like object recognition. Even simple tasks like lifting weights involve motor learning.

dilawar• 3 years ago

We'll borrow the work from future to keep people busy. There will always be plenty of work. IlIn addition to organic vs other food, there will be stall of human grown vs machine grown food etc etc..

When first tractor arrived in my village, when grandfather joked that all the landless labourers will dies of hunger now since there won't be work for them. Manual ploughing reduced but a number of other work became routine. These days it's hard to find labour in my village (western UP).

mortureb3• 3 years ago

They all seem to be laborers in Karnataka now. Almost all laborers seem to be from UP and Bihar.

Tade0• 3 years ago

Populations of middle and high income countries are aging so there aren't that many takers for these jobs anyway.

eachro• 3 years ago

As far as I understand, getting robots to do reasonable things when faced with out of distribution experiences is super hard. How do they manage this?

belter2• 3 years ago

"Visuomotor Policy Learning via Action Diffusion" - https://news.ycombinator.com/item?id=37581866

paulsutter2• 3 years ago

It's on

Really - start thinking carefully about what you’re working on. Until now the new AI have been language only, not spatial. That’s over

zzzeek2• 3 years ago

why, are a lot of Hacker Newsers pursuing short order cook careers?

paulsutter2• 3 years ago

Essentially, yes

How can we be sure?

bluerooibos• 3 years ago

Been thinking about this a lot over the last year. It's becoming very difficult to plan for the long term future, career-wise.

zzzeek2• 3 years ago

we use the term "Large language models" because the entire world wide web, library of congress, etc. have produced a truly vast amount of written content such that LLMs have massively large datasets available for learning. That's what I understand to be the "large" part. We have an unbelievable amount of written content available from a huge amount of datasets (both public domain and more questionably public domain).

When this video refers to "large behavioral model", where's the "large" part? Where are they getting a similarly "large" amount of behavioral input data? It looks like they have a big lab with a few dozen people modeling behaviors. that's great but it's not like this number of people could produce as much content as all of digital written content.

tamimio• 3 years ago

I wouldn’t call it breakthrough, a step forward yes, but nit a breakthrough, since these movements are recorded and then replicated, it will be a breakthrough if for example you only teach the robot how to hold a spoon, and then it can figure out by itself how to use it.

momofuku2• 3 years ago

Having been a member of the robot learning community both in grad school and now in industry, I'd actually like to rightfully attribute something here since it seems that TRI is (deservedly so, I will agree wholeheartely) receiving most of the praise:

The core of these advancements are powered by Diffusion Policy [1], which Prof. Shuran Song's lab at Columbia (before she moved recently to Stanford) developed and pioneered. I'd suggest everyone to view the original project website [2], it has a ton of amazing real world challenging experiments.

It was a community favorite for the Best Paper Award at the R:SS conference [3], this year. I remember our lab (and all other learning labs in our robotics department), absolutely dissecting this paper. I know of people who've entirely pivoted away from their projects involving behavior cloning/imitation learning, to this approach, which deals with multi-modal action spaces much more naturally than the aforementioned approaches.

Prof. Song is an absolute rockstar in robotics right now, with several wonderful approaches that scale elegantly to the real world, including IRP [4] (which won Best Paper at R:SS 2022), FlingBot [5], Scaling Up Distilling Down [6] and much more. I recommend checking out her lab website too.

[1] https://arxiv.org/abs/2303.04137

[2] https://diffusion-policy.cs.columbia.edu/

[3] https://roboticsconference.org/program/awards/

[4] https://irp.cs.columbia.edu/

[5] https://flingbot.cs.columbia.edu/

[6] https://www.cs.columbia.edu/~huy/scalingup/

mdonahoe• 3 years ago

our lab

Which lab are you referring to?

momofuku2• 3 years ago

I meant the academic lab that I was a part of while in grad school (would like to keep that anonymous for now, it's a smallish community).

Btw, I was at R:SS 2022 and meeting the Skydio autonomy team was one of the highlights of my career as a robotics engineer!

srsqsonyl• 3 years ago

You may as well credit the information theorists, mathematicians, and physicists who laid out the fundamentals that brought us here.

They died before hardware achieved their decades old visions. Not much of this work is net new description, moreso normalizing old descriptions with observation now that we can actually build the old ideas.

3seashells• 3 years ago

Prompt: ChatGpt write a generic anti academia rant against robotics research

snor• 3 years ago

It should be noted that Diffusion Policy (not to mention IRP) was also apparently joint work with TRI.

yellow_lead• 3 years ago

To be fair, they do credit Professor Song and the paper you linked. TRI is also listed as a collaborator on the paper.

Diffusion Policy: TRI and our collaborators in Professor Song’s group at Columbia University developed a new, powerful generative-AI approach to behavior learning. This approach, called Diffusion Policy, enables easy and rapid behavior teaching from demonstration.

prox• 3 years ago

Interesting that we have a genius robotics Dr. Song for real vs Star Treks Dr. Soong :)

dotancohen2• 3 years ago

I'm glad that she skipped Dr. Soong's ambition that preceded his work on robotics!

ben_w2• 3 years ago

My dad, who worked on military IFF before he retired, met someone in the UK intelligence community whose actual name was James Bond.

Or so he said…

3abiton• 3 years ago

It seems some researchers in her lab were also involved with Toyota.

tomp• 3 years ago

Can anyone ELI5 (well, or, "explain like I'm someone who understands how autoencoders, transformers & convolutional networks work") diffusion?

What makes it work so much better than alternatives mentioned above?

Paluth• 3 years ago

I haven't read the paper on Policy Diffusion yet so I don't know what they do differently. But I can ELI5 image diffusion models, like stable diffusion. Essentially you add random noise to an image, and the ask the model to predict the noise, such that if you remove that noise detected by the model, you obtain the original image. After the model has been trained enough, int the noise removal task, you can pass just random noise, ask the model to remove noise from the noise only image, then remove a little bit of the noise the model suggested, and do it again. And again, for multiple steps, eventually all the noise is removed and you end up with an image "dreamed" by the model from random noise. You can also condition the noise removal with things like text or other images to guide the noise removal process toward a certain target image.

kneel• 3 years ago

Sex robots!

Real world adoption will catalyze with the company brave enough to train these behaviors in.

AtlasBarfed• 3 years ago

Jokes aside, IMO the next great "appliance" is the personal robot. It basically follow you everywhere, witnesses/records everything you do (for only your benefit/access hopefully), carries/lifts/moves things for you, fetches things for you, holds things for you, can do emergency services. Can walk a mountain with you.

There was once a cartoon where a robot is instructed to protect it's "primary" from rain and proceeds to swat away all the individual drops of rain from it. A ridiculous parody, but... think about it.

But yeah ... sex robots will absolutely be a thing. Might HAVE to be the thing for the female clientele with insemination abilities to continue the existence of the human race.

midoridensha3• 3 years ago

Marriage rates are already falling; when sex robots get good enough, they'll collapse entirely. I'm not even sure if this is a bad thing.

jdkee• 3 years ago

"“Our research in robotics is aimed at amplifying people rather than replacing them,” said Gill Pratt, CEO of TRI and Chief Scientist for Toyota Motor Corporation."

Why do CEOs make public statements such as this when the goal is humanoid robots to replace human labor, particularly in countries with declining birthrates?

mortureb3• 3 years ago

Softening the blow. No one wants to hear a CEO say the robots are coming for their jobs.

bman_kg• 3 years ago

Most of you guys have deep and practical experience with robotics and robots, for me anything that a robot is demonstrating by doing is a magic and scary thing...now I am having mild paranoia due all these progress in LLM and now these activities done by robots...What future might roll in front of us? What are your opinions on that?

ChatGTP2• 3 years ago

Star Wars ?

DaiPlusPlus• 3 years ago

Star Wars is set in the distant past, not the future.

SanderNL4• 3 years ago

This will become increasingly normal, but it’ll take a while before massive impact (ie taking your and your friends’ jobs). I’m not sure of the timeline. Humans are slow to adapt, but in 20-30 years I think the pace will pick up.

I’m not too worried about the current generation, but my kids. Don’t know what to tell them TBH.

I guess it’ll all be fine though. We techies tend to have a paranoid streak which isn’t becoming.

Animats2• 3 years ago

Google was doing something similar, and it was on HN about a month ago.[1]

I wonder how much force feedback they have. Is that big round squishy thing in the videos sort of like a big finger, with lots of pressure sensors? People have built area pressure sensors before, as far back as the 1980s, but nobody knew what to do with all that data back then. Today, too much sensor data is far less of a problem.

I once took a crack at this problem by equipping a robot arm with an end wrench. The idea was that it would feel around for a bolt head, get the wrench onto it, and turn. A 6 DOF force sensor is enough for that. But this was pre deep learning, and I didn't get very far. although I did build the wrench robot setup.

[1] https://news.ycombinator.com/item?id=37167698

v9v• 3 years ago

The squishy thing is essentially an inflated balloon with a camera inside, which monitors the balloon's deformations: https://punyo.tech

Animats2• 3 years ago

Nice. Cameras are so small and cheap now you can do that. Computing optical flow is cheap now, too. This is something you can make from cell phone parts. Early designs were sheets of resistive plastic between crossing grids of flexible printed circuits.

gtzi• 3 years ago

When it comes to giving robots a "sense of touch", you may want to check out Bota Systems (botasys.com) – already deployed in hundreds of research and industrial robots globally.

dinvlad• 3 years ago

If this is using gen-AI, I wonder how it "hallucinates" - would that result in jerk movements that could break something or hurt someone?

Toyota Research claims breakthrough in teaching robots new behaviors

Related Stories