Towards robot accelerators, democratizing hardware acceleration in robotics

https://news.accelerationrobotics.com/towards-robot-accelerators-democratizing-hardware-acceleration-in-robotics/

by vmayoral • 4 years ago

59 45 4 years ago

news.accelerationrobotics.com

meowtastic2• 4 years ago

Looks impressive to a layman like myself. I'm guessing it addresses the global shortage of single-board computers like Raspberry Pis right? Curious to know how long it'll take to do that as well.

vmayoral7• 4 years ago

Well, yes but not only. There's only so much you can do with Pis and other SBCs that offer a CPU-centric compute solution. Robots are real-time systems and CPUs don't excel at that.

When optimizing dataflows for lower latencies and higher throughputs, you typically seek specialized compute architectures and that's wherein GPUs and FPGAs come in. That's what this work really enables.

meowtastic2• 4 years ago

I see, thanks for the explanation!

archivisti• 4 years ago

comparison using Jetson TX2 or Xavier could have provided better insight instead of Nano 2G which struggles even in basic task but otherwise looks quite promising

vmayoral7• 4 years ago

It was benchmarked against Jetson AGX Xavier as well. See https://news.accelerationrobotics.com/hardware-accelerated-r....

Results using AGX Orin are in the pipeline and will come out soon as well.

j-pb5• 4 years ago

While hardware accelerators in robotics are very cool, and needed. The fact that the entire workflow is ROS2 centric is a bummer. FPGA and Accelerator Programming is already a huge PITA, and despite it's success ROS is really one of the worst frameworks I've ever used in terms of DX. It makes an already hard problem, needlessly harder.

I wouldn't be surprised if they developed their own FPGA programming environment based on C++ and CMake, and all kinds of other custom language and tool shenanigans.

octocop• 4 years ago

Do you mind sharing some of the issues that you are having with ROS2? I'm also using it and i find it quite straight forward.

pabs3• 4 years ago

What would you use instead of ROS?

thendrill2• 4 years ago

arduino or rpi with something like mqtt, PyRemoteObjects on top , just a few off the top of my head.

cmontella4• 4 years ago

Indeed, ROS set out to create an ecosystem of reusable robot software infrastructure, but a monoculture has grown instead.

I started building a ROS alternative called Mech (see my profile) after becoming frustrated with ROS. It’s getting there but not ready for prime time yet.

To answer the sibling comment, my biggest problem with ROS is how inscrutable it is for undergrads. I’ve witnessed students struggle to even install it while following directions step by step. It is a huge mess of incidental complexity, layering the complexity of C++ build systems on top of an IPC system on top of an expansive middleware.

The learning curve is so high that when I spend a semester teaching robotics, students still struggle with ROS basics at midterms. The course turns into more about learning ROS rather than learning actual robotics.

I’m hoping one day Mech will work along side and in conjunction with ROS to provide an easier way of doing things for those who want to get right to the heart of robotics.

vmayoral7• 4 years ago

Use ROS 2. The misconceptions about ROS have been mostly addressed with ROS 2.

Check out industry adoption of ROS 2 in various industries including automotive, healthcare and warehouse automation. ROS 2 is (and will remain as) the de facto standard for robot application development.

iancmceachern3• 4 years ago

Custom written stack

thendrill2• 4 years ago

I have to agree with you on this one.

Yes Ros looks hood on paper but the moment you open it, it just screams academia and academics.

Also yes FPGA is faster for in general. But who is gonna program it ?

vmayoral7• 4 years ago

Move to ROS 2 :)!

Regarding who's going to create the accelerators, enablers like this allow any ROSser to jump into creating their own accelerator from C++ by purely adding some metadata into their CMakeLists.txt files (which every ROS package has already).

On top of that, organizations are creating professional-grade kernels meant for production. NVIDIA and AMD are investing in that direction and so are we at Acceleration Robotics. Here're a few of the first ones announced: - Accelerated perception stack (ROS 2 API-compatible) https://accelerationrobotics.com/robotcore-perception.php - Accelerated coordinate system transformation (ROS 2 API-compatible, tf2) https://accelerationrobotics.com/robotcore-transform.php

pclmulqdq• 4 years ago

You have to remember that in a professional setting, the alternatives to ROS are atrocious pieces of crap like LabVIEW.

From there, ROS is a step up! Both are programming environments designed by mechanical engineers, who care more about making easy problems really easy than making hard cases any better (after all, to a mechanical engineer, hard programming problems are really hard on their own).

vmayoral7• 4 years ago

That's exactly what this work does while remaining ROS 2-centric (with C++, CMake extensions and other friends). Goal in this research was to provide a way for the path of least friction for roboticists.

There's a huge gap between robotics development (more sw-centric today) and the average hardware engineer in big semiconductor companies. Totally different perspectives and viewpoints. We felt bridging the gap was necessary.

iandanforth2• 4 years ago

Did you know it takes almost a hundred ms for a signal to go from your fingers to your brain? Did you know that the brain is constantly predicting the future to make up for its slow reaction time? Did you know even small computers are really really fast? If you did then you'd probably guess that initiatives like this are a waste of time and money. The difficulty is in the software, not the hardware, or the compute layer. With good algorithms our current crop of robots are already fully capable of producing trillions more in economic value than they do today.

We do not need hardware acceleration in robotics. Can we use it if it exists? Sure, but do we care about "democratizing" it? No.

There is a place for hardware acceleration in robotics that is making a huge difference and that's in simulation. GPUs, the once and future king, make training robust neural networks possible and fast. If you're an investor reading this who needs a reminder, bet on software.

fxtentacle• 4 years ago

Did you know that for predicting what you'll see 100ms into the future you need a level of real-time 3D recognition and world knowledge that we haven't ever solved acceptably, not even offline and parts of it in isolation?

While I agree with you that in 50 years, advances in AI will make fast hardware obsolete, it may well be that in 5 years, fast hardware will be able to avoid problems that we cannot yet solve in software. That would mean this company has 45 years to cash in before they get replaced by software.

MichaelCollins2• 4 years ago

The human brain does what it does with roughly 20 watts, if we take the popular figure of the brain taking 1/5th the power of your whole body and the average human body burning 100 watts.

How much neural net can you run on silicon with 20 watts?

mynegation• 4 years ago

TBF it takes human brain anywhere from several months to decades to train for its skills. Plus several hundred million years spent on pre-training.

vmayoral7• 4 years ago

I agree that one of the major aspects to improve in robotics is software. There's still a lot do there and many are working towards it. Most leading initiatives around ROS 2. I also believe that we need faster robotics simulation, and hardware acceleration will be valuable there, not only with GPUs, but also other accelerators (e.g. FPGAs outperform GPUs in many benchmarks both performance and energy consumption-wise).

I strongly disagree with the rest and your comments indicate a clear lack of (hardware) understanding. I often encounter this in the AI world. Where folks building ANNs just "forget" that this computational abstraction only "grew in popularity" (there were previous accelerators though) when CNNs got implemented as an accelerator in a GPU empowering newer results. The same's likely to happen in robotics.

Robots are real-time systems. Meeting time deadlines in their computations is the most important feature. (robot) Behaviors often take the form of computational graphs, with data flowing between compute abstractions (Nodes), across physical networks (communication buses) and while mapping to underlying sensors, compute technologies and actuators. ROS enables you to build these computational graphs and create robot behaviors by providing libraries, a communication infrastructure, drivers and tools to put it all together.

From a more technical compute architecture perspective, ROS 2 presents an event-driven programming interface. The resulting computational graphs built with it are "event-driven software architectures". Mapping these event-driven software architectures to hardware using CPUs leveraging classic control flow architectures (von Neumann architectures) leads to various issues. A key challenge in applying classic event-driven programming is that CPUs hardly provide real-time and safety guarantees. The de facto strategy in industry to meet the timing deadlines is a laborious, empirical, and case-by-case tuning of the system. This “whack-a-mole” approach is being realized by some, but unsustainable and hard to scale due to the lack of a hardware-supported timing-safe even driven programming interface. This is where accelerators come in. As already adopted in other industries including aerospace, automotive and healthcare, through the creation of custom compute building blocks (accelerators), a hardware/software co-design strategy provides clean behavioral specifications, describing clearly its guarantees in terms of timing (in other words, avoiding memory-centric von Neumann architectures).

Hardware is important. Note the current scarcity of semiconductors, which is one of the drivers of this research. Note also that creating custom accelerators for robotics application not only can lead to performance improvements (i.e. software that runs faster!) but also to more deterministic responses (which affects the downstream pipeline of software!). Rephrasing Alan Kay's quote, if you're serious about robotics, you should care about hardware ;).

iandanforth2• 4 years ago

Yeah, started my own robotics company, early engineer at another that sold for >$200M. You seem to have misinterpreted my statement of "hardware is already good enough" to mean "hardware is unimportant."

LeifCarrotson3• 4 years ago

As a controls engineer who has implemented and assisted with dozens of robotic manufacturing cells, the problem isn't the motion planning or compute, it's legacy product support and ease of onboarding new hires.

Any given cell may be in production for 10 years - some only for 5, but some for 20 or more - and you need <24 hour response times to any mechanical issues that may arise over that duration. Also, one can't expect that the original programmer will be available to diagnose and debug issues that arise in the field after a year or two, so you need a large pool of technicians and maintenance personnel that can understand the software and be productive in the first hour after being introduced to the machine. None of this is valued by academics, instead, the fundamentals of the process of a single lab trying to prove merit as PhD students and, once that credential is acquired, move on to greener pastures is anathema to both of these ideas.

The problem is that academia is working in ROS (now ROS 2), writing Python and VHDL, while the real work in the industry is being done in proprietary vendor tooling that's backwards compatible with training from the 80s. They're proudly advertising touchscreens and full-color 5.7" displays on their teach pendants, as if their customers were still 20 years in the past.

Yes, academia, I do want sub-millimeter repeatability, and flawlessly smooth motion planning. I want collision detection that accommodates both the inertia of the arm itself and any end-of-arm tooling, including cable harnesses with distributed loads. I want automatic singularity avoidance and trivial point and frame manipulation. I want 24-bit encoders for resolution, and I want angular velocities to the moon, and to make those work together I need servo loop rates and high-speed skip responses at 1, 2, or 10 kHz. I want 4D stop position prediction to avoid intersection of the robot arm with complex assemblies and safety zones imported from CAD. I want all these things cheaper and faster, with more storage.

But I need these to be accessible by Billy Joe, whose only credentials that got him the job in the maintenance department is that he helped out his daddy working with a welder and an old clapped-out Bridgeport on the farm growing up. Billy is probably my best tech on 3rd shift, and he's never learned to type on a full-size keyboard, just his smartphone... and on a robot teach pendant.

cs7022• 4 years ago

After reading this, I'm persuaded this space would benefit greatly from having a few dominant open-source software frameworks and a few dominant open-source hardware standards, making product support and new-hire onboarding much easier for vendors of all sizes. Alas, I recognize that no established vendor in the space would ever want to see the emergence of industry standards. :-P

bismuthcrystal• 4 years ago

What this field really needs is a massive push. The reason it always fail to deliver is that the incentive system is screwed. We need something like a Manhattan project capital allocation to a few very talented people to make progress. Building endless frameworks and trying to get thousands of people to cooperate on high level decision making dooms the whole endeavor. The risk for product development is too high for private industry to take and academia can't deliver products.

LeifCarrotson3• 4 years ago

It definitely would. RoboDK is one product that is moving in the right direction.

In the PLC space, Beckhoff controllers using commodity x86/64 hardware and standardized IEC 61131 programming languages are gobbling up Rockwell and Siemens' market share.

Given they're perpetually about 20 years behind the times, perhaps this year someone at Fanuc, ABB, Kuka, and Yaskawa will catch up to the year 2002 and read Spolsky's admonition [1] that says:

Smart companies try to commoditize their products’ complements.

Stop trying to turn every division of your company into its own product center. I loathe paying $800/year to Rockwell for the privilege of reading "update your firmware" (and subsequently bricking my controller as an unwilling beta tester) on TechConnect. They're a PLC vendor, they should focus on selling PLC hardware, not making their customers hate them by nickel-and-diming them for software licenses and support contracts. They were down for FOUR DAYS this weekend for planned maintenance. FOUR DAYS!

Likewise, Fanuc should focus on selling servos and castings - robotic manipulator hardware - and stop trying to get the training division to turn a profit. If, instead of a 40 hour course costing $2,000, it was free, there would be more people able to use their $40,000 robot arms! If Roboguide software was free instead of $2,650, clients could more efficiently build more robot cells, and sell more yellow paint! Gah!

[1] https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/

iancmceachern3• 4 years ago

What you describe is ROS. The issue is that industrial manufacturers of things like arms or motion controllers all have their own proprietary stuff.

vmayoral7• 4 years ago

Well said. Robotics is indeed the art of system integration and such remains (and will) one the biggest tech hurdles.

We wrote about this, and raised-funding, and worked for years on it. Hell, we even created an extension of ROS for it called the "Hardware Robot Operating System" (H-ROS) that aimed to address many of these issues you describe using ROS as the common language and Programmable Logic (FPGAs) to deal with physical interfaces. More on this at https://ieeexplore.ieee.org/document/8046383.

Problem was that the market didn't really accept it. I still believe on the tech problem landscape in here. I'm just not so sure anymore if there's any real market/business for such a solution. After all, vendor lock-ins are great business, and generate business.

antoniuschan99• 4 years ago

What are your thoughts on OPCUA? That's the direction they're trying to get to having different vendors be able to talk to each other as it's all pretty much federated.

LeanderK• 4 years ago

I may be misunderstanding your comment, but as I understood it you are talking about industry failures academia can't do anything about. If industry is massively behind on outdated, closed environments and academia work's with different, open tools then that's not academias fault. Like:

it's legacy product support and ease of onboarding new hires.

I don't see how academia can help with this, this so very much an industry problem.

But I need these to be accessible by Billy Joe, whose only credentials that got him the job in the maintenance department is that he helped out his daddy working with a welder and an old clapped-out Bridgeport on the farm growing up.

While accessible interfaces and forms of human-computer interactions are researched by academia, they are not building products. Research has to be relevant, but it's still research. Those problems, they might be better addressed by a startup than academia, if that's even realistic. Academia can't solve a problem if it's not a research-problem.

iancmceachern3• 4 years ago

That's what he is saying. ROS is made, lead, by academia. He is saying that ROS isn't terribly useful in industry for the reasons he lists.

metadojo2• 4 years ago

you are leaving out a whole host of use cases bro. maybe in the past robotics was just for industrial use cases. but there are orders of magnitude other use cases that a ROS can help solve. simple things around the house , in the neighborhood, and other mechatronic integrations that would not properly called a robot. I would say the ones you are addressing with billy bob on the 3rd shift are like 1 percent of the cases. There are plenty of things you can actuate in the world that dont require sub mm repeatability. like what about tending my hydroponic vertical garden? theres nothing about the ubiquitous robotic future of humankind that needs any of the things you mentioned. thats more of an improvement on current use cases.

Towards robot accelerators, democratizing hardware acceleration in robotics

Related Stories