The software fixed the hardware

http://www.jwz.org/gruntle/roomba.html

by J3L2404 • 17 years ago

162 42 17 years ago

DannoHung• 17 years ago

So, does it actually "fix" the hardware by say, oscillating the wheel in such a way as to clean or jiggle the sensor, or does it update the firmware to compensate for the wheel sensor's spotty readings?

djcapelis7• 17 years ago

The entry itself speculates that the new firmware simply compensates for the spotty sensor(s).

djcapelis7• 17 years ago

This type of thing happens all the time with embedded platforms. Software fixes for hardware defects are common. When we were building robots invariably the electronics team would always wire some motor backwards. Since the motors had similar power in both directions they would always ask me to just write the code to run motors they wired the wrong way backwards.

It was simple enough to do, but it always feels a bit wrong.

Other types of software workarounds are possible though too. I once modified the acceleration curve of the robot (clamped down the jerk) so that there would be reduced stress on the poorly made drive train and under many failure modes this changed the robot from essentially dead in the water to workable.

It seems surprising how many issues you can solve with code if you aren't used to it. But these hacks are actually everywhere in a lot of embedded systems and allow teams to continue to quick, cheap and dirty work when needed that can get patched up later.

HeyLaughingBoy4• 17 years ago

You describe them as workarounds, but often this the intended design. If you only have manual control, then perhaps a driveshaft must be made larger and heavier for the case of the ham-fisted user. If the transmission is under software control, the software can be careful about accelerations to avoid overstressing the driveshaft.

The point is that when you consider the system as "mechanical, electrical and software" you get a lot more flexibility than if you think of it as "mechanical and electrical" but we do some stuff in software.

djcapelis7• 17 years ago

Certainly I agree the system is more complex and we could have designed it to work this way. But the stark reality was that these components were supposed to be wired a certain way in our design and the drive system was supposed to stand up to a lot more torque, as per the intended design.

So I think it is fair to call these particular fixes workarounds.

Your point that it is silly to look at these systems as anything other than integrated (often quite deeply) is well taken however. The fact that one can fix a mechanical defect using the software section of the system speaks volumes about just how integrated they are.

dkarl• 17 years ago

it always feels a bit wrong

It helps to think of it as "don't care" code rather than "error compensation" code. Which way should the engine be wired? It's cool; the software doesn't care. Just wire it up however. What order should I start these services? It's cool; they don't care. Just start them up however. As soon as your users stop caring about the stuff you can take care of, they'll have time to care about something else that's more important.

ramchip2• 17 years ago

And in this case, don't forget to do a #define INVERSE_MOTOR_POLARITY 1 or similar, so that the hack is clear and doesn't bite you when porting to another platform.

It's not a bug, it's a customizable feature :)

djcapelis7• 17 years ago

Well, the values actually sent out to the PWM were backwards. Thankfully I could just invert it at the last step in the code where it got sent and so while writing the actual code I could pretend everything was correct. It's hard to think of -127 going forward and 127 going backwards as anything but a dirty hack.

While the underlying electronics thankfully didn't care, the digitized values set up certain expectations.

Quarrelsome• 17 years ago

It can get a bit mad sometimes though. The main issue is that if a hardware fault is found it is sometimes too late to rectify without a massive cost.

My management was recently trying to convince me to rewrite a FAT file system in high level code due to issues with power management, corruption and the disk we store our most precious data on.

vijayshankar• 17 years ago

Once our hardware team mounted the camera in a cell phone device upside down. Later we had to write code to rotate the image 180 degree to compensate for it.

kaib• 17 years ago

I just recently wrote some code to do pcb milling with pretty close tolerances. The actual milling table wasn't straight however, so when the substrate was clamped down it would have Z variations 3x larger than the desired cutting depth. A 5 minute touch probe run and a simple 2D compensation table removed the need for a time consuming tramming operations and expensive clamping hardware.

This wasn't so much fixing as augmenting. Software has the unique benefit that it can replicated essentially free while the physical improvements need to be administered separately in each and every instance.

Re. the backwards wired motors. Storing a config table in EEPROM seems to be a simple and portable solution to this. Do it in the logical way until you hit the output pins.

HeyLaughingBoy4• 17 years ago

Do it in the logical way until you hit the output pins.

Ding, ding, ding!!! We have a winner. This is exactly correct: write your code the way everything should be in a perfect world, BUT have a hardware abstraction layer to separate that from reality. The upper layer code should just know that when the end sensor is reached, the hydraulic ram should be reversed. The HAL knows what state the end sensor must be in when it's triggered, and what polarity to send to the valve controlling the hydraulic ram.

Even in small embedded systems, a properly tiered architecture can save a lot of trouble.

blackguardx• 17 years ago

Don't worry, it works the other way as well.

At my last job, I had to fix software defects in hardware.

djcapelis7• 17 years ago

I think I'd like to file a hug report...

Maciek416• 17 years ago

I had a similar problem with my Roomba. Recently it began to bump into phantom walls where there were none, to the point where it would get stuck in the middle of the floor, turning around and around, stuck in a jail of invisible walls.

The Roomba detects this problem and emits a failure code of 9 beeps. I contacted iRobot about the problem, and their response was to ship me a fresh Roomba (without the battery + charging station) right away. Here are some pictures of the battery transplant:

http://dl.dropbox.com/u/1149620/roombasurgery/index.html

From the troubleshooting instructions that iRobot gives you upon encountering the 9-beep code, I suspect that there's probably just a faulty or possibly dirty (likely very inexpensive) sensor on the front bumper of the robot. I wonder if there's a similar fix to jwz's Roomba where I could patch for the bad sensor somehow (the roomba appears to have more than one way to detect it's stopped moving), and perhaps accept a slightly "dumber" but otherwise working Roomba.

jrockway• 17 years ago

My roomba has this problem from time to time. It's because the front wheel accumulates goo and stops rotating. The sensor that detects the wheel rotation then thinks the wheel is not rotating, and the robot moves erratically to free itself from the wall that it thinks is in its way.

Clean out the goo, and the robot is fine again.

HeyLaughingBoy4• 17 years ago

Meh! If I had a dime for every time that I was told to fix a hardware problem in software, or provide a "limp home" mode if software detected a hardware fault, well... I'd have a lot of dimes. It's pretty standard behavior for electromechanical embedded systems.

The really important take home here is that the manufacturer sent out a software fix to a consumer product that is not online! That's a huge leap forward and is something the industry has been pondering for a long time.

jeffbarr• 17 years ago

Can they send one to Mars to fix the rover?

dandelany• 17 years ago

They can and have! I always thought this was very cool. However, the current Mars rover problem is not a hardware problem, per se, it's a stuck-in-the-mud problem.

bockris• 17 years ago

An elderly robot that walks with a limp that's stuck in the mud problem.;-)

po2• 17 years ago

The bad wheel that forced them to drive backward was a hardware problem. They should have designed it with 2 axes of symmetry!

torpor4• 17 years ago

I just love the hacking that space hardware designers are allowed to do ..

"oh, its already bagged up and installed on the rocket, we've got 2 years to fix it until its going to suffer 45 seconds of insane conditions, so just let it slide for now .."

jtnak• 17 years ago

This is nothing new. Read about Voyager 2. Specifically its platform problems between Saturn and Uranus in 1981, about 3 decades ago.

pcc• 17 years ago

Also it should be noted that devices like FPGAs allow for directly altering certain hardware structures via software.

In these cases it can actually be possible to fix a hardware bug (or add a hardware feature) via software, where the hardware itself actually gets fixed (ie its not just firmware working around an ongoing hardware constraint).

DavidSJ2• 17 years ago

Why can't this be done with a download and USB?

kelnos• 17 years ago

Does the Roomba have a USB port? jwz mentioned a 'serial' port...

brk• 17 years ago

It does not have a USB port.

The serial port is also a non-standard (for serial ports) connector (it is a standard DIN connector). My guess is that the average Roomba owner doesn't have a PC with a serial port, and probably wouldn't want to acquire an uncommon connector and wire up a serial converter cable, and then run some program on their PC to dump the firmware image in to the Roomba. The osmo dongle makes the process near foolproof for soccer moms and grandmothers and so on.

DavidSJ2• 17 years ago

Part of my question is why it doesn't have a USB port.

andrewljohnson• 17 years ago

Roombas are no match for Golden Retrievers.

astine• 17 years ago

As pets maybe... but I've never seen a Golden Retriever clean a floor.

tricky• 17 years ago

Wrong breed. I've got a food-motivated wiener dog that'll destroy any roomba in a floor cleaning contest.

RiderOfGiraffes• 17 years ago

Another example occurs on this thread: http://news.ycombinator.com/item?id=1032063

Here's the link: http://news.ycombinator.com/item?id=1032152

torpor4• 17 years ago

I work in embedded. Primarily, my work involves fixing hardware bugs in software. This is not surprising to anyone who works in embedded, but for those of my programmer friends who just see the hardware and have no clue just how much software is under the hood, it always seems to be a big shock to them that I have had to spend a few weeks fixing a hardware design that had one pin wired around the wrong way, instead of just getting the hardware fixed.

That said, one of the things I really like to see these days is new generations of extremely competent hardware designers who have absolutely no problems with the expenses of hardware iterations .. even though it means less work for me in the end, its good to see. I have lost count of the times I've had to write some god-awful driver code that just makes me want to puke my lungs out, just because the hardware guys were too lazy to breadboard something ..

e40• 17 years ago

When I saw it was jwz being linked to, I was worried I would have to suffer through a green on black web page. Oh, the relief.

The software fixed the hardware

Related Stories