XNect: Real-Time Multi-Person 3D Motion Capture with a Single RGB Camera

https://gvv.mpi-inf.mpg.de/projects/XNect/

by nabla9 • 6 years ago

62 20 6 years ago

gvv.mpi-inf.mpg.de

This is a great way that various disciplines in sports, dance, martial arts etc. could be captured and preserved. A great first step. Adding in support for additional programming languages like say python could open this up to a wider audience of DIY mocap.

rednab• 6 years ago

Unfortunately, it appears this technology is made available only for research purposes¹)/people with an institutional email address.

Which means that the best way for DYI mocap that I know of is still getting hold of an increasingly rare Kinect One.

¹) https://gvv-assets.mpi-inf.mpg.de/xnect/?page_id=12

DoingIsLearning• 6 years ago

Is there a reasoning to limit the distribution to other researchers only?

It feels even more out-of-place considering most of the authors are affiliated with public institutes and likely benefited from public funding.

I actually thought this would be super neat for a poorman version of MS Kinect and hobbie gaming.

maxst2• 6 years ago

So many algorithms insist on using "single RGB camera" approach, when it would be much more practical to use 2 cameras.

kristopolous• 6 years ago

The power is the existing giant corpus of video. When I'm in front of a computer I'm going to run this up against the dancing of James Brown and Michael Jackson. Should be interesting.

Maybe an Olympic gymnast as well.

dTal• 6 years ago

Practical for an algorithm implementer, maybe - deeply impractical for real-world use. Stereo cameras are rare and nontrivial (you have to synchronize shutters). Monocular algorithms can be applied to the millions of hours of existing footage, or used with the billions of cameras, smartphones, robots, drones, and fancy doorbells that already exist right now.

RavlaAlvar• 6 years ago

For “slow” moving object like human body, does synchronised shutter matters that much and if so , is there any tricks to compensate it if synchronisation is not possible?

maxst2• 6 years ago

There are dozens of 360 cameras on the market, so I think shutters synchronization is not that difficult to implement.

derefr• 6 years ago

If you can calibrate the time-delay between the two cameras, can you not just interpolate one or the other signal backward or forward in time so that it aligns with the other? (By "interpolation", here, I mean the sort of thing the Oculus does on the display side, generating frames "between" frames, to smooth motion during head rotation. Take one real frame from one camera, and build an interpolated frame between two real frames from the other camera to match it.)

regularfry2• 6 years ago

Only for near-field. Further out than roughly an arm-span, your brain itself doesn't use binocular vision for 3D estimation because there's not that much information in the parallax.

dungdang2• 6 years ago

do you have a source for that? because my source: one closed eye, tells me very clearly i do use binocular vision for 3d - 3d being how far something is.

you don't need 'much' information. you know the distance between the eyes, and then you have the 2 lines from each eye to the object. that's called a triangle. do you know how to calculate the height of a triangle? because that's your distance.

Mashimo• 6 years ago

Does the video have interlacing problems? How does that happen?

chmod775• 6 years ago

How long until technology such as this is used for automatic "crime" detection with surveillance cameras?

Not looking forward to having that in the west too. This stuff is the wet dream of every authoritarian looking to effectively control a populace.

We really ought to amend our constitutions, adding protections against this, while we still can.

XNect: Real-Time Multi-Person 3D Motion Capture with a Single RGB Camera

Related Stories