Development in Motion Capture: Are the Life Science and Entertainment Markets Out of Sync?

Out of Sync. Jealous.

This is how I would describe the Life Sciences and Entertainment markets for motion capture. The Life Sciences market is peering over the partition at its flashy colleague, thinking how exciting their life must be, meeting movie stars and attending premieres. Meanwhile, the Entertainment market struggles through their morning coffee wishing they had a regular 9–5 job like the more stable Life Science market.

You would think our customers from different markets are poles apart. I think they are just out of sync. What is important for the Entertainment market right now is almost the least important thing for the Life Science market, but in the next period that changes.

I have always loved the line from the Simpsons, “Very few cartoons are broadcast live, it’s a terrible strain on the animator’s wrists.” And I have always thought that it sums up our LIFE SCIENCE customers… Very few papers are published in real time.

Traditionally, the difference has always been real time. Real time is/was of the utmost importance to the Entertainment market and the Life Science market couldn’t care less. Now the advances in real time requested by the Entertainment market are really driving advances in the Life Science market. Now that we have real time sorted for the Entertainment market with Vicon Shōgun, the Entertainment market is really interested in the biomechanics advances from the Life Science market to solve their animation skeletons. One market’s advances have always driven the other, but just out of phase.

I would say that one of the highlights for all Vicon employees is visiting and being a part of so many different applications. Personally, in the last 12 months I have been up in a stable, applying markers to a racehorse; floated on a boat in a wave tank; made my directing debut by filming four professional dancers with a virtual camera; and even put the mocap suit on myself for a couple of sword fights (there was a particularly satisfying goblin fight)!

Traveling to Shanghai, Beijing, Tokyo, Singapore and Melbourne isn’t half bad either.

The Vicon staff and I know best what all our customers are doing, and when we can leverage all of this information it is truly one of the most satisfying moments of my job (Peking Duck at Da Dong in Beijing is a close second).

One of those moments for me was at the ISB conference in Brisbane earlier this year. We wanted to take the real world question of, “What if we immersed an athlete into a Virtual Reality scenario to measure their ability under changing game situations?” Think that sounds pretty easy? Just an extension of Wii tennis? Not exactly.

The hard part was making sure we didn’t change the game/athlete behavior, but kept the ability to measure at biomechanics lab levels. We were able to combine Life Science standards perfectly with an Entertainment application and make it valid for both.

We took third-party information from force platforms and EMG via the Vicon Lock+, synced with the Vicon motion capture data in Nexus, overlaid synced HD video from the Vicon Vue, used the Vicon real time SDK to pipe the information to the Unreal Game Engine, calibrated the game world to be in the same world space as the real world, displayed this in real time through the Vive VR headset, and displayed force vectors and biomechanical data in real time through a game engine, but most importantly, met the goal of immersing an athlete into the game situation while still maintaining the situation as ecologically valid.

The advantage of using a game engine is being able to change the athlete’s perspective, all at the press of a keyboard. The Life Science community is able to leverage advances in programming and tools created for games like The Killing Floor or Batman Arkham City.

It is truly mind-blowing that a game engine that was developed to kill virtual creatures with your Enforcer Sub Machine Gun could be driving advances in athlete performance and rehabilitation.

Clearly, I am breezing over some of the challenges and we were able to stand on the shoulders of giants. However, we reached a moment where the two main markets for Vicon were no longer out of sync – they were actually perfectly aligned.

I know examples of this are going to become more and more frequent.