Change font size
It is currently Fri Sep 18, 2020 12:22 am


Post a new topicPost a reply Page 1 of 2   [ 13 posts ]
Go to page 1, 2  Next
Author Message
PostPosted: Mon Jan 13, 2020 9:09 pm 

Joined: Mon Jan 13, 2020 8:54 pm
Posts: 6
Hello all,

I've spent all afternoon researching various motion capture solutions, and I'm very impressed with the ipi motion capture solution using the Kinect. I had a few basic questions f you wouldn't mind helping me figure it out.

I noticed that 2 kinects are recommended. I was curious, what is the practical reason for this?

My other question, is there anything I could to supplementally to the Kinect camera that would grant me additional usage of the hands? From my research I discovered that hands and fingers are particularly expensive to get right, but I saw some people using some PS move controllers and I'm just wondering how that would combine with the Kinect cameras (Since it's Microsoft vs. Sony), do they play nice?

Thanks for your time!


Top
 Profile  
 
PostPosted: Wed Jan 15, 2020 4:07 am 
iPi Soft

Joined: Wed Jan 11, 2012 6:12 am
Posts: 2251
Location: Moscow, Russia
Hi

Single depth camera will only allow you to capture a limited range of motions, basically when an actor is primarily facing the camera all time. Problems arise when motions involve occlusions of some body parts. For instance, when an actor turns around. In that case certain body parts are not visible to the software and it loses tracking which requires manual cleanup.
Introducing a second camera increases a number of POVs which in turn reduces a chance of occlusions. As most of the time all body parts are visible to at least one of the cameras. Also, with 2 cameras the accuracy of capture may be improved. Thus dual depth camera setup is suitable for a wide range of motions not requiring a big capture area (the size is limited due to the operation range of depth sensors).

Our software doesn't track hands from a video (neither with depth nor with color cameras). However, it is possible to track wrist rotations with the help of motion controllers (which are not limited to PS Move but it's considered the best of supported models). You can use any motion controllers with any cameras. Also, iPi Mocap Studio features key framing tools for fingers.


Top
 Profile  
 
PostPosted: Mon Jan 20, 2020 11:22 am 

Joined: Mon Aug 03, 2009 1:34 pm
Posts: 2410
Location: Los Angeles
Being a depth sensor, a Kinect device captures 3D data. Naturally, this makes a single Kinect more suitable for capturing 3D motion than a single 2D video camera, like the PS3 Eye. However, even a single Kinect is still capturing motion from one point-of-view, and with only one perspective Mocap Studio can’t capture what it can’t see.

What this means is that any body parts that are hidden or occluded by the performer’s body cannot be precisely captured. Where this becomes more apparent is when the actor is performing complex movements like when walking, spinning, folding the arms, reaching behind, and crouching low. A single Kinect is fine for capturing the actor facing the device directly and within a limited range of motions but it's not so great when the actor has to turn away.

Adding a second or third sensor overcomes this limitation, allowing iPi Mocap studio to see what’s going on in these ‘hidden’ areas and capture motions more precisely.

BTW, this is not to say motion data captured using with multiple Kinect sensors is necessarily more accurate that what you get with multiple 2D video sensors. The general rule of thumb is that the quality of data captured using two Kinect sensors is comparable to data capture using four PS3 Eye cameras. However, using more 6 or more PS3 Eye cameras gets quite a bit better than using multiple Kinect sensors.

Here are the trade-offs:

Multiple Kinect Sensor
Pros:
Easier to setup and take down; doesn’t require special lighting or special color clothing; works well inside a small space.
Cons: More costly than PS3 Eye cameras. May require multiple computers when using Kinect 2. (Recent improvements to iPi Mocap Studio offer a possible solution for this latter issue.)

Multiple PS3 Eye
Pros:
With six or more sensors, produces higher quality capture data; cameras requires fewer computer resources; works will in larger area; can capture multiple actors simultaneously.
Cons: strict lighting considerations (mostly ambient with no dark directional shadows) and special color clothing. Potentially longer time to process the data.

More info:

The original Kinect for XBox 360 was fine for its time but the data is considerably noisier than what you get with more recent Kinect sensors, and can require heavy application of jitter removal. They're still relatively inexpensive. Kinect 2 (aka, Kinect for XBox One) produces cleaner data but for a lot time, it requires multiple computers to use multiple sensors. Recently, iPi Soft figured out how to work around this MS imposed limitation.

Reportedly, the latest Azure Kinect produces significantly higher quality data than any previously available depth sensors. I'm currently using Kinect 2 sensors but I'm going to upgrade to multiple Azure Kinect sensors later this year.

Regarding hands, there are two parts to consider. First, you need to capture the wrist motions, and to do this, you need devices that record rotation data. i use PS Move Controllers for this. To record with Move Controllers, you simply grip them firmly while you perform. A Move can also serve as a wireless remote control for iPi Recorder, and the globe on the end works well during calibration. I also use a third Move controller to capture my head movements. The downside with Move is that it's a bit bulky.

Nintendo Switch JoyCon is a smaller alternative the Move. I own a set but haven't tried them for comparison yet. I'll try to do that in near future and post my findings.

The next part is finger animation. To be blunt, you can't record live finger motions using iPi Mocap Studio. However, you can add finger poses from a library of hand poses inside the program. The program lets you interpolate between each keyframed pose and you can add your own custom finger poses to the library. I personally find using the library system in Mocap Studio works well for getting credible hand animations on a character, but more specific 'hand acting' should be added using an animation program.

If you need actual live finger capture, one possible solution is to use iClone 7 with Pipeline and a Leap Motion sensor to record your fingers and wrist motions in a separate session. Then you can apply them to your iPi data inside iClone. With this system, it may not be necessary to record the wrist motions since the Leap captures that. In fact, the Leap can even capture forearm rotation. However, I don't own the Leap Motion setup and I don't have any direct experience with this yet, so you'll need to do some research on your own.

Leap Motion is something I've casually thought about for a while but more seriously I'd rather get two Azure Kinect sensors for iPi Mocap Studio first.

Hope this helps.

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)
Image
Watch a one minute excerpt on Vimeo now!


Top
 Profile  
 
PostPosted: Mon Jan 20, 2020 2:19 pm 

Joined: Mon Jan 13, 2020 8:54 pm
Posts: 6
Wow thank you so much that helps immensely! Great minds think alike as I JUST bought the leap motion controller for hands, but I've been busy learning animation nodes, so I haven't had the chance to try it out yet, but plan to soon. I'm going to get the workflow for the hands down first before I dive into the body mocap and which software to buy. I'm going to use some freely available motion cap .bvh files and experiment with putting hand animation on them.

Money is certainly an issue so, while in theory I LOVE the idea of getting 2 azure camera's, I looked into those and they're 400$ each new, where I can get a used Kinect v2 for under 100$ used, so I'd lean towards 2 of those.

As far as software, I haven't made any commitments yet, but this guy named Brekel makes some interesting looking budget friendly software https://brekel.com for the leap motion controller. I was looking at the iclone software, and you can get the hand motion cap software for 100$ (motion live), but to export it to blender requires an extra 500$(3dXchange) so I kind of turned away from that, and am leaning towards purchasing the Brekel software. Although I'd love recommendations or advice if you had any. I'd rather buy the best and cry only once if you know what I mean.

Thanks so much for the info and the help!


Top
 Profile  
 
PostPosted: Mon Jan 20, 2020 8:50 pm 

Joined: Mon Aug 03, 2009 1:34 pm
Posts: 2410
Location: Los Angeles
Glad to help! Some more tips...

In recent months, I've been learning to use iClone 7 w/Pipeline to edit my iPi Mocap Studio data and I happened to read about the Leap Motion support a few weeks ago. For iClone, Leap Motion support is an add-on so, in addition to buying the Leap Motion device, I'd have to purchase the extra plugin and support software. It's not terribly expensive but I decided it's be better for me to save up for the Azure Kinects instead. That said, I'm interested in learning more about the Leap so please post about your progress when you can.

So far, iClone 7 w/Pipeline has been a good 'affordable' mocap editing system for working with iPi Mocap Studio data. Reallusion has regular sales for iClone, so if you watch out for it, you can get a good price. Just be aware that to use iClone with Mocap Studio, you need to get the Pipeline edition...the regular edition of iClone has no way to import and export custom data.

But, TBH, if you're on a tight budget, it's not really necessary to use an intermediate program like iClone or Motion Builder. I don't use Blender (yet) but I'm sure it's fully capable of editing iPi Mocap Studio data.

As for Kinect 2 sensors, I can confirm they work well with iPi Mocap Studio. In recent years, I was mostly just experimenting with Kinect 2 when I could. That hasn't been as often as I like but this year I'm planning a return to making short films using iPi Mocap Studio.

Years ago, I used multiple 'Kinect For Windows' sensors (an enhanced version of the XBox 360 Kinect) with iPi Mocap Studio to make my two 'Brudders' cgi short films and to create vfx for a handful of SyFy Channel movies. (If you're curious, some of the vfx footage with iPi mocap can be seen on my old demo reels: Little Green Dog Channel On Vimeo). Reportedly, these early models are not nearly as accurate as the new Azure Kinects, but they're still pretty darn good sensors. (Good enough for me to use in real studio work anyway)

My current setup is one Kinect 2 connected to my desktop and a second to my laptop. I have USB repeater cables running from our computer room to the living room where I record mocap data when I need it. I use the built-in Bluetooth in my laptop to connect the PS Move controllers for my hands and head. This has been a convenient setup for our somewhat small-ish home for a few years now, and my wife doesn't mind so long as I put away the gear when s I'm done with it.

Trust me, this was not as easy to do back when I was used the PS3 Eye cameras because that setup required setting up special lighting in the living room and we don't have the space for six cameras to make it worthwhile. But if you have the space for it, six or more PS3 Eye camera is certainly worth considering. (Or Azure Kinects whenever you can afford them.) :)

Good luck!

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)
Image
Watch a one minute excerpt on Vimeo now!


Top
 Profile  
 
PostPosted: Mon Jan 20, 2020 9:55 pm 

Joined: Mon Jan 13, 2020 8:54 pm
Posts: 6
Wow your demo reel is quite impressive! I'm amazed at what you can accomplish on this software setup and kinects. It's insane how expensive a full professional body suit is!! We are so lucky to live in an age where technology has brought down the costs of so much of this stuff, it's incredible.

I would LOVE to be able to afford the whole mocap studio, and it's good to know it goes on sale! Maybe I'll save up for Black Friday so I can gain some extra capabilities, but for now I'm dipping my toes in the water. I'm not trying to be an animation studio for other people, my motivation is to learn how to do some basic animations for some music video's I'm creating for myself, so I'm taking it step by step. I got the hand controller first because there's a scene I wanted to do where I'm lifting up a cereal box and I want to animate my pouring a bowl of cereal and I'm hoping I can accomplish that with Brekel hands and I'm hoping I can figure out how to stream the information through OSC into blender so I don't have to deal with intermediate software. I will definitely follow up if I'm able to get it to work!

I am also hoping to do some "zero gravity" mocap with the Kinect, and I'm planning on building a rig to levitate a dancer off the ground. Hopefully it looks ok when all is said and done. I have a vision, but it can be hard to accomplish when you're not sure what you're doing. I've been at this for about a month now and feeling overwhelmed, but there's a quote from Elon Musk I love where he says "Struggling with something is the first step to understanding it" and it's always proven true for me. In another month I'm sure I'll look back and it will all feel so simple.

Keep up the great work! If I ever become successful perhaps I can hire you one day as I'm also in Los Angeles.


Top
 Profile  
 
PostPosted: Tue Jan 21, 2020 9:39 am 

Joined: Mon Aug 03, 2009 1:34 pm
Posts: 2410
Location: Los Angeles
sargentpilcher wrote:
Wow your demo reel is quite impressive! I'm amazed at what you can accomplish on this software setup and kinects.


Thanks! To be clear, only specific shots on the demo reels were done using iPi Mocap Studio, not everything on the reels. On the Asylum reel, it's mainly the giant robot creature and some digital stunt double work. Of my shorts, Happy Box and the music video excerpt were mostly animated using Mocap Studio. Prior to that, I used it to previs characters for a couple of commercials.

I had to set aside that music video project a while back but in recent months, I've been busy revising the project. Since my 'free' time comes in small bursts, I'm re-thinking my workflow and schedule to complete the project in smaller chunks. That worked for other personal projects last year, so I might actually get these videos done this year. Fingers crossed! :)

Quote:
I am also hoping to do some "zero gravity" mocap with the Kinect, and I'm planning on building a rig to levitate a dancer off the ground.


Free fall and zero-g effects are fun to do with iPi Mocap Studio; you get to be clever with the setup and performance. Tip: for falling, try leaning back on a stool like a drummer's throne with your feet in the air. I did this to animate a family falling into a cave in a low-budget disaster movie years ago. I think I posted some examples in this forum a while back. I've also used iPi for a 'space walk' by simply balancing on one foot and correcting the foot angle in the rig. At the time, I considered shooting in our garage and wearing a harness--that probably might have looked convincing but I just didn't have the time for it.

Oh, also check out the work of 1k0. I don't think he uses iPi Mocap Studio anymore but years ago he made a cool trailer video which had a very believeable free fall effect. To do this, he balanced on one foot (which is where I got the idea for my own free fall effect.) His videos will probably turn up if you do a search in this forum.

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)
Image
Watch a one minute excerpt on Vimeo now!


Top
 Profile  
 
PostPosted: Wed Jan 22, 2020 9:10 pm 

Joined: Mon Jan 13, 2020 8:54 pm
Posts: 6
Greenlaw wrote:
Glad to help! Some more tips...

In recent months, I've been learning to use iClone 7 w/Pipeline to edit my iPi Mocap Studio data and I happened to read about the Leap Motion support a few weeks ago. For iClone, Leap Motion support is an add-on so, in addition to buying the Leap Motion device, I'd have to purchase the extra plugin and support software. It's not terribly expensive but I decided it's be better for me to save up for the Azure Kinects instead. That said, I'm interested in learning more about the Leap so please post about your progress when you can.


Hey Greenlaw! Would you be able to give me some insight into what the editing process is like using ipi mocap and iclone? I realize that all in all it's going to run me close to a thousand bucks, but I'll do it if it's worth it. I just spent the last few days struggling trying to get the leap motion controller to play nice with blender and it's been a nightmare. I am thinking of asking for a refund for Brekel pro hands (If he will honor that) and put the money towards ani clone purchase.

Blender (Which I'm using because it's free) does NOT handle editing .BVH animations very well at all. I was trying to for example, take a hand animation that I recorded through Brekel, and attach the hand animation to a .BVH file from the internet without hands, and I could not for the life of me figure out how to do it.

I am not a quitter though, and I'm kind of attached to the idea of doing mocap, so I'll pay some money if it will get me the quality, but I want to know I won't run into the same issues. Brekel hands for example is designed to work with animation designer which costs well over a thousand a year to license it. How is iclone 7 in regards to editing animations that you've recorded in ipi with the Kinect? Does it "play nice" so to speak? I tried going to their website, but it doesn't seem to be their main selling point so they don't really talk about it much. I'd love to hear some firsthand experience!


Top
 Profile  
 
PostPosted: Thu Jan 23, 2020 8:23 am 

Joined: Mon Aug 03, 2009 1:34 pm
Posts: 2410
Location: Los Angeles
Sorry, I wish I had something to show using my new pipeline but I'm still figuring things out with iClone. But technically speaking, it's not that different from when I was using Motion Builder 2010 to edit my motions between iPi Mocap Studio and LightWave. The key is in being able to re-target the data from one rig to another, which requires using rigs with the same bone hierarchy. Even the names can be different because you can remap them, but using the same hierarchy is critical. You can have other bones in the rig but the flow of the expected bones needs to be unbroken. The starting rotation of the bones is critical too; ideally everything should be zero-ed out in the setup frame.

The workflow I was using with MoBu was: clean up the motion in iPi Mocap Studio, export a .bvh, import to MB where I would retarget to my rig from LightWave. This was critical for my 'cat' characters because of the 'non-human' skeletal structure. Note that the main bone hierarchy was still the same, but there are tricks I can do in MB than I can't easily do in Mocap Studio or LightWave to make that work. I got iClone to replace my aging copy of MoBu, and ATM I think this should work fine for this.

Sometimes I would just go directly from iPi Mocap Studio directly to LightWave. If the character has a normal biped structure, this works pretty easily. Just save out an FBX and use Merge Only Motion Envelopes to transfer the data to a rig in my LightWave scene. This is what I typically did in some of my vfx work for the commercials and b-moves I used to work on.

Last night I checked a couple of iPi Mocap Studio to Blender videos on YouTube and it seems like they have a similar workflow. Probably worth checking out. I would try getting sample output and walking through the process to see if it will work for you first, and then decide if you need additional or different software.

Also, search these forums...other artists are using iClone, Blender, Max, Maya, Webanimate, etc., to edit their iPi Mocap Studio data, and some have posted workflow videos.

Hopefully I'll have demos and test examples to show again soon. After a long hiatus from my personal projects, I'm actually getting stuff done again and that feels pretty good. :)

(Note: One of my personal projects is tutorials for 2D animation (which is my main work now) but I'm planning to make a 3D tutorial series soon, including videos for iPi Mocap Studio. Naturally, I'll post here when I get around to that.)

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)
Image
Watch a one minute excerpt on Vimeo now!


Last edited by Greenlaw on Thu Jan 23, 2020 10:19 am, edited 4 times in total.

Top
 Profile  
 
PostPosted: Thu Jan 23, 2020 9:34 am 

Joined: Mon Jan 13, 2020 8:54 pm
Posts: 6
Thank you so much for the followup! If iclone is anything at all like motion builder, then I think I'm going to be very happy. Now that I've accepted that I'm going to have to spend a few hundred to accomplish what I want to, I've been watching some iclone videos and it has some great looking features! I'm going to play around with the demo in the following weeks, and they have a sale going on through the end of the month for 530$ for iclone 7 and 3d change pipeline, so I'm going to get that bundle in particular. If everything works out, I'll be buying a Kinect and ipi and going back and forth between them. Thanks again so much for all the help!


Top
 Profile  
 
Display posts from previous:  Sort by  
Post a new topicPost a reply Page 1 of 2   [ 13 posts ]
Go to page 1, 2  Next


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron


Powered by phpBB® Forum Software © phpBB Group
610nm Style by Daniel St. Jules of Gamexe.net