after i start to track sometimes ipi loses arm and cannot catch it again. after long calculation i have to rewind and set manually arm pose and recalculate the rest. microsoft api doesnt need t pose and can catch all suitable poses. can ipi auto detect as microsoft api can detect? i mean is it possible to integrate microsoft sdk to auto fit some frames?

If Kinect SDK was so good on automatic pose detection why would anyone use our software at all? There are many programs for motion capture that rely on KinectSDK skeletal tracking. We think our software is valueable because it provides better accuracy and detects more complex poses then Kinect SDK can. So when tracker in iPi Mocap Studio loses something, there is absolutely no guarantee that Kinect tracker finds a correct pose in this situation.
However, at some later time, we may start using data provided by Kinect skeletal tracker as supplementary information for our own tracker.
To avoid massive retracking with iPi Mocap Studio, it's better to watch (at least periodically) how the tracking is going, and correct an error as soon as it is noticed. Tracking in backward direction is also a much help for corrections.

i posted a reply but now i cannot see it so i repost it;

i did not say ms sdk is more accurate. i catch poses in the middle of capture session. it can lose all but i has ability to catch it again. when ipi studio lose a pose it is not possible for it to catch it again. if it were like that i would recalculate just part of take. under these conditions i have to recalculate the rest. i suppose it will better when you integrate ipi studio with ms kinect library.

do you plan to add face capture to ipi studio? i mean just face not both pace and body at the same time.

For simple motions, I usually just allow the tracking system to run through on its own and then I go back and fix things--this sounds like your workflow. However, for more complex motions, I prefer to supervise the tracking a bit more progressively, fixing errors as they occur.

The 'supervised' workflow I use is to track up to the point where I encounter the error, go to a later frame after the error and make the correction, Refit Pose, then track backwards. This usually solves the problem. Then I move back to the last good frame (where I corrected the pose) and continue tracking from that point again. This method works most of the time, and if you have a reasonably good card, it can go very quickly. (FYI, I use a GTX 460, which is a fairly modest card these days but it can track a frame in 0.67 seconds--IMO, good enough for production. Newer cards will be much faster.)

If the error is so bad that the entire pose is messed up, I use Copy Pose at the last good frame and advance to just past the bad frame, paste the copied pose. Then I make a manual tweak if necessary and press Refit Pose. Sometimes, I can just press Pose Refit without making a manual correction--this depends on the error of course. Next, I track backwards through the error.

In rare or extreme situations, I may need to repair a pose manually and move on without pressing Pose Refit. This is usually okay because you can smooth the transition (usually a 'bump' in the motion) by 'touching' the Trajectory Filtering. (Move the slider to a different value and then slide it back.)

Sometimes, when I run into an error, I will copy a good frame, paste the good frame after the bad frame, press Refit Pose and move on with the intention of fixing the error later. This will at least get me through the entire sequence so I have a better idea of what I'll be dealing with. If I feel there are way too many errors to fix, then I will use a different take or re-record the motion. Doing this quick 'first pass' also provides me with a temp mocap file that I can use for Previs purposes--I can replace the temp mocap later in production, after the motion has been properly tracked.

When you're done tracking, be sure to run Configurable Jitter Removal. CJR runs only through a set ROI, so you can change settings as you work your way from start to finish. If you use different settings of CJR throughout the take, remember to 'touch' the Trajectory Filtering slider--this smooths out the transition between different settings.

Tip 1: If you use CJR, I generally find I need fairly aggressive JR on the legs and much less for the other parts of the body. Don't use too much CJR on the entire body--it can make your motion look 'robotic'.

Tip 2: Remember, you have two very useful tools on the Timeline: ROI and Takes. Set the Take to the full range of what you want to track and export. (If you have multiple segments, you can add takes by right-clicking.) YOu can use ROI for the segment you are currently working on, which is typically a small range of frames--this feature can help you avoid overwriting frames you are satisfied with. When your done working in the current ROI, move the range (click and drag in the middle of the ROI) to the next range you wish to work on. Of course, you can tweak the ROD by moving the I/O points as well.

When you're all done, double-click on the Take. This automatically sets the ROI to the length of the Take and you can preview only this range (the take) independently of other takes or the entire duration of your captured data. To export the take, right-click on the take and choose the desired option.

Hope this helps. There are many other tracking tips and tricks I could post--when I have time, I'll put together a short series a videos some day to cover them.

G.

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)

Watch a one minute excerpt on Vimeo now!

thanks for tips. we are all agree on workflow. it is no problem if i get short takes. problem occurs when i try to take a whole dance capture :-) especially on one kinect recordings arms get less depth points and ipi studio confuse positions. maybe 4 kinects will be the best for turning movements :-) in fact this motion capture is for a camera. so what is important is to get the best for camera angle. so 180 degree setup is not meaningful. what i need is what the main camera sees. it is not important for me if ipisoft calculate wrong behind my actor like i turn my back to camera and i hide my arms from camera. so i do not need my arm calculations. as a result there is no full cover solution like very very expensive 3d censored motion capture systems. we have to plan and choose the better solution and workarounds.

I disagree, it is certainly possible to get fairly complex full coverage captures using two Kinects. Here's a test I did with my daughter a while back using two Kinects with the 90 degree setup (not 180):

Mocap Test with Small Child (1 meter tall)

What you see here is entirely inside of iPi Studio and has not been tweaked or otherwise cleaned up in another animation program. This kind of tracking takes a little practice to get accurate results, but after you've used iPi Studio awhile it becomes a fairly quick process. I recognize that if this motion was for production work, it would need a bit more tweaking and finessing in an animation program, but for a entry-level program that is not designed to do mocap editing tools this is really not bad.

I agree that it's not necessary to track every detail if you're not going to see it on camera. When we created 'Happy Box', we were careful not to spend too much time fussing over limbs we never saw. For example, in the chainsaw dance scenes, you never see Sister's feet, so we didn't bother enabling feet tracking--this sped up tracking a bit and we focused on other things. For many of the close up shots, we didn't worry too much about the lower body tracking--Studio tracked these motions fairly accurately anyway but if there had been lower-body errors, we didn't worry about it since it was all 'off-camera'.

That said, there are times when you absolutely want a full body capture, iPi DMC is certainly able do this. But to do it well, it does take practice. :)

Now, after saying ALL THAT, there are some limits to be aware of. The current iPi rig has built in limits that can make certain poses difficult to track accurately. In these situations, you need to switch from IK mode to FK mode when adjusting rotations--this is mostly true in the shoulders and clavicle joints. If that fails, you may have to accept the best that the system can do and then correct or enhance the motion using a mocap editing program like Motion Builder, Ikinema Web Animate, or iClone Pro with 3DXchange. I use Motion Builder but will be experimenting with 3DXchange after we finish our current film project.

BTW, if the dancing you wish to capture is very complicated, you might consider going with six PS3 Eyes--this configuration will give you better more accuracy thanks to higher frame rate and more complete coverage of the capture space. However, it does require more space and it is a little more complicated to setup. However, you still need to master tracking skill to get good results from the program, so practice, practice, practice!

G.

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)

Watch a one minute excerpt on Vimeo now!

One more tip:

Sometimes Feet, Shoulder and/or Head tracking can cause problems because these features impose limits that may prevent some poses to be accurately tracked. When tracking, I will sometimes disable one or more of these features for a specific range of frames and renable them when I think they will be useful again. If you do this, remember to 'touch' the Trajectory Filtering when you're done to smooth out the transitions.

For example, in the test with my daughter, I had to disable feet tracking when she gets very low to the floor to get good results, but then I re-enabled this feature once she stood up again to get good results in this range of frames. (Note: there is no 'Head Tracking' in this video because this feature did not exist back then.)

Hope this info is helpful.

G.

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)

Watch a one minute excerpt on Vimeo now!

Some really nice tips in there Greenlaw, it seems like you really have your technique down to an art!

Oli

In fact, Kinect SDK finds pose for each frame independently. So the term "lose" is not quite appropriate in its case. Either Kinect SDK "knows" this specific pose from its database, or not.
iPi Mocap Studio's pose tracker is incremental - next frame position is based on position from previous frame. Indeed, it can catch up the pose after some minor errors like loss of one arm. However, if pose gets completely wrong, it's hard for the software to recover without manual help. So leaving the tracking process unattended is not recommended due to the fact the errors happen at the current level of software's capabilities. We are constantly improving the tracking quality to decrease the number of possible errors, but it's still a long way to fully automatic tracking.

Currently we have no such plans. We have a lot of things to do with full body capture. But if eventually we have an interest and resources to engage in facial capture, most probably it will be a separate product, not iPi Mocap Studio.