Hey,

Mr, Bones is doing some incredible stuff with IPIsoft, I'm very impressed, wow.
https://www.youtube.com/watch?v=Nfg0ssR5mtY



Thanks.
Best Regards;
David

_________________

Though I haven't done it myself yet more than once long ago, I think he's simply importing a model of his own and, if the heirarchy is the same as the ipisoft avatar, it will mirror his performance. None of this is real-time though it would be boss if it were.

...

Just not showing the manual corrective measures taken where the tracking is lost due to no point cloud data to the sensor, it's pretty easy to do, then the refine process takes it from there to get pretty good results, but there is not zero occlusion during tracking, that's a pipe dream if you believe that, you would see this if he would show the actual tracking where there would be occlusion, or probably know this already if you use a single Kinect 1 or 2.

ONLY pull the IK close to where the positioning should be, DO NOT hit the refit pose button at that time, unless you do have enough cloud data for it to lock on to, which you most likely won't, The Refine and JR process will pretty much correct any bad rotations, but if not you just have to do a bit more adjusting and run the Refine again, it just takes some working with the system to get fairly good and faster at doing it, but it does work pretty well, even with a single Kinect 360 tracking.

The tracking processing will be needed to be stopped when the hand, or leg comes back in trackable view, Pull the IK back in place and refit the pose again, then re-start the tracking, this will need to be done at any lost tracking points throughout the tracking, fairly quick and easy process though.

This can only be done with continuous motion, if you stop at 90 degrees to the sensor and remain there, this trick is almost impossible to use and make it look good, without adjusting your performance to keep the body parts a bit more viewable to the sensor at all times.

There is no "Secret Sauce" just some extra manual pre-adjustments using frame forward advancing and pulling the IK of the bones back to close of what they would be if tracking data was there before running the Refine process.

Make all needed corrections throughout the entire failed portion of the ROI before running Refine, you don't need to do all the areas of the full ROI at once, but you can.

This is why 2 sensors are better than one, even if it's 2 - Kinect 360, you get a full body point cloud and for many other reasons other than just 360 spins also, but for basic motions, the post tracking adjustments to make it look like there is no occlusion is fairly simple.

Here is a quick sample only using a single Kinect 360 using this technique inside iPi Studio 3 only, (it had occlusion errors during tracking process of course), exported video using iPi Export Video feature right from the view port: https://youtu.be/F37DzBm3M2U

No Moves were used, for best results, any model Kinect sensor works better with 3 used, hands and head.

No recording trickery was used, this is the actual result, surprisingly better than a Kinect 360 used to do with older versions of iPi Studio too, the updates have shown much improvement, but I still will never use Kinects 360 or XBox One for my recordings, just not my preference.

The rest is done in final clean up in an external 3D editor.

...

...


After looking at it again, you can hit the refit button after pulling the IK handle and it should untwist the arm bone better, but remain in the position you pulled it to in most cases where the bones have no data to go on, (may have to pull it and hit refit more than once to get proper untwist), but you have to pull the IK handle again off it's position a little bit, not just hit refit button again.

Also best after the point cloud is visible to re-connect to after tracking loss, to run track backwards as far as it will run before loosing connection again, use < or > keys on keyboard to advance back to last good frame and re-start tracking forward.

DO NOT re-run track forward on these manually corrected areas once all are fixed pretty closely, just run Refine Forward or Backwards, then when satisfied with results, you can run JR with whatever settings you want to use.

You have to run the full track forward processing before you can run refine anyway, so no need to fix the areas until after all visible data is tracked, then go back and fix errors.

Just takes some experience with doing it and anyone can get it looking fairly good, I don't work with Kinects much, but this technique will help to resolve these areas to look much better.

...

I have to admit, not using IPI much these days but, did try doing a full body turn with one Kinect and to my surprise, it was very close.



Best;
David

_________________

...

Yes, as long as you keep continuous rotation you only need to fix some areas, especially with Kinect 2, and the new algorithm catches back up to lost connection to the cloud a bit better, but still if it gets disconnected fully, you have to stop and reset those areas to continue the tracking.

Without data from a sensor the system has no way of knowing properly where the appendage would be, but can try to make a guess as to trajectory with prediction and sometimes that works, but most times a manual re-adjustment is needed.

...

Gotcha.



Thanks;
David

_________________