...

Ahhh, ok, now I better understand your reasoning for needing the wire frame to match, but as for needing the iPi Actor to proportion a mesh would only be used as a time saving measure?

If the iPi animation is processed with a Kinect and the video recorded with a DSLR camera, the Kinect would have to be positioned roughly the same location then?

Or, are you only using the Kinect RGB video?

I don't do film based VFX, but now that explains your scenario.

I always thought that kind of stuff was done as added 3d mesh objects and camera tracked to the video, but I suppose there are many different ways.

...

...

I would like to see a sample if you do get your idea to work with how you explained using the Kinect data to accurately match move a mesh to the video.

Would be cool if you could use it to get effects like these to work: https://www.youtube.com/watch?v=GyFo559P2DQ

Good luck, sounds interesting.

...

...

Wondering if you ever tried a 3D scan of an actor using a Kinect sensor and then a program to auto-rig it for animation, which this one saves mesh in .DAE format, which iPi Studio accepts for importing and characterizing.

http://smartbody.ict.usc.edu/autoriggerandreshaper and it's Freeware.

There are several Kinect 3D scanning program options if you search the net.

...

...

Have you tried the BioMech add-on to see if the point cloud image can be exported and converted for your use that may match your recorded motions better?

I see you posted for how to get the trial working.

I can see where an exported character mesh isn't what would work for absolute background image match moving that you want.

...

...

https://www.youtube.com/watch?v=Ixg6EaLkhgo 3ds max and VFX .... impressive.

...

Yes I'm trying to import depth data using biomerch not to be use as actual working mesh (that would be impossible) but to have a good reference as i'm trying to replicate my ipi scene in max. However that doesn't look right ether. I'll do another post about it after i wrestle with it some more .but short of it - Depth cloud imported via biomerch and skeleton imported as fbx from ipi look slightly different in relationship to each other in Max compare to the way it appears in ipi.


The reason depth mesh can only be used as a reference is that for any animation vfx task you need clean topology mesh - depth mesh is simply a collection of vertices in space that also don't have continuity from frame to frame.

...

I see your issues now, and I tested the BioMech point cloud export also, just to see for myself, but I only had 1 Kinect 360 and the point cloud verts were shotty, xbox 1 sensor is probably much cleaner.

Any re-targeted mesh inside iPi doesn't follow the motion as closely as the iPi actor does, so I see the trouble in match moving any external mesh to the video for your purpose.

I have looked to see how they do it for the marker type motion capture, but I don't see how they parent the meshes to the live action mocap for compositing, i see you did it with markers on your samples, so probably how they do it also.

I don't know if creating a simple male/female mesh and binding it to the default iPi fbx exported skeleton would help you any to match move to the video better, you could save the created male/female meshes and merge them each time for binding, or possibly just using add and update animation from fbx importer you could just change out the video on the back plate plane that matches the animation session.

Could easily create morph meshes for each one so they could be proportionally adjusted to a performer just like iPis does.

I don't know totally what you need, just spit balling to see if anything sticks :)

...