...

Calibration is off a lot then, did you set cam 1 height to exact real world height of the center of the camera 1 lens after calibration process completed? That must be right!

After that is set right, all other cameras should fall into height within a few cm of what the real world actual height is, but they don't have to be exact, just not wayyy off real world height.

You also CAN NOT have moved any cameras after you calibrated them, or you MUST re-do the calibration process again, or the recordings will never match in iPi Studio.

Also having a center point marked on the floor so the cameras can be pointed at it, but at chest level of the performer helps to keep all cams and the iPi Actor to come in more centered to the grid.

You should also manually spin the grid to run with the Actor, if it doesn't come into scene that way, some times it will, some times it won't, but making that change will help when manipulating the IK later.

Your actor scale in scene should be no more than 1 or 2 cm from the original performers actual height, even if the actor looks a little smaller than that when you load in, you can size up to get colors selected better, but scale it back to performers height before you hit the refit button and all should resize and snap right to the video correctly.

Then hit the refit button a few more times back to back so it gets a better fit, before starting to track.

...

I didnt set the height of camera 1 to anything..nor have knowledge of defining a center point ..didnt know I had to do that I will have to read the manual better. Let me do all the things you said and see how things go. Also having a contrasting background is new to me. Got rid of a messy bookcase behind my stage.. now its clean white wall.

Let me try again and I will repost. Thanks a million Snapz

...

Yes, going over the Docs from iPi is a good getting started point, it covers the basics and you can adjust to your own preferences from there.

I don't know iPi mentions a center floor marking, as this isn't a mandatory requirement, that is just a common method in most optical capturing systems, 1) for the actor to know where they are in relation to center 2) for the system cameras to align better with during calibration.

(Don't point the cameras at the actual mark on the ground, point them to intersect there at the approximate chest height of the performer, this will still work even if performers vary in height from average performer heights of 5 ft - 6 ft, if you have a session recording children, or very tall performers, you will probably want to adjust the cams for that.

iPi does a fairly good job at separating the background from the performer when tracking, but it is always best to have a less cluttered and contrasting environment with any optical system, it will save time later in clean up.

iPi also doesn't require intense lighting of the performer, it will wash the colors in the recordings, just use enough light to illuminate the performer to get a fairly accurate actual color of the clothing worn in the video.

You can also adjust the camera settings so each camera has a similar view port appearance and I personally turn off the auto-gain of all cameras.

With some performers having lighter skin tone, especially for the hands, it may be beneficial to have them wear colored gloves of a lightweight material for better tracking, this also isn't a requirement, but it does help, as the hands seem to be the hardest thing to maintain tracking of with iPi, even if you are using Move controllers.

Note: White T-shirt on a white wall is bad contrast.

Also, the iPi Actor skeleton accuracy to the video is important, for arm, (wrist joint to match video performer), and leg lengths, positioning the hips bone more toward the center of the butt of the performer, (low point of the hip bone at the groin), as well as the overall iPi Actor mesh proportions to match the performer closer on the video, better to have the mesh thinner than thicker in most cases.

Once you get it solidly set up and replicate the steps for each session, especially if must break down the set up after every session, you will get better tracking and do it much faster without worry.

Good Luck!

...

I'm tracking right now at half resolution, head enabled, shoulders from video, Flexible lower spline.. with 3 cams. It's tracking perfectly now at 6 fps.

Setting the height of the first camera accurately fixed the dummy size.

The main thing I was overlooking was the accuracy of the dummy to the actor. leg length (proportinally accurate in comparison to upper body length), feet size, body mass size, head size, etc.. once it was adjusted to be close to the actor, it now tracks well. I can see how doing 360s even will be very possible if I had enough space to put cameras behind the actor. I can put 200 degrees set up with cams.. so I could do maybe 180 rotation. it's definately more accurate than one Kinect v2. With Kinect, I was getting lots of occlusion errors.

and Yes, I figured white t-shirt with white background wouldn't be good, so I'm wearing yellow T-shirt, blue long sleeve underneath and black pants with white socks. I think I'm going to try with an orange shirt.

Even at 320 resolution it's tracking quite nicely.. no errors so far. I can see why you do most of your trackings on Low resolution. for me it's 6.1 fps vs 4 fps. Hope IPI will optimize tracking for multiple GPUs so that it can utilize my FOUR 1070 gtxs.

I'm getting my active USB cables tomorrow, so I will try with 6 cams on rectangular set up and two high up for feet tracking as you suggested.

Can't wait to try some nice motions on my own characters and Daz custom characters with cloth simulations and highend rendering (redshift and octane). : )


thanks Snapz again. Will keep updating.

...

Great! Good to read you are figuring some things out, once you realize how to do things the same consistently, you will see how easy it really is to get very high quality tracking.

You don't absolutely need a rear camera, especially with your room size, I sent a camera configuration drawing to your email for a suggested set up that will allow you to get full 360 rotations easily.

Yes, colors used should be as saturated as possible, yellows and whites aren't good and you want them deep enough to not receive many self shadows.

With only 3 cams you will get faster tracking fps, but you will still, or should be above 5 fps with 6 cams @ 60 fps using 640 x 480 resolution, although iPi doesn't need much resolution for it's tracking, it really just needs good saturation of colors to lock onto throughout the video.

When I stated use Low Res speed, I meant the setting on the Tracking options, where it says "High Res-Recommended" switch that to "Low Res", not the actual video recording resolution, but you can probably use a combination of both also.

You are on your way now it seems, now you just have to optimize your workflow and things will go much smoother once exported.

...

Here's my first capture using 5 ps3 cams. half circle (snapz, I couldn't find your email with your recommended camera set up, so I just went with what I though was good. I'm waiting for more active usb cables, so when I get them I'll add the last one for total of 6 cams). I can rearrange the camera set up then.

when exporting form IPIstudio, I changed from 50fps to maya's 24fps.. so it's a bit jittery. also only used one move controller for the right hand. the head tracking via recorder is a bit jittery as well. oh well, this is my first capture.. so I'm sure it will get better.

Rendered in Redshift 3d using four GTX 1070 at 11sec a frame. outputted from AE with text, etc


https://www.youtube.com/watch?v=_V8_VCY ... e=youtu.be



this one is from Kinect v2 capture I did a while ago.. around 34sec, you can see the hit/torso bends weird.. don't know how to fix that.

https://www.youtube.com/watch?v=4jSLPll-qvo


This one is an older one also with kinectv2 but with qualoth cloth simulation. I'm still learning qualoth and haven't figure out all it yet. it's penetrating at some part of the clip.

https://www.youtube.com/watch?v=kv4tjxI ... e=youtu.be

...

Looks like you didn't have head tracking on during this tracking?

Noticeable elevation in animation quality for just the first attempts over use of 1 Kv2.

Hmm, not sure why you didn't receive that email, I will try again, but I didn't know which way your room L/W was, so drew it as 3.5 m (L) x 5 m (W), but if it runs the opposite way, that diagram probably won't work anyway, it looks like you had no issue with a full 360 rotation how you have them set up now though.

Like you said, just 1st tests, you will get things optimized to look much better, especially once get into more extreme, and actually usable motions.

It is also better to perform your motions with a bit more over acting, iPi doesn't really track subtle motion all that well and it may look a bit stiff when exported if you don't make you actions just a bit more exaggerated than natural would be, also best to maintain a perimeter of approx. 4-6 inches off the body with the arms and hands in relation to any other part of the body, it will track better which is most important and can easily be adjusted in an external editor later.

With the recent iPi Studio update, can't you export at 60 fps, even if a 50 fps animation?

Maya will automatically bring animation files at 1/2 the frames of what they were recorded in, but it will or should appear at correct speed, if you manually switch it to 50, or 60 fps, it will just add frames to the animation timeline, it shouldn't increase the animation speed.

Without a Move controller on the head, you will get some glitching on the head, but it is usually easy to fix on a layer in editing in external 3D editor, a tip also it to restrain the hair within a tie type dark color head wrap, and keep hair off the sides and back of the neck also, you can try using full (5) on the head during Jitter removal and set Trajectory Smooth up more before exporting.

...

I set it up 6 cams as per your diagram. Had a hard time with choosing the right clothing (kept getting tracking errors on left arm and on foot, so ended up wearing black socks and changing long sleeve to dark blue)

Anyways, also tried tracking with 3 playstation movers. head one mounted and tied to a headphone and the two for hand just holding them. Took a couple of tried to get the position right, but I think I sort of got hang of what to do.

Here's my first 3 move controller tracking (with 6 cams)

https://www.youtube.com/watch?v=5-GrpP6uRZY\


I'll keep optimising, and honning on details. I realized that with my set up (half circle), it's much better staying facing frontal and staying maybe below 170 degrees. without the back camera, if my arm is totally facing back, it kept getting tracking errors. maybe it's because the background is too mess for y sideview cameras and also lighting is not evenly diffused for the whole room.


I'll keep at it.. but I do like those movers!

...

That looks fine, Not sure why your loosing tracking on a rotation, maybe slide the side angled cams straight forward, to see if they can see more of the arm and should pick up the other as you rotate along with the 2 rear cams, object is to get 3 cams in triangulation to always see a body part, but straight to the side of body at 90 degrees may still cause a loss, just let it go and pick it up on the other side by stopping the tracking and refit the pose after it's visible well again and continue tracking forward.

Possibly, by looking at the mini view ports of scene, your cam 1 and cam 3 are a bit off center, not that it matters greatly, but you can try to get your center ground mark to be closer in all view ports, even if you have to place something more visible on the floor to show the center point, like a can or a small chair, just remove it before you do your calibration.

Also, you may try dark solid colored gloves, you have a white wall behind you and lighter skin tone, the Moves don't get picked up as color during tracking.

Overall, it looks fine, but you are looking a bit unnatural in your motions, (stiff), I understand just tests now, but later on don't watch yourself on the screen when acting, it will mess you up :) Just do the performance with a bit of fuller motion, even exaggerate the motions a bit, it will track better and actually look better on the animations.

Looks like the head Move is working good for you also, but be warned the headphones may not keep them stable during extreme head motions, like throwing head down or back, the hands should always maintain pretty good results, just try to match the actor hands to the video as close as possible before final application of the Move data before exporting, now later you can learn setting hand gestures and making your own gestures and saving them for later recall.

It looks like you have plenty of room to move around, as long as you keep the feet when on the ground in at least 3 cams views when moving away from any cam, the closer cams can usually loose full view of the feet a bit, (maybe try to mark a toe limit perimeter on the ground also), and the upper body and arms can move even closer to the cam without tracking loss, but you should keep at least a 2m space from any one cam, even with the hands or feet, like in a kick or punch action.

The lighting looks fine also, maybe even a bit too bright, try turning your camera gain down slightly maybe, to around 15, it may look a bit dark on the camera view port, but the studio will lighten it back up more in tracking video.

Yep. just keep playing with things, looks like you are making good progress, these are all just suggestions, you will figure out more as you work more with the program and with other performers other than yourself, if you ever do that.

...

Snapz, not sure what helped, but I did a few things that improved my tracking quite a lot.

-centered the cameras (put an object in the middle and tried centering it in all cameras)
-wore gloves
-put white cloth on top of things in the background (monitors and desks etc.. which were dark colored)
-turned off custom color/hue/exposure and set everything back to automatic

I got this tracking without any major errors (some feet distortions had to be manually repositioned) and came out a bit smoother than before.


https://www.youtube.com/watch?v=uQ0HtAD ... e=youtu.be

I think once I layer in some animations on top of the mocap in maya (hand/fingers, neck etc) it can really be good.