Hello,

As far as our tests go, with a 6 PS Eye camera setup, everything gets tracked very good, no body part gets lost, even for a couple of frames. Also the tracking is pretty accurate, except for the torso. It is the part that has the poorest fidelity, in comparison with the movements captured, when compared to the rest of the body parts (well the head also, but with a motion controller that problem is solved). By poor fidelity I mean things like:

- on jumpy moves for example, you can see the ipi actor going less in height then the actual footage, this applies also for some lateral movement also;
- most of the twists and bends "offsets" that naturally occur between the chest and the hips are lost, making the whole torso behaving like a solid object. Only on very extreme motions do I see an actual bending or twisting of the spine. And this also happens with the very flexible spine option when tracking.

I was thinking of a couple of things that could improve the torso tracking, things like :

-having as tight as possible clothes on the chest. We did the test with a normal short sleeve t shirt over a semi fit long sleeve t shirt;
-maybe having a full circle camera setup ? although everything else is tracked very good, sometimes the feet don't even need a jitter removal;
-increasing the number of cameras to 8, or even more when IPI will support it :)

Also is there a chance that the torso tracking part of the algorithm needs a bit of improvement :) ? I would imagine it is the hardest part to evaluate accurately since it has a poor movement to volume ratio, making a subtle move harder to track than say an arm or a leg ...

This is my personal experience, hope it helps:

Most of my captures are of dance movements with complex movements, fast hand and feet movements and torso and hips twisting oppositely alot.(Even the individual spine bones).

My studio is 4m (12ft) wide by 7m (22ft) long and I use 2 cams on one end and 2 on the other, (your configuration may vary), really no need for more cams if 4 are positioned well, but thats personal choice, easier calibration and as it is now... saves $500 a yr. on subscription costs.

I know what you are talking about, I was running into the same issue, I don't know the quality of your actual tracking, (you need pretty good quality tracking first), but picking colors with the color picker again after initial refit is better than auto-analyze.(even if it looks not the right color to you, try it, it will most likely work better). You should also re-pick colors if you switch cameras while tracking.

After tracking, fixing any errors, it is as good as you think you want, try using Zero on the torso in JR and only 1 on the rest of the parts and turn Trajectory filter to Zero when running JR.(I avoid running refine forward/backward at all, but your choice).

(Below is my personal experience on my Win 7 machine and system configuration, your result may vary, if you don't agree, don't do it, I'm just posting the info.):

(I don't know if Win 8.1 is the same, I don't have a Win 8.1 machine with iPi on it to test it, but if you run Win 8.1, it's worth to try, it won't mess up anything you have already done to re-run a calibration video, and if it works, it's a substantial difference). If it helps, please post a response here, so I will know.

I run Win7 and I had to personally do a 5mm camera calibration adjustment after system showed "Perfect" calibration and actual cam 1 hieght set, (changed from what iPi instucts to do), and adjusted my actor height accordingly by 5mm and raise the floor grid in scene 1mm and that improved my tracking and smoothness of the final export dramatically. With my machine, the camera calibration was off by 5mm too high when set to camera 1 actual height, might not seem like much, but made dramatic improvement. No other manual camera adjustment was required after calibration, just used slider to cam 1 height 5mm lower and actor height 5mm shorter in scene and floor raised 1mm.
(Based on a 5'8" and up performer, I did notice the shorter the performer, I had to adjust actor height lower accordingly to work).

(Don't worry so much that the actors heels are exactly flat on the floor on initial refit, just reasonably close, the program will adjust better as it tracks to work out fine).

Without doing this correction, I was getting mediocre to OK tracking, but not what I really was looking for, I knew something was a little off, (too much post-process clean up), but now I am convinced I found the issue and made the right correction.

You may never get full 100% first pass tracking quality, especially on complex movements, or if batch processing, but the above helped me and it's 75% better than it was.

I did the same reduction on the cam 1 height on a cal. video file where all my cameras were 2.42m with the same good result, so that's all I do now with my camera calibrations.

I have used several different camera set ups, I just re-ran the cal.video and made the re-cal change for each session files, worked every time.

I may put together some video demos to show how the camera calibration adjustment worked for me, now I have about 20-30 mins clean-up per capture, instead of excessive as it was, and I will show side by side exported bipeds with iPi instructed calibration and how I had to change it and the outcome. It is very noticable to me anyway, and thats who it matters to, as well as much less clean up required.

I will also show the difference in different lighting conditions and how it reacts to the colors chosen while tracking, if camera is switched during tracking process, (which is sometimes needed if "babysitting" tracking process).

Whatever camera you are tracking with sees the color of the clothing different, especially if lighting is a bit different from that cams angle, (which is common with PS Eyes), so if you choose a different camera while tracking, stop the tracking, then switch, and use the color picker to choose the colors again of the torso/sleeves and pants, picking a color area not too light or too dark. (trial and error). The auto-analyze is a bit off to what the colors should be for better tracking to me, use the color picker, if you don't and the colors are off, it could cause jumps or bone clicks as it adjusts the bones for that camera.

After running this V3 program for several months now, I find it best to use as little of JR as possible, as long as your tracking is pretty good to start with, (usually just 1 on all settings), and Trajectory smoothing about 2 or 3, just depends on your preferrence.

I finally got the program following almost the exact movements of the video dancer, all the jumps, bounces, butt twerks and torso/hip twists and bends very close, while feet stay as they should on the ground for the most part, there will be some manual clean up, but I try to limit it in iPi as much as possible.

I use mostly shoulders tracked by video and flexible lower spine (not dancer spine) and head tracking on, (set this way before I even initially refit the actor).

You can very well if needed, stop mid tracking and switch to dancer spine and click refit to track some areas without issue, but use it sparingly as it can cause issues, and then switch back to flex spine the same way.

The head will be an issue with internal tracking on, its ok, but usually looses it some, it seems to like the face a little lighter color, but good thing its the easiest bone to adjust after export, so don't beat your head in trying to make the head tracking perfect, I don't know about using a Move for the head, I don't worry with it, too easy to fix slight errors on export.

Try to get your first forward tracking pass to be as cleanly tracked as possible, try to limit the back-tracking/refining, as it does affect the integrity of the animation file, if you care to have animation as clean and smooth as I do from this mocap system.

There are some other small glitches in the program, as you use it, you will notice them and adjust accordingly.

Hope this helps..

Good Luck!

Tnx for your input :).

I did a new calibration and I fine tuned the camera heights based on their real ones. I also bought tight clothing in nice pretty colors :).

I saw a big improvement on the shoulders, although sometimes they get bad tracking but that's because of my camera setup. When they pop they are usually caught properly by only 2 cameras, so that is fixable with a 360 setup.

Also a big improvement on the spine twist and roll (meaning bending sideways), BUT

As far as I can tell, the spine completely lacks any forward/backward bending. Here is a video:

https://www.youtube.com/watch?v=yUhCe2BibXs

As it can be seen, when I start to bend forward, it follows until my back starts to become really arched and then it just refuses to bend. You can see the jitter increasing as the system tries to follow the arms and shoulders while the spine stubbornly refuses to bend. In the end, when my bending becomes too much, the system compensates by rotating the hip too much, resulting in an unnatural pose. Same thing happens when I bend back but to a lesser extent since the human spine cannot bend backwards as much as it can bend forwards (obviously :) ).

Now, besides the main problem, the hands get dragged upwards also, no matter how much the shoulders are compensating.

So in conclusion, I think this is a "solving" problem of the spine system in IPI and not a tracking one. In my humble opinion the tracking is very good.

Could you please tell me Snapz, this bendless spine behavior, was it like this in V2 also? Perhaps it's a bug with V3? If not, this is a serious problem because although above it's an extreme pose, that lack of bending of the spine creates an unnatural stiffness in common movements, although more subtle but still, it's there and it takes away from the motion. And it's such a pity because everything else looks great :(

Also, if someone from IPI could shed some light on this I would be very grateful

Well, spine looks ok bending back to me the way you did on video, unless you're talking contorsion artist moves :)

I am not sure about the Move controller on the head affecting the back, (I don't use it), have you tried just internal head tracking and see if there is any difference?

I know my dancers do somw strange bends and twists and as I have my set up now, it pretty much tracks them.

The dancers spine setting releases the pevis bone to be more free moving, so really no need to use that setting unless really needed or it will cause adverse effects.

The problem I was having before was the feet wanting to leave the ground on simple hip lifts and twists and performer stretching high in a dance, I solved that now though.

Not sure what you're looking to find out, if the actor mesh is following the video pretty close, thats all you're going to achieve seeing it in iPi, the test comes when you apply it to a custom character mesh.

You also have to remember it's a 3 bone spine, not like a human, 3 to 4 spine links is common for games, too many and it smashes the mesh weirdly and just plain unnecessary anyway.

There is bugs, but I don't think the spine is one, you will see when you run it on a character mesh or even on a biped skeleton in 3ds max, it should work fine, just needs a bit of manual tweeking in an adjustment layer.

I analyzed your video closer and can see the actor mesh isn't aligning correctly with you, it shouldn't ghost the performer (you) as it is, should be a lot closer anyway.

The arm twitching is a common affect of either mis-calibration, Moves not used for wrist tracking, or over-extended reach, iPi does have an issue with far reaches for some reason, especially up, it tries to pull the hips up and feet off the floor, but again, I made that less of an issue with custom calibration.

Your head was still kind of looking upward in the video when bending forward, which tells the back to remain more straight at the chest, try a full arch with head looking at knees once and see if any difference.

I see other slight twitching in the hips that shouldn't appear that way when everything is right, especially at 3 on TS, even with Zero in JR. you will see some movement, but shouldn't be a twitching action.

If the actor mesh doesn't stay closely locked to the performer in all cams, as it should, something isn't quite right and will cause shoulder/other problems and arm twitches could be because you didn't use Moves for wrist tracking, in V3 they are pretty much needed for hands, in my opinion.

I hear that more than 4 cams is a bit tricky to get calibrated properly, but I haven't tried personally.

I will try to send you a video link of a dance movement through email, see if you can see the difference.

Well, problem solved :)

It was all about the head tracking. I couldn't believe how much that influences ..well, everything:

- the spine now bends and twists almost 100% accurately;
- the shoulders are better;
- even the hip gets some rotations that were missing before;

I really can't believe how much the head tracking improved everything

Of course, I still apply the motion controller data afterwards on the head, you should really try using motion controllers, they are great.

So to conclude, NEVER track without head tracking, although it doesn't track the head rotations like a motion controller it does wonders through its positional tracking for the whole torso (chest, hips, shoulders)

Right now, if I would have had two motion controllers for the hands I strongly believe that no jitter removal would even be necessary, only a bit of trajectory filtering should be enough. And that's even without refine, just normal tracking. Again, I'm really stunned :) And these great results are with the stiff spine, I will do some tests with the flexible and very flexible settings to see what I get. There is also a cartwheel in the capture(it's not in the attached video) that now looks as it should.

One thing though, is it me or JR on 1 is too aggressive? It actually creates identical keys on the torso, on subtle neighboring poses.

OK, well good, glad you worked it out to your liking.

Yes, I thought head tracking off was affecting things.

1 in JR shouldn't be too aggressive, if tracking is very good, I guess just use what you feel you need for the quality you want to achieve with how good your tracking is.

Nah, not strapping a Move to my or my dancers heads, internal tracking works well enough, what it does loose can be cleaned up too easy in 3D editor.

I hope eventually iPi will get a better tracking system for the hands, as the Moves aren't all that accurate, in my opinion, but it is what it is now.

As long as "YOU" are happy with the results, is all that matters :)

Good Luck!

Had a very similar experience. Based on a previous recommendation by Snapz, I ... was also somewhat shocked by how much head tracking helped the torso, having seen no reference anywhere to indicate this would be true. In my case, I was dealing with motions with a lot of knee bend, and although the head tracking, (and shoulder from video) helped things hugely, the flexible spine (the middle choice), gave me the last bit of improvement I was hoping for.

Based on your post, j_brujah, I think I have the answer to another question.

I've set up a motion controller rig, for my head. I speculated whether this would remove the need for the head tracking (which is fairly expensive ... processing-wise), or if I would still need it, for the benefits to spread out through the rest of the skeleton. From what you say, I'll still need to do the software based head tracking, with the motion controller getting used afterwards, to do more of an isolated head orientation improvement.

Since the processing is fairly brief, compared to the time to integrate the motions, at least for me, I'm going to be maxing out the setting, from now on.

Thanks.
Mike

The explanation for why we need head tracking on is that the motion controller records only rotations, so the position for the head must be tracked "traditionally" and on top of this it also affects the spine :).

Now about those subtle spine movements (weight shifts and the such) I did some tracking on footage from both the sample on the iPi wiki and from someone here on the forum. Thing is I got great results with the spine on these and the only difference that I can see are the bigger camera distances. Right now for my test setup I barely can fit a capture area of about 2x2 m and obviously my 6 cam near half of circle setup makes the cameras pretty cramped up. What I'm saying is, I think my 6 cameras are to closely positioned to offer the tracking algorithm the proper differences it needs to solve the skeleton at the quality that it should.

Could this be a reason too? That's why I keep saying that with an 8 camera 360 degrees with larger distances between the cameras, I will get better results for the torso.

On my camera calibration, it works fine as long as I have at least 1 camera set at 1.50m and use that as the known camera height, doesn't matter what camera number it is, but if I try using a different camera height as camera one, it throws my calibration off by 5mm, but that 5mm makes a big difference in my tracking quality.

I have a fixed camera at 1.50m, but wasn't camera used as camera 1 when I did some calibrations, I fixed it stationary at that height, because V2 used that height well, and in V3 the camera height is 1.50m shown on the calibration screen as the camera 1 height.

So, I at least will use 1.50m as my camera 1 height now, no matter what heights it actually sets for the rest of them, as long as they are close and all should be ok.

I just know it didn't work using any other height for cam 1 and track correctly as it is now.

On another file where all cams were high 2.425m pointing down on actor, I had to adjust by 5mm also and it straightened the tracking right out, actor centered to performer throughout, where it wasn't when used the actual cam height as cam 1.

In any regard, it's tracking well again with minimal errors, so I am happy.