Thanks folks:

I had anothe shoot last night, and it still won't calibrate. The capture area is 12 feet by 16 feet. I did the table leg LED motion from Greenlaw's calib video, and did spirals throughout the space (I walked along the outside of the capture space and did the spirals as I went - I could only reach about 6 feet inside from the edge,but the full capture space was covered).

I'm sending the links to the capture video to ipisoft so they can troubleshoot the issue.

If worse comes to worse, I can use Svboda's matlab programs, but I may need some pointers from ipisoft on how to convert his camera matrix into the scene.xml file format.

Wish me luck,

Michael 2

I was wondering if this might be a hardware issue because I understand that iPi Studio leverages the graphics card to do all it's 3D processing.

What type of card do you have in your computer? iPi Soft recommends an Nvidia GTS 250 or Radeon HD 4850 or better. If you have an appropriate card, make sure the drivers are up to data.

FWIW, I've used older video cards with success but performance very slow compared to I've been getting with more modern cards. Cards that are even older or other makes (like Intel Graphics) might not be compatible at all.

G.

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)

Watch a one minute excerpt on Vimeo now!

Naw, it's not the video card. It's a new generation GTX 560Ti, and when I used it to track the mocap I did in my living room, it got between 1.6 and 1.8 seconds per frame with both shoulder and foot tracking turned on. There wasn't much of an issue with the calibration with the living room mocap once I switched to your table leg pattern with spiral center for the LED movement.

It's either bad video interpretation, bad RANSAC cleanup, a bad LED movement pattern, or a bug or instability in the program. Frankly, from previous experience, I'd vote for instability in the non-linear optimization section of the calculation process. I experienced the same thing with the Svboda matlab programs. However, I'm actually going to try using the Svboda programs to see if they can give me a calibration.

Or maybe ipisoft will give me the magic bullet to kill the monster,

Michael

Thanks for the debug video. We analyzed it. It is indeed difficult to calculate calibration from this video.

The main problem is the positions of the cameras - not enough overlap of the space seen by cameras. When you move the marker, you (and the marker) are out of view of one or more cameras all the time. Our software is not sophisticated enough to handle calibration for such a loosely-coupled multi-camera system. We will work to improve our algorithms to be able to process calibration videos like this one, but it may take us unknown amount of time (at least two weeks) to come up with new version of algorithms. Your video is a very interesting test case and we appreciate your contribution, it will help us to make software more robust.

For now, I recommend the following workaround. In your next video (mocap) session, point the cameras in such a way that they see the same space. Put something on the floor so that it is easier to see what's going on in each camera view. For example, put four bottles of Coca-Cola into four corners of your capture area, and make sure all four bottles are visible in all cameras.

By the way, what do you use for a glowing marker? Do you use a flashlight with a xenon light bulb, like Mini Maglite? Or a LED? Your marker looks very small on video. Most of the time, our software is able to track it, but there are some frames on your video where marker is barely visible (like, really barely visible, not only for software but also for a human eye). I recommend that you use a Mini Maglite, it is brighter and looks like a bigger white point on video. Mini Maglite should be easy to get in USA. Our software is tuned for typical visual size of Mini Maglite. If you use a LED, software should be tuned for much smaller marker size. I think we may make marker size range user-configurable in future versions but I'm still curious to know why it looks so small on your video. Overall, I don't see a big problem with marker tracking on your calibration video - in some frames maker was difficult to detect, but in majority of frames it was OK.

Hi Michael:

Thanks for looking into this for me. I'm used to marker based mocap systems that don't require more than 3 cameras to see a marker at any one time. I've got a link to Dr. Svboda's calibration programs in one of my posts.

I may still be able to use his algorithms to do a calibration, but it seems like your software won't work if it can't see the dancer in all the cameras.

Thanks,

Michael 2

Sorry to bug you but I'm still curious, do you use a LED or a Mag Lite for calibration?

Theoretically, it should work. Actor tracking is coded in a universal way and it does very little assumptions on where the actor is on video. When actor is not visible in some cameras, this may affect the weighting factor for visual components of cost function (calculated from video) versus non-visual components of cost function like joint limit violation cost (calculated from skeleton pose). But this is a very slight effect. Losing half of the cameras distorts the weight of visual components by 50% which is very little and the optimizer will not even notice it.

In practice, this does not work well as of yet. We have some tests were actor is out of view of a few cameras for short periods of time. Even with 6 cameras, there is a problem when actor is out of view of some cameras. Our current calibration algorithm results in less accurate calibration near the borders of camera view. When actor is visible in all cameras, such inaccuracies average out. When actor is out of view of some cameras, this usually means actor is close to borders of some other camera views. The combined effect of this is lost visual information plus distorted visual information. As one might expect, this results in much less accurate and much less reliable tracking. This will improve in future versions as we develop more accurate calibration algorithms.

No, the non-linear optimization section of the calculation process uses advanced global optimizer and is very reliable (at the cost of computation time). The problem is in our RANSAC implementation (the "Auto detect initial camera positions" option). It is very primitive and sometimes fails to calculate initial camera positions correctly. Once we get correct initial camera positions, the rest should not be a problem. Unfortunately, it is very difficult to manually set initial camera positions in your video because camera configuration looks very unusual and there are not visual cues for human operator. We will work on better RANSAC implementation. For now, the recommended workaround is to set your cameras in a more predictable pattern, so that it would be possible to set initial camera positions manually.

Hi Michael:

Thanks for all your attention to this issue:

To answer your query, I posted 2 calibration sessions, I think the first one might have the maglight calibration. The second calib video uses a LED flashlight which I agree is only bright when it is facing upwards. Some of the second video is done with a white LED, and some is done with a blue LED. I tried the maglight, but when I placed it against the ground, the bulb bent in its holder. I'll definitely use it for my next calibration session in the rental space.

I went back to the site today and took some measurements, this should allow me to create a more accurate initial camera position.

I was wondering about a couple of questions that I want to ask you about:

1) To salvage the old video, would it possibly work for me to edit together a 4 camera video of both the calibration sequence and the mocap sequence? I believe that I could calibrate based on points that are in all 4 cameras.

2) To get coverage on the space, I'm thinking about putting the 2 lower cameras very close to the ground beneath two opposite corners of the room (below the larger tripods, or just off to their side). This would guarantee more calibration points in all the cameras, but no side perspective coverage. Would this work, or do you anticipate fatal problems?

If I wasn't so crunched with the film deadline, I would find this fiddling around with videos and calibration data to be kind of fun.

Thanks for yor help, Michael 2

LOL! I had a similar problem - only it was not me who damaged my Mini Maglite. The person who damaged the Maglite felt so guilty that I had hard time convincing him that it's no big deal and it's too early to commit ritual suicide. I was eventually provided a replacement on his employer's expense. My original Maglite is still alive, only the bulb is still bent a bit. It is not possible to fix it completely but it is still usable (I have to be very careful when I remove the reflector).



Why not just send me a schema of your camera placement (a very primitive hand-drawn view-from-top should be enough). It is very difficult for me to understand your camera placement by looking at the video, while it should be easy for you since you installed the cameras. I will set initial camera positions manually and will send you the resulting calibration.

Eventually we'll fix the software to handle this automatically but the fix may take much time. Currently we are on a crunch to deliver Kinect-related features since we are under significant pressure from a large number of users interested in singe camera version.



I recommend that you just point all 6 cameras to the center of your room. As I described in one of earlier posts, one easy way of doing it is putting four bottles of Coca-Cola into four corners of your capture area, and making sure all four bottles are visible in all cameras.