I'm wearing white socks. I think (and I may be wrong about this) the contrast helps iPi DMC distinguish my feet and ankles from my legs. FWIW, it seems to be working so far. The socks also contrast very well with the green backdrop and floor tiles.

Where the socks failed me, as I mentioned earlier, was when my pants cuffs covered the heels of the socks. I think this confused iPi DMC about the length of the feet and the location of the ankles. But if I keep my heels visible by rolling up the cuffs, motion tracking is very good.

The other reason for the socks (and color has nothing to do with this) is to protect the foam rubber floor tiles. The tiles are supposed to be rugged (they're designed for kids playrooms,) but I want to keep them nice and clean.

That's an interesting idea about wearing socks or gloves over the hands. I guess it might help iPi DMC find the wrist location more easily, but since the hands aren't being tracked anyway there might not be much benefit in it (yet). :)

I"m curious to hear Michael and Andrew's opinion on the matter.

G.

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)

Watch a one minute excerpt on Vimeo now!

I seem to remember being told at one point to hold the hands out flat facing the camera (or maybe it was flat facing the ground), in the T-pose?

That would seem to indicate they are being tracked to some degree?

Are you setting the colors for each item? That Jovian screen grab for the color of a location certainly comes in handy. I wish I could copy and paste into iPi.

I think the hands are acknowledged by iPi DMC to figure out where the wrist and hand joints should go, but that's as far as it goes in the current version.

My guess is that when it tracks the T-Pose, it uses the large 'round' shape made by the hands when they're palms forward and the contrasting color between the hand and sleeve to determine placement of the joints. Oh, and of course it assumes the average human proportions of the arms. iPi DMC may not animate those joints, but the software will at least position them in the right place so you can animate them in another program.

As for actor colors, no I do not bother to set them manually. I prefer to let Analyze Actor determine what the colors should be, and I don't see any reason to question iPi DMC's color judgement.

Another reason I don't bother with manual color adjustment: after the initial tracking is done and I want to 'tighten up' the results, I typically find I need to reset the actor colors using Analyze Actor. Even on a controlled stage I find that the color values may shift during the shot probably because of natural changes in light and shadow that can occur by simply walking around in the room.

FYI, during this mocap refinement stage, whenever I click Refit Pose, I usually click Analyze Actor once and then click Refit Pose again once or twice before clicking Track Backwards and/or Track Forwards. When I do this iPi DMC seems to make an extra effort at accuracy in positioning the joints.

For example, after the initial tracking pass of the three different character performances, I was missing some of the smaller ukulele strums, cowbell hits, and a few subtle harmonica arm gestures, but after doing the above, I was able to bring those small but important details back into the motion data.

G.

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)

Watch a one minute excerpt on Vimeo now!

Greenlaw - I did some testing with setting the colors using Jovian, and I got much better tracking results. It was WAY off on the 'shoes' when I wore white socks for instance.

I'm bouncing my softboxes off the ceiling, and unsure if I have as much variance in color when walking around. I think that may diffuse the light a lot more and result in greater color consistency?

Suggestion: If this is not already the case, if the user supplies color data for the parts, these should NOT be overridden when 'analyzing actor appearance' (or, is that the only thing that button does)? Maybe a checkbox for 'do not reset colors'?

BTW - I have it working almost decently now. I've had to adjust the 'actor's height to match the video instead of using my actual height. I maybe have adjusted it too far this time, as the knees always seem to be bent?

But, it's tracking fairly well in most cases. A number of issues I'll comment on later when I have a better grasp and can figure out exactly what's happening.

BTW - I hope no one makes the same mistake I made and uses the 'scene' tab's move/orbit/etc. controls instead of the controls on the far upper left of ipi when loading in the actual tracking scene video.

That's interesting. I was assuming that iPi DMC averages the colors from all four camera images and then uses this information for, well, I'm not sure. I'll guess it's for calculating a volume and then separating sections of the volume to position the bones in? Anyway, I'm still trusting how the program does this because I'm not fully convinced that telling the program what the colors should be as opposed to what it's actually reading in the video images is really helping it.

(I wish the processes and features were documented a little better; I might spend less time second guessing what the ideal input for the program is.)

That's probably a good idea if you have a white ceiling. I'm set up in a two-car garage that has no ceiling, just rafters, so there is no surface to bounce light from. But since they are softboxes, I haven't found it problematic to point them directly at the set and performer.

FYI, when I'm shooting for chromakey, I usually have two of the lights favoring the backdrop behind the performer to keep shadows very diffuse and to maximized color vibrancy; I'm doing the same for shooting the motion capture, and I think it's helping the tracker by clearly defining the performer's shape. Normally I position the third light off to the side as a keylight, but for motion capture I've been using it head on to even out the lighting. I think that helps the tracker too. (Although, iPi DMC does allow you to resposition the key light in the scene to match the video lighting; in my case, the virtual key light is positioned high center like the real light.)


I think it uses this data to find 3D volume or maybe to isolate parts of the body in 2D space. I'm not sure, it's just a guess. And I think the colors are valid only if the colors remain very consistent throughout the video, which I don't think is always possible. This is why I believe I'm able to get a little more tracking accuracy when I reset it at certain points. Again, this is mostly guesswork about how DMC does its magic, but it seems to work for me at the moment.


It's now suggested that the height be set to slightly taller than the performer's actual height. I was experimenting with different height settings the other day, and the taller setting did seem to work best for me, resulting in better joint placement. I imagine results will be different for different body proportions, and I was wondering if custom skeletons would help. Still waiting to hear from the iPi guys about this.


Cool! Good luck and keep us posted. :)

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)

Watch a one minute excerpt on Vimeo now!

For chroma keying, have you tried adding green gel filters to the backdrop lights?

I've gotten very good results with using the green gels for chroma key work, but I'm only using two of 5 lights for motion capture, pointed straight up at a 22 foot white ceiling.

I'm really hoping to be able to capture dance moves, so having cameras in front and back is important to me. My downstairs is rather narrow, and luckily I literally have no furniture except a grand piano and a kitchen table - so I almost have enough room to make this work.

I'll try to post some images in another thread to see if I can get ipi's advice on the best placement of cameras.

My last track went pretty well, and only went completely bonkers towards the end when I'm lifting one leg as high off the ground as I can. I expect someone performing ballet would go a bit higher, so I'm hoping ipi is not assuming the feet stay on the ground?

My main problem is that when I use IK to position (is there FK?), when I start tracking forward or even backward from that point, it seems to ignore my correction rather than learning from it? Also, at times it might be nice to re-position the pelvis and other parts than just the hands and feet?

BTW - as you were shopping for USB3 cards, are there cards that will handle multiple USB connections at full bandwidth? If I took out a Matrox MX02 mini which I've been less than happy with, I'd have a free 4X pciE slot.

[url]For chroma keying, have you tried adding green gel filters to the backdrop lights?[/url]
That's normally a good idea, but my space is pretty small (It's only about 8 x 8 feet) and using green gels is likely to cause extra green bounce, which of course means more spill to deal with. If I had a bigger 'stage', I'd probably do that though.

As you can see in the small stage pic shot with a still camera, the green is pretty intense and even, and I didn't really light it for chromakey here. As is, the result is still better than many green screen plates I've had to deal with from large film and commercial productions.


Cool! I tried front and back set ups way back in early beta, which didn't work so well at the time. Of course, the software is significantly better now so let me know how well it works for you.


Did you try tracking backwards from the high point? Go to that frame, reposition the leg or just press Refit Pose a few times, and then press Track Backwards for a bit. Depending on the action, Tracking Backwards can sometimes produces better results.

If that doesn't help, maybe it's a joints position issue. Try scaling the rig a little bigger or smaller and tracking again. (I haven't seen the video, but it could also be caused by the background blending with the actor or too much motion blur. Just a couple more things that came to mind.)


Yes, FK is the default mode. Just click a joint and then click the Rotate button.

[url]...when I start tracking forward or even backward from that point, it seems to ignore my correction rather than learning from it? Also, at times it might be nice to re-position the pelvis and other parts than just the hands and feet?[/url]
Hmm. That's happened to me the other night. I think after a while, the software can get 'stuck' and it stops acknowledging manual poses. Save your project, quit, and restart and reload your project. That got it working for me again.


I have two USB 3.0 cards; one came with my Western Digital USB 3.0 External Drive, and the other is made by Asus. They're different types, which was actually my mistake but luckily it worked out perfectly. (The Asus card is a 4X pcIE and it was bought for my old computer but it wasn't compatible, and my new computer just happened to have the correct slot for it. I think this is the card I have the SATA 3 'video' drive connected to.) I'll get the specs when I get a chance.

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)

Watch a one minute excerpt on Vimeo now!

My observations are that tracking either way is generally the same but with opposite problems. Whenever there are major problems with the tracking, I do tend to try it in backward tracking from the last correct refit, and it does work many times. I just think that it is recieving better images from those frames in certain circumstances. Maybe a camera view loses the view of the arm and now only 3 cameras are seeing that arm, for instance. More cameras would also help correct some of this. I like 4 cameras though, and 4 cameras keeps the whole system portable for me.

I've got two of these on the way they are going to sample me as I've had
issues with their USB2.0 PCI card in my free PCI-X slot.

They are both for pciE, so I should be able to handle 5 cameras on just one 5 port USB card, and use my internal USB for a 6th.

http://www.koutech.com/proddetail.asp?linenumber=513
http://www.koutech.com/proddetail.asp?linenumber=388

I'm not sure there are 4 port USB3 cards yet, so I think I'll wait for the dust to settle a bit and pick one up when they are cheaper and limp by for now with the 5 port USB2 card.

Hopefully, at the full 6o fps, my tracking will be better (it's not that bad now), and if I can add 2 more cameras, that certainly can't hurt.

I think you're right. I thought it might have something to do with the motion itself, but image clarity makes a lot more sense.

I'm happy with my four camera set up too. Mostly because it's working and I simply don't want to spend more time messing with it; I just want to get this project done. :)

Also, even though four cameras are working well for me, I suspect six will be more than my system can handle. Just wondering, has anybody tried five cameras? I think maybe my system could handle five.

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)

Watch a one minute excerpt on Vimeo now!