I have been trying to follow tutorial on tracking (iPi Studio). My UI does not seem to be that same as any of the tutorials.

The tut states to load the scene from the Scene tab after loading the video. As soon as I load the video, that Scene menu completely changes and there is no load button and I can't adjust anything. The Actor menu also changes when I load a video. Everything below "Legs Morph" disappears.


Any ideas?
pixlmonky

Maybe a graphics card issue? Check the specs of your card against the software requirements. The most important thing is that the card supports SM 4.0. If it's not compatible, iPi DMC will not run properly.

Also, are you capturing with Kinect or PS3 Eye? If you used PS3 Eye, do you have a valid calibration video? With either system, you will also need a few seconds of clean background video at the head of the footage.

Hope this helps.

G.

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)

Watch a one minute excerpt on Vimeo now!

It is a quadro fx1800 with an sm of 4.0.

I captured the avi with a Kinect.




pixlmonky

That should work fine.

I did a lot of Kinect capture this weekend (three characters for a 1 minute short,) and the tracking results were excellent. (I still need to retarget the data.) My workflow was as follows:

1. Setup the Kinect in an appropriate environment. You want to avoid a lot of solid black surfaces as the IR sensors will not work with black surfaces. Likewise, you want to avoid wearing shiny black clothes (I had a slight problem wearing a black belt.) You will also want to shoot in a space large enough that you are away from objects behind or near you, like a wall or large furniture which may interfere with the actor volume.

2. Make sure you have plenty of diskspace and disable any processes or programs that can interfere with capture. I typically disable network and Wireless connection just to be safe...I've had software (most recently, Adobe Acrobat and Flash) try to 'auto update' during a capture and they messed a session up.

3. When shooting, avoid occluding your limbs. iPi cannot track what it can't see. (This is wear the PS3 Eye system shines since it can over a much wider range than Kinect.) I also found that if you bring your forearm too close to your chest, the Kinect volume data can 'blob' in that region and confuse the tracker, so avoid doing that.

4. Before performing, allow iPi Recorder to capture some 'clean' background video This is used by the software to help separate the actor from the environment. After a couple of seconds, step in and record your T-Pose, then start your performance. Make sure you do multiple takes to minimize the chance of having to come back for 'do overs.'

5. After capturing, simply open a movie file in iPi Studio. Studio will automatically calibrate Kinect data. Make sure the Background range is properly set. This should happen automatically unless there is something unusual about how you shot the beginning of the video, in which case you will need to manually set it to a clean range.

6. Scrub to the T-Pose. I usually pick a good frame towards the end of the pose. Press 'i' to set your in frame. Click Refit Pose a few times. If the actor doesn't automatically position itself correctly, you will need to make manual adjustments to the actor height, shape (morphs) and/or position. Once you have a good fit, save your Actor. This Actor settings file can be reused for later sessions.

7. Scrub to the end of the video and find a desired out frame. This would typically be at the end of your action and before your 'walk off'. Press 'o' to set your out frame. Press 'm' to go to the in frame or click the Go to First Frame button.

8. Now you can start tracking. Normally, I will scrub to the first frame of what I want to track and set a new in point from here. If you do this, you will need to move and repose your Actor. No need to be exact, 'close' is usually good enough. Then click Refit Pose. Alternatively, you can just start tracking from the T-Pose frame, though if you this will probably be tracking a lot of data you don't need.

9. After the 'first pass' is done, play back the footage an look for egregious tracking issues. The motion will be 'jerky' throughout but don't about that; we'll fix that later using the Configurable Jitter Removal. To fix the major problems, sometimes it's easiest to go past the error, hit Refit and track backwards through the error. If this doesn't work, you'll need to pose manually in the middle of the error and track backwards and forwards from that frame. You want to use Refit Pose but in certain situations, this might not work well and you'll have to go with the manual pose. Tip: you can copy a pose and paste it to a frame that got mistracked. Use Refit Pose if you paste a pose to a different frame.

10. When you're satisfied, it's time to use Jitter Removal. I like to run JR throughout the entire motion the first time and then tweak as necessary. JR is added non-destructively on top of the tracking data in a separate layer so it can be redone over and over again without needing to retrack the scene.

Open the options pane for JR. Typically, I like to use just a little JR for the arms, and just slightly higher for the torso, and go very heavy on the legs. But you should adjust to to suite your own tastes: too little JR may look 'fidgety' and too much can make the motion look 'robotic', so you'll want to find what you like by experimenting.

You can use different settings at different points of the animation. To do this, you may want to set new in and out points for the new settings. After using different JR settings you will want to se Trajectory Filtering to 0 and then back up to your desired setting (typically 1 or 2.) This helps smooth the transition between the different JR settings. If you don't do this, you may see a pop where you changed settings.

11. You will want to set your 'Takes' for each motion clip. You set as many takes as you need in a scene, and even give the take a unique name. To set a Take, right-click on an empty white area in the bottom timeline and choose Add Take. Then adjust the in and out points for the new Take; you'll need to set this manually. Tip: the end points will snap to the ends of the upper timeline's in and out points, so you can adjust this to frame precision by stepping through the motion using the < and > keys. If you right click on a Take, you can rename it or export the range as an animation or motion file.

12. Finally, I will retarget and edit my data in MotionBuilder and apply the results to a rig in Lightwave. I've done this many times using the PS3 Eye system and the procedure is the same for Kinect. Some people prefer to do this directly in Max, XSI, Blender, Maya. Another option is to use the free Animeeple program which can also retarget for other programs. You can can even do this within iPi Studio by importing a rig via Collada or FBX. See the Wiki for more details.

Well, that's about it. Hope this info helps.

G.

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)

Watch a one minute excerpt on Vimeo now!

Thanks for the detailed description. That worked perfectly first time thru.

The tutorial I have seen were horrible. This description should be sticky'd. Very helpful.



Do you use multi-cams? Have you used this at R&H, if you are allowed say?





cheers
pixlmonky

A quick question on re-targeting. Do you do any targeting in iPi or is it all done in MB(or other)? I was trying to import skeletons from Maya into iPi but nothing was working (FBX, BVH). Then I realized that targeting in iPi might not be the best solution.

Thoughts?
Pixelmonky

Up to now, I've been using mutliple PS3 Eye cameras, which are the way to go for best quality but it's also much more complicated to set up. My current project uses only the Kinect; it's a test to see how quickly and simply I can create a high-quality short film using iPi DMC. So far the results are very good. We're committed to finishing this film in just a few weeks.

My other project is a bit more ambitious and it used the six camera PS3 Eye setup extensively. It was started almost exactly a year ago and I'm still not sure when we'll be finished making it. Right now this project is on hold while we finish the smaller project, and we'll probably jump on finishing another film we started long before we even got involved with iPi DMC and mocap. Now that we have established a good workflow, we feel we can finish this older film very quickly too, and very possibly with using only the Kinect.

As for using iPi at R+H, no. It's been discussed but I think my boss first wants to see me actually finish a project using the system. :p

Typically we have our mocap at R+H created by Giant, the same folks who did the work for Avatar.


I rig in Lightwave since that's my final renderer and export an FBX for MotionBuilder. Then from iPi I export a .bvh, and characterize both files in MB and retarget there. The reason this is that my character and rigs have odd proportions and non-standard skeletons, and this is easier to deal with in MB. When I'm done in MB, I export an FBX and using Lightwave's Merge Only Motion Envelopes to import the data back onto my original LW rig. This has worked flawlessly with data from the six camera setup and should work very well with the Kinect data too.

I have yet to try importing the rig directly to iPi Studio and retargeting there. I'll give it a shot after we're finished with our current project.

G.

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)

Watch a one minute excerpt on Vimeo now!

Sounds interesting. Wouldn't mind getting your thoughts after finishing the projects.

We use Maya here and haven't had a need for MB yet. If we start using mocap, we would need MB. Plus, I think the animators might enjoy some of the features of MB.

Mocap data would be targeted to an FK skeleton right? Is it possible to target to a skeleton with an IK setup. I can't think of a way. But it would make it easier for the animators to adjust especially if the character is jumping or picking something up.



cheers
pixlmonky

MB has it's own IK/FK control rig which is applied to any FK skeleton using drag and drop. It's very easy to use and works well for a traditional human biped character. The controls can be modified as needed--for example, I'm using it for biped 'cats' with procedural tail animation, and the mods were pretty easy to do once I was able to wrap my head around how things worked in MB.

I don't know for certain but I think the new Human IK system in Maya is based on the MB counterpart. At a glance they appear to share many similarities in application and behavior (based on a video I saw earlier this year.) I haven't used this feature in Maya yet as I don't have the latest version.

G.

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)

Watch a one minute excerpt on Vimeo now!