I have three PS3 cameras and the tracking is not working with my action. As you can see via the image, the floor is not level with my feet either. Any tips or suggestion would be great.

To align the floor with the ground plane you need to touch the floor with a marker at several points when shooting a calibration video. Then mark these ground points when doing calibration. Did you do that?
Please read Calibration section in the docs thoroughly
http://docs.ipisoft.com/User_Guide_for_Multiple_PS_Eye_Cameras_Configuration#Calibration

Next, about the tracking:

• Due to incorrect ground plane the tracking doesn't work correctly when Enable ground collisions option is on. As it tries to place an actor above the ground.
• Loose clothes are not an option as they do not allow the software to get a good sense of body pose.
• For better results it's recommended to have distinct solid colors for pants, torso and sleeves. Otherwise the software can barely distinguish body parts when they overlap.
• Lighting conditions are far from ideal. The main problem is it's too dark so the image is very grainy making solid colors look patchy. Also the lighting is not even producing dense shadows.

Check these sections in the docs
http://docs.ipisoft.com/User_Guide_for_Multiple_PS_Eye_Cameras_Configuration#Actor_Clothing
http://docs.ipisoft.com/User_Guide_for_Multiple_PS_Eye_Cameras_Configuration#Environment

PS. 3 cameras is an absolute minimum for tracking. However, with this amount you may expect a big number of tracking errors which would require cleanup. A recommended set for a single actor is 4-6 color cameras.

Out of all of this, the only one that seems to may be the problem is the calibration marking. I say that because:

-Majority if not all the tutorial pictures and videos have the actors in the same type of clothing as I
-Majority if not all the tutorial pictures and videos are either working with one Kinect or three PS3 cameras
-The website even says 3 PS3 cameras minimum. If in fact we need four PS3 cameras then it should be said as such
-Majority if not all the tutorial pictures and videos have similar lighting if not darker than mine

Something else that the tutorials are really bad at-they jump from one section to another without telling you how to transition. For example, you link to me that calibration with PS Eye cameras yet it doesn't say what to do after you put the markers in place. Okay so what? Do I act in the same video as I did the marking? It doesn't tell you. It jumps right into your 'action' video. =/ Like I said, these tutorials are horribly written and the videos are horribly done and outdated.

I also think the free trial needs to be at least 60 Days due to having to search through all of this outdated material.

To be clear, it's recommended that you wear close fitting clothing because the system needs to recognize a clearly human form to track it accurately. Bulky or loose fitting clothing masks the body shape, making it more difficult for the software to track your body.

It's probably ok if the clothing is a little loose. iPi can probably handle that but why make the task more difficult than necessary? Based on the image you posted, I'd say the shirt is way too baggy and makes it difficult to read where the upper arms and shoulders are. It also masks the shape and rotation/bending of the torso. In short, iPi can't track what it can't see.

For PS3 cameras, you should wear clothes that contrast your torso from your arms. I recommend a black short-sleeved t-shirt over a bright green or red long-sleeved t-shirt. (Actually, for the long-sleeved shirt, almost any bright solid color with do--the key word is contrast.) The color shirt makes the arms show more clearly against the torso, and the black shirt minimizes the shadows cast by the arms. Blue jeans are fine but not baggy pants--again, the more iPi Mocap Studio can see of your body shape the more accurately it can track it. Regular fitting dark shoes is good. Back when I used PS3 Eye cameras, this attire combo always worked great for me.

Lighting is also critical--you want mostly ambient, non-directional lighting. Hard dark shadows will interfere with tracking by making it difficult for the software to read the arms and legs. Back when I used PS3 Eye cameras, I used a couple of soft boxes which provided plenty of light and eliminated hard shadows. This setup might sound like overkill but I got good results with these lights.

If you decide to go with depth sensors (i.e., Kinect or Azure Kinect,) the clothing and lighting criteria is a little different. You still need close fitting clothes but they do not need contrasting color because iPi Mocap Studio does not use the rgb channels when tracking depth data. (The one exception is when you're calibrating the space with a light, but it's not used at all for body capture.) I use depth sensors exclusively these days and I just wear comfortable jeans (regular fit) and a fairly close fitting t-shirt, long- or short-sleeve, so long as the sleeves are not baggy or too loose. Depth sensors seem to track the feet better if there is more volume, so I wear soft boots or bulky sneakers. Also, avoid black with depth sensors because black absorbs IR.

With either type of device, avoid shiny clothing, hats, and unnecessary accessories that confuse the body shape. Also, avoid shiny floors, and reflective backgrounds. Reflections can cause serious problems for the tracker. Oddly shaped backgrounds are not a problem, in fact, they may help with calibration.

For PS3 Eye avoid having the same colors in your clothing as the background environment. If there is too much of the same matching color in the background, you may appear invisible to the software. (Note: This is not an issue with depth sensors since it's not looking for physical volume not color.)

For either type, the calibration stage is critical! Make sure you're able to get good calibration data before potentially wasting time with body capture. Without good scene calibration, every motion you record will be garbage. When I'm recording, I may sometimes record calibration data again after motion capture, just in case something changed in the environment or device positions.

IMO, the info in the Wiki isn't outdated but I agree that it's a bit scattered. But the process really isn't that complicated once you've done it a couple of times. My advice is to think about what the software is looking for when it's recording and tracking data, and optimize your environment and performance for it to get the best results. As the saying goes, 'garbage in, garbage out.'

I think you can continue to use the demo after the expiration, you just lose the ability to export the data. Maybe one the devs can verify this into.

I hope this is helpful. Good luck! :)

(Edit: sorry, I had to change some info about the shirt colors for PS3 Eye. It's been a long while since I used this type of camera.)

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)

Watch a one minute excerpt on Vimeo now!

Yes, that's right. Expired trial keeps most functions so one can evaluate tracking results within Mocap Studio.

I do believe it is the marking and/or lighting.

I don't think it's the clothing because again, all the tutorials say other wise. There are also contradictions to the tutorials. Since the tutorials are published, I can't help but choose to follow the tutorials or why else would iPiSoft publish them? They say the clothing can be black, blue, or green yet one poster said "no black".

I shall do some experimenting with the marker although no one answered my question: do I act in the same video as I did the marking?

This is something I will also be experimenting with. Either way, I shall update the thread as progress is made...if it is made.

Calibration and action video should be shot separately as they require different environment. Dimmed lighting for calibration and bright lighting for action. General workflow of a recording session is

• Setup cameras
• Shoot calibration
• Process calibration to check it's good
• Shoot one or more action videos
• Shoot another calibration. This is optional but provides you some insurance. In case cameras have been moved during the session having final calibration will allow you to process action videos shot after the move.

As I said, the exact clothing colors aren't important. It's more about contrast, in particular the arms against the body and the entire body against the environment. The colors I suggested are what worked well for me, and for many users of iPi Mocap Studio for many years. You definitely want to avoid clothing that masks the body shape and reduces iPi Mocap Studio's ability to understand what it's supposed to track. Black clothing is fine with the exception mentioned a few paragraphs below. You should still have a different color for the sleeves or bare arms so the arms can stand out from the body. If the arms and torso are the same color, it makes the arms more difficult to track accurately.

(Note: if you go with bare arms, I think iPi may want a different color for the hands so maybe you need to wear gloves? If you do wear gloves, make it a single color, distinct from the arms and torso color. Just a thought...I've never actually tried tracking with bare arms before. Maybe it doesn't matter and a dev can verify this?)

Regarding your question about acting for calibration: no, there is no 'acting' in the calibration process. For calibration, you just stay out of the way of the light so all the cameras can see it. During calibration, iPi Mocap Studio is not looking for a performance, it's only looking for the point of light, and all the cameras must be able to see the light as much as possible. The reason for calibration is to teach iPi Mocap Studio the parameters of the room and positions of each cameras so it can recreate it. It's also used to subtract the environment from the motion that you will record later. But at this stage, it doesn't care about the actor at all.

The light can be moved in any random pattern but three rules should be observed: the point of light must be clearly visible to all cameras, the pattern should cover a large enough area (under 6 x 6 ft should be good, just don't go off-camera,) and the light must touch the ground at multiple points. The pattern I liked to use was a spiral and a 'table', with the light touching the ground for 'leg' of the table. The actual pattern isn't important but this one worked well for me and I was able to do this consistently.

When using a light, a darker room is ideal. There is no need for complete darkness, the light just needs to be the brightest point for the software to recognize what to track. In fact, for safety, I would avoid recording in a completely dark room. This light should be an exposed led or bulb, and a small point. You don't want a large overly bright light as that would be harder to track. The globe on the PS Move works pretty well too (this is what I used nowadays.)

BTW, this is one of the reasons you don't want shiny or reflective surfaces for the floor or in the room (like glass windows, mirrors, or polished metal). If the light point is reflected in the environment, iPi Mocap Studio may lose track of which point of light to follow and your tracking data will fail.

You can also use a bright ball in place of the light. Users who record mocap outdoors often use a ball since the light may not show up well outdoors. A white styrofoam ball on a stick works well since it doesn't show dark shadows. (If you decide to record outdoors, there are other factors to consider. Just ask if you need this info.)

For motion capture with PS3 Eye, you want a well lit environment because with RGB cameras, iPi needs to see the actor, and it needs to see the limbs and torso CLEARLY. All the lighting rules described earlier (i.e., soft, non-directional shadows,) should be followed. Trust me, harsh, directional lighting or poor weak lighting can really mess things up when recording with PS3 Eye.

With depth sensors however, you don't need to follow the same lighting rules because iPi doesn't used the RGB data when capturing motions with depth sensors and doesn't care about shadows. However, shiny surfaces can bounce IR rays and degrade the quality of the data. In my case, we have a shiny wooden floor so I like to place a large rug on the floor when recording.

With depth sensors, you want to avoid black clothing because it absorbs IR rays. This is different from clothing worn for RGB cameras.

Also, when recording for calibration with depth sensors, you can optionally use a flat plane to create calibration instead of a point light. I still use this method sometimes. It's not as accurate as the light but it's quick and easy to do, and the result is reasonably accurate. (Yes, I really like depth sensors...less hassle and it's much easier.)

That's really about it. If you follow the explanations from the above posts, you should be able to record good results every time. As mentioned in an earlier post, it's been several years since I used PS3 Eye but I believe the info about is still accurate and up-to-date for this type of camera.

Of course you should feel free to experiment with iPi Mocap Studio as you see fit, and please post any useful and interesting discoveries. This is how many of us learned to use the software.

Good luck!

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)

Watch a one minute excerpt on Vimeo now!

I recently updated another thread of mine. I think the problem may be that I'm not able to import my videos as 'action'. I've only ever been able to import my videos as 'calibration', and then import it into 'action'. I only have the format option of iPiVideo and iPiCalib. I do not have iPiMotion which apparently is needed for 'action'.

As long as Tracking tab is available - you're working with an action project. Tracking is simply not available for non-action projects.
Just try to follow advices you've got here about environment and clothing to improve tracking with your videos.