Hello,

1. As there are limitations for Kinect (4 Devices) and PS Eye (16 cameras), are there any limitation for webcams with 720p at 60 fps and 1080p at 120 fps? How many webcams can be used at one? (I will be using webcams with Directshow driver)

2. Has anyone tried recording performance by wearing Florescent color cloths and under UV lights(Example as photo below, in this clothing Face, Hair, Chest, Forearm, Upper arm, Hands, Thighs, Lower Leg and Feet will be of different neon colors ). As this will help distinguish each limbs as separate part?

3. If I wear Each parts that are Hair, Chest, Forearm, Upper arm, Hands, Thighs, Lower Leg and Feet of different colors, will this help better in tracking?

4. If i record calibration video at no lights and then switch on all lights for recording actor, will this work as there will be two different lighting conditions?

5. If I record calibration at 50 fps and then shoot actor at 60 fps, will this work?

Thank you,
Parth Shah

You could try it, although I'm not sure there are any currently recommended webcams.

This last time I used webcams was the old Logitech cameras for iPi DMC 1.0 and, IMO, that was too finicky for practical use. This big issue was getting a consistent image quality from the cameras which required you to constantly tweak them individually.

When PS3 Eye support came along, usage got much simpler because the cameras were much more consistent with their video quality. (The PS3 Eye cameras were also considerably cheaper, but that's probably besides the point since it sounds like you're looking for something with better quality.)

I haven't heard of anybody trying newer alternative webcams, but I would be interested in hearing from adventurous users with experience too.



Mocap Studio doesn't distinguish L/R limbs by color, but color does help to distinguish arms from the torso. This is why a contrasting long sleeved shirt under a black T-shirt is recommended. It doesn't really matter what the color is so long as it contrasts against the black. (I use a bright red or green long-sleeved shirt under the black shirt--the bright color I choose depends on which contrasts better with the background.) Black, BTW, is used for the short sleeved T-shirt because this color is least likely to show cast shadows from the arms and the torso itself. Any lighter color will show shadows which may affect tracking.

Short answer: using all those colors for every body part as described may actually result in poorer quality tracking.

IMO, it's better to simplify rather than complicate what the software needs to work with. Increasing the number of colors in the performer's appearance may potentially increase conflict or confusion with background colors.



In most situations, darkening the room not necessary. That said, back when I used PS3 Eye cameras, I would sometimes dim the room light to improve visibility of the Maglight--the calibration mode is only looking for the point of light, so seeing the room or floor really isn't important.* One thing you want to avoid is a room with a shiny floor--even in the dark, it may reflect the point of light, potentially causing calibration to fail. In that case, it will work better if there is carpet on the floor.

Technically speaking, it doesn't matter if the room lighting for the calibration is different from the motion recordings because calibration is a separate and different process from body tracking.

Room lighting for background subtraction, on the other hand, must to be exactly the same for the motion capture, otherwise 'differencing' cannot happen accurately.



Calibration gives you the locked camera position which will be used for the body tracking process--the result has no motion, just the static location of all cameras. This data is brought into your scene for tracking.

Short answer: it doesn't matter.

Just curious but why would record the calibration video at a different rate anyway? You might be making the process more complicated by introducing more variables to the process.

---

*This is in contrast to using multiple Kinect sensors, which absolutely needs to see the floor during calibration. Interestingly, room lighting is mostly irrelevant even when recording motions. However, for obvious safety reasons, I don't recommend performing in the dark. :)

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)

Watch a one minute excerpt on Vimeo now!

One important consideration: If you go with a web camera that has a higher resolution and framerate, and you wish to use as many as 16 of these cameras, you will likely need to use additional computers to support that many. Mocap Studio, does, fortunately allow that.

I would suggest trying one alternative webcam first, just to see if iPi Recorder will even recognize it. If you're successful with one camera, then try adding a few more so you can record calibration and motion data. If you can get the system to work well with 4 cameras, it will probably work with additional cameras.

BTW, if the cameras are only 60 fps, I don't think you're going to see much improvement in mocap quality over what you get with PS3 Eye, which are also 60 fps. The higher resolution is not likely to improve motion capture quality, but it may allows you to capture in a larger space.

An even higher framerate should let you record very fast motion but this depends on the image quality at that speed. Technically, PS3 Eye camera can record at 120 fps, but the image quality becomes too degraded at that fps for accurate motion tracking.

That said, multiple PS3 Eye cameras are able to capture a 20' x 20' space, which is pretty big when you think about it. If you want the alternative camera to capture a larger space, you will need to consider how to light the space. Remember, Mocap Studio prefers even room lighting with soft non-directional shadows.

Hope this helps. If you get a chance to try a different camera, please post what you learn.

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)

Watch a one minute excerpt on Vimeo now!

Hello,
I totally confirm Greenlaw's answers.
A bit of clarification + short answers.



Comarison of Kinects vs Sony PS3 Eye cameras

Higher resolution is not important for markerless mocap. Image quality (low distortion and motion blur) is really important. That's why we recommend Sony PS3 Eye cameras. Using other cameras is not recommended, as it will make tracking worse in most cases.



It will not work with luminiscent light I think. It is very important colors would be the same in all cameras, at all angles. Different colors of feet, legs, arms, torso, hair is recommended. Left and right should be the same.



Calibration is done to detect positions and rotations of cameras. So you can do it at any fps. Darkening of the room is recommended because it makes flashlight marker detection easier. However, if if you room / lighting the market is detected without darkening, you can record calibration under regular light. In the end of calibration the system shows calibration quality, so if it is perfect / good in all cameras, you can use the calibration.

_________________
iPi Soft

Thank you Greenlaw and Ipisoft for replying and it was very informative.

1. I have tried to capture 10 meters by 10 meters using 16 cameras. It didnt work well because is was of low resolution(640p) and thus I want to increase capture volume by 10 by 10 meters. Therefore i asked for higher resolution webcams with 120 fps. I just want to know if we can increase Webcams or PS3 eye cameras more than 16 and what can be maximum limit for both? If only 16 can be used, then can I use 16 webcams? I do have 5 workstations and 3 laptops which are very sufficient for me to record more than 20 cameras at higher frame rate.

2. The reason I asked for florescent colors is because Ipisoft tracks actor based on video and background subtraction. So if i just use black enviroment with UV lights on high ceiling, after evaluation enter capture volume and will keep (Hand,Head,Torso, Arm, Leg and feet) of 6 different florescent color and this will give me 100% in background vs Actor. And also the colors will be prominent to cameras.

3. I will be definitely keeping left and right same but the reason I asked for different limbs - different color is because sometimes in my test, software finds it hard to track shoulder movement which can be solved if different colors are wore for Torso, Upper arm and lower arm.

4. I tried doing calibration in no lights and then shot in all lights on. It indeed great GREAT for me and results were perfect for calibration and also for action file.

5. We did this test. Recording in low light with 50 fps will give no flickering in video whereas at 60 fps we do get flickering even after turning on Anti-Flicker at 50 and then 60Hz. Solution: We found out that recording in night or low light can be done well with 50 fps.

P.S : Just for clarification, I am not working with Kinect but only with PS Eye camera.

That's very interesting. Thanks for the explanation.

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)

Watch a one minute excerpt on Vimeo now!

...

Just to add my 2 cents worth...

iPi is a very simple set up, to do things to complicate how it was designed to operate will be fruitless, I have tested many different set ups and methods and this is what I found works consistently better.... "for me".

1) You seem to say you get perfect calibration doing it however you are now, but turning all lights off is actually bad during calibration, just use the extra darkening option and leave lights on and it isn't necessary to do very high sweeping motions with the light, just cover the majority of the capture volume, height of light need only be 2-3 ft. off floor during transitions with only needing 5 floor touches, 4 corners and center staying out of the way with body to all cameras throughout.

2) Clothing: I have my performers ONLY wear:
Black: for shoes/socks, tighter fitting dance pants or leggings, head wrap, or solid color dark hair works the same, gloves. (solid black, no markings like stripes).
Saturated blue, short sleeve form fitted shirt. (Any contrasting color to background would work, but a bit darker and saturated works better for me).
Arms are left uncovered, (using only skin color, I tried the long sleeve colored shirt under the short sleeve one, but works just as well with arm color in my case).
Light to no facial hair, head hair pulled up on top, or back of head (iPi must see front and back of neck to properly track it, or will get weird positioning).
I get the normal tracking arm issues with some extreme motions, but not much when very visible moving fast with arms, legs/feet. There will be some, expect it.
I record single performer high speed dancing regularly at real time music speed with very good results and even high speed fight actions with little tracking fault with only a 6 PS Eye set up, using a 4m x 4m capture volume on the floor.
There should be no reason to need fluorescent colors, but can't say I personally have tried them.

3) Lighting:
iPi tracks better with lower lighting directly on performers, back lighting or downward facing lighting works better, and better to face actual light away if possible, PS Eyes are very sensitive to direct light in scene, it causes a light halo in recording and can block the color on the glove, arm, head out when passes by them, could cause absolute loss of tracking if lights are large.

4) Background:
Just try to limit clutter in direct camera to performer capture view, iPi does very well at distinguishing colors in the depth to colors on the actor, even if the same and without a highly reflective surface directly behind performer in the cams view, like a mirror.
Avoid "hot spots" on floor from lighting, they appear much brighter in iPi Studio than you may think and high shine flooring isn't optimal.

DSLR cameras:
As long as all cams use Direct Show drivers and of same make, this should work. (Although I have not tested this, because iPi told me resolution wasn't what the program was designed for and action cams have a perimeter distortion issue).
Just seems you have a large studio with Optical marker real time tracking for very large coverage areas, so why need this type set up, I don't know, maybe just for the "marker-less factor" and easier portability I would assume.
More than 60 fps recording with iPi is also going to add double the key frames to track and may slow processing to a crawl, even with a high powered machine and graphics card(s).

with 16 PS Eyes you should easily be able to get a capture volume of 7m x 7m or a bit more, and remember these are free positioning cameras, they will record in any vertical position, even straight down from above with the proper clearance above to fit the actor and maintain coverage of majority of the capture volume.

Also, you do not have to stay in view of all cameras at the same time, as long as you can get all cams calibrated properly. (seeing light marker throughout calibration in all cams).
An example would be if you want to position in a hall set up, you can walk out of the cams view and into another without affecting the tracking process at all, this works well for maintaining better hips tracking depth and position.
This is more complicated in camera positioning and calibrating, so most would avoid it.
Also, if you have enough computers to handle more than 16 cameras, you can cover a much larger area using a two Recorders set up, two Master Recorders that is.
To do this, (You would need 2 Pro systems licenses at least), you can set up 16 max. cams on one side of a stage area, and 16 max cams on the other, calibrate each side separately ONLY.
To record, simply start/stop both recorders at same time, may require 2 operators, but shouldn't be necessary.
This way you can walk out of one cam stage into the other and never loose tracking capabilities, but you will have to track each video separately and join the action later in external editors.
There is NO ABSOLUTE REQUIREMENT to start in a T-pose with iPi Recordings, you can always export with a T-pose on export for other applications characterizing/linking later in post.
You can get the actor to "lock on" from any positioning in the studio processing as performers walk between stages, you just need to manually position the actor to the recording at that point.

Use camera set up parameters to get each camera to look approximately close, using a lower exposure will probably help also, especially with motion blur. (try 60 on all cams, then use Gain to adjust brighter if needed).

Many users try to over think the set up needed for iPi Recording and most is really unnecessary, or returns fruitless results in the end. (Clear you thoughts on what "works" with other systems and you'll be better off).

PS Eyes will record fine outside, "IF", the capture area can be shaded well, or on overcast days, and light winds if not using a very stable mounting system.

iPi has gotten a lot better and continues to increase its capabilities, so I would expect more features to come, for what it was designed for it works well, high end studio applications it may struggle for what you expect.

Hope any of this makes sense... Good Luck!

...

...

Just a clarification on the dual recorder operation, an active license is only needed for the recorder if you want to use it's editing features.

You can install and operate the basic recorder for free on 2 computers at the same time using distributed recording.

I use the editing features on the record a lot, so I keep including that as needing a license to operate.

If you say you want more than 16 cams, the described set up will work with 10 and 10 PS Eyes or any variations up to 16 PS Eyes on one recorder and distributed recording.

Processing the videos will still need to be done separately in iPi Studio, which would only require the one Pro license.

You can run 10 PS Eyes off one computer set up to handle it though and would work easier than distributed recording from several machines, that would give you 20 cams and a larger capture volume.

...

Actually, adding more cameras wouldn't necessarily give you a larger capture volume. The range with PS3 Eye is limited by the device's video resolution so moving the camera farther away would make the performer more difficult to track.

For video-based capture, moving the cameras farther away does require higher resolution. But higher video resolution will naturally increase the bandwidth requirements and fewer cameras will be usable for each computer.

_________________
Greenlaw
Artist/Partner - Little Green Dog | My Demo Reels (2013,) (2015,) (2017,) and (2019)

Watch a one minute excerpt on Vimeo now!

...

I have never seen anyone posting they tried all 16 DSLR cams for recording, so the jury is still out on this topic, but probably true, if were going to use DSLR cams, bandwidth could be an issue for most users, shouldn't be an issue with PS Eyes if the computers will handle 10 using up to the 4 port add on USB controllers with 4 separate chips, I can use 8 on my machine with just one 2 port, single chip add on controller without frame drops, but not needed and Basic version won't use them anyway, my machine has an i7 6800k now with 28 psi-e lanes, so probably not all machines will run this set up, but from what was stated, their studio has sufficient machine capabilities.

They stated they wanted a larger capture volume using 20 cams, so using 2 recorders with 10 cams on each, each stage can then be within the max. camera spacing for PS Eyes, and will give a larger overall capture volume, the camera positioning config. and the tracking of the videos would be done differently of course, then spliced back together later in post to achieve the full stage animations, this is commonly done to append other animations to one another, so why not this way.

Trying to get a capture volume of 20m x 20m, or more with PS Eyes is probably unachievable at a trackable resolution, agreed, the actor would be too small.

True, I haven't tested it with DSLR cams, but I reasonably speculate it would just add another layer of compatibility issues that aren't needed for how the program was designed in keeping to it's recommended single recorder PS Eye 7m x 7m capture volume max. or maybe a little more 8m x 8m is feasible and the fact that many DSLR cams would be very expensive to just up and buy, even to try the Logitech webcams that use 1080p resolution would be a hefty expense, for me anyway, and I don't think it would give that much better of tracking results, if any better, the video would just be a higher resolution.

It was explained to me that iPi tracking doesn't operate off great video resolution in its design, it really just needs to maintain an undistorted video and good saturation of color of the performer throughout the recording and a few other controlled measures to work fine, this is also where I believe some users are using too much direct lighting on the performer during recording, or incorrect camera parameter settings and washing out the video colors, or just wrong clothing in general if they are getting very poor tracking quality, along with improper skeleton set up, or scale of the actor, or just over expectations of the capabilities of the program.

It will track just as well with the skin turned off only showing the bones, the skin slider is just an alpha channel to make it disappear on-screen, once it gets the colors selected and locks onto the performer, it's locked as good as it is going to get, maintaining the the saturation of video colors used throughout the performance is more important, as well as having a very good calibration for all cams is very important and limiting shadowing on the body parts with good equal ambient lighting and better yet to use backlit walls, or draped enclosure, using just enough light on the performer to see the clothing colors as they actually are, or very close, a little darker is fine actually, it will appear differently inside the studio during tracking anyway.

I could however understand where the DSLR cameras could be set farther away from the actual volume for more FOV and using the focal point of the cams set to zoom in closer to the performer, but is that really necessary for most uses, or even better use of the program and would it even scale correctly for calibration? There isn't enough data from this type set up to know, although it would give a better resolution by default.

I ran a test set up outside setting the PS Eye cameras spacing to the 30 ft. (10m) front to rear cam spacing, basically the same config. I use inside, just further apart and I saw no real resolution issues when tracked, it just made the performer a bit smaller in scene at the center point of the volume, but not untrackable, and I was just recording using a Haswell dual core i3 HT processor with a 2 port add on controller and a cheap $50 DX 11 graphics card, with no recorder issues, the recorder really doesn't drain resources like the studio does, it was just a bit time consuming to set up and take down, so I haven't pursued that any further for now, but I do know it works fine, the capture area just needs to be shaded, or used out of direct sunlight.

I actually looked into buying a 30 x 30 canopy set up for $350 to modify for this use to make it easier and leave set up, would just have to remove the cams after use.

...