...

I believe you can run 3 cams on one controller at a lower frame rate, I saw it somewhere stated by iPi in the forum, but I haven't personally tested it.

If you have the cams already, give it a go, that's the best way to find out, don't need to string them into position to test that, if they all get recognized by the recorder and then when running maintain say 50 fps when set to that, then they should while recording motion after placed also.

Yeah, I don't like used for that type electronics, if I knew the person, then yes, but too easy to get scammed online nowdays.

I think I will do the 1070 route, like I did the 970 over the 980 Ti route before, your stats seem plenty fast enough and hopefully iPi can get a fast track option for PS Eyes eventually, but since you are tracking a video and not a point cloud as with Kinects, it is probably much harder to achieve.

I meant to ask if you can try your USB 3.1 controller and ports, I did on mine, but couldn't get it to be recognized by the recorder, maybe you can, I don't know.

...

I have a question.

I see that even when moving slow.. Kinectv2 doesnt provide a very accurate or pricise joint movement tracking.. for example.. subtle small hip movements or poses of shoulder n arm are not followed in detail.

If I use ps3eye cams.. 6 of them.. will I be able to track those subtle movements n poses in more detail than with Kinectv2? I dont need to track fast movements nor need a big movements jumping or going around..just need acurate posing.

...

True, Kinects can not get the same tracking fidelity as PS Eyes, even if you were to use 4 of them, but definitely not with just 2, or 1.

There is limited clavicle movement, hips can get a bit wonky easier, the spine react differently even when set to different spine bend options, and occlusion is very easily encountered.

Neither system is just record and track and expect a high fidelity outcome, there is steps needed to get a much better tracking outcome, but it can be done very closely with just the integral auto-correct features iPi includes, and a bit of manual manipulation at times, but those features work well and help a lot when used right before export.

It may take you a while to get a good work flow with the program that's works quickly for you, but it can be done, as most of my videos I posted are all only done with iPi auto correct methods, but you will always have some post work to do.

There is also much stricter clothing and lighting parameters to follow with PS Eyes, and you need a much tighter error rate on the calibration video, meaning your per camera errors should be as small a number as possible and all basically in the same vicinity for each cam, wide error percentages will affect your tracking, even if the program shows a Perfect calibration.

Camera placement and how big of an area of capture volume you are trying to perform in wiil also make a difference, larger areas may cause less accuracy in the tracking, especially if lighting isn't very ambient uniform around the capture area, bright overhead lighting is bad for PS Eyes also, really any overly bright lighting is.

There is also camera settings that should be manually adjusted for recording a performance with PS Eyes, to achieve better tracking.

You will have to put some work into tracking with iPi Studio and best to monitor each video tracked while it's running, stopping and starting to correct any errors as they occur, and move on, but mostly it takes experience in usage, just like any 3D package, I am sure you didn't just jump in and get the best results with Maya either :)

Give it time and you will achieve a very acceptable pre- export animation with iPi and lucky you, you didn't have to work through all the previous versions issues HA! I will add that iPi has really done well with their updates and corrected a lot of the problems, so recording and tracking is much easier and better now with PS Eyes, at least for myself anyway :)

Also, you should probably always use Move controllers on the hands, (head Move is optional for PS Eyes, but not optional for Kinects to get proper head tracking), but should always turn head tracking on either way, have good Actor to video alignment, proper skeleton adjustments to video performer, proper floor height set, proper clothing and lighting. proper size area and positioning for the amount of cams used, always use the Refine feature, always re-add the Moves data as the very last step before playback test in iPi, or exporting the animations, you can set finger gesture animations as desired and select the AMT box if importing a test character for export tests.

There are many ways to skin a cat, so I don't want to sound like my work flow is the be all, end all way to do anything, I just use what works best for me and I feel I get very good results, with just the iPi auto tools and a bit of time inside iPi first before exporting.

iPi tracking does usually work better with constant motion, over basically still, or idle motions, but most can be cleaned up fairly easily, but you will never get 100% animation to video recording accuracy, not with any MoCap system, even very high end ones, with 100 cameras, so keep that in mind also, the new Vicon system has made impressive strides in tracking, but at far more expense!

Hope this helps explain a little, but you can search the forum to find more posts on PS Eye recording and tracking tips, but you are going to have a lot of trial and error until you get things working how they should and be aware you may get some self occlusion with 6 PS Eyes, but should be far less than Kinects and keep your performance surrounding clean and clutter free of background objects, especially if they conflict with the clothing colors chosen.

...

Thanks for your detailed n well explained reponse Snapz. I like your videos the most and I think youve got it to the point of being the best it can be with current ipi software. I saw the video about that studio that worked on Game of thrones and they used ipi for their mocap on some scenes.. so I guess it can provide decent enough capture for certain not so critical shots even for high end films.

As far as clothing and lighting, once I get down to setting it up correctly then its just a matter of doing it. But for placement and calibration..etc.. I just have to keep trying and experiment.

What is the MINIMUM distance for PS3 cam to fit the whole body head to toe?. Im going to try to put the cams in my small room.. 4 at the corner on the ceilings and 2 more on each the longer sides of the walll. I dont evn need much movement..just need accurate poses so that I can use them to pose my characters in 3d in maya. Heck I dont even need 360 rotation.. so I could place the cams in half circle since I dont need to take back shots.. Found a some good price on ebay from hongkong..at around 8 dollars a pop.

...

Yes, it's all speculation until you actually get the stuff, set it up, and start using it the way you feel works best, but you will make corrections needed to get better with it I am sure.

Sure several higher end studios use iPi, but they do have a team of animators to back up the export quality afterwards, that makes a difference.

Room size and camera positioning will have to be tested, but always have at least 1-90 degree positioned to the performer for better depth tracking, especially for the hips, a rear camera is helpful just for the fact if you turn 90 degrees to the front and an arm lays behind the body closely, it will be occluded at some point.

If you really only need a 2m x 2m capture volume, I would say maybe a 3.5m-4m x 3.5m-4m would be minimum, but I can not say exactly, as I have never used that small of an area with PS Eyes, but you may not get full arms raised above head view with that small of a space.

If you are just kind of using iPi for easier pose to pose animations style, with limited motion, that should work ok, but I would think you would not want to limit your set up to just that in the future.

I am not sure on pricing over seas, but PS Eyes are fairly cheap now yes, and so are the USB 2.0 repeater cables needed. (Note: buy pretty good cabling!)

...

Snapz,

I finally got my PS3 eyes (got 7 of them, one extra just in case one goes bad).

I tried different variations of connecting to my 4x1070 computer and I ended up doing this:

2 cameras connected via external USB3 hub connected to RED USB 3.1 output

3 cameras connected to 3 usb3 ports (blue)

1 camera connected to 1 of the usb2 port

Bluetooth wireless device to one of the usb3 port


And when I do a record test, 2 of the cams stay at steady 60fps(640x320) I think those two are from the external USB3 hub connected to usb3.1RED port

the rest of the four cameras stay between 60fps to 59.2


Are frame rates above 59.2--60.. usable for Mocapstudio to track motion accurately? or do they have to stay at absolute 60fps on all 6 cameras?

I tried decreasing the frame rate to 50fps, and it's weird how it still fluctuates between 50and 49.xx

I read somewhere that the modern controllers for USB2-3 seems to have broader bandwidth.. thus explaining how my set up with 4 cameras on USB2 and 3 in the same computer gives me near 60fps.. I know I have two controllers in my computer, however, due to the FOUTH 1070, it blocks the connector from linking the cable for my frontal USB ports which probably is from my second controller.


So, my question is, is 59.2-60fps okay for tracking?

...

Yeah, you may get that warning, but that's from the shared controllers with newer CPU and boards, but cams may still show up ok in the view ports.

I am not sure USB 3.1 works at all, if you get a black view port from that camera, it won't, which is what I got when I tried it, but I didn't need that controller for cams anyway.

Also not sure a USB hub will work, I haven't tried it, although somewhere in the forum posts iPi may have stated it would, but not sure which type would work.

The frame rate will fluctuate a little, 60-59.2 is fine, won't be noticed in the recording for tracking.

You say you don't have front USB 3.0 ports wired up? Because, although they run off same 3.0 controller, they come off the board header and you can possibly use 2-USB 2.0, 2-USB 3.0 on back I/O and 2-USB 3.0 on front panel to get 6 cams at 640 x 480 @ 60 fps, if somehow you can get them wired in.

You have 3 controllers on that x99 mobo, but USB 2.0 and 3.0 are shared through the rear ports and 3.1 is its own.

Yes, keep posting your updates.

...

Did some tests with 4 cams. 2cams from 3.0hub connected to one3.1redport and 2more connected to 2 usb 3.0ports. No warning and they hold up 60fps perfectly. I only have one active usbextender cable so I put one cam far up left and the rest close by near the computer. I can still get whole body head to toe clearly..so no problems with room size since Im not doing any acrobatic moves.

Had some hard time calibrating using the move controllers white ball light (didnt know what to do) but after manually selelcting four ground points and also doing window wipe motion on some of the outer plane of my volume areas I was able to match the recorded videos in all 4 views and test tracking. Snapz if you could simply tell me how is your routine calibration is done.. please give me some tips. Also do you calibrate once and use it for several days/weeks? Or do you recalibrate regularly?

As far as tracking the recorded videos.. I tried without wearing a shirt.. but didt work at all. So put on a blue long sleeve n white tshirt with black tights and white socks.. then did the auto actor color detection and it worked.

Had to manually IK and rotate limbs to match first frame on all cam views.. but after that it started tracking.

Which settings do you use? I know you use autohead tracking. But how about shoulders?. Do you use from arm position?. Or do you use from camera?. I put flexible splines.. and after first pass Im doing refine forward before I do Jitter removal.

Let me do some more tests and I will post some results. Definately less occlusion errors compared to one Kinect V2. But tracking speed is dead slow at around 4fps.. compared to 40fps on Kinect.. but I prefer quality over quantity.

I can see that without big movements my small space (3.5m x 5m) is pretty good.. with half circle set up. When I get my cables come I will try little over half circle set and see how it does).

...

OK, good, sounds like you are on the right track now... 4-5 fps is most likely max for 6 cams with one 1070 graphics card running it on High Resolution speed selection, but yes, that is fine for the better quality tracking you will get, I am not sure using dual cards will help much anyway.

(You could try running tracking on Low Resolution speed setting for a bit more fps, which is mostly what I track on now, but you may get more errors during the tracking on more complex motions where you may need to stop the tracking and readjust and refit the actor before continuing the tracking, but still shouldn't be awful, it's not for me anyway running at 2.7 fps Low Res speed tracking with 6 cams).

You can also switch between Low Res and High Res speed during the tracking process, if you have some trouble error areas where Low Res speed is causing them, once by those areas, stop tracking and switch back to Low Res and continue tracking again.

I think I read what you posted wrong before, it was late here when I replied.

You stated you have all 6 cams working @ 60 fps @ 640 x 480, so that's great:

Usually there can't be 3 cams on one controller to get steady 60 fps (59) @ 640 x 480, guess you figured that out though.

Tripods aren't a necessity for attaching cameras to, you can attach them to anything stable, and high enough off the ground to at least the chest of the performer at a level height, 1.2 m high is a good low height off the floor, and tape them down solidly to get some test recordings. (Be sure to have accurate measurement for Cam 1 to set it for calibration purposes).

With final cam positioning once to that point, you should always have a least 1 very high cam looking down on the performer for better accuracy of the foot placement on the ground. (I use the 2 high side 90 degree cams for this, as well as for better depth and hip tracking).

A good tip if using a Mag light for calibration is to get a ping pong ball and cut out a spot with an exacto knife to snuggly make it fit over the threads after removing the bezel and lens from the Mag light, this helps contain the light better within the ball and works well, also attaching it to a longer dowel extender will help so you don't have to move your body around the capture volume.

I will have to dig into why my USB 3.1 wasn't sending a signal to my cam, if your is working, mine should be.

I have 12 PS Eyes, and with using an add-on USB 3.0 controller and the USB 3.0 mobo header front ports, I can get 8 cams to work on my machine at 60 fps 640 x 480, and if my USB 3.1 port would work as you state with a hub, or a C to A adapter, I should be able to run 10 cams on my one machine, but I only use the Basic version of iPi, so I can only track 6 cams at a time anyway.

I always use head tracking ON, shoulders by video, very flex spine, (even if just basic motions recorded), because it works fine for me and the calibration phase is very important to the quality of the tracking, as well is a clean background highly contrasted to the performer clothing colors!

Yes, after tracking is finished fairly cleanly, you should run your Move controller data, (applying the Move data first will keep the hand rotations in the Refine processing for better overall outcome), and then the Refine procedure, I run it backwards, but either way is fine and if you run into where the tracking is off, say an arm, stop the Refine and use the frame forward process using the < or > keys (next to the M key) and use the Move tool on each bad frame to pull the arm, (or leg as needed), more back into proper location, (DO NOT Rehit the REFIT Button Again!), then restart the Refine to go back over those areas and it will clean things up much better.

Then run the Jitter process, and if using Move data, always run that as the final step before exporting, or even testing the motion inside iPi Studio.

At least 2 Move controllers for the hands should be used for best results!

I hope this helps.

...

Snapz,

I'm having very hard time getting the cameras to line up so that the dummy guy is centered and in place in all cams.. is it maybe because I'm not calibrating the cameras right? When I align the dummy in one camera, it goes off on the others..

Also the dummy comes in waaay smaller than the video actor. I have to agment him to like 4.5 meters talls to make the dummy fit the video man. Why is that? it worked fine with Kinect..