I have a copy of iPi Motion Capture PRO together with the iPi BodyMesh addon. I am using a 3 x Kinect2 setup. Is there any way that I can export the raw depth that is captured by one of the Kinect cameras? It is obviously being captured and used internally.

Thanks
Ian

Hi
There is no way to export raw depth. However, you can export point cloud from any combination of Kinects using Biomech addon. Various formats are supported.
I have a couple of questions for you.

• What do you intend to use raw depth data for?
• If raw depth export was available, what format would you expect from it?

Thanks for your reply. I thought that would be your answer :)

I am looking to create ground truth skeletons and body meshes for input color and depth. This will then be the starting point for further research. With the point cloud export and the kinect camera positions from the calibration it will be possible for me to project the cloud back to depth camera space and re-create the depth. But the point cloud you create is a merged one from all the cameras. That's great for creating your body mesh, but not great for re-creating the depth. Maybe there's an option to export the point cloud from only one camera?

I think the real question your asking is what's the business case for exporting the raw depth? I work for one of the largest IT firms in the world and we have a large research department, spread globaly. When I've talked to fellow researchers about your product they get excited, until I tell them they can't have the raw depth and the data is captured in a proprietry format. That's when they stop being interested. If you could make the raw depth available, even if it was only to people who have purchased the PRO version of your software and the BioMech addin (an outlay of almost £2,000), then you'd sell a lot more copies of your software to other researchers like myself.

You asked about the format of the raw depth. Having it back in UINT16 format and millimetres would be great, just like the input.

As I've said earlier, you can export point cloud from any combination of depth sensors in iPiVideo. There are checkboxes Camera 1, Camera 2 etc in the Export area on the Biomech tab.


I was hoping you would give me a thought about some already existing file format you prefer that is suitable for depth data. For instance, an uncompressed AVI with 16-bit grayscale image, or a sequence of PNG files. Of course, we can design another custom binary format for this (with open spec) - but is this what you actually need?

I had a go at the export from the BioMech addon and spotted the checkboxes for the different camera feeds, which you've mentioned. In terms of the export I'd prefer 16-bit PNG files for each frame, if possible. If you gave me the uncompressed AVI file, I'd just split out the individual frames to PNGs anyway. So the PNG files would save me that step.

Thank you for your reply. I appreciate your help.

Thanks for your input, Ian.
Implementation of export of depth data to PNG does not seem to require much effort. Therefore I suppose we'll be able to include this task in our development schedule for the remainder of this year. This is not an official commitment though :)

That would be great! I understand that's not an official commitment, but knowing it's in the pipeline and will be available sometime soonish is a huge plus. It will also mean that I will have something to show the other researchers and promote your product. Thank you :)

Depth export feature is included in iPi Recorder v3.3.1. Enjoy!

Thank you so much for this! I thought I'd get an automatic notification when you posted, but I didn't so I've only just seen this. I've tried it out and the export PNGs are exactly what I need :)

Ian