Higher Resolution Color Images for Kinect Scans

I bought the Kinect v2 sensor.  And it has been a bit of a let down but probably not for the reasons you might think. Well, there are two main reasons; (1)one is that vendors have been confounded by it and slow to support it and (2)two because it still has the same lame color resolution.  While I have no answers on how to speed up vendors to support it sooner, I have been able to attack the problem of getting higher resolution color images for my Kinect scans.  And that is what I'd like to share with you.

My first brush with the idea of affixing higher quality color imagery to Kinect scans came when I had the good fortune of connecting with the engineers over at Lynx labs.  They were just a Kickstarter yet to be at the time and we talked at length about my fanaticism for depth mapping (and my many failed attempts at scanning for film-making) and their plans for give a Prime Sense sensor great software powers and making it an all-in-one appliance type solution.

As their development progressed we discussed desired features quite a bit.  As a film maker my demands can be summarized as needing great (and sophisticated) color/texture resolution and detail and needing a little bit of depth.  Where as an engineering heavy group might need super accuracy in their 3D information and not need color at all.  Lynx Labs deployed a Raster Alignment feature in the software system of their A-cam and it was simply brilliant.  It was based loosely on a similar Raster Alignment tool (which I have had very little success with) that was available in Meshlab.  The results of applying high fidelity imagery to a mid-quality 3D scan was fantastic.  So how do I go about reproducing this result with my Kinect scans?

Step One

Gather Photo Reference Under Ideal Lighting

An overcast day is great but selecting an object in shadow is equally valuable.  The reason is that we are trying to capture a subject in ambient lighting so that we get a clean diffuse texture (without lighting perspective) that can be synthetically re-lit once it is on a 3D model.

Step Two

Create Photo Scan and Fail

Usually this is where I attempt a full photo scan with Agisoft and possibly fail (thus driving me to turn to a Kinect scanning approach).  If you don't fail, then you can proceed to create a fantastic scan using just photos!

Read More