Video-based reconstruction offers the potential of faithfully communicating both appearance and attention. Video conferencing can faithfully communicate what some looks like but not what he/she is looking at. Immersive collaborative virtual environment do the opposite. We are both building and studying the use of free viewpoint 3D video reconstruction from multiple 2D video streams. We have implemented both view dependent volumetric and view independent polyhedral approaches to reconstructing form.
Team
Salford Team: Toby Duckworth, Carl Moore. John O'hare, Rob Aspin and David Roberts (Principle Investigator).
Industrial Collaborators / Sponsors
OMG Vicon, BBC and Electrosonic.
Approach
We have implemented both reconstruction algorithms and multi-stream capture that allow real time frame rates for reconstructions from above 10HD colour cameras. We have built a unique test facility, the octave, which allows both immersive display and capture to support this research.
3D reconstruction of David Roberts using Carl's image manipulation and Toby's polyhedral based reconstruction.
The above 3D model is created at real time frame rates using our (Toby's) parallelised implementation of an EPVL algorithm. Images are captured and backgrounds segments both in real time by our (Carl's) multi-video capture/manipulation system.
We have been studying the impact of camera placement on the ability of Vision Based Reconstruction to faithfully communicate appearance and attention. Much of this work is currently being submitted to review and so we don't want to give away too much here.
The above reconstructions were created from images captured within our octave display and capture simulation. The silhouette cones are shown here:
This is the impact of good camera placement on shape reconstruction:
This is the impact of poor camera placement on shape reconstruction:
Publications:
Duckworth, T & Roberts, D J 2011, Accelerated Polyhedral Visual Hulls using OpenCL, in: 'IEEE Virtual Reality', IEEE, Singapore, Singapore, pp.203-204.
Duckworth, T & Roberts, D J 2011, Camera Image synchronization in Multiple Camera Real-time 3D Reconstruction of Moving Humans, in: 'Proceedings of 15th Int. Symp. On Distributed Simulation and Real Time Applications', IEEE, Salford, UK, pp.138-144.
Moore, C & Duckworth, T & Roberts, D J 2011, Investigating the Suitability of a Software Capture Trigger in a 3D Reconstruction System for Telepresence, in: 'IEEE/ACM Proceedings of 15th Int. Symp. On Distributed Simulation and Real Time Applications', IEEE, Salford, UK, pp.134-137.
Aspin, R & Roberts, D J 2011, A GPU based, projective multi-texturing approach to reconstructing the 3D human form for application in tele-presence, in: 'ACM Computer Supported Co-operative Working', ACM, New York, USA, pp.105-102
Aspin, R & Roberts, D 2010, An Exploration of Non-tessellated 3D Space Carving for Real-Time 3D Reconstruction of a Person through a Simulated Process, in: 'Visual Media Production (CVMP), 2010 Conference on', IEEE, London, UK, pp.151-160. Conference details: Visual Media Production (CVMP), 2010 Conference on, London
Moore, C & Duckworth, T & Aspin, R & Roberts, D 2010, Synchronization of Images from Multiple Cameras to Reconstruct a Moving Human, in: 'Distributed Simulation and Real Time Applications (DS-RT), 2010 IEEE/ACM 14th International Symposium on', IEEE/ACM, Fairfax, VA, USA, pp.53-60. Conference details: 2010 IEEE/ACM 14th International Symposium on Distributed Simulation and Real Time Applications (DS-RT), October 2010.