The 3 years research project "GYROVIZ" addresses the challenge of automatic modeling of 3D physical scenes from located frames. The central motivation of our proposal stems from the observation of the current limitations of image-based modeling systems, which require a substantial amount of user interaction for matching the images. This limitation becomes even critical when dealing with massive datasets to match.
While a considerable amount of efforts has previously been put at solving this issue purely from the algorithmic point of view, we propose to address it primarily from the technological side. Our solution hinges on a set of new accurate inertial sensors (specific fiber optic gyroscopes and accelerometers) which once coupled with an image acquisition device, provide a very precise measure of its physical location and pose.
Both robustness and efficiency of the image matching problem are this way substantially improved, as the solution space of the matching problem is drastically reduced, and the matching is always provided with good initial guesses
The algorithmic aspect of our proposal relies on state-of-the-art computer vision and geometric computing techniques for extracting and matching characteristic points and features from images, and for reconstructing 3D models with color attributes. Real-time reconstruction as well as feature and symmetry-preserving reconstruction are listed as topics for 2 Ph.D. theses (CEA/LIST and INRIA).
The complete system should be mobile and portable, and should allow a fully automatic 3D modeling of complex scenes. Two complementary laboratory models using the same localization unit will be developed in the framework of the project to make various sets of data acquisition for algorithm tests as well as to enable a practical evaluation of the potential of the technology with end-users:
- A 3D real time video camera tracking system for virtual studio and postproduction purposes;
- A 3D static scene and object modeling system for multimedia and engineering applications.
The project GYROVIZ is supported by the French Research Agency - ANR.
- 3D scene reconstruction from pictures
- Pictures and videos acquisitions
- 3D tracking
- 3D scene reconstruction in real time
- Self-localization module able to providing 6 location information for each image
- Geometric features reconstruction of from point clouds
All modules of the processing chain have been taken into account; from the sensors to the software integration:
- New specific high precision inertial sensors development;
- Development of a new method to merge inertial data and vision applied to the real-time 3D reconstruction;
- Development of new 3D high resolution reconstruction algorithms preserving edges.
Two laboratory models have been implemented to evaluate the contribution of the concept GYROVIZ and potential applications of the technology developed.
Scientific production and patents
Gyroviz concept is under international patent. 3D modelisation in real time, with features detection and preservations by differed reconstruction are specific themes which have been held by two doctor thesis and led to several international publications.
The process of merging data from image and motion or position sensors has also been protected by a patent filed in 2010.