Approach Using Color Images and 3D Models

This branch of development is represented by object pose recognition from regular color images with the direct use of corresponding 3D models of sought objects. This approach is based on the utilization of image information in the form of key points and related descriptors. These features are extracted from both real images and object templates yielding a set of matches from which the relative pose between the object and the observer can be estimated. Once this pose is known, we can easily highlight the whole object or its parts. Motivation behind this approach is the wide base of proven methods and algorithms used to process and analyze color images as well as the maturity level of traditional color sensors, which is very good compared to the sometimes-disputable quality of the outputs from just emerging depth sensors. The current implementation is able to find one object in a medium complex environment with a moderate frame rate on a multicore system. With the aim of the build-in tracking capabilities of the HoloLens device, it was possible to overcome the time-complexity of the devised tracking algorithm and to provide smoother user experience while observing the augmented reality through the headset.

Video Samples