Augmented reality for non-rigid surfaces

摘要: Augmented Reality (AR) is the process of integrating virtual elements in reality, often by mixing computer graphics into a live video stream real scene. It requires registration target object with respect to cameras. To this end, some approaches rely on dedicated hardware, such as magnetic trackers or infra-red cameras, but they are too expensive and cumbersome reach large public. Others based specifically designed markers which usually look like bar-codes. However, alter objects be augmented, thereby hindering their use application for visual design matters. Recent advances Computer Vision have made it possible track detect relying natural features. no method commonly used AR community, because maturity available packages not sufficient yet. As far deformable surfaces concerned, choice even more limited, mainly initialization so difficult. Our main contribution therefore new framework that can properly augment deforming real-time. Its platform standard PC single webcam. does require any complex calibration procedure, making perfectly suitable novice end-users. satisfy most demanding designers, our scene engineering, renders illuminated light, let occlude ones. meet challenge, we developed several innovative techniques. approach real-time surface wide-baseline feature matching. traditional outlier elimination techniques RANSAC unable handle non-rigid surface's number degrees freedom. We proposed robust estimation scheme allows both 2–D 3–D registration. Another issue critical importance achieve realism illumination handling, existing setup procedures devices reflective spheres. By contrast, includes methods estimate rendering purposes without sacrificing ease use. Finally, handling occlusions multiple cameras only deal occluding modeled beforehand. one camera models at runtime. incorporated these components consistent flexible framework. many different T-shirt sheet paper, under challenging conditions, real-time, correct occlusions. also technique measure shape deformed sails. validated deployment distributing software package letting an artist create two applications.