Real-time Vision-Based Camera Tracking for Augmented Reality Applications

Dieter Koller , Gudrun Klinker, Eric Rose, David Breen,
Ross Whitaker, and Mihran Tuceryan

Abstract

Augmented reality deals with the problem of dynamically augmenting or enhancing (images or live video of) the real world with computer generated data (e.g., graphics of virtual objects). This poses two major problems: (a) determining the precise alignment of real and virtual coordinate frames for overlay, and (b) capturing the 3D environment including camera and object motions. The latter is important for interactive augmented reality applications where users can interact with both real and virtual objects. In this paper we address the problem of accurately tracking the 3D motion of a monocular camera in a known 3D environment and dynamically estimating the 3D camera location. We utilize fully automated landmark-based camera calibration to initialize the motion estimation and employ extended Kalman filter techniques to track landmarks and to estimate the camera location. The implementation of our approach has been proven to be efficient and robust and our system successfully tracks in real-time at approximately 15~Hz.

The document is available online in application/postscript (853324 Bytes)

Last modified on Tuesday, November 25, 1996, Dieter Koller (koller@vision.caltech.edu)