Tracking for Mobile Projector Camera Systems
The recent trend towards miniaturization of mobile projectors is allowing new forms of information presentation and interaction. Projectors can easily be moved freely in space either by humans or by mobile robots. We present a technique to dynamically track the orientation and position of the projection plane only by analyzing the distortion of the projection by itself, independent of the presented content. This enables distortion-free projection with a fixed metric size for moving projector-camera systems. For this purpose, an optical flow-based model is extended to the geometry of a projector-camera unit. Solving an overdetermined system of equations on pixel level leads to the pose offset between two images. In order to reach a high invariance to illumination changes we use adaptive edge images. Image pyramids allow a fast pose estimation. Because of the global optimization, there is no dependence of the availability of local feature points.
Model-based pose estimation. In (a) the observed image is not aligned with the expected image, highlighted by the green frame. After our correction (b), the expectation and the observation are aligned.
The general principle has been tested with synthetic and real data and can be used in projector-camera systems to display e.g. user interfaces on arbitrary planes even while the device is moving. The results have been integrated in an interactive software implementing the closed-loop principle and running with multiple frames per second. Since the equations are defined on a pixel level, the algorithm can easily be parallelized and ported to the GPU to allow true real-time performance.
Geometry of a projector-camera system. The expected plane differs from the observed plane by a certain transformation (R, t). The displacement in image space is related to v’ and the result of the function f. The displacement vectors in image space are evaluated to calculate the pose offset and to correct the projection.
In future, we will extend the presented principle to the tracking of rigid objects, enhanced by a projective texture. The projection distortions, which emerge if the enhanced object is moved, are analyzed to calculate a movement with six degrees of freedom. Since the retextured object is known its surface normals can be integrated into the equation system to estimate a consistent motion.
Projects
This topic is funded in the project EASY-COHMO
Publications
N. Gard, P. Eisert,
Markerless Closed-Loop Projection Plane Tracking for Mobile Projector-Camera Systems, IEEE International Conference on Image Processing (ICIP), Athens, Greece, October 2018.