Humboldt-Universität zu Berlin - Mathematisch-Naturwissenschaftliche Fakultät - Visual Computing

The Virtual Mirror is a system for the visualization of customized shoes that a person can try on virtually and watch their appearance after having designed them individually at a terminal. This is achieved by using augmented reality techniques which combine real video with virtual objects represented by 3-D computer graphics models. The use of a mirror environment enables the augmentation of the user with artificial objects without the user being forced to wear special glasses.

Left: Virtual Mirror system presented at IFA 2008; right: Virtual Mirror installation in Paris

 

Our system has been created for adidas and was also installed in stores, e.g. at the Champs Elys´ees, Paris. At their innovation center, a customer could not only chose shoes from the shelf but design personalized models. Besides particular fitting to the left and right foot, the client was able to change the design and colors of a shoe model at a special terminal and add individual embroideries and decorations. With our system, a customer can design personalized shoe models by changing the design and colors at a touchscreen terminal and add individual embroideries and decorations. In order to give the customer an impression how the shoes will finally look like after being manufactured, the user can step in front of the Virtual Mirror and watch him/herself wearing the new shoes. A camera captures the customer wearing regular shoes. In order to achieve the desired mirror effect, the display outputs the horizontally flipped camera image. The display is mounted such that the shoes/legs appear at the same position, where the user would expect to see them when looking into a real mirror. In order to enhance the virtual feeling of the framework, the background is segmented and replaced by a synthetic environment. A novel 3-D motion tracker estimates the position and orientation for each foot using a model-based approach that is very robust and can easily be adapted to new shoe models. In contrast to many existing approaches, no markers are required on the shoes. Once the exact feet positions and orientation in 3-D space are known, the computer graphics models, that have been configured and colored according to the customer’s wishes, are rendered and integrated into the video stream such that the real shoes are replaced by the virtual ones. Special care has to be taken for this augmentation, since the real scene in the 2-D video should occlude parts of the virtual 3-D shoe models. Therefore, visibility for all parts of the shoe has to be computed for a given position. Since all algorithms have been implemented with real-time constraints, the customer can move freely and watch himself/herself with the new shoes that have been designed just some moments earlier.

Upper row: original frames; lower row: augmented shoes

 

Videos

Virtual Mirror for Shoes

 

Publications

P. Eisert, P. Fechteler, J. Rurainsky
3-D Tracking of Shoes for Virtual Mirror Applications, Proc. IEEE Conf. on Computer Vision and Pattern Recognition, Anchorage, Alaska, June 2008. [PDF]