User wearing the iSTAR device.

So far no light-weight head mounted device combines eye tracking and augmented reality, i.e. imaging the eye and displaying information simultaneously. A current Fraunhofer research project called iSTAR (“interactive See-Through Augmented Reality”) aims on filling this gap. In this project a bi-directional sensor (OLED and photo diodes) has been developed allowing simultaneous imaging and displaying of data. The sensor is combined with special optics (prism and micro lens-array) and integrated in a light weight glasses frame, in order to enable eye-tracking and augmented reality simultaneously. Recently, first prototypes of this device have been presented.

In the ARtSENSE project the iSTAR device will be extended with a scene camera in order to estimate the users gaze on the artworks. Moreover the scene camera will recognize gestures of the user, allowing the user to interact with the device.

The development of iSTAR is ongoing. In order to demonstrate a basic ARtSENSE system, see through augmented reality glasses from Vuzix (STAR 1200) are used, until iSTAR is finished. However, Vuzix glasses do not provide eye tracking functionality. Therefore basic eye tracking functionality is realized by user tracking in the first ARtSENSE prototype. That means, the orientation of the scene camera is used to determine the visitors’ focus of interest.