Public display systems are increasingly becoming part of the urban landscape, with systems being deployed in venues such as railway stations, shopping malls and city squares. It is not hard to imagine that soon these displays will be networked and empowered with interaction technology, e.g., touch, thus constituting a novel and powerful communication medium – networked public displays. One of the biggest problems this medium is facing is determining measures of success for different application that will be running on these displays. In other words: How many people pass by the display? Out of those how many glance at the screen? Out of those how many actually interact with the application? Is there any influence on the audience size on the interactions with the screen?

The aim of this project is to develop a set of tools and metrics that will allow automatic detection and clustering of the audience in front of the display into groups such as 1) passers-by, 2) people who glance at the screen, and 3) people who interact with the screen. As these groups can emerge all at once around the scree in different display zones it will be important to map the audience/users to different interaction zones.

At the outset, 1) using the Kinect sensor and live video feed a simple algorithm that detects the number of people in front of the screen should be developed. Then using the information from depth-camera a classifier should be developed that clusters detected users into different groups, i.e., 1) passers-by, 2) people who glance at the screen, and 3) people who interact with it. At a certain point in time users should receive IDs and transitions between the interaction zones and user groups should be detected (e.g., a person passing by the screen then moving a bit closer and finally interacting with a display application. In order to confirm the accuracy of the cluster video recordings from another camera will be used to manually label and classify the audience and their interaction zones.