Student Projects

A Crowd-Sensor for Context-Aware Public Displays

Type: Master
Status: Completed September 2011
Student: Pietro Formenti

Thanks to the technologic progress, we are moving towards an era in which computers are go- ing to gradually disappear, melting themselves into our everyday life. This evolution requires novel methods to interact with computers that are almost invisible to users. One of the new challenges is how to make the system and user aware of each other, to let them communicate in a easy and natural way.

As an example of digital system that disappear, the Public Displays Network project proposes to design and implement a large networks of pervasive public displays as a new media for in- formation access and interaction. This master project, part of the PD-NET project, consists in a crowd sensor that aims to make the system aware of the people who stand or simply pass in front of the display. The final goal is to let the system change its behavior accordingly to the information perceived from the people in its vicinity.

The context awareness of the system is achieved using the Microsoft Kinect sensor, a com- mercial gaming device that records streams of 2D and 3D frames. Images were collected using the libfreenect development library released by the OpenKinect community that provides basic functionality to connect to the camera, set configuration values and retrieve images. The tech- nique that allows the recognition of the degree of crowdness of the scene is the self-similarity algorithm, that computes internal features of color images to find recurrent patterns like the human shape.

Tests are applied both on pre-constructed test images, images captured from the Kinect sensor and images downloaded from the web. The results show the suitability of the approach for the problem of crowd recognition and the correct classification of crowded and not-crowded images with peculiarities and limitations.

For more information contact: Ivan Elhart