Student Projects Archive |

A Classification Scheme of Lifelogging User Interfaces

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Completed September 2015 Student: Daniel Cardozo

Nowadays lifelogging is increasingly becoming part of people’s everyday life. The continuous advancements in different sensing and information technologies are facilitating the application of lifelogging to several domains. These domains describe scenarios where human memory can be supported to recall information of past experiences. However, one of the main challenges of lifelogging concerns how to access information when this is located in complex and vast depositories of data called lifelogs. Without the appropriate user interface to interact with lifelogs, the retrieval and visualization of information becomes overwhelming. What’s more, the different applications of lifelogging pose diverse contexts and particular needs of information.

This thesis regards the construction of a classification scheme of lifelogging user interfaces. The approach of reference to the construction of the scheme was Faceted Analysis. This approach permitted to integrate different perspectives from which lifelogging user interfaces can be described. The domain where lifelogging is applied was defined as the main classification feature. The rest of features were derived from the combination of design principles of lifelogging systems and design goals for presentation and visualization of lifelogs. The criterion to define and develop these features was the scope of lifelogging user interfaces, which was defined in terms of their boundaries when supporting the synergy between human memory and lifelog. The result of this thesis was a classification scheme composed of seven features that allow characterizing lifelogging user interfaces for different application domains.

For more information contact:

A Mobile Privacy Assistant for Sharing Personal Memories

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Master UROP Status: Draft Student:

Recent improvements in mobile technologies (better and richer sensors, cheaper and faster storage) allow us to record different aspects of our daily activities, e.g. work meetings, discussions during coffee breaks, family moments, etc. Within the EU research project we investigated how such data captured from mobile and wearable devices can serve as “memory cues” to help individuals remember certain past events – e.g., a set of images can help one remember an evening with friends or pictures showing white board content can help one remember what was discussed in the last work meeting.

Having already developed the RECALL system for collecting and storing such personal memories, now we are interested to understand how such data can be safely shared with others without violating users’ privacy.

The goal of this project is to develop a mobile application for allowing users to control (at some extent) how such data is shared with others. For example, one can specify that they want to share all their work-related data with work colleagues, or data captured during last hiking trip with close friends, etc. We envision the following three functionalities/features of the Mobile Privacy Assistant app:

an interface for allowing users to specify what parts of their captured data can be shared with which people and under what conditions;
create a model for storing users privacy specifications from step (1) – using e.g. JSON or XML – such that they can be easily processed by other components of our RECALL system
since this app will not have any list of friends that a user can share data with, we envision to start out by connecting it with the user’s social network accounts (e.g. Facebook, Linkedin) and use their APIs to retrieve user’s social connections together with their classification (e.g. “friends”, “family”, “colleagues”).

The app development will be done in Android, hence it is preferred if the student has already some basic Android programming skills (if not then willingness to quickly learn it).

In case for a master thesis, the student will first perform a literature investigation (on the different privacy interfaces for controlling data sharing) and come up with a set of requirements that can be then added to the Android app.

For more information contact: No items found

A Novel Approach Using Bilateral Data Fusion for EDA Data Classification

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Master UROP Status: Available

This thesis addresses the impact of lateralization on Electrodermal Activity (EDA) sensors in wearable devices. Lateralization, influenced by brain hemisphere activation, affects the accuracy of EDA readings based on the device’s placement on a specific body side. Despite recent studies highlighting this issue, there is limited exploration of the potential benefits of using EDA devices on both sides simultaneously. The research aims to fill this gap by investigating how leveraging data from both sides concurrently can enhance classifier accuracy through machine learning. The focus is on datasets in the lab, with implications for medical-grade applications affected by lateralization. Success in demonstrating improved accuracy may revolutionize the field, particularly in sensitive medical tasks, offering more reliable predictions for tasks impacted by lateralization.

For more information contact: ,

A tangible interface for controlling capture and sharing of personal data

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Completed September 2017 Student: Matteo Pontiggia

The goal of this project is to design and prototype a tangible device for controlling the capture and exchange of “lifelog” data – e.g., photos captured by a wearable camera, or audio recorded by a wrist-worn audio-capture device – with other, co-located peers. The overall vision of lifelogging is that captured experiences can help us remember better our past, and thus improve our cognitive skills and overall memory performance. By supporting the dynamic exchange of such captured data when co-located with other (e.g., in a meeting) we can also have access to our own experiences from someone else’s vantage point. The to-be-designed device would allow one to control both the capture and the subsequent sharing of experiences not only in a tangible way, but also act as a social marker that would allow all parties involved in a meeting to understand when their discussions would be captured and shared. For example, recording would only take place if the device is placed on a table; exchange with others would only proceed if other devices are placed next to it; shaking the device would delete the last minute of captured data, etc. The first stage of this project will study the different requirements for a privacy friendly data recording device. The second part will be about applying those requirements and constructing a first prototype of a physical recording gadget. Willingness to learn advanced prototyping skills (3D printing, embedded systems development) is required, prior actual experience with embedded systems programming (Arduino or Raspberry Pi) is a plus.

For more information contact:

A Toolchain for Visual Memory

Mon, 01 Jan 0001 00:00:00 +0000

Type: UROP Status: Completed August 2015 Student: Lucas Pennati

Within the EU research project RECALL we are investigating how captured images can serve as “memory cues” to help individuals remember certain past events (e.g., meetings, or an evening with friends). Existing wearable “lifelogging” devices allow for the automated capture of photographs throughout the day, e.g., every 30 seconds. However, in order to use these images as memory cues, we need to have an idea of what’s “on” these pictures. This UROP project will investigate opportunities to automatically classify such images along various dimensions, e.g., the amount of blurriness that they contain, their light levels, but also higher-level contextual information such as the number of people (faces) in the image, or if it contains a cup, etc. The student will develop a modular toolbox using OpenCV that can run an arbitrary set of sensors over a folder of images in order to extract contextual information from these images.

For more information contact:

A Visualization Tool for Smartphone-acquired Activity Traces

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Master UROP Status: Draft Student:

Every day we spend an increasing amount of time interacting with our smartphones, while chatting, checking e-mails or simply browsing the web. Even if smartphone usage is highly segmented, it nevertheless results in a considerable volume of personal data, such us call, text, app and GPS logs, pictures, videos etc. This combined with the strong habit-forming aspect of mobile devices can reveal precious insights of one’s daily life and habits. However, what is particularly intriguing is the potential of such information to support memory recall. For example, it has been found that selfies hold a significant amount of visual cues that can lead to richer recollections. This bachelor project should create a smartphone app to collect and visualize one’s day through the lens of smartphone activity, e.g., GPS traces, SMS and call logs, slide-ins, USB dockings, app invocations, etc.). Aggregated data should be clustered and visualized in different ways (e.g. along a timeline). Basic Android programming skills required, strong Web programming skills an asset. Strong interest in visualization a plus. Data collection will be done using an existing mobile phone collection framework such as funf.

For more information contact: No items found

A Web-based Application for Visualizing Personal Memories

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Status: Completed February 2018 Student: Federico Pfahler

Using wearable and mobile devices people now can fully log their life in pictures, audio or even video recordings. Reviewing such “life-logs” can support users in recollecting memories of past events and potentially improve their overall cognitive abilities. In this project, we focus on visual logs, namely images, as this type of data offers the strongest influence in recalling past memories. In addition to first-person view images that one can capture with their wearable camera (e.g., Narrative Clip or Microsoft SenseCam), users can also obtain first-person view images captured by others or even third-person view images recorded by fixed infrastructure cameras.

Once such data is captured, they have to be displayed to the users in useful and attractive visualizations. The aim of this project is in designing and developing an interface for visualizing the pool of visual traces as captured by users’ wearable cameras as well as images obtained from other sources. Starting from a pre-designed mock-up and the results from a usability test of the mock-up, the student will develop a web-based application that visualizes captured images in a compact but still useful way. The application should follow a responsive design, i.e., render well in both big sized screens (e.g., laptop and desktop) and smaller variants (tablets and phones). Moreover, the application should be integrated with the back-end database that stores captured images. Solid web-development skills required, while strong interest in data visualization is a plus.

For more information contact:

Activity recognition using ear-worn sensors and machine learning

Mon, 01 Jan 0001 00:00:00 +0000

Type: UROP Status: Completed September 2022 Student: Davide Casnici

Working toward context recognition for human memory augmentation systems, we developed Human Activity Recognition (HAR) machine learning pipeline using data from earable devices. In this paper, we analyze how the earables can be used to detect different types of verbal (e.g. speaking) and non-verbal (e.g. head shaking) interactions and other activities (e.g. standing still). We collected a dataset of ear-located IMU sensors from 30 participants and compared classical machine learning methods with state-of-the-art deep learning models to classify the raw data into a set of predefined activities. We explored how different parameters — including sampling frequency, window size, and type of sensors — influence the models’ performance. The best-performing model in the leave-one-subject-out evaluation, a spectro-temporal residual network (STResNet), achieved an F1-score of 0.69 (in recognising seven different activities: nodding, speaking, eating, staying, head shaking, walking, and walking while speaking.

For more information contact:

Affect and Learning in the LAUREATE Dataset

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Completed March 2024 Student: Enqi Fu

Affective states and cognition share a close relationship, influencing each other in several ways. In particular, our affect, i.e. our feelings and mood, impacts our perception of the world, our memory, and our decisions. This relationship also influences the cognitive process of learning. Working towards personalised student interventions for improving learning and mental well-being, we conducted a longitudinal study during a university semester. The resulting dataset, called LAUREATE (the Longitudinal multimodAl student expeRience datasEt for AffecT and mEmory research), consists of daily self-reports from students, audiovisual material from lectures, physiological signals from students and lecturers, students’ grades, and more. In this work, we present an initial analysis of lecturers’ and students’ experiences during lectures in terms of self-reported affect, especially positive activation (PA), negative activation (NA), valence (VA), and engagement.

For more information contact:

Analysis of Human Memory and Physiological Signals

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Status: Completed June 2023 Student: Andrea Prato

Memory Augmentation Systems could be capable of selecting specific to-be-remembered events during an experience, e.g., by detecting the individual as distracted or unengaged. Such information could then be used to generate memory cues for the specific periods during which the user was distracted. One way used to detect the cognitive state of users, e.g. if the user is focused on a task, is through physiological signals, like electrodermal activity (EDA) and interbeat interval (IBI). The goal of this thesis is to evaluate the potential of physiological signals captured by a wrist-worn device to detect the cognitive load of individuals presented with a memory task. The final purpose is to use this work as an additional input in future Memory Augmentation Systems. The specific tasks of the project are: (i) Overview methods for physiological signal processing and review memory augmentation literature; (ii) Develop the necessary code for processing the EDA and IBI data; (iii) Design and run an experiment to test the setup and pipeline.

For more information contact:

Attentive Public Displays: Understanding Audience Moving Patterns in front of Interactive Public Displays

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Completed September 2016 Student: Cristian Gomez Mora

Public displays are used in many public places such as city squares, universities, malls, libraries or bus stations. The public display content may vary depending on their purpose and the type of presentation. However, it is not always clear how public displays and their content influence everyday activities in the environment where they are deployed; how people behave, move around, and use the displays and their content; and how effective is the display content according to requirements of different stakeholders.

This thesis presents Attentive Public Displays (APD) – a tool that captures movements of people in front of public displays using depth sensor data and provides visualization and descriptive statistics of the audience movements. First, the thesis shows the design and implementation of the tool. It describes the software components and algorithms for capturing depth information, detecting people and their movements from the collected depth data, and representing and storing the movements in a database. The database entries containing people movements are further structured in a more understandable way, using additional filtering and data mining techniques to present the paths in a simple to use Web interface. Then, the thesis presents the evaluation of the performance of the developed tool based on three metrics: Accuracy, Precision and Sensitivity. The evaluation of the tool is based on 15 captured videos, each of 5 minutes duration. The number of people in the videos is manually counted and contrasted with the data captured by the tool. In total the tool correctly detected 211 people out of 226 people counted manually in front of the display. The average accuracy of the tool is 90.53%, average precision is 95.97%, and sensitivity is 94.21%. Then, the thesis presents a real-world deployment of the tool in front of a public display located in a busy hall in front of the university canteen, where the tool collected 6 weeks of data. Finally, the thesis discusses the limitations of the tool and present possible improvements.

For more information contact:

Co-Located AR Based Display Sharing

Mon, 01 Jan 0001 00:00:00 +0000

Type: UROP Status: Completed August 2015 Student: Alexander North

Winter sports like skiing and snowboarding are often group activities. Groups of skiers and snowboarders traditionally use folded paper maps or board-mounted larger-scale maps near ski lifts to aid decision making: which slope to take next, where to have lunch, or what hazards to avoid when going off-piste. To enrich those static maps with personal content (e.g., pictures, prior routes taken, or hazards encountered), we developed a wearable augmented reality system that allows groups of skiers and snowboarders to share such content in-situ on a printed resort map while on the slope.

For more information contact:

Collaborative Economy Practices and Communities in Switzerland – A Case Study

Mon, 01 Jan 0001 00:00:00 +0000

Type: UROP Status: Completed June 2018 Student: Masako Kitazaki

The general principle of “collaborative consumption” enables the effective and efficient coordination, acquisition, distribution, and sharing of many kind of different resources, e.g., vehicles, housing, or fertile land. Apart from the well-known for-profit sharing services such as Airbnb, Uber, and TaskRabbit, an increasing amount of community groups and organizations have established not-for-profit cooperatives that often prioritize environmental, social, and cultural values within their local communities.

The goals of the master project are (1) to conduct an empirical research study in the form of in-depth interviews and observations in both commercial and non-for-profit organizations (e.g. in the context of sharing personal artifacts and/or bike sharing); and (2) to compare and contrast sharing practices throughout these services and provide a comprehensive interpretation of the results. Particularly, we would be interested in utilizing practice-based approaches (e.g. Shove’s theory of social practice) within the data analysis. This project requires strong analytical skills and a willingness to learn about a novel and emerging research field. Experience with empirical research is a plus (e.g., ethnography, seminar work in human-computer interaction) though supervising guidance is available. Proficiency in any of the Swiss official languages (mostly German, French, or Italian) is an asset.

For more information contact:

Counterfactual Explanations for Unsupervised Learning

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Completed Student: Aurora Spagnol

Artificially Intelligence (AI) is slowly but surely becoming an integrated part of our daily lives. However, with decisions derived from AI systems ultimately affecting human lives (e.g., medicine and law), there is an emerging need for understanding how such decisions are made by AI methods. Furthermore, the “right to explanation” foreshadowed by the General Data Protection Regulation (GDPR) [1] challenged the Machine Learning (ML) community to build explainability into predictive models and their outputs. This paradigm shift – where predictive performance is no longer the only (and main) objective – gives rise to two distinct viewpoints. One argues that algorithmic black boxes should continue to be optimised for predictive power with explainability needs, possibly, fulfilled through post-hoc methods due to an apparent incompatibility of these two goals, thus forcing one of them to be sacrificed for the other [2]. The second standpoint disputes this trade-off as purely anecdotal and persuasively argues for building inherently transparent models, especially for high-stakes decisions [3].

Counterfactuals are an explainability approach uniquely positioned in this space as they can be generated post-hoc but remain truthful with respect to the underlying black box (i.e., exhibit full fidelity). They enable ML users to understand what the output of a predictive model would have been had the instance in question changed in a particular way. This type of counterfactual analysis helps the explainees to simulate certain aspects of the ML model, thus improving its interpretability [4]. Notably, evidence from psychology and cognitive sciences suggests that people use counterfactual reasoning daily to analyse what could have happened had they acted differently [5].

The goal of this project is to explore existing XAI methods and possibly develop a new method that would improve existing solutions, with a focus on counterfactual-based XAI.

Specifically, this Master thesis has four main tasks:

Explore the related scientific studies (papers) on: Counterfactual explanations for unsupervised models; and counterfactual explanations in wearable sensor data.
Preprocessing and exploratory data analysis over the LAUREATE dataset [6].
Develop Unsupervised BayCon – an unsupervised version of the supervised counterfactual generator BayCon [6]
Perform a small user-study with 5-10 participants to evaluate the quality of the explanations generated by Unsupervised BayCon

References:

Goodman, S.Flaxman, European union regulations on algorithmic decision-making and a “right to explanation”, AI Magazine 38 (3) (2017) 50–57.
Rudin, C. Stop explaining black-box machine learning models for high-stakes decisions and use interpretable models instead. Nature Machine Intelligence 1, no. 5:206-215, 2019.
Robert Hoffman, Tim Miller, Shane T. Mueller, Gary Klein, and William J. Clancey. Explaining explanation, part 4: A deep dive on deep nets. IEEE Intelligent Sys-tems 33, no. 3:87-95, 2018.
Ruth M. J. Byrne. The rational imagination: How people create alternatives to reality. MIT Press, Cambridge, Massachusetts, 2005.
Romashov, P., Gjoreski, M., Sokol, K., Martinez, M. V., & Langheinrich, M. BayCon: Model-agnostic Bayesian Counterfactual Generator.
In-house dataset (30+ participants) that contain longitudinal data from physiological sensors (heart rate, sweating rate, skin temperature and acceleration). To be provided by the supervisors:

For more information contact:

Creating an Experimental Apparatus for a Digital Health Literacy Experiment

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Status: Completed January 2021 Student: Robert Jans

The faculty of communication science is planning an experiment to investigate the digital health literacy of participants in the area of sleeping disorders. As part of this project, we are seeking a student to help implement the experimental apparatus that will allow the research team to record data and test its hypotheses.

Specifically, this Bachelor projects has three main tasks

Spidering and indexing (e.g., using Apache Solr/Lucene) a pre-defined set of website that provide sleeping disorder information as an experimental corpus
Creating a Google-like search interface to the corpus that can be configured to selectively show/rank different sets of corpus sites, according to the experimental conditions under investigation
Setting up an experimental environment (e.g., using the SafeExamBrowser) to conduct controlled experiments using the corpus (e.g., to prevent participants from accessing non-corpus sites) and to record salient data (e.g., search logs, clickstream)

For more information contact:

Deep learning models for human mobility modeling

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Master UROP Status: Completed June 2021 Student: Roland Holenstein

Human mobility modeling is widely recognized as being key to providing new services and solutions in many application domains. For example, tracking viral diseases dynamics (e.g., the COVID-19 pandemics), tracking changes in behavior which can be used to recognize impending mental health episodes, deliver more effective advertising and retail experiences, enhance security and shape the provision of urban services, etc.

In the context of a BSc project, you will implement an existing state-of-the-art method for human mobility modeling (e.g., [5],[6]) and you will apply the method on a new dataset (e.g., [1] or [2]). Also, you will analyze the behavior of the method for a variety of parameters (e.g., dataset size, model size, etc.). Finally, you will report on the results and propose possible improvements/changes that could be applied on the method for improving the results.

In the context of an MSc thesis or UROP project, you will: (i) research datasets and machine learning methods for human mobility modeling (e.g., [1][2][3][4][5][6]); (ii) pre-preprocess/normalize several mobility datasets to a common format (e.g., [1][2]); (iii) implement advanced deep-learning methods for human mobility modeling (e.g., [5],[6]) and compare their results on at least two datasets; (iv) summarize the results and propose future work.

References:

Mokhtar, Sonia Ben, Antoine Boutet, Louafi Bouzouina, Patrick Bonnel, Olivier Brette, Lionel Brunie, Mathieu Cunche et al. “PRIVA’MOV: Analysing Human Mobility Through Multi-Sensor Datasets.” 2017 [https://hal.inria.fr/hal-01578557/document]
Moro, Arielle, Vaibhav Kulkarni, Pierre-Adrien Ghiringhelli, Bertil Chapuis, and Benoit Garbinato. “Breadcrumbs: A Feature Rich Mobility Dataset with Point of Interest Annotation.” arXiv preprint arXiv:1906.12322 (2019) [https://arxiv.org/pdf/1906.12322.pdf]
Luca, Massimiliano, Gianni Barlacchi, Bruno Lepri, and Luca Pappalardo. “Deep Learning for Human Mobility: a Survey on Data and Models.” arXiv preprint arXiv:2012.02825 (2020). [https://arxiv.org/pdf/2012.02825.pdf]
Feng, Jie, Zeyu Yang, Fengli Xu, Haisu Yu, Mudan Wang, and Yong Li. “Learning to Simulate Human Mobility.” In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 3426-3433. 2020. [https://www.youtube.com/watch?v=sj4UCW0P6Ks&ab_channel=AssociationforComputingMachinery%28ACM%29]
Feng, Jie, Yong Li, Chao Zhang, Funing Sun, Fanchao Meng, Ang Guo, and Depeng Jin. “Deepmove: Predicting human mobility with attentional recurrent networks.” In Proceedings of the 2018 world wide web conference, pp. 1459-1468. 2018. [https://github.com/vonfeng/DeepMove]
Yu, Lantao, Weinan Zhang, Jun Wang, and Yong Yu. “Seqgan: Sequence generative adversarial nets with policy gradient.” In Thirty-first AAAI conference on artificial intelligence. 2017. [https://github.com/LantaoYu/SeqGAN]
scikit-mobility: mobility analysis in Python
Brown, Tom B., Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan et al. “Language models are few-shot learners.” arXiv preprint arXiv:2005.14165 (2020). [https://arxiv.org/abs/2005.14165]
Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. “Attention is all you need.” In Advances in neural information processing systems, pp. 5998-6008. 2017. [https://arxiv.org/abs/1706.03762]
Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. “Bert: Pre-training of deep bidirectional transformers for language understanding.” arXiv preprint arXiv:1810.04805 (2018). [https://arxiv.org/pdf/1810.04805.pdf?source=post_elevate_sequence_page—————————]
“Open Sourcing BERT: State-of-the-Art Pre-training for Natural Language Processing”. Google AI Blog. Retrieved 2019-11-27.

For more information contact:

Design and Evaluation of a Smartphone App to Support Sharing Physical Objects (SHA21.C)

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Master UROP Status: Draft Student:

Many persons are willing to contribute to the community by sharing objects they own, such as household items, tools and media items. The goal of the project is to develop an application that supports this by connecting lenders and borrowers in an easy way. Beyond the obvious features of a standard “classifieds” app (i.e., a potential lender provides information about objects that he is willing to lend; the application provides an easy way for the borrower to find the items he is interested in and, once found, contact the lender) the app should focus on supporting the interaction inherent in such physical lending, i.e., the physical exchange of the item in question. For example, a simple “bump” could allow a borrower to acknowledge receipt of an item, or an embedded NFC tag in the item itself could be used. Optionally, the app should be evaluated within a short study with several participants. Willingness to learn qualitative research methods in Human-Computer Interaction, as well as basic iOS programming skills required; strong Web programming skills an asset. Hardware such as a mobile phone and a smartwatch will be provided.

For more information contact: No items found

Detection of Social Engagement Interaction using Smartphones

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Master UROP Status: Draft Student:

Recent improves in mobile technologies (better and richer sensors, cheaper and faster storage) allow us to record different aspects of our daily activities, e.g. work meetings, discussions during coffee breaks, family moments, etc. Within the EU research project RECALL we are investigating how such data captured from mobile and wearable devices can serve as “memory cues” to help individuals remember certain past events – e.g., a set of images can help one remember an evening with friends or pictures showing white board content can help one remember what was discussed in the last work meeting.

We are interested to understand how such data can be safely shared with other co-located people – e.g. with all the colleagues that participated in the meeting – in order to construct a better representation of what really happened in an event. Hence, our bigger goal is to build a solution for peer2peer exchange of captured memory cues or moments. The goal of this bachelor project is to develop an Android mobile app that uses audio data (and/or other mobile sensors) to detect when two people are socially interacting (e.g. speaking). This will be used as a trigger to decide when to start our peer2peer moment sharing. If time permits the student can also experiment and try to construct a solution for detecting the presence of nearby peers using audio signals. The project requires solid Android programming skills, preferably some knowledge on how to work with audio data and willingness to quickly learn new things.

For more information contact: No items found

Displays Mobile Application: A Mobile Interface for Customizing Public Displays

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Status: Completed June 2015 Student: Talal El Afchal

Display blindness is a known problem for interactive public displays since passers-by ignore or pay little attention to displays in public. Various theories argue that people ignore displays not because they are not interested in interacting with them, but because they are not interested in all the displayed information. A mobile application that personalizes the displays could increase people’s attention to the displayed information. In order to allow the displays personalization this project implements an android mobile application where users can choose applications to be displayed on public displays.

For more information contact:

EDA Lateralization during Sleep

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Master Status: Draft Student:

Advances in wearable technologies have made possible the ubiquitous sensing of physiological signals, like Blood Volume Pressure (BVP) or ElectroDermal Activity (EDA). However, some signals, like EDA, can change the value that is recorded depending on which side of the body the sensor is placed on. The medical literature has explored extensively this phenomenon, called Lateralization, in both EDA and other physiological recordings, but it is not clear the impact it might have on wearable-based applications. In this thesis, the objective is thus to analyse the lateralization effect from physiological signals recorded while people are sleeping. As such, the first a data collection will be run, to integrate some existing dataset. Then, the student will analyse the raw signal as well as extract some hand-crafted features. A Machine Learning task will be then investigated, to gauge the impact of lateralization, e.g., higher or lower performance of the classifier depending on which side the data is trained on.

This project is available as a MSc thesis or as a BSc thesis (with reduced tasks).

For more information contact:

Embodied Large Language Models for Personalized MeetingSummarization

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Master UROP Status: Available

The rapid advancement of pervasive systems and wearables has paved the way for personal informatics systems, e.g., personal assistants and chatbots. Such systems gear towards enhancing users’ productivity in both work setting and personal life. In work settings, users leverage personal informatics systems in managing their tasks, tracking their progress and scheduling their meetings.

Meetings are crucial for communication, decision-making, and brainstorming. The effectiveness of meetings relies on participants’ ability to remember key topics, often using meeting minutes or notes as memory cues. However, this approach is time-consuming, demands high attention levels, and may lead to varied perceptions among participants, causing a lack of synchronization.

In this project, we aim at developing a personalized meeting summarization tool that aims at optimizing the meeting experience. The proposed tool integrates users’ affection from wearables with large language models, e.g., ChatGPT.

This project is available as a MSc thesis, as a BSc thesis, or as a UROP internship project.

For more information contact: ,

Empathetic Virtual Agents using Large Language Models

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Master UROP Status: Available

The rapid advancement of pervasive systems and wearables has paved the way for personal informatics systems, e.g., personal assistants and chatbots. These systems can be deployed in different personal and professional settings. Such personal assistants aim at personalizing the user ‘s experience by taking into consideration the user’s affection status and respond based on the user’s emotions and mental state status.

In this project, we aim at developing an empathetic personal assistant tool that provides an affection-aware user experience. The proposed tool integrates users’ affection from wearables with large language models, e.g., ChatGPT.

This project is available as a MSc thesis, as a BSc thesis, or as a UROP internship project.

For more information contact: ,

Evaluating a Human Memory Augmentation App (RECALL.C)

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Completed June 2017 Student: Matias Laporte

For more information contact:

Evaluating Secure Personal Memory Sharing with Co-Located People

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Completed February 2018 Student: Jacob Jeyara

Using smartphones and wearable devices people now can fully log their life in pictures, audio or even video recordings. Such data – “life-logs” – can help evoke past memories and potentially improve our overall cognitive abilities. One interesting opportunity in highly networked environments is the ability to share parts of one’s life-logs with others, in order to benefit from recordings of each-other (e.g., by having access to a third person view of oneself in a meeting). In order to avoid any privacy violations, life-logs should only be shared with co-located people – as soon as people leave or join the meeting, the exchange of lifelogs should be stopped or initiated, respectively. In prior work we have designed an initial prototype of this system, running on several Nexus 5X smartphones. The aim of this project is to evaluate the Android app – called ‘MemShare’, in order to understand usability requirements and use, and to further refine the overall system (including the backend server and web-based control and inspection tools) based on collected feedback and observed use. This project required a highly motivated student that will co-design, develop, and trial the MemShare system. Basic Android programming skills and willingness to learn about human subject research required; knowledge of human-computer interaction methodologies and visual design skills are a plus.

For more information contact: ,

Explainable AI for Federated Models in Wearable Sensing

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master UROP Status: Available

Federated learning and its combination with differential privacy is the latest technique for building privacy-aware machine-learning models [1]. Its primary assumption – no data leaves the local data storage, has enabled its application in a variety of privacy-sensitive domains: mobile keyboard prediction [2], human mobility modeling based on GPS data [3], modeling from electronic health records [4] , etc.

Artificial Intelligence (AI) methods can bring significant and sustainable improvements to our lives. However, end-users must be able to understand those systems. Unfortunately, today’s groundbreaking AI methods are black-boxed (i.e., the decision model and the process are not understandable). The increased complexity of AI algorithms has made previous eXplainable AI (XAI) tools unsuitable, including the fact that most of the XAI solutions are not designed to operate under privacy constraints.

This project will investigate XAI techniques compatible with privacy-aware approaches (e.g., federated learning). The focus will be on counterfactual explainers [55] for wearable sensing data. Specific project tasks are:

Analyze XAI tools that can operate under privacy constraints, focusing on counterfactuals.
Pre-process one dataset from wearable sensing systems. Example datasets include emotion recognition, activity recognition and energy expenditure estimation [6, 7, 8].
Develop machine learning models for one of the datasets in step 2, and apply existing XAI tools on the models developed, including the method for generating counterfactual explanations, BayCon [10, 11].
Develop XAI tool for counterfactual explanations that can operate under privacy constraints.

This project is available as a MSc thesis and as a UROP internship project.

Literature:

[1] J. Konečný, H. B. McMahan, F. X. Yu, P. Richtárik, A. Theertha Suresh, and D. Bacon. “Federated learning: Strategies for improving communication efficiency.” arXiv preprint arXiv:1610.05492 (2016). [2] Andrew Hard, Kanishka Rao, Rajiv Mathews, Françoise Beaufays, Sean Augenstein, Hubert Eichner, Chloé Kiddon, and Daniel Ramage. Federated learning for mobile keyboard prediction. CoRR, abs/1811.03604, 2018. URL 1811.03604. [3] Ezequiel, C. E. J., Gjoreski, M., & Langheinrich, M. (2022). Federated Learning for Privacy-Aware Human Mobility Modeling. Frontiers in Artificial Intelligence, 5, 867046. [4] Brisimi, T. S., Chen, R., Mela, T., Olshevsky, A., Paschalidis, I. C., & Shi, W. (2018). Federated learning of predictive models from federated electronic health records. International journal of medical informatics, 112, 59-67. [5] Jiang, J. C., Kantarci, B., Oktug, S., & Soyata, T. (2020). Federated learning in smart city sensing: Challenges and opportunities. Sensors, 20(21), 6230. [6] T. Miller, “Explanation in artificial intelligence: Insights from the social sciences,” Artificial intelligence 267: 1-38, 2019. [7] M. Laporte, D. Gasparini, M. Gjoreski, and M. Langheinrich. “Exploring LAUREATE-the Longitudinal multimodAl stUdent expeRience datasEt for AffecT and mEmory research.” in the UbiComp/ISWC’22 Adjunct Proceedings, 2022. [8] Gashi, S., Min, C., Montanari, A., Santini, S., & Kawsar, F. (2022). A multidevice and multimodal dataset for human energy expenditure estimation using wearable devices. Scientific Data, 9(1), 1-14. [9] Gjoreski, M., Kiprijanovska, I., Stankoski, S., Mavridou, I., Broulidakis, M. J., Gjoreski, H., & Nduka, C. (2022). Facial EMG sensing for monitoring affect using a wearable device. Scientific reports. [10] M. Gjoreski M, V. Kuzmanovski V, and M. Bohanec “BAG-DSM: A Method for Generating Alternatives for Hierarchical Multi-Attribute Decision Models Using Bayesian Optimization,” Algorithms, 2022. [11] P. Romashov, M. Gjoreski, K. Sokol, M. V. Martinez, and M. Langheinrich. “BayCon: Model-agnostic Bayesian Counterfactual Generator,”. In IJCAI 2022.

For more information contact:

Explainable AI through counterfactual explanations

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Master UROP Status: Completed August 2022 Student: Piotr Romashov

Artificially Intelligence (AI) is slowly but surely becoming an integrated part of our daily lives. However, with decisions derived from AI systems ultimately affecting human lives (e.g. medicine and law), there is an emerging need for understanding how such decisions are made by AI methods [11]. Furthermore, the recent General Data Protection Regulation (“GDPR”) [1] tasks the machine-learning (ML) community to enable explainability of the models and their output. More specifically, according to the GDPR, the ML models should offer the possibility to answer/provide explanation such as: “You were denied a loan because your annual income was £30,000. If your income had been £45,000, you would have been offered a loan.” [1]. In the ML domain, this task is referred to as “search for counterfactual explanations”. The idea is that, besides the model’s output, additional counterfactual information should be provided of how the world would have to be different for another (e.g., more desirable) outcome to occur. This new requirement has some researchers argue that a model’s accuracy should be sacrificed, and interpretable models should be preferred over black-box ML models for high-stake decisions [2].

In order to augment existing AI systems with explainability, “Explainable Artificial Intelligence (XAI)” methods are being developed actively both in academia [3-9] and industry (e.g., IBM, Microsoft, Facebook and Google). XAI deals with the creation of machine learning techniques that enable end-users to understand, trust, and possibly manage the emerging generation of so-called “artificially intelligent partners” [10].

The goal of this project is to explore existing XAI methods and possibly develop a new method that would improve existing solutions, with a focus on counterfactual-based XAI.

Specifically, this Master thesis has four main tasks:

Surveying existing algorithms for generating counterfactual explanations (e.g., [3-9, 12]);
Implementing at least one XAI method and analyzing its performance (benefits and drawbacks) on several datasets (preferably in comparison with at least one related method from related studies).
Publicly available XAI service (or python library) to be used with scikit-learn models.

References:

[1] Wachter, S., Brent M., and Chris R. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harv. JL & Tech. 31 : 841, 2017.

[2] Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence 1, no. 5:206-215, 2019.

[3] Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence 1, no. 5:206-215, 2019.

[4] Joshi, S., Oluwasanmi K., Warut V., Been K., and Joydeep G. Towards realistic individual recourse and actionable explanations in black-box decision making systems. arXiv preprint arXiv:1907.09615, 2019.

[5] Karimi, A.-H., Gilles B., Borja B., and Isabel V. Model-agnostic counterfactual explanations for consequential decisions. arXiv preprint arXiv:1905.11190, 2019.

[6] Wexler, J., Pushkarna, M., Bolukbasi, T., Wattenberg, M., Viégas, F., and Wilson, J. The what-if tool: Interactive probing of machine learning models. IEEE transactions on visualization and computer graphics, 26(1):56–65, 2019.

[7] Tolomei, G., Silvestri, F., Haines, A., and Lalmas, M. Interpretable predictions of tree-based ensembles via actionable feature tweaking. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 465–474. ACM, 2017.

[8] Ustun, B., Spangher, A., and Liu, Y. Actionable recourse in linear classification. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 10–19. ACM, 2019.

[9] Dandl, Susanne, Christoph Molnar, Martin Binder, and Bernd Bischl. “Multi-objective counterfactual explanations.” In International Conference on Parallel Problem Solving from Nature, pp. 448-469. Springer, Cham, 2020.

[10] D.Gunning, Explainableartificialintelligence(xAI),TechnicalReport,DefenseAd- vanced Research Projects Agency (DARPA), 2017.

[11] B.Goodman,S.Flaxman,Europeanunionregulationsonalgorithmicdecision-mak- ing and a “right to explanation”, AI Magazine 38 (3) (2017) 50–57.

[12] Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., … & Herrera, F., “Concepts, taxonomies, opportunities and challenges toward responsible AI,” Information Fusion, 58, 82-115, 2020

For more information contact:

Exploring LAUREATE- the Longitudinal multimodAl stUdent expeRience datasEt for AffecT and mEmory research

Mon, 01 Jan 0001 00:00:00 +0000

Type: UROP Status: Completed June 2022 Student: Daniele Gasparini

Studies in the lab have shown that affect recognition using physiological data is feasible with machine learning methods. Datasets collected in-the-wild can further improve such methods’ robustness and applicability. This study presents LAUREATE, a Longitudinal mUltimodal student expeRience datasEt for AffecT and mEmory research. The dataset was collected throughout a university semester with 44 participants (including two lecturers) in two courses totalling 52 sessions, including classes, quizzes, and exams. We recorded participants’ physiological signals with a wristband device and collected daily survey answers about participants’ behaviour (e.g., study hours, smoking habits, physical activity, caffeine and food intake) and their perceived engagement, attention, and emotions after class. As a proxy for evaluating the quality of the physiological data, we present preliminary findings about the relation between the physiological signals and the different session types.

For more information contact:

Fed-CogLoad: Federated Cognitive Load Estimation

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Status: Assigned March 2023 Student: Mattias Formo

Federated learning (FL) is a state-of-the-art machine-learning technique developed by Google, where the users’ privacy is guaranteed by implementing one simple rule: “No personal data leaves the user-device”. This project will investigate FL techniques for cognitive load estimate. Cognitive load can be estimated through the analysis of from pupillometry data, brain activation data (EEG), breathing rate, heart rate, heart rate variability and other related physiological responses.

Specifically, this thesis has four main tasks:

overview excising datasets for cognitive load estimation;

develop a centralized machine learning pipeline for cognitive load estimation;

develop a FL pipeline for cognitive load estimation;

The main challenge of the work will be to develop enhanced privacy-aware models for cognitive-load modeling using federated learning.

For more information contact:

FunSquare: The Context Collector

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Status: Completed June 2011 Student: Matteo Bellan

In a world where social networks diffusion is limiting face-to-face interactions, a dedicated technology can bring them back in the place where they originated: squares and cities. Public Displays networks, as a growing research field with a very high potential, are the perfect tool for such a purpose. Within the FunSquare project, the goal is to build a public display application that provides conversational topics and, by becoming a special feature of the place, helps stimulating socialization through triangulation, an effect where a special feature of a place acts as a link between people.

The FunSquare Context Collector (FCC) is a core module of the FunSquare project. Its goal is to automatically gather -from different sources across a city and on the web- information about the places where the displays are located, in order to attract people with something they feel to be part of.

For more information contact:

FunSquare: The Fun Fact Generator

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Status: Completed June 2011 Student: Thomas Selber

Funsquare is a public-display service to facilitate social interaction between strangers, acquaintances and friends. It displays context-aware “fun facts” about a place, using an arbitrary range of topics, e.g., sustainability, history, science. By providing conversational topics and becoming a special feature of the place, Funsquare can help stimu- lating socialization through “triangulation”, an effect where special feature of the place acts as a link between people.

The goal of this project is to implement the F3G module of the FunSquare application, i.e., an engine that is able to pair various “facts” from a database with “context information” and generate lists of “fun facts” that are written back into the database, so that another module can display them.

For more information contact: ,

Generative Adversarial Networks (GANs) for human mobility trajectory generation

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Completed July 2022 Student: Ivan Fontana

GANs are a type of neural networks that once trained on a specific dataset, they can generate new samples, similar but not same, as the samples in the training dataset (e.g., see 15). This generative characteristic can be used to generate human mobility trajectories, and thus enhance existing labeled human mobility datasets, in order to improve the performance of existing methods for human mobility modeling.

In the context of a BSc project, you will: (i) overview methods and datasets for trajectory generation (e.g., [4]); (ii) use an existing GAN method for trajectory generation on new datasets; (iii) compare the generated trajectories with original trajectories and analyze the behavior of the network for a variety of parameters; (iv) summarize the results and propose future work.

In the context of an MSc thesis or UROP project, you will: (i) overview methods and datasets for trajectory generation (e.g., [4]); (ii) pre-process datasets appropriate for the task (e.g., [1][2]); (iii) create a novel GAN method for trajectory generation; (iv) use the generated trajectories to enhance existing datasets and evaluate machine-learning models for next-task prediction on enhanced vs. original dataset; (v) summarize the results and propose future work.

References:

Mokhtar, Sonia Ben, Antoine Boutet, Louafi Bouzouina, Patrick Bonnel, Olivier Brette, Lionel Brunie, Mathieu Cunche et al. “PRIVA’MOV: Analysing Human Mobility Through Multi-Sensor Datasets.” 2017 [https://hal.inria.fr/hal-01578557/document]
Moro, Arielle, Vaibhav Kulkarni, Pierre-Adrien Ghiringhelli, Bertil Chapuis, and Benoit Garbinato. “Breadcrumbs: A Feature Rich Mobility Dataset with Point of Interest Annotation.” arXiv preprint arXiv:1906.12322 (2019) [https://arxiv.org/pdf/1906.12322.pdf]
Luca, Massimiliano, Gianni Barlacchi, Bruno Lepri, and Luca Pappalardo. “Deep Learning for Human Mobility: a Survey on Data and Models.” arXiv preprint arXiv:2012.02825 (2020). [https://arxiv.org/pdf/2012.02825.pdf]
Feng, Jie, Zeyu Yang, Fengli Xu, Haisu Yu, Mudan Wang, and Yong Li. “Learning to Simulate Human Mobility.” In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 3426-3433. 2020. [https://www.youtube.com/watch?v=sj4UCW0P6Ks&ab_channel=AssociationforComputingMachinery%28ACM%29]
Feng, Jie, Yong Li, Chao Zhang, Funing Sun, Fanchao Meng, Ang Guo, and Depeng Jin. “Deepmove: Predicting human mobility with attentional recurrent networks.” In Proceedings of the 2018 world wide web conference, pp. 1459-1468. 2018. [https://github.com/vonfeng/DeepMove]
Yu, Lantao, Weinan Zhang, Jun Wang, and Yong Yu. “Seqgan: Sequence generative adversarial nets with policy gradient.” In Thirty-first AAAI conference on artificial intelligence. 2017. [https://github.com/LantaoYu/SeqGAN]
scikit-mobility: mobility analysis in Python
Brown, Tom B., Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan et al. “Language models are few-shot learners.” arXiv preprint arXiv:2005.14165 (2020). [https://arxiv.org/abs/2005.14165]
Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. “Attention is all you need.” In Advances in neural information processing systems, pp. 5998-6008. 2017. [https://arxiv.org/abs/1706.03762]
Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. “Bert: Pre-training of deep bidirectional transformers for language understanding.” arXiv preprint arXiv:1810.04805 (2018). [https://arxiv.org/pdf/1810.04805.pdf?source=post_elevate_sequence_page—————————]
“Open Sourcing BERT: State-of-the-Art Pre-training for Natural Language Processing”. Google AI Blog. Retrieved 2019-11-27.

For more information contact:

Group Activity Journal – Shared Feed for Outdoor Sports

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Master UROP Status: Draft Student:

Skiing and snowboarding are highly social activities. Winter enthusiasts capture and share vast amount of pictures and videos during outdoor vacations. To support information exchange among groups of skiers and snowboarders we propose a group daily feed that could be automatically populated with media, POIs, and contextual and statistical details of the skiing activity, and which would be accessible online through a mobile phone app or a web-page. The feed would show user-captured events and present them on a timeline. Events could be added instantly via a simple tap using a smartwatch and/or using mobile phone interface. Following the recent trend in instant messaging services such as Snapchat or iMessage, where a message can expire after some time, we would like to associate an expiration tag to any events added to the feed. This “group feed” populated by custom events of participants could be used as a trip report or a blog to create a narrative about ski vacations for the group members. The feed would be automatically shared among participants with an option to grant access to external observers who want to follow a particular participant or the entire group in near real time.

Basic iOS programming skills required, strong Web programming skills an asset. Hardware such as a mobile phone and a smartwatch will be provided.

For more information contact: No items found

Investigating the dichotomy of sharing practices in virtual and physical realms: from theoretical overview to design considerations (SHA21.A)

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Completed July 2017 Student: Jeremías Albano

Online social networks have made sharing personal experiences with others – mostly in form of photos and comments – a common activity. Nowadays the scope of user-generated and shared content on the net varies vastly from personal media to individual preferences to physiological information (e.g. in form of daily workouts). Popular “sharing economy” services (e.g. AirBnB, Uber) and connected devices are expanding the set of “things” one can share. Given that a new generation of sharing services is about to emerge, it is of crucial importance to understand how traditional sharing practices inform and support designers of those services. This project will look into consolidating the existing body of work on both sharing personal digital content (e.g., on social networking sites, through photo sharing apps) and personal physical possessions (e.g., apartment sharing). The project aims to identify commonalities and differences between digital and non-digital context sharing, in particular summarizing existing research on motivations to share, audience management, privacy and trust issues, and user experience requirements. If possible, these findings should be connected to contemporary theories of social psychology and practice theory. The final results of this project would be: (1) a comprehensive account of the existing body of knowledge on content and resource sharing practices; (2) a set of design considerations that allows designers and developers to build future sharing services to enable sharing activities bridging virtual and physical realms. This projects requires strong analytical skills and a willingness to learn about a novel and emerging research field. Experience with interdisciplinary research literature a plus (e.g., seminar work in human-computer interaction) though supervising guidance is available.

For more information contact:

Just Share It: A Decentralized Autonomous System to Support Sharing Physical Objects Using Blockchain and Smart Contracting

Mon, 01 Jan 0001 00:00:00 +0000

Type: UROP Status: Completed May 2018 Student: Egor Ermolaev

Many persons are willing to contribute to the community by sharing objects they own, such as household items, tools and media items. For example in Switzerland an online service provides a set of stickers for a mailbox to let people see what household items one can borrow from their neighbours. However, the service does not support the actual act of sharing those items – how borrower and lender meet, agree, and exchange. “Just Share It” is meant as an application that provides such a service, by connecting lenders and borrowers through mobile technology.

A potential lender provides information about objects that he is willing to lend. The application provides an easy way for borrowers to find the items they are interested in. Once a borrower has found an item, the application provides a way for the lender and the borrower to communicate and come to an arrangement. An underlying layer of blockchain-driven smart contracting technology facilitates online contractual agreements (e.g., to record sharing transactions). In order to maintain a positive and friendly environment, “Just Share It” will also need to provide means by which users can build up trust. For example, the application should allow the borrower to leave a short feedback about the experience with the item in the form of a short notice and a picture.

This project is part of the Swiss National Science Foundation funded SHARING21 research project, where we are looking into new ways of supporting sharing both digital information and physical objects. The goal of this project is to implement the “Just Share It” application on a mobile platform, incorporating blockchain and smart contracting technologies (e.g., using Etherium, an open-source distributed computing platform). Master students applying for this project in addition to developing the prototype are expected to conduct its evaluation with a dozen of users.

Strong Web programming skills (Ajax frameworks) are required, basic Solidity and/or C++ programming skills are an asset, prior experience with iOS and/or multi-platform smartphone app development (e.g. Ionic framework) is desirable. Hardware such as a mobile phone and a smartwatch will be provided. Supervising guidance on qualitative research methodology (Anton) and distributed information system architecture (Agon) is available.

For more information contact: ,

MoviePlus – A Platform for Sharing Digital Content For Augmented Reality Displays

Mon, 01 Jan 0001 00:00:00 +0000

Type: UROP Status: Completed September 2018 Student: Bianca Stancu

Nowadays people are using tablets, laptops and various digital devices to work collaboratively on the same project while being present physically or remotely. Imagine to work on the same design or analytical task, or just simply watch an entertaining video using Google Glass. Annotating and interacting with virtual content became a daily activity of many architects, engineers, designers, artists, etc. We offer a student to join on exciting opportunity to design a prototype, which enables digital content sharing and interaction in Augmented Reality. The student design and develop the shared movie viewing system using VideoPlus, a platform for augmented video content ( .

We propose to use the combination of a head-mounted display (e.g., iPhone X/ Pixel 2 with Daydream or Microsoft Hololens) and a wrist-worn controller (e.g., Apple Watch) to develop an interactive system, which would enable tagging favorite scenes, sharing subtitles, sending ephemeral messages among viewers and/or taking collaborative trivia quizzes. We envision prototyping on the mobile device as part of the project work, so we would encourage students with iOS/Android or C#/C++ development skills to apply. Ultimately, we would like to conduct controlled experiments to evaluate a designed system.

This project is hosted by Faculty of Informatics and Institute of Computational Science. Thus, the student will be co-supervised by the responsible investigators from both sides.

For more information contact:

Multimodal Federated Learning for Sensor Data

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Status: Assigned February 2023 Student: Alessandro Gobbetti

This project will investigate single modality vs. multi-modality federated models. This is an important issue for wearable sensing systems that utilize multiple sensing devices, e.g., smartphone and smartwatch. Each device, and each sensor in the devices, may have a different availability –– data coming from the smartwatch may be unavailable at certain periods (e.g., while charging). To enable the collaborative learning of joint models between users with a variable data/modality availability, we will investigate several multi-modal schemes.

Project tasks:

Pre-process one dataset from wearable sensing systems. Example datasets include emotion recognition, activity recognition and energy expenditure estimation [6, 7, 8].
Build centralized multimodal and single-modal models using the dataset from step 1.
Build federated multimodal and single-modal models using the dataset from step 1 and compare them with the centralized models from 2.
Develop novel multi-modal federated learning method considering device/sensor availability, computational cost, model accuracy.

This project is available as a MSc thesis (all four tasks), as a BSs thesis (only task 2 and task 3), or as a UROP internship project (all 4 tasks).

Literature:

[1] Konečný, Jakub, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. “Federated learning: Strategies for improving communication efficiency.” arXiv preprint arXiv:1610.05492 (2016). [2] Andrew Hard, Kanishka Rao, Rajiv Mathews, Françoise Beaufays, Sean Augenstein, Hubert Eichner, Chloé Kiddon, and Daniel Ramage. Federated learning for mobile keyboard prediction. CoRR, abs/1811.03604, 2018. URL 1811.03604. [3] Ezequiel, C. E. J., Gjoreski, M., & Langheinrich, M. (2022). Federated Learning for Privacy-Aware Human Mobility Modeling. Frontiers in Artificial Intelligence, 5, 867046. [4] Brisimi, T. S., Chen, R., Mela, T., Olshevsky, A., Paschalidis, I. C., & Shi, W. (2018). Federated learning of predictive models from federated electronic health records. International journal of medical informatics, 112, 59-67. [5] Jiang, J. C., Kantarci, B., Oktug, S., & Soyata, T. (2020). Federated learning in smart city sensing: Challenges and opportunities. Sensors, 20(21), 6230. [6] M. Laporte, D. Gasparini, M. Gjoreski, and M. Langheinrich. “Exploring LAUREATE-the Longitudinal multimodAl stUdent expeRience datasEt for AffecT and mEmory research.” in the UbiComp/ISWC’22 Adjunct Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2022. [7] Gashi, S., Min, C., Montanari, A., Santini, S., & Kawsar, F. (2022). A multidevice and multimodal dataset for human energy expenditure estimation using wearable devices. Scientific Data, 9(1), 1-14. [8] Gjoreski, M., Kiprijanovska, I., Stankoski, S., Mavridou, I., Broulidakis, M. J., Gjoreski, H., & Nduka, C. (2022). Facial EMG sensing for monitoring affect using a wearable device. Scientific reports, 12(1), 1-12.

For more information contact:

NDA: Developing Networked Public Display Analytics Based on Classifying the Audience in Front of a Display and Mapping Them to Interaction Zones

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Completed September 2014 Student: Miloš Nikolić

Public display systems are increasingly becoming part of the urban landscape, with systems being deployed in venues such as railway stations, shopping malls and city squares. It is not hard to imagine that soon these displays will be networked and empowered with interaction technology, e.g., touch, thus constituting a novel and powerful communication medium – networked public displays. One of the biggest problems this medium is facing is determining measures of success for different application that will be running on these displays. In other words: How many people pass by the display? Out of those how many glance at the screen? Out of those how many actually interact with the application? Is there any influence on the audience size on the interactions with the screen?

The aim of this project is to develop a set of tools and metrics that will allow automatic detection and clustering of the audience in front of the display into groups such as 1) passers-by, 2) people who glance at the screen, and 3) people who interact with the screen. As these groups can emerge all at once around the scree in different display zones it will be important to map the audience/users to different interaction zones.

At the outset, 1) using the Kinect sensor and live video feed a simple algorithm that detects the number of people in front of the screen should be developed. Then using the information from depth-camera a classifier should be developed that clusters detected users into different groups, i.e., 1) passers-by, 2) people who glance at the screen, and 3) people who interact with it. At a certain point in time users should receive IDs and transitions between the interaction zones and user groups should be detected (e.g., a person passing by the screen then moving a bit closer and finally interacting with a display application. In order to confirm the accuracy of the cluster video recordings from another camera will be used to manually label and classify the audience and their interaction zones.

For more information contact:

Personalized Public Displays: Designing Interactive Display Applications for Active and Walk-by Content Personalization

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Status: Completed June 2017 Student: Francesco Saverio Zuppichini

Public display systems are increasingly becoming part of the urban landscape, with systems being deployed in venues such as railway stations, shopping malls, city squares, and universities. Most public displays today are simple slide-show systems that broadcast content based on a pre-selected schedule. However, public displays envisioned in the near future will provide a platform for running diverse interactive applications with highly personalized content. These applications will allow viewers to express their preferences both explicitly through interactive touch interfaces (active personalization) and implicitly using mobile handsets (walk-by personalization).

The aim of this project is to explore the development, deployment, and actual use of web-based interactive display applications that can show highly personalized content such as content from online social platforms on public displays. The main focus of this project is on 1) designing and implementing minimum three web-based applications that can show personalized and user generated content (e.g., Facebook stream) and show such a content on public displays based on a set of personalization parameters, 2) implementing mobile interfaces for the developed apps that can be “loaded” into an existing Android-based “controller app” prototype (called “Tacita”) that allows users to set the personalization parameters, and 3) extending the existing Tacita application with the indoor localization using Bluetooth Low Energy (BLE) devices that will support implicit walk-by personalization.

For more information contact:

Personalized Public Displays: Uncovering Personalization Needs

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Master UROP Status: Draft Student:

With the significant price drops of large LCD panels, public displays are increasingly dominating the urban landscape. It is not hard to imagine that in the near future, public displays will shift from showing predefined still images or videos to becoming more interactive and highly personalized. By providing custom content to individual passers-by such as upcoming bus schedules, relevant news items, and even personalized messages (e.g., Twitter, Facebook, or Google+ posts), public displays may increase their utility and become more appreciated in our environment. However, showing such a contextualized and personalized content increases privacy concerns and may impact the use of public displays.

The main goal of this project is to explore, uncover, and understand personalization needs of potential and actual users of public displays. The project will start with an online survey/questionnaire for assessing current understanding of personalization needs of public displays in a university environment. Following the initial survey, the student will conduct in-depth interviews with the student community members using snow-ball sampling method or conduct interviews next to the displays at the point of interaction. The main task is to identify different “work roles” of the community members and uncover their information needs and technologies used while “on the go” on-campus. The initial survey and in-depth interviews will be followed by an extensive data analysis looking into personalization needs and concerns of the display users and understanding what technologies are used, when, and how to satisfy the information needs on-campus. A possibility to strengthen the data analysis would be to conduct a second set of interviews after installation and initial use of implicit personalization extensions into the existing display system. These personalization extensions will allow users to personalize display applications and their content using a mobile application called “Tacita”.

For more information contact: No items found

Personalized Shared Public Displays: Supporting Multi-user Walk-by Content Personalization

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Completed September 2018 Student: Filip Zivkovic

The aim of this project is to explore the development, deployment, and evaluation of web-based interactive display applications that can show personalized content influenced by the presence and preferences of multiple mobile phone users. The main focus of this project is on: 1) Designing and implementing two web-based applications, based on Public Transportation and USI Class Schedule information available through online API services, that can visualize and personalize such a content on public displays based on a set of personalization parameters. The apps should provide novel content visualization and scheduling algorithms that can support multiple, potentially conflicting, walk-by content personalization requests on shared public displays. 2) Integrating and deploying the new display applications within an existing touch-enabled display system at USI and supporting an existing Tacita personalization framework by implementing mobile interfaces that can be “loaded” into an existing iOS “controller app” and can be used to set personalization content parameters within the apps. 3) Evaluating the stability, latency, and overall performance of the apps within the existing display network. The evaluation will be performed based on the system log-files, time-stamps of events and personalization requests that occur within the apps, performance of the developed content visualization and scheduling algorithms, and the usage of system resources, e.g., CPU usage. The applications should be developed using HTML5 technologies within Play (Java) framework.

For more information contact:

Personalized Sleep Quality Prediction Model Using Wearable Devices

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Master Status: Draft Student:

Mobile and wearable devices can nowadays be used as highly available, non-invasive tools to monitor human behavior. Their ubiquitous characteristics encourage their employment in personal health monitoring systems. Such systems provide the user with continuous feedback about daily behavior, productivity, well-being, etc. Daily sleep is considered a pivotal factor in daily routine due to its role in the rejuvenate of brain and body from accumulated daily basis fatigue. The continuous quantification of sleep quality helps in assessing human health and life patterns. However, the interpersonal variability hinders this task by creating a significant gap between the user-reported sleep quality and the objective sleep quality. The goal of this thesis is to analyze the impact of different personal characteristics on the user-reported sleep quality. Then, to make use of this knowledge along with physiological signals from wearable devices to build a personalized sleep quality predication model.

This project is available as a MSc thesis or as a BSc thesis (with reduced tasks).

For more information contact:

Physiological Synchrony and its Effects on Recall in Peer Meetings

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Completed March 2021 Student: Luca Albinati

Meetings form an integral (if often boring) part of our work. A common approach to improve the effectiveness of such meetings is to create a record of it for later review, thus allowing participants to prepare for the next meeting. Being able to effectively follow up a meeting is crucial to its success. Technology can make record creation easier, e.g., by automatically creating a meeting transcript from a video recording. However, reading a full summary of a meeting can be cumbersome. Furthermore, such an “objective” view of the meeting does not consider the individual experience of each participant.

An alternative approach is to provide participants with so-called “memory cues” – either semantic (e.g., words clouds showing key concepts discussed in the meeting) or episodic (e.g., short audio clips, pictures) – that allow them to recall, from their own memory, what transpired during the meeting. Today’s technology makes both the collection, preparation, and presentation of memory cues possible. The ubiquitous availability of screens (e.g., laptop screen savers, tablet and smartphone lock screens, wall-mounted displays) enables the ambient presentation of memory cues, while wearable recorders and sensors facilitate the comprehensive capture of one’s meeting experience. Wearable biophysical sensors (e.g., as integrated in smart watches) allow for a highly personalized capture process that “understands” how a meeting affects each participant.

One interesting social process in a meeting is called “synchrony” – a cognitive, affective, and behavioral connection between two or more participants that creates a sort of “connection” or “vibe”. This connection has been shown to greatly improve the effectiveness of a meeting, as well as one’s recall. Social synchrony manifests itself physiologically in our brains: “synchronized” people have their brains react to a task or event simultaneously. While using an fMRI machine to measure each participant’s brain activity clearly is infeasible, prior research has shown that electrodermal activity (EDA) is a good proxy for brain activity. People who are synchronized not only have “synchronized” brain activity, but also show simultaneous changes in EDA.

The goal of this Master’s thesis is to design, develop, and execute a small pilot experiment to study how the physiological synchrony of peers during a meeting affect their memory of the event, and how this can be used to create a recall-based productivity system.

For more information contact:

Privacy-aware human mobility modeling

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Master UROP Status: Completed June 2022 Student: Jose Castro Elizondo Ezequiel

Human mobility can be modeled using smartphone data, however, such data represents sensitive information, and the collection of those threatens the privacy of the users involved. The recent introduction of federated learning, a privacy-preserving approach to build machine and deep learning models, represents a promising technique to solve the privacy issue.

In the context of a BSc project, you will: overview methods based on federated-learning [8][9][10] with the aim to use them for human mobility modeling; (ii) use an existing dataset to train a deep learning model for human mobility modeling (e.g., next task prediction) [12][13] using federated learning; (iii) analyze the behavior of the model for a variety of parameters (e.g., dataset size, model size, etc.); (iv) summarize the results and propose future work.

In the context of an MSc thesis or UROP project, you will: (i) create an overview of privacy-aware human mobility modeling approaches; (ii) pre-preprocess/normalize datasets to a common format (e.g., [1][2]); ; (iii) create advanced privacy-aware models for human mobility modeling (e.g., Transformer networks [8][9][10] using federated learning [14]); (iv) compare the results to existing baselines (e.g., baselines based on RNNs [5],[6]); (v) summarize the results and propose future work.

References:

Mokhtar, Sonia Ben, Antoine Boutet, Louafi Bouzouina, Patrick Bonnel, Olivier Brette, Lionel Brunie, Mathieu Cunche et al. “PRIVA’MOV: Analysing Human Mobility Through Multi-Sensor Datasets.” 2017 [https://hal.inria.fr/hal-01578557/document]
Moro, Arielle, Vaibhav Kulkarni, Pierre-Adrien Ghiringhelli, Bertil Chapuis, and Benoit Garbinato. “Breadcrumbs: A Feature Rich Mobility Dataset with Point of Interest Annotation.” arXiv preprint arXiv:1906.12322 (2019) [https://arxiv.org/pdf/1906.12322.pdf]
Luca, Massimiliano, Gianni Barlacchi, Bruno Lepri, and Luca Pappalardo. “Deep Learning for Human Mobility: a Survey on Data and Models.” arXiv preprint arXiv:2012.02825 (2020). [https://arxiv.org/pdf/2012.02825.pdf]
Feng, Jie, Zeyu Yang, Fengli Xu, Haisu Yu, Mudan Wang, and Yong Li. “Learning to Simulate Human Mobility.” In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 3426-3433. 2020. [https://www.youtube.com/watch?v=sj4UCW0P6Ks&ab_channel=AssociationforComputingMachinery%28ACM%29]
Feng, Jie, Yong Li, Chao Zhang, Funing Sun, Fanchao Meng, Ang Guo, and Depeng Jin. “Deepmove: Predicting human mobility with attentional recurrent networks.” In Proceedings of the 2018 world wide web conference, pp. 1459-1468. 2018. [https://github.com/vonfeng/DeepMove]
Yu, Lantao, Weinan Zhang, Jun Wang, and Yong Yu. “Seqgan: Sequence generative adversarial nets with policy gradient.” In Thirty-first AAAI conference on artificial intelligence. 2017. [https://github.com/LantaoYu/SeqGAN]
scikit-mobility: mobility analysis in Python
Brown, Tom B., Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan et al. “Language models are few-shot learners.” arXiv preprint arXiv:2005.14165 (2020). [https://arxiv.org/abs/2005.14165]
Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. “Attention is all you need.” In Advances in neural information processing systems, pp. 5998-6008. 2017. [https://arxiv.org/abs/1706.03762]
Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. “Bert: Pre-training of deep bidirectional transformers for language understanding.” arXiv preprint arXiv:1810.04805 (2018). [https://arxiv.org/pdf/1810.04805.pdf?source=post_elevate_sequence_page—————————]
“Open Sourcing BERT: State-of-the-Art Pre-training for Natural Language Processing”. Google AI Blog. Retrieved 2019-11-27.

For more information contact:

Privacy-aware Video-based Sensing for Affect Recognition

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Master UROP Status: Assigned February 2024 Student: Eleonora Bardelli

Affective computing is an interdisciplinary field that aims at the development of computer science techniques that enable machines to recognize, understand and simulate human affective states [1, 2]. A fundamental assumption is that different mental states (e.g., emotions and stress), and different intensities of those states, manifest through physiological and behavioral changes. A variety of sensing modalities can capture these changes [3, 4].

Video-based sensing is one promising approach for affect recognition, however, it is also privacy intrusive. Thus, this project will develop a method for privacy-aware personal-video sensing for affect recognition. The method will utilize personal camera (e.g., smartphone or laptop camera) and will include privacy-aware features such as: to record only when the users grant permission, to record only when a specific user is in front of the camera (user identification). Once the video is collected in a privacy-aware manner, existing video-based affect recognition software will be sued to extract affect related information.

Project tasks:

Review existing software (e.g., GitHub and Google scholar) for user identification, software for counting faces in a video, and software for Facial Action Coding System (FACS) [5].
Implement privacy-aware user identification;
Implement privacy-aware method for extracting Facial Action Units (based on FACS);
Test the overall processing pipeline is a small user-study (e.g., 5 to 10 participants).

This project is available as a MSc thesis (all four tasks), as a BSc thesis (only task two and task three), or as a UROP internship project (all four tasks).

LiteratureL

[1] R. Picard. “Affective Computing,” Cambridge, MA: MIT Press., 1997. [2] J. Tao, J., and T. Tan, “Affective computing: A review. In International Conference on Affective computing and intelligent interaction,” Springer, Berlin, Heidelberg, 2005. [3] Y. Wang, W. Song, W. Tao, A. Liotta, D. Yang, X. Li, S. Gao, Y. Sun, W. Ge, W. Zhang, and W. Zhang, “A systematic review on affective computing: Emotion models, databases, and recent advances.” Information Fusion, 2022. [4] R. Arya, S. Jaiteg Singh, and A. Kumar. “A survey of multidisciplinary domains contributing to affective computing.” Computer Science Review 40 (2021): 100399. [5]

For more information contact:

Protect privacy in wearable devices using data anonymization

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Available

Wearables facilitate continuous data collection to monitor diverse human behaviors, covering activity, health, and stress. This personal data, including electrocardiogram, movements, and heart rate, is often accessible online for research. Despite the common practice of masking names with random identifiers, studies show that this is insufficient for user identity protection due to the subject-dependent nature of physiological data. This research utilizes existing data and models to implement and assess anonymization techniques such as noise addition and synthetic data generation. The objective is to determine the extent of effective user identity protection while minimizing disruption to human behavior prediction.

This project is available as a MSc thesis or as a BSc thesis (with reduced tasks).

For more information contact: ,

Public Reactive Displays

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Status: Completed September 2010 Student: Federico Caputo

In the modern age of communication and transport, it’s an increasingly common scenario for the average person to have friends scattered all around a country, if not around the world. Despite the modern social networking platforms and the increasingly common use of VoIP software, it is still rather difficult to keep in touch with friends and colleagues, mainly due to the differences in people daily schedules. The goal of this system is to help “friends” to remain in touch despite distances and time constraints, using a same-time-free-time approach.

The same-time-free-time approach is based on the chance of two friends having some free time in the same moment, thus having the opportunity to (virtually) meet each other; what it’s missing is a system that is able to exploit these free-time at the same time occurrences, allowing friends to keep in touch. The basic idea is to have a computer connected to a motion sensor, a blue-tooth antenna and a webcam; this way, the system is able, respectively, to detect users’ presence, to recognize users through their bluetooth devices’ unique MAC address and to automatically open a sort of “virtual window” with another similar system in a different location where another user, recognized as a friend of the first user, is recognized through the same process.

For more information contact:

PulseCam: Capture your truly significant moments

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Status: Completed May 2015 Student: Christian Vuerich

Life logging emerged as a way to capture and remember more mainly through pictures. However, as life log- ging becomes increasingly mainstream, the volume of captured content also increases but our capacity for reviewing diminishes. In order to limit picture taking on such devices to only the most memorable moments, we propose a biophysical driven capture process that adapts the camera capture rate based on one’s heart rate. In our prototype called PulseCam an Android smart phone worn on the body acts as the picture capture device, adjusting its capture rate based on one’s heart rate as measured by an Android-based smart watch.

The purpose of this work is two fold: a) we examine the potential of PulseCam to capture pictures of significant moments and b) investigate the potential of such pictures to improve one’s ability to remember. This paper introduces the general approach, describes our current prototype, and outlines the planned study design.

For more information contact:

Re-Live the Moment: Visualizing Run Experiences to Motivate Future Exercises

Mon, 01 Jan 0001 00:00:00 +0000

Type: UROP Status: Completed August 2014 Student: Jesper Findahl

Contemporary psychology theory emphasizes that people are more likely to achieve planned behavior if they are reminded of previous good experiences of that behavior. Within the EU research project RECALL, we are investigating the effect of past memories on behavior change, i.e. make people run more by reminding them the fun moments they experienced during their previous run activities.

This project focuses on existing technological improvements in data collection (advanced mobile and wearable sensors) and data visualization in order to create an experimental prototype for capturing and visualizing ones physical activities. The outcome will be an an Android app that records one’s physical workouts in the form of pictures and music one was listening to at the time, and, process those data to create a slide show of the experience.

For more information contact: ,

Reactive Displays: Supporting Dynamic Display Behavior Using a Rescheduling Approach

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Completed September 2014 Student: Alessandro Gusmeroli

As a follow-up to the international research project “PD-Net”, we are deploying a number of large public displays on the USI campus. These displays will form a part of an open display network that can show interactive and concurrently running applications and allow display users to influence application and content scheduling. Spontaneous user interactions are hard to predict and can occur at any moment in time requiring public displays to adjust, or reschedule, applications and their content. Thus, the displays need to produce a new schedule, or a playlist, every time there is an input from the users.

The main goal of this project is to develop and evaluate the performance of a “rescheduling component” for public displays that produces a new schedule every time a user starts interacting with a display based on a given set of applications, associated constraints, and previous states of the display.

The software basis for this work should be a) a web-based template for developing public display applications called “WE-BAT” developed at USI, and b) the SMT solving framework for expressing and solving scheduling problems. SMT solvers combine a rich and flexible high-level modeling language with efficient decision procedures. In the course of this project, an SMT solver, such as OpenSMT developed at USI, should be used to produce a display schedule based on the WE-BAT application template.

For more information contact:

Roaming Objects – Object-encoded Digital Experiences

Mon, 01 Jan 0001 00:00:00 +0000

Type: UROP Status: Completed August 2016 Student: Lucas Pennati

In the era of Internet of Things traditionally “disconnected” objects such as household items, domestic electronics and even vehicles will have their own unique identity and can be addressed through the Web anywhere, anytime. With the advent of “sharing economy” services those objects could be easily found and shared among users (e.g., book sharing, ). Once shared the object provides a unique experience with a borrower, often could be described with a story. Remember, those checkout cards from the library, that indicates a name of a previous owner? The character of the Miyazaki’s ”Whisper of the heart” turned this simple discovery into a fascinating quest. Using this metaphor we would like to embed a digital tag (e.g. using NFC, QR code, augmented reality marker less tracking) on a “roamed” physical objects. The tags could be addressed digitally through a mobile phone or a web interface. Within the SHARING21 research project, we are looking into new ways of supporting sharing both digital information and physical objects. The bachelor project require to setup a mechanism to add, browse and retrieve digital experiences encoded into physical objects. Those experiences can be represented in form of textual, media or contextual (e.g. location, weather, mood) information associated with the object.

Strong Web programming skills required, basic iOS programming skills an asset. All hardware will be provided.

For more information contact:

Self-supervised Domain Adaptation for Sensor Data

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master UROP Status: Available

Wearable devices, combined with Artificial Intelligence (AI) methods, can bring significant and sustainable improvements to our lives – from improved patient monitoring and decreased healthcare costs to enhanced sports performance and improved quality of life. Standard approaches involve Machine Learning (ML) techniques applied to the data captured from body-worn sensing devices. The ML techniques can be based on classical (feature-based) ML, or Deep Learning (DL) applied on the raw sensor data (end-to-end learning). A typical weakness that all ML-based HAR systems have, regardless of whether they are classical or DL-based, is the domain shift that can be caused by different sensor placements [1, 2].

This project will explore personalization and domain-adaptation techniques to address important challenges in wearable computing: noisy data, limited data, and domain shifts in the labels and the sensor data due to subjectivity. ML processing pipelines (including deep learning techniques) will be augmented with the latest unsupervised and self-supervised learning techniques, including contrastive learning [3]. These advanced techniques should produce more robust and data-efficient models (i.e., requiring fewer person-specific labels). Diffusion-based approaches [4, 5] could also be considered.

Project tasks:

Overview of existing self-supervised learning approaches [6, 7, 8, 9, 10].
Pre-process one dataset from wearable sensing systems. Example datasets include emotion recognition, activity recognition and energy expenditure estimation [11, 12, 13]
Build baseline ML models using the dataset from step 2.
Develop self-supervised ML approach and compare self-supervised models with the baseline ML models from step 3.

This project is available as a MSc thesis and as a UROP internship project.

Literature:

[1] Gjoreski, M.; Gjoreski, H.; Luštrek, M.; Gams, M. How Accurately Can Your Wrist Device Recognize Daily Activities and Detect Falls? Sensors 2016, 16, 800. . [2] Kalabakov, S.; Stankoski, S.; Kiprijanovska, I.; Andova, A.; Rešˇciˇc, N.; Janko, V.; Gjoreski, M.; Gams, M.; Luštrek, M. What Actually Works for Activity Recognition in Scenarios with Significant Domain Shift: Lessons Learned from the 2019 and 2020 Sussex-Huawei Challenges. Sensors 2022, 22, 3613. . [3] Haresamudram, H., Essa, I., & Plötz, T. (2021). Contrastive predictive coding for human activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 5(2), 1-26. [4] K. Rasul, C. Seward, I. Schuster, and R. Vollgraf. Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting. In International Conference on Machine Learning, 2021. [5] L. Yang, Z. Zhang, Y. Song, S. Hong, R. Xu, Y. Zhao, Y. Shao, W. Zhang, B. Cui, and M.-H. Yang. “Diffusion models: A comprehensive survey of methods and applications.” arXiv preprint arXiv:2209.00796 (2022). [6] J.-B. Grill, F. Strub, F. Altché, C. Tallec, P. Richemond, E. Buchatskaya, C. Doersch et al. “Bootstrap your own latent-a new approach to self-supervised learning.” Advances in neural information processing systems 33 (2020): 21271-21284. [7] D. Wei, J. J. Lim, A. Zisserman, and W. T. Freeman. “Learning and using the arrow of time.” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8052-8060. 2018. [8] C. Tong, J. Ge, and N.D. Lane. “Zero-Shot Learning for IMU-Based Activity Recognition Using Video Embeddings.” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, no. 4 (2021): 1-23. [9] H. Kwon, C. Tong, H, Haresamudram, Y. Gao, G. D. Abowd, N.D. Lane, and T. Ploetz. “IMUTube: Automatic extraction of virtual on-body accelerometry from video for human activity recognition.” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, no. 3, 2020. [10] Y. Jain, C. I. Tang, C. Min, F. Kawsar, and A. Mathur. “ColloSSL: Collaborative Self-Supervised Learning for Human Activity Recognition.” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, no. 1 (2022): 1-28. [11] M. Laporte, D. Gasparini, M. Gjoreski, and M. Langheinrich. “Exploring LAUREATE-the Longitudinal multimodAl stUdent expeRience datasEt for AffecT and mEmory research.” in the UbiComp/ISWC’22 Adjunct Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2022. [12] Gashi, S., Min, C., Montanari, A., Santini, S., & Kawsar, F. (2022). A multidevice and multimodal dataset for human energy expenditure estimation using wearable devices. Scientific Data, 9(1), 1-14. [13] Gjoreski, M., Kiprijanovska, I., Stankoski, S., Mavridou, I., Broulidakis, M. J., Gjoreski, H., & Nduka, C. (2022). Facial EMG sensing for monitoring affect using a wearable device. Scientific reports, 12(1), 1-12.

For more information contact:

Shadow: An Application Selection Interface for Pervasive Public Displays

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Completed August 2014 Student: Federico Scacchi

Public display systems are increasingly becoming part of the urban landscape. Most public displays today are simple slide-show systems that broadcast content based on a pre-selected schedule. However, public displays envisioned in the near future will provide a rich platform for running diverse interactive and concurrently running applications that display viewers can actively select for presentation.

The aim of this project is to explore the development of an application selection interface for public displays in the form of user’s silhouette based on depth data and a touch screen interface. The main focus of this project is on: a) visualizing the presence of viewers in front of a display using depth information, b) adding icons of available display application to the user’s silhouette and change their size (silhouette and icons) depending on the users distance, and c) evaluating the interface. Application icons could be placed on the edges of the silhouette and by touching the icons viewers could select individual applications for an immediate on-demand presentation. The project should be developed using advanced web technologies that allow for streaming depth data and overlaying the shadow interface over the existing display content.

For more information contact:

Smart Group Activity Journal – Creating an Automated Activity Feed for Outdoor Sports (SHA21.D)

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Draft Student:

Skiing and snowboarding are highly social activities. Winter enthusiasts capture and share vast amount of pictures and videos during outdoor vacations. To support information exchange among groups of skiers and snowboarders, this project seeks to create a semi-automated group activity journal. The journal would be automatically shared among group members with an option to grant access to external observers who want to follow a particular participant or the entire group in near real time. The student should implement an app that allows one to both post to and visualize a shared “event stream”. Events posted should be supported with a simple plug-in system, e.g., one could imagine a “slope tracker” that posts to the stream whenever one has finished a ski run. Types of content added to the shared stream should include, but are not limited to (1) a pin on a map with geolocation information to setup a meeting point while on the slope; (2) captured media content (photos, videos, an optional live-stream) of the run; (3) reference information in a form of text necessary for descent (e.g., the time left until sunset, the operational hours of a ski lift at a particular location, conditions of the slope with detailed information about potential hazards during descent). Following the recent trend in instant messaging services such as Snapchat or iMessage, where a message can expire after some time, optionally one would be able to associate an expiration tag to any events added to the feed. An additional smartwatch interface should support quick entry of items, e.g., one could post a “hazard” on the slopes by selecting a hazard type and the system automatically adding time and location. The shared activity journal will be hosted on a central server and will be accessible by group participants or observers through the app. An optional integration with an optical head-mounted display for augmented reality (RideOn, Recon Snow 2) is encouraged. Intermediate to strong iOS/Android programming skills required, strong Web programming skills an asset. Hardware such as a smartphone, smartwatch, and augmented reality gear will be provided.

For more information contact: No items found

SmartHelmet: Using LIDAR sensing to alert skiers

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Status: Completed September 2015 Student: Filippo Ferrario

Helmets are a necessity on today’s ski slopes, yet it still happens that one is not aware of an approaching skier from behind (in particular when one listens to music on headphones), leading to potentially dangerous collisions. The SmartHelmet project should explore the use of multiple LIDAR sensors (e.g., the LIDAR Lite by PulsedLight) in order to provide skiers with visual (LEDs) and/or auditory (headphones) warning of skiers approaching from behind.

For more information contact:

Statistical Analysis for Affect and Learning in the LAUREATE Dataset

Mon, 01 Jan 0001 00:00:00 +0000

Type: UROP Status: Completed September 2022 Student: Cindy Guerrero Toro

For more information contact:

Table Tennis Paddle Detection Library for Microsoft Kinect

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Status: Completed June 2011 Student: Fabio Rambone

The commercial introduction of a technology as the Microsoft Kinect at its relatively low cost continues the rev- olution of how humans can interact with Computers. This revolution has originated with the Nintendo Wii, allowing a user to interact with the a system with hand gestures through a controller called “WiiMote”. The WiiMote is held in one hand and through the use of an accelerometer and optical sensors it is capable of motion sensing.

The Microsoft Kinect brought these capabilities to a new level, eliminating the need of a hand-held controller and allowing a precise position detection of the user in 3D space. This evolution extended greatly the possibilities of such a device from simple video games to much more, including its usage in virtual reality simulation. The goal of this bachelor project is to implement a table tennis paddle detection and tracking library for the Mi- crosoft Kinect. Which in turn will be used to make a realistic virtual table tennis application.

For more information contact: Randolf Schärfig

Tacita-Mobile: Controlling Content Presentation on Personalized Public Displays Through Mobile Phones

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Status: Completed June 2012 Student: Mattia Candeloro

Public display systems are increasingly becoming part of the urban landscape, with systems being deployed in venues such as railway stations, shopping malls and city squares. Most public displays today are simple slide-show systems that broadcast content based on a pre-selected schedule. However, public displays envisioned in the near future will provide a rich platform for running diverse interactive applications with highly contextualized and personal- ized content. These applications will allow viewers to express their preferences both explicitly through interactive touch interfaces and implicitly using mobile handsets. This project explores the development of an Android mobile application that allows viewers to influence the content of the application displayed on public displays. The mobile application that has been developed is based on an HTML5 component that is the user interface in conjunction with native Android components that monitors locations of the mobile phone, implements the interaction with the displays using the WebSocket protocol and store users data.

For more information contact:

The Human Dash Cam: A Human Memory Prosthesis in the Era of Ubiquitous Computing

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Master UROP Status: Draft Student: TBA

The goal of this project is to develop a mobile app that collects continuously pics and contextual info from any possible source (e.g., a wearable camera, a smartwatch) and makes it immediately accessible to the user in a “messenger style” UI (i.e. using bubbles on the screen) that stacks context up to a 5 min buffer. When something “significant” happens, the user can go back and revisit the most recent pics/data captured and augment the data explicitly by adding tags, or by semi-automatically adding further context information (e.g., add GPS data, people vicinity as indicated by Bluetooth, or one’s emotional state as measured by a medical wristband). The so-tagged “memory” will be moved to storage directory, where meaningful events are stored. Eventually, this will form part of a larger endeavor to help people to better remember the significant moments in their day. This project requires excellent mobile development skills, and the willingness to quickly learn novel APIs that allow one to connect the mobile phone to wireless devices, such as a medical wristband (Empatica E3) or a wearable camera (Narrative Clip2). All hardware will be provided.

For more information contact: No items found

Transformers for human mobility modeling

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Completed Student: Riccardo Corrias

In this BSc project you will implement a state-of-the-art model for human mobility modeling. The goal is to use Transformer networks (e.g., GPT-3, BERT), which are deep learning language/sequence modeling methods that have a potential to improve upon the previously used RNN- and LSTM-based methods. More specifically, in this project you will: (i) overview methods based on Transformer networks [8][9][10] with the aim to use them for human mobility modeling; (ii) use an existing dataset to train a Transformer network for human mobility modeling (e.g., next task prediction) [12][13]; (iii) analyze the behavior of the network for a variety of parameters (e.g., dataset size, embedding dimension, number of heads, etc.); (iv) summarize the results and propose future work.

In the context of an MSc thesis or UROP project, you will develop state-of-the-art models for human mobility modeling. The goal is to use Transformer networks (e.g., GPT-3, BERT), which are deep learning language/sequence modeling methods that have a potential to improve upon the previously used RNN- and LSTM-based methods. More specifically, in this project you will: (i) overview methods based on Transformer networks [8][9][10] with the aim to use them for human mobility modeling; (ii) pre-preprocess two mobility datasets (e.g., [1][2]); build a novel Transformer network for human mobility modeling (e.g., next task prediction) [12][13]; (iii) compare the Transformer network to an existing baseline method for human mobility modeling [5][6]; (iv) summarize the results and propose future work.

References:

Mokhtar, Sonia Ben, Antoine Boutet, Louafi Bouzouina, Patrick Bonnel, Olivier Brette, Lionel Brunie, Mathieu Cunche et al. “PRIVA’MOV: Analysing Human Mobility Through Multi-Sensor Datasets.” 2017 [https://hal.inria.fr/hal-01578557/document]
Moro, Arielle, Vaibhav Kulkarni, Pierre-Adrien Ghiringhelli, Bertil Chapuis, and Benoit Garbinato. “Breadcrumbs: A Feature Rich Mobility Dataset with Point of Interest Annotation.” arXiv preprint arXiv:1906.12322 (2019) [https://arxiv.org/pdf/1906.12322.pdf]
Luca, Massimiliano, Gianni Barlacchi, Bruno Lepri, and Luca Pappalardo. “Deep Learning for Human Mobility: a Survey on Data and Models.” arXiv preprint arXiv:2012.02825 (2020). [https://arxiv.org/pdf/2012.02825.pdf]
Feng, Jie, Zeyu Yang, Fengli Xu, Haisu Yu, Mudan Wang, and Yong Li. “Learning to Simulate Human Mobility.” In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 3426-3433. 2020. [https://www.youtube.com/watch?v=sj4UCW0P6Ks&ab_channel=AssociationforComputingMachinery%28ACM%29]
Feng, Jie, Yong Li, Chao Zhang, Funing Sun, Fanchao Meng, Ang Guo, and Depeng Jin. “Deepmove: Predicting human mobility with attentional recurrent networks.” In Proceedings of the 2018 world wide web conference, pp. 1459-1468. 2018. [https://github.com/vonfeng/DeepMove]
Yu, Lantao, Weinan Zhang, Jun Wang, and Yong Yu. “Seqgan: Sequence generative adversarial nets with policy gradient.” In Thirty-first AAAI conference on artificial intelligence. 2017. [https://github.com/LantaoYu/SeqGAN]
scikit-mobility: mobility analysis in Python
Brown, Tom B., Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan et al. “Language models are few-shot learners.” arXiv preprint arXiv:2005.14165 (2020). [https://arxiv.org/abs/2005.14165]
Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. “Attention is all you need.” In Advances in neural information processing systems, pp. 5998-6008. 2017. [https://arxiv.org/abs/1706.03762]
Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. “Bert: Pre-training of deep bidirectional transformers for language understanding.” arXiv preprint arXiv:1810.04805 (2018). [https://arxiv.org/pdf/1810.04805.pdf?source=post_elevate_sequence_page—————————]
“Open Sourcing BERT: State-of-the-Art Pre-training for Natural Language Processing”. Google AI Blog. Retrieved 2019-11-27.

For more information contact:

Uncertainty-aware Deep Learning in digital healthcare

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master UROP Status: Available

In this thesis, the objective is to investigate the use of Monte Carlo Dropout, a Bayesian Deep Learning technique, to make uncertainty prediction on Neural Network outputs when dealing with physiological data. In particular, the student will investigate the creation of a system using Early Exit to adjust the computational power based on the uncertainty prediction. The student will leverage state-of-the-art Deep Learning models to make predictions on publicly available datasets for health applications, e.g., ECG data from healthy and unhealthy individuals.

This project is available as a MSc thesis or as a EUROP project.

For more information contact: ,

Uncovering needs and requirements for data sharing activities in sports

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Completed August 2015 Student: Nadeen Alkaydi

Digital cameras, mobile phones, portable music players, activity trackers, and similar devices make creating, uploading, and interacting with digital content easier today than ever before. Users are increasingly interested in sharing content across computers and devices and across an ever-increasing number of online platforms (e.g, YouTube, Facebook, Google Docs, Instagram) that make data potentially available anywhere.

The proposed master thesis aims to discover needs and requirements of sportspeople for sharing activity-related personal data, both biometrical and non-biometrical. We invite a master student to conduct an empirical study within the extended USI community, encompassing students, their families and friends. The student should focus on comparing sharing requirements and practices across different sport activities (e.g. runners vs alpine skiers), document differences in sharing habits. The ultimate research question to be answered is: How do sportspeople practices and the context of an activity influence the communication needs and practices of sharing digital sports-related data?

For more information contact:

Understanding Practices and Motivations for Sharing Physical Resources through Digital Services (SHA21.B)

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Draft Student:

Today, vast amounts of user-generated and user-mediated content populates social networks. Current research has focused extensively on needs, practices, and concerns surrounding the sharing of photos and videos, textual information (e.g., status updates), and documents. However, in recent years, the scope of what is “shareable” has greatly increased, comprising not only audio-visual content but also preferences and tastes (e.g., playlists, food), physiological data (e.g., workouts), trips, and even information about and access to real-world artifacts (e.g., “couchsurfing”). A recent market trend is to share personal physical possessions, initially rooms and apartments (e.g., Airbnb), but more recently rides (Uber), cars (Getaround) and household items (Snapgoods). The goal of this project is to design, conduct and subsequently analyze an online survey that attempts to elicit current practices of usage of selected sharing economy services, as well as identify motivations to participate in such services. Additionally, the student should conduct contextual interviews involving different stakeholders of such service (e.g. users, non-users, owners, suppliers) to further understand the economic role of using such sharing-economy services. This projects requires strong analytical skills and a willingness to learn about a novel and emerging research field. Experience with performing survey research and/or interviews a plus.

For more information contact: No items found

Understanding Purpose-Driven Use of Location Sharing Services

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Master UROP Status: Draft Student:

So-called “geosocial” applications allow one to share one’s current location with friends, families, or even the public. This project investigates how the particular location sharing functionalities of a geosocial app, namely continuous, proximity-based and ad-hoc sharing, affect its actual use in daily life. To this end, we drew up a Location Sharing Acceptance Model (LSAM) that comprises user acceptance factors for location sharing from previous works, and conducted its initial validation by means of a 4-week study involving 36 participants where we collected survey data and actual application use. In a second step, we now seek to validate these findings in a larger follow-up study, conducted via a crowdsourcing platform. This requires a highly motivated student that is (a) willing to learn about survey research and the statistical models involved, and (b) devises, creates, conducts, and analysis a crowdsourcing study using an online survey platform.

For more information contact: No items found

User identification in real-world settings using wearables

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Assigned January 2023 Student: Emilio Lingenthal

Wearable devices provide the means for continuously identify users by performing biometric recognition. The highly subject-dependent nature of physiological data (e.g., heart rate, electrodermal activity, skin temperature) makes it possible to identify the user by tracking those traits over time, as it is already done nowadays with the fingerprint or iris recognition. This project aims to understand to what extent user identification is possible and if different condition can hamper that capability. After performing data exploration on an existing data, provided by the supervisor, the student will develop a machine learning model to identify users from wearable data. In addition, he/she will test the efficiency of common anonymization techniques, such as noise addition and synthetic data generation.

For more information contact:

USIDisplays: Designing a Multi-Application Display Experience

Mon, 01 Jan 0001 00:00:00 +0000

Type: Bachelor Status: Completed June 2012 Student: Marcello Romanelli

The main focus of this thesis is to create a new architecture capable of supporting a wide range of applications that can be deployed on a distributed display network. Moreover, users will have a chance to influence applications displayed the screens either with their presence or by direct interaction with the screens.

In this report, I will explain the system’s architecture and how we used the latest web 2.0 technologies provided by HTML5 API in order to provide the desired functionalities. The components that have been developed are a display manager, an application repository, an initial set of applications and of course a display component.

The result of this work has been an initial set of application that can run concurrently inside a single display. An additional characteristic of these applications, is that they can be influenced both in their aspect and in their content via the interaction with the mobiles phones of the nearby users.

For more information contact:

WeatherUSI: Crowd Sourcing the Weather on Public Displays

Mon, 01 Jan 0001 00:00:00 +0000

Type: Master Status: Completed January 2016 Student: Ahmed Fouad

Public display systems are increasingly becoming part of the urban landscape. Most public displays today are simple slide-show systems that broadcast content in the form of static images. However, public displays envisioned in the near future will not only integrate content from a number of different sources, but also serve as data collection stations. Indeed, research has shown that public displays may hold a significant potential in “crowd sourcing”, when motivational design and feedback validation mechanisms are employed.

Crowd sourcing is a process where a large number of volunteer users contribute information through an online platform. A well-known example of crowd sourcing is Wikipedia. In the context of this thesis, the student should explore the design, implementation, and evaluation of a crowd sourcing public display application for weather information. This application builds upon the existing infrastructure of “Atmos”, a participatory sensing app that allows users to contribute subjective weather experiences, as well as their personal predictions, through their mobile phones. WeatherUSI should port the main principle of Atmos to a public display app and use the Atmos backend services to store user feedback, ideally merging both mobile (i.e., through a smartphone) and stationary (i.e., at a public display) Atmos reports. The main challenge of the work will be: (a) a design that will draw in passery-by, e.g., by using gamification elements; (b) an implementation that cleverly supports the multi-region public display engine in use at USI; and (c) an evaluation that verifies the feasibility of crowd sourcing weather information through public displays.

For more information contact:

Student Projects Archive |

A Classification Scheme of Lifelogging User Interfaces

A Mobile Privacy Assistant for Sharing Personal Memories

A Novel Approach Using Bilateral Data Fusion for EDA Data Classification

A tangible interface for controlling capture and sharing of personal data

A Toolchain for Visual Memory

A Visualization Tool for Smartphone-acquired Activity Traces

A Web-based Application for Visualizing Personal Memories

Activity recognition using ear-worn sensors and machine learning

Affect and Learning in the LAUREATE Dataset

Analysis of Human Memory and Physiological Signals

Attentive Public Displays: Understanding Audience Moving Patterns in front of Interactive Public Displays

Co-Located AR Based Display Sharing

Collaborative Economy Practices and Communities in Switzerland – A Case Study

Counterfactual Explanations for Unsupervised Learning

References:

Creating an Experimental Apparatus for a Digital Health Literacy Experiment

Deep learning models for human mobility modeling

References:

Design and Evaluation of a Smartphone App to Support Sharing Physical Objects (SHA21.C)

Detection of Social Engagement Interaction using Smartphones

Displays Mobile Application: A Mobile Interface for Customizing Public Displays

EDA Lateralization during Sleep

Embodied Large Language Models for Personalized Meeting Summarization

Empathetic Virtual Agents using Large Language Models

Evaluating a Human Memory Augmentation App (RECALL.C)

Evaluating Secure Personal Memory Sharing with Co-Located People

Explainable AI for Federated Models in Wearable Sensing

Literature:

Explainable AI through counterfactual explanations

References:

Exploring LAUREATE- the Longitudinal multimodAl stUdent expeRience datasEt for AffecT and mEmory research

Fed-CogLoad: Federated Cognitive Load Estimation

FunSquare: The Context Collector

FunSquare: The Fun Fact Generator

Generative Adversarial Networks (GANs) for human mobility trajectory generation

References:

Group Activity Journal – Shared Feed for Outdoor Sports

Investigating the dichotomy of sharing practices in virtual and physical realms: from theoretical overview to design considerations (SHA21.A)

Just Share It: A Decentralized Autonomous System to Support Sharing Physical Objects Using Blockchain and Smart Contracting

MoviePlus – A Platform for Sharing Digital Content For Augmented Reality Displays

Multimodal Federated Learning for Sensor Data

Literature:

NDA: Developing Networked Public Display Analytics Based on Classifying the Audience in Front of a Display and Mapping Them to Interaction Zones

Personalized Public Displays: Designing Interactive Display Applications for Active and Walk-by Content Personalization

Personalized Public Displays: Uncovering Personalization Needs

Personalized Shared Public Displays: Supporting Multi-user Walk-by Content Personalization

Personalized Sleep Quality Prediction Model Using Wearable Devices

Physiological Synchrony and its Effects on Recall in Peer Meetings

Privacy-aware human mobility modeling

References:

Privacy-aware Video-based Sensing for Affect Recognition

LiteratureL

Protect privacy in wearable devices using data anonymization

Public Reactive Displays

PulseCam: Capture your truly significant moments

Re-Live the Moment: Visualizing Run Experiences to Motivate Future Exercises

Reactive Displays: Supporting Dynamic Display Behavior Using a Rescheduling Approach

Roaming Objects – Object-encoded Digital Experiences

Self-supervised Domain Adaptation for Sensor Data

Literature:

Shadow: An Application Selection Interface for Pervasive Public Displays

Smart Group Activity Journal – Creating an Automated Activity Feed for Outdoor Sports (SHA21.D)

SmartHelmet: Using LIDAR sensing to alert skiers

Statistical Analysis for Affect and Learning in the LAUREATE Dataset

Table Tennis Paddle Detection Library for Microsoft Kinect

Tacita-Mobile: Controlling Content Presentation on Personalized Public Displays Through Mobile Phones

The Human Dash Cam: A Human Memory Prosthesis in the Era of Ubiquitous Computing

Transformers for human mobility modeling

References:

Uncertainty-aware Deep Learning in digital healthcare

Uncovering needs and requirements for data sharing activities in sports

Understanding Practices and Motivations for Sharing Physical Resources through Digital Services (SHA21.B)

Understanding Purpose-Driven Use of Location Sharing Services

User identification in real-world settings using wearables

USIDisplays: Designing a Multi-Application Display Experience

WeatherUSI: Crowd Sourcing the Weather on Public Displays

Embodied Large Language Models for Personalized MeetingSummarization