Student Projects

Deep Emotion Recognition in Text

Duration: One Semester new
Goals: Build an emotion recognizer for text using deep learning methods
Supervisor: Dr. Pearl Pu
Student Name: Open
Keywords: Emotion recognition, classification, deep learning, machine learning

This project aims at analyzing the emotions presented in some given text using deep learning approaches. This is usually cast as a multi-class (or multi-label) classification problem, where a given piece of text (for example a tweet, a review for some movie, or a piece of news) is classified into one (or several) emotion category (categories). Current deep learning methods for the task of text classification include convolutional neural networks, recurrent neural networks, or hybrid models.

During the project, the student is supposed to:

  • understand various emotion models;
  • understand word embeddings;
  • understand some of the state-of-the-art deep learning methods for text classification;
  • devise a neural network structure;
  • find some suitable dataset(s) for training and evaluation;
  • compare the deep learning method with traditional approaches.
Related Skills: Background or interest in machine learning, deep learning, and natural language processing
Suitable for: Master Student. Interested student should contact Dr. Pu along with a copy of your CV. 

Emotion Recognition in Human Dialogs

Duration: One Semester new
Goals: Build a specialized emotion recognition engine (ER) for human dialogs
Supervisor: Dr. Pearl Pu
Student Name: Open
Keywords: Emotion Recognition, sentiment analysis, machine learning

Built on our lab’s competence in emotion recognition, this project aims at building a specialized emotion recognition engine for dialogs obtained from a big TV subtitle dataset.

The responsible student will:

  • Survey the state-of-the-art methods in text-based emotion recognition 
  • Build an ER from basic lexicons to gradually more domain-specific ones
  • Semi-supervised machine learning
  • Validate the method 
Related Skills: Strong background in data mining, machine learning, and statistics. Interest in natural language processing and emotion modeling. 
Suitable for: Master Student. Interested student should contact Dr. Pu along with a copy of your CV. 

Visualizing Personal Nutrition Intake and Emotions

Duration: One Semesternew
Goals: Using tools like D3 to build an environment to let users explore, interact, and understand what their food journey has been in the past year, and how they are correlated with their emotion and mood.
Responsible TA: Onur Yuruten (onur DOT yuruten AT
Student Name: Yumeng Hou
Keywords: information visualization, machine learning, sentiment analysis, personal health

Existing apps do not do justice to the amount of attention we pay to personal food and nutrition intake. We spend three times a day to select, prepare, and consume our meals. And if we are able to share with our friends and family members, these meals can contribute to the most memorable moments of our lives. How can we make a digital diary of our food consumption and the emotions that accompanied them? how can we make these memories as delicious as the food we ate? This project aims at exploring both information visualization issues as well as personal health issues. Along the way, we want our students to have fun with the D3 tool and learn a few things about color design. 

The responsible student will:

  • Survey what has been in this field including work from our lab
  • Build the food and mood visualizaiton environment in D3
  • Make it available to the Web users
Related Skills: Human computer interaction, information visualization, interest in personal health, web programming, passion to succeed in multidisciplinary work
Suitable for: Master Student. Please contact Onur (above) or along with your CV. 

Predicting Seizures in EEG recordings

Duration: One Semester
Goals: Create and evaluate an EEG-based seizure forecasting system.
Responsible TA: Igor Kulev (igor DOT kulev AT
Student Name: Yuguang Yao (exchange student from Tsinghua University)
Keywords: time series, classification, forecasting, healthcare

Epilepsy afflicts nearly 1% of the world’s population, and is characterized by the occurrence of spontaneous seizure. Seizure forecasting systems have the potential to help patients with epilepsy lead more normal lives. In order for EEG-based seizure forecasting systems to work effectively, computational algorithms must reliably identify periods of increased probability of seizure occurrence. If these seizure-permissive brain states can be identified, devices designed to warn patients of impeding seizures would be possible. Patients could avoid potentially dangerous activities like driving or swimming, and medications could be administered only when needed to prevent impending seizures, reducing overall side effects. In 2014 and 2016 Kaggle completed two seizure prediction challenges. The first challenge primarily involved long-term electrical brain activity recordings from dogs. The second challenge focuses on seizure prediction using long-term electrical brain activity recordings from humans obtained from the world-first clinical trial of the implantable NeuroVista Seizure Advisory System. In this project you will apply and compare different methods on the EEG data obtained from both Kaggle challenges.

The responsible student will:

  • Survey time series classification techniques
  • Apply existing techniques on the EEG data
  • Analyze the performance of these techniques
Related Skills: Knowledge in Data Mining and/or Machine Learning; Programming skills
Suitable for: Master Student

Gamified Food Logging System

Intended for: Master project or thesisnew
Goals: Develop a playful mobile app motivating its users to log their food intake over a long period of time.
Supervisor: Dr. Pearl Pu
Student Name: Open
Keywords: Gamification, Prototyping, Nutrition, Well-being, User Study

Unbalanced diet is a major risk for chronic diseases such as cardiovascular disease, metabolic diseases, kidney diseases, cancer and neurodegenerative diseases. Diet is complex and rapidly changing. In addition to inter-person variability in dietary patterns there is important intra-person variability, including day-to-day and seasonable variability. Current methods to capture dietary intakes in epidemiologic studies include food-frequency questionnaires, 7-day food records and 24-hour recall. These methods are expensive to conduct, cumbersome for participants to use, and prone to reporting errors.

In this project, your goal is to re-design the experience of diet reporting. You will develop a mobile app to provide a joyful experience for users to log their food intake over a long period of time. As a first step, a survey of gamification design for such apps will be conducted, followed by the identification of successful criteria. Based on such findings, you will then apply design-thinking methods to create several solutions that satisfy both the design context and the criteria. After several iterations, the design will evolve into a highly interactive prototype before it is tested by real users. Other techniques used include stakeholder analysis, user empathy, experience mapping, and ideation of design solutions.

Related Skills: Design Thinking; Principles of Human-Computer Interaction; Mobile app development experience is a plus; Interest in Well-Being, Nutrition, or Gamification
Suitable for: Master students at their final stage of studies. Please send email to pearl pu along with a copy of your CV. 

Happy Food: Exploring Effects of Nutrition on Emotional Well-being

Duration: One Semester
Goals: Analyze relations between nutrition and emotional well-being based on the experiences reported in social media.
Responsible TA: Pearl Pu
Student Name: Othman Benchekroun
Keywords: Emotion Recognition, Twitter, Data Analysis, Statistics, Social Media Analysis, Nutrition, Well-being

Nowadays, much about human behavior can be discovered from their online traces in social media. People describe their lives, personal events, their reactions to products and global events, and also reveal their eating preferences. Research studies suggest that having a healthier lifestyle, including eating healthier food, can improve personal well-being and make a person happier. This project will analyze the relations between nutrition and emotional well-being as detected from social media. It will aim to derive more fine-grained patterns of relations between the products we eat and the emotions we experience.

The responsible student will:

  • Survey the state-of-the-art methods in text-based emotion recognition and related work on personal well-being relation with nutrition
  • Collect the tweets for studying relations between reported emotions and mentioned food types
  • Adapt (and potentially improve) existent tools for emotion recognition and food mention classification to extract required information from the collected tweets
  • Perform statistical data analysis to discover patterns of relations between emotions and specific food types
Related Skills: Programming skills; Statistics; Basic knowledge of Data Mining and/or Machine Learning; Interest in Social Media Analysis, and/or Computational Linguistics
Suitable for: Master Student

Evaluating Interfaces of Visual Analytic Tools

Title: Evaluating Interfaces of Visual Analytic Tools
Goals: Facilitate visualization techniques to explore “big data”, and provide a coherent method to evaluate information visualization tools.
Responsible TA: Onur Yuruten
Student Name: Taken
Keywords: Visual analytics, visualisation, interface evaluation

The amount of data generated in scientific studies (such as in life sciences and pervasive healthcare applications) is growing to an extent that it will be too time consuming, if not completely impractical, to apply manual analysis alone to derive significant conclusions and to verify hypotheses. To address this problem, many studies aim to deliver information visualization tools to aid scientists to explore their data. However, the information visualization field lacks a common methodology to evaluate and compare such interfaces for their usefulness and usabilities. This is an excellent opportunity to investigate existing methods, and either determine the best method, or if possible, find a way to unify them.

The responsible student will:

  • Survey information visualization techniques, and qualitative and quantitative evaluation methods
  • Develop an information visualization tool (or improve an existing interface) with algorithmic functionalities
  • Employ the evaluation techniques to validate the tool, and elaborate on the usefulness of each technique
Related Skills: Web development (Javascript, HTML, PHP, SQL); Visualization libraries for Javascript (such as D3, Rickshaw, etc.)

Design Guidelines for Crowdsourcing Emotion Annotations [Archived]

Duration: One semester
Goals: Develop guidelines for designing a crowdsourcing task for textual emotion annotation.
Responsible TA: Valentina Sintsova
Student Name: Not assigned yet
Keywords: Emotion Annotation, Crowdsourcing, Design Guidelines, Design Recommendations, Task Design

Annotation of emotions in collected text documents is an essential step for developing the reliable emotion classification system. It can serve as the ground-truth for training and/or testing emotion recognition models. Current crowdsourcing approaches involve annotation of text documents for emotions and emotion indicators. Yet, subjectivity in understanding of the emotion concept and lack of clear instructions may result in high inconsistency of collected data. There is a need to elaborate the appropriate design guidelines in order to ensure the collection high-quality annotations.

The responsible student will:

  • Survey the existent approaches to crowdsourcing sentiment and emotion annotation of text, as well as the method of their qualitative and quantitative evaluation
  • Analyze data collected from the previous crowdsourcing experiments on emotion annotation
  • Based on that analysis, derive design guidelines for crowdsourcing tasks on textual emotion annotation
Related Skills: Strong analytical skills; Statistical analysis; Interest in Crowdsourcing, Qualitative Research, and Human-Computer Interaction Design
Suitable for: Bachelor or Master Student

Modeling Effects Of Modifiers on Emotional Statements

Duration: Two semesters (February – June 2015, February – June 2016)
Goals: Investigate the effects of different linguistic modifiers on emotional expressions, and suggest how to model those effects within emotion recognition system.
Responsible TA: Valentina Sintsova and Pearl Pu
Student Names: Margarita Bolívar Jiménez (2015) and Nataniel Hofer (2016)
Keywords: Emotion Recognition, Text Classification, Social Media Analysis, Modifiers Effects

People express their emotions and feelings in multiple subtle ways. Even when they use explicit emotional terms, such as “happy” or “sad”, the emotional meaning of statements can change because of the variety of linguistic modifiers. Those include negation, intensity shifting, modality, and others. So far the researchers have investigated the effects of those modifiers on polarity of terms (positive vs. negative). However, their effects on more fine-grained emotion categories remain understudied. The first part of this project investigates the effects of different modifiers on emotional meaning of the terms via data analysis techniques. The second part studies to what extent the better modeling of modifiers improves emotion classification quality.

Advancing Human Computation for Emotion Detection

Duration: One semester (February – June 2015)
Goals: Design a human computation task that would allow collecting affective knowledge of better quality, and develop the evaluation techniques to quantify the impact of the task design.
Responsible TA: Valentina Sintsova and Pearl Pu
Student Name: Séphora Madjiheurem
Keywords: Human Computation, Emotion Recognition, Crowdsourcing, Amazon Mechanical Turk, Experiment Design, Quantitative Evaluation Techniques

Social media are filled with emotional content, which many researchers and companies seek to analyze. However, automatic methods for emotion recognition are far from the level of human ability to understand emotion language. Human computation techniques are seen as a way to help machines learn how to detect emotions. Online labor platforms such as Amazon Mechanical Turk allow to use individual humans to obtain answers to such judgment tasks as emotion detection in text. One strategy to obtain quality answers is to combine answers from different workers. Yet, in order to make use of the wisdom of the crowd, human answers must be comparable. This can be achieved by providing clear instructions and designing tutorials for the task. Moreover, if the human computation task is subject to systematic bias, using multiple workers is not enough to obtain quality answers. In this project, two experiments were conducted in an online labor platform. The first experiment aimed to evaluate the impact of tutorials on the quality of the answers provided by workers and on their engagement in the task. The second experiment was focused on comparing the workers’ output quality and engagement when using different incentives for motivating workers. The results show that tutorials with limited instruction do not necessarily lead to poorer performance. The results also demonstrate better quality work from workers under certain treatment conditions for motivation depending on the difficulty of the task.