Discovering User Motivations and Experience of Open-Domain Chatbots Through App Reviews (PAST)

May 12, 2020
Duration: One Semester
Goals: Identify the main drivers of people’s engagement with open-domain chatbots and explore various aspects of current interaction experience with them based on user reviews from Google Play market and Apple app store.
Assistant: Ekaterina Svikhnushina (ekaterina DOT svikhnushina AT epfl DOT ch)
Student Name: Alexandru Placinta (Taken)
Keywords: Qualitative research; social chatbots; motivations; remote experience evaluation

Due to recent advances in neural network-based language generation the area of open-domain chatbot development has become increasingly active. To ensure compelling user experience the design of conversational agents should meet eventual user goals and expectations. A number of studies in HCI community explored user motivations, needs, and perceptions of chatbots through a variety of methods, including surveying (Brandtzaeg, 2017), interviewing (Jain, 2018; Clark, 2019), diary studies (Muresan, 2019), and review analysis (Purington, 2017). However, previous works were either focusing on chatbots in general (also including task-oriented ones) or conducting a case study of a single specific agent. The aim of this project is to identify core user motivations and significant interaction experience aspects with open-domain chatbots based on elaborate analysis of reviews and ratings that users provided for a range of trending conversational apps on popular software distribution platforms.

The student is expected to:

  • Survey existing popular open-domain chatbots
  • Scrape user reviews and ratings of surveyed chatbots from Google Play market, Apple app store, or other platforms
  • Analyse the reviews using both ML-based automatic techniques (e.g., filtering methods, sentiment analysis, topic analysis, etc.) and qualitative and/or quantitative methods (e.g., coding, affinity diagramming, thematic analysis; correlation analysis, statistical analysis, etc.) to elicit main user motivations, experiences, and concerns of open-domain chatbots
Related Skills: Knowledge in Data Mining and/or Machine Learning; strong analytical skills; familiarity with natural language processing; statistical analysis basis
Suitable for: Master student. Interested student should contact Ekaterina Svikhnushina (ekaterina DOT svikhnushina AT epfl DOT ch) and Pearl Pu (pearl DOT pu AT epfl DOT ch) along with a copy of your CV.