Discovering User Motivations and Experience of Open-Domain Chatbots Through App Reviews (PAST)

Due to recent advances in neural network-based language generation the area of open-domain chatbot development has become increasingly active. To ensure compelling user experience the design of conversational agents should meet eventual user goals and expectations. A number of studies in HCI community explored user motivations, needs, and perceptions of chatbots through a variety of methods, including surveying (Brandtzaeg, 2017), interviewing (Jain, 2018; Clark, 2019), diary studies (Muresan, 2019), and review analysis (Purington, 2017). However, previous works were either focusing on chatbots in general (also including task-oriented ones) or conducting a case study of a single specific agent. The aim of this project is to identify core user motivations and significant interaction experience aspects with open-domain chatbots based on elaborate analysis of reviews and ratings that users provided for a range of trending conversational apps on popular software distribution platforms.

The student is expected to:

• Survey existing popular open-domain chatbots
• Scrape user reviews and ratings of surveyed chatbots from Google Play market, Apple app store, or other platforms
• Analyse the reviews using both ML-based automatic techniques (e.g., filtering methods, sentiment analysis, topic analysis, etc.) and qualitative and/or quantitative methods (e.g., coding, affinity diagramming, thematic analysis; correlation analysis, statistical analysis, etc.) to elicit main user motivations, experiences, and concerns of open-domain chatbots