Pearl Pu will give a tutorial on “Design and Evaluation of Recommender Systems – Bridging the Gap between Algorithms and User Experience” with Paolo Cremonesi and Franca Garzotto
Abstract: Recommender Systems (RSs) help users search large amounts of contents by allowing them to identify the items that are likely to be more attractive or useful. RSs play an important role in many domains (e.g., e-commerce, e-tourism, social networks, entertainment), as they can potentially augment the users’ trust towards an application and orient their decisions or actions towards specific directions. The goal of this tutorial is to give participants a solid background of how to design and evaluate RSs, with a particular focus on user experience aspects, and to provide pragmatic guidelines to perform these activities more effectively. The tutorial is structured into three parts, and the teaching style will be “by-examples”. In the first part, after a general overview of recommender systems, we will analyze their design issues and will analyze different definitions of design quality. In the second part we will analyze “off-line” (system-centric) evaluation techniques. We will describe different quality metrics and how to measure them. We will highlight the benefits and pitfalls of off-line evaluation. The third part will explore “on-line” (user–centric) quality evaluation methodologies, focusing on the users’ perceived values and how these affect their behavioral intentions. We will describe in some details a unifying user-centric evaluation framework, called ResQue (Recommender systems’ Quality of user experience), and psychometric methods, including structural equation modeling (SEM).