Exploring the properties of data applicable for training socialbot evaluation model

April 12, 2019
Duration: One Semester
Goals: Identify the properties of training data that yield the strongest correlation of automatic socialbot evaluation model with human.
Assistant: Ekaterina Svikhnushina (ekaterina DOT svikhnushina AT epfl DOT ch)
Student Name: Open

This project aims at investigating what kind of data should be used for training a socialbot evaluation model, so that its predicted scores for bot’s responses meet the human expectations.

The student is expected to:

  • train the socialbot evaluation model with different datasets (Twitter, OpenSubtitles, Movie Dialogs)
  • frame and conduct a human experiment to collect human scores for a test set of responses
  • compare the correlations of scores predicted by different evaluation models with the human
  • retrieve the outstanding properties of the dataset that resulted in the highest correlation of automatic score with the human
Related Skills: Background or interest in data mining, machine learning, natural language processing
Suitable for: Master student. Interested student should contact Ekaterina Svikhnushina (ekaterina DOT svikhnushina AT epfl DOT ch) and Pearl Pu (pearl DOT pu AT epfl DOT ch) along with a copy of your CV.