Optimization of large-scale subjective preference evaluation of synthetic media and its application to corpus construction of preference data

Period: April 2024 – March 2029
Grant category: Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research for Young Researchers
Issue number: 24K20792
URL: https://kaken.nii.ac.jp/ja/grant/KAKENHI-PROJECT-24K20792/

Subjective evaluation is essential for evaluating the quality of generated media. In recent years, the remarkable improvement in the generation quality of generative models has made it increasingly difficult to evaluate generated media. As such, preference evaluation, which selects the better of two samples, has garnered attention as a highly accurate method for subjective evaluation. However, due to the large number of combinations of evaluation pairs, large-scale preference evaluation, such as evaluating the entire corpus, has not been possible until now. In this research, we perform a large-scale subjective preference evaluation by optimizing the combinations of evaluation pairs for preference evaluation through online learning. This aims to contribute to the sound development of generative technology as a whole.