Project Approach
We will learn the mappings using annotated/labeled training data. A new dataset called Wikiart Emotions (Saif Mohammad, 2018) contains paintings labeled by humans with associated emotions. We can also use general image datasets that do not focus on art. The popular image database ImageNet (Jia Deng, 2009) contains 15 million images with 20k different objects. We will explore the possibility of using this data directly or if needed apply neural style transfer (Yongcheng Jing, 2019) to create annotated paintings.
We will adapt the standard GAN architecture to incorporate information about emotions and objects. Conditional GANs have been successful in the past for similar tasks (Bo Dai, 2017). We will develop a similar approach to solve our task.
We are planning to write a research paper at the end of the project for submission to a conference. If you are interested in doing research in deep learning at the intersection of
- Computer vision,
- Text mining, and
- Digital cultural heritage,
then this is the right project for you. It would be helpful if you have some prior knowledge about deep learning architectures and deep learning development frameworks such as PyTorch.
Contact
This project will be jointly supervised by the Information Systems chair and the AI and Intellgient Systems chair. If you have any questions, please do not hesitate to contact Dr. Ralf Krestel, Prof. Dr. Gerard de Melo or Alejandro Sierra-Múnera
References
Jia Deng, e. a. (2009). Imagenet: A large-scale hierarchical image database. IEEE conference on computer vision and pattern recognition (CVPR).
Saif Mohammad, S. K. (2018). Wikiart emotions: An annotated dataset of emotions evoked by art. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC).
Yongcheng Jing, e. a. (2019). Neural style transfer: A review. IEEE transactions on visualization and computer graphics.
