Looking Beyond Sound: Unsupervised Analysis of Musician Videos

TitleLooking Beyond Sound: Unsupervised Analysis of Musician Videos
Publication TypeConference Proceedings
Year of Conference2013
AuthorsLiem, CCS, Bazzica, A, Hanjalic, A
Conference Name14th International Workshop on Image and Audio Analysis for Multimedia Interactive services (WIA2MIS)
Date Published07/2013
Conference Location Paris, France
In this work, we focus on visual information conveyed by performing musicians. While musicians are playing, their movement relates to their musical performance. As such, analysis of this information can support structural characterization and timeline indexing of a recorded performance, especially in cases when such analyses are not trivially computed from the musical audio. We propose an unsupervised visual analysis method, in which visual novelty is inferred from motion orientation histograms of regions of interest. Considering our method in a case study on audiovisually recorded jam sessions, we show that our analysis of the visual channel yields promising and meaningful performance-related information, including information complementary to the audio channel.