This note presents a system that learns expressive and idiosyncratic gesture variations for gesture-based interaction. The system is used as an interaction technique in a music conducting scenario where gesture variations drive music articulation. A simple model based on Gaussian Mixture Modeling is used to allow the user to configure the system by providing variation examples. The system performance and the influence of user musical expertise is evaluated in a user study, which shows that the model is able to learn idiosyncratic variations that allow users to control articulation, with better performance for users with musical expertise.
JF - CHI - Human Factors in Computing Systems PB - ACM Press CY - San Jose, CA ER - TY - CONF T1 - A Personality-based Adaptive System for Visualizing Classical Music Performances T2 - Proceedings of the 7th ACM Multimedia Systems Conference (MMSys) Y1 - 2016 A1 - Markus Schedl A1 - Mark Melenhorst A1 - Cynthia C.S. Liem A1 - Agustín Martorell A1 - Óscar Mayor A1 - Marko Tkalčič JF - Proceedings of the 7th ACM Multimedia Systems Conference (MMSys) CY - Klagenfurt, Austria ER - TY - CONF T1 - Personalized Retrieval and Browsing of Classical Music and Supporting Multimedia Material T2 - Proceedings of the ACM International Conference on Multimedia Retrieval (ICMR) Y1 - 2016 A1 - Marko Tkalčič A1 - Markus Schedl A1 - Cynthia C.S. Liem A1 - Mark Melenhorst JF - Proceedings of the ACM International Conference on Multimedia Retrieval (ICMR) CY - New York, USA ER - TY - CONF T1 - Using Instagram Picture Features to Predict Users' Personality T2 - Proceedings of the 22nd International Conference on MultiMedia Modeling (MMM 2016) Y1 - 2016 A1 - Ferwerda, Bruce A1 - Schedl, Markus A1 - Tkalčič, Marko JF - Proceedings of the 22nd International Conference on MultiMedia Modeling (MMM 2016) CY - Miami, USA ER - TY - CONF T1 - Improving Music Recommendations with a Weighted Factorization of the Tagging Activity T2 - Proceedings of the 16th International Society for Music Information Retrieval Conference (ISMIR) Y1 - 2015 A1 - Andreu Vall A1 - Marcin Skowron A1 - Peter Knees A1 - Markus Schedl JF - Proceedings of the 16th International Society for Music Information Retrieval Conference (ISMIR) CY - Malaga, Spain ER - TY - CONF T1 - On the Influence of User Characteristics on Music Recommendation T2 - Proceedings of the 37th European Conference on Information Retrieval (ECIR 2015) Y1 - 2015 A1 - Markus Schedl A1 - David Hauger A1 - Katayoun Farrahi A1 - Marko Tkalčič JF - Proceedings of the 37th European Conference on Information Retrieval (ECIR 2015) CY - Vienna, Austria ER - TY - CONF T1 - Iron Maiden while jogging, Debussy for dinner? - An analysis of music listening behavior in context T2 - Proceedings of the 21st International Conference on MultiMedia Modeling (MMM 2015) Y1 - 2015 A1 - Michael Gillhofer A1 - Markus Schedl JF - Proceedings of the 21st International Conference on MultiMedia Modeling (MMM 2015) CY - Sydney, Australia ER - TY - CONF T1 - I-Vectors for Timbre-Based Music Similarity and Music Artist Classification T2 - Proceedings of the 16th International Society for Music Information Retrieval Conference (ISMIR) Y1 - 2015 A1 - Hamid Eghbal-zadeh A1 - Bernhard Lehner A1 - Markus Schedl A1 - Gerhard Widmer JF - Proceedings of the 16th International Society for Music Information Retrieval Conference (ISMIR) CY - Malaga, Spain ER - TY - CONF T1 - Listener-aware Music Recommendation from Sensor and Social Media Data T2 - Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2015) Y1 - 2015 A1 - Markus Schedl JF - Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2015) CY - Porto, Portugal ER - TY - CONF T1 - Listener-aware Music Search and Recommendation T2 - Proceedings of the 6th International Workshop on Mining Ubiquitous and Social Environments (MUSE 2015) Y1 - 2015 A1 - Markus Schedl JF - Proceedings of the 6th International Workshop on Mining Ubiquitous and Social Environments (MUSE 2015) CY - Porto, Portugal ER - TY - CONF T1 - Music Retrieval and Recommendation – A Tutorial Overview T2 - Proceedings of the 38th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) Y1 - 2015 A1 - Peter Knees A1 - Markus Schedl JF - Proceedings of the 38th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) CY - Santiago, Chile ER - TY - JOUR T1 - Personality Correlates for Digital Concert Program Notes JF - UMAP 2015, Springer LNCS 9146 Y1 - 2015 A1 - Tkalčič, Marko A1 - Ferwerda, Bruce A1 - Hauger, David A1 - Schedl, Markus KW - classical music KW - digital program notes KW - personality ER - TY - CONF T1 - Personality & Emotional States: Understanding Users’ Music Listening Needs T2 - Extended Proceedings of the 22nd International Conference on User Modeling, Adaptation and Personalization (UMAP 2015) Y1 - 2015 A1 - Ferwerda, Bruce A1 - Schedl, Markus A1 - Tkalčič, Marko JF - Extended Proceedings of the 22nd International Conference on User Modeling, Adaptation and Personalization (UMAP 2015) CY - Dublin, Ireland ER - TY - CONF T1 - Personality Traits Predict Music Taxonomy Preferences T2 - Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA '15 Y1 - 2015 A1 - Ferwerda, Bruce A1 - Yang, Emily A1 - Schedl, Markus A1 - Tkalčič, Marko JF - Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA '15 SN - 9781450331463 UR - http://dx.doi.org/10.1145/2702613.2732754 http://dl.acm.org/citation.cfm?doid=2702613.2732754 ER - TY - CONF T1 - PHENICX: Innovating the Classical Music Experience T2 - Proceedings of the IEEE International Conference on Multimedia and Expo (ICME 2015) Y1 - 2015 A1 - Cynthia C. S. Liem A1 - Emilia Gómez A1 - Markus Schedl JF - Proceedings of the IEEE International Conference on Multimedia and Expo (ICME 2015) CY - Torino, Italy ER - TY - CONF T1 - Tailoring Music Recommendations to Users by Considering Diversity, Mainstreaminess, and Novelty T2 - Proceedings of the 38th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) Y1 - 2015 A1 - Markus Schedl A1 - David Hauger JF - Proceedings of the 38th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) CY - Santiago, Chile ER - TY - CONF T1 - Timbral Modeling for Music Artist Recognition Using I-vectors T2 - Proceedings of the 23rd European Signal Processing Conference (EUSIPCO 2015) Y1 - 2015 A1 - Hamid Eghbal-zadeh A1 - Markus Schedl A1 - Gerhard Widmer JF - Proceedings of the 23rd European Signal Processing Conference (EUSIPCO 2015) CY - Nice, France ER - TY - CONF T1 - Towards Personalizing Classical Music Recommendations T2 - SoMeRA 2015: Proceedings of the Second International Workshop on Social Media Retrieval and Analysis Y1 - 2015 A1 - Markus Schedl JF - SoMeRA 2015: Proceedings of the Second International Workshop on Social Media Retrieval and Analysis CY - Atlantic City, USA ER - TY - CONF T1 - Beat Tracking from Conducting Gestural Data: A Multi-Subject Study T2 - Proceedings of the 2014 International Workshop on Movement and Computing Y1 - 2014 A1 - Sarasua, Alvaro A1 - Guaus, Enric KW - beat tracking KW - classical music KW - conducting KW - expressive performance KW - motion capture AB -The musical conductor metaphor has been broadly used in the design of musical interfaces where users control the expressive aspects of the performance imitating the movements of conductors. Most of the times, there are predefined rules for the interaction to which users have to adapt. Other works have focused on studying the relation between conductors' gestures and the resulting performance of the orchestra. Here, we study how different subjects move when asked to conduct on top of classical music excerpts, with a focus on the influence of the beat of the performance. Twenty-five subjects were asked to conduct on top of three classical music fragments and recorded with a commercial depth-sense camera. We evaluated predicted beats using ground truth annotations from score-performance alignment by an expert musicologist and a modified F-measure that is able to account for different tendencies on beat anticipation across subjects. The results show that these tendencies can be used for possible improvements in the design of conducting musical interfaces in terms of user adaptation.
JF - Proceedings of the 2014 International Workshop on Movement and Computing PB - ACM CY - Paris, France SN - 978-1-4503-2814-2 UR - http://doi.acm.org/10.1145/2617995.2618016 ER - TY - CONF T1 - Dynamics in Music Conducting: A Computational Comparative Study Among Subjects T2 - Proceedings of the International Conference on New Interfaces for Musical Expression Y1 - 2014 A1 - Álvaro Sarasúa A1 - Enric Guaus AB -Many musical interfaces have used the musical conductor metaphor, allowing users to control the expressive aspects of a performance by imitating the gestures of conductors. In most of them, the rules to control these expressive aspects are predefined and users have to adapt to them. Other works have studied conductors' gestures in relation to the performance of the orchestra. The goal of this study is to analyze, following the path initiated by this latter kind of works, how simple motion capture descriptors can explain the relationship between the loudness of a given performance and the way in which different subjects move when asked to impersonate the conductor of that performance. Twenty-five subjects were asked to impersonate the conductor of three classical music fragments while listening to them. The results of different linear regression models with motion capture descriptors as explanatory variables show that, by studying how descriptors correlate to loudness differently among subjects, different tendencies can be found and exploited to design models that better adjust to their expectations.
JF - Proceedings of the International Conference on New Interfaces for Musical Expression PB - Goldsmiths, University of London CY - London, United Kingdom UR - http://nime2014.org/proceedings/papers/464_paper.pdf ER - TY - CONF T1 - Genre-based Analysis of Social Media Data on Music Listening Behavior T2 - Proceedings of the 1st ACM International Workshop on Internet-Scale Multimedia Management (ISMM 2014) Y1 - 2014 A1 - Markus Schedl A1 - Marko Tkalčič JF - Proceedings of the 1st ACM International Workshop on Internet-Scale Multimedia Management (ISMM 2014) CY - Orlando, FL, USA ER - TY - CONF T1 - Impact of Listening Behavior on Music Recommendation T2 - Proceedings of the 15th International Society for Music Information Retrieval Conference (ISMIR 2014) Y1 - 2014 A1 - Katayoun Farrahi A1 - Markus Schedl A1 - Andreu Vall A1 - David Hauger A1 - Marko Tkalčič JF - Proceedings of the 15th International Society for Music Information Retrieval Conference (ISMIR 2014) CY - Taipei, Taiwan ER - TY - Generic T1 - Improved musical onset detection with convolutional neural networks T2 - Proceedings of the 39th International Conference on Acoustics, Speech and Signal Processing (ICASSP 2014) Y1 - 2014 A1 - Jan Schlüter A1 - Sebastian Böck JF - Proceedings of the 39th International Conference on Acoustics, Speech and Signal Processing (ICASSP 2014) ER - TY - CONF T1 - Location-Aware Music Artist Recommendation T2 - Proceedings of the 20th International Conference on MultiMedia Modeling (MMM 2014) Y1 - 2014 A1 - Markus Schedl A1 - Dominik Schnitzer JF - Proceedings of the 20th International Conference on MultiMedia Modeling (MMM 2014) CY - Dublin, Ireland ER - TY - CONF T1 - Mobile Music Genius: Reggae at the Beach, Metal on a Friday Night? T2 - Proceedings of the 2014 ACM International Conference on Multimedia Retrieval (ICMR) Y1 - 2014 A1 - Markus Schedl A1 - Georg Breitschopf A1 - Bogdan Ionescu JF - Proceedings of the 2014 ACM International Conference on Multimedia Retrieval (ICMR) CY - Glasgow, UK ER - TY - JOUR T1 - Music Information Retrieval: Recent Developments and Applications JF - {Foundations and Trends in Information Retrieval} Y1 - 2014 A1 - Markus Schedl A1 - Emilia Gómez A1 - Julián Urbano VL - 8 ER - TY - CONF T1 - Social Media and Classical Music? – A first analysis within the PHENICX project: “Performances as Highly Enriched aNd Interactive Concert eXperiences” T2 - SoMeRA'14: Proceedings of the First International Workshop on Social Media Retrieval and Analysis Y1 - 2014 A1 - Markus Schedl JF - SoMeRA'14: Proceedings of the First International Workshop on Social Media Retrieval and Analysis CY - Gold Coast, Australia ER - TY - CONF T1 - SoMeRA 2014: Social Media Retrieval and Analysis Workshop T2 - Proceedings of the 37th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) Y1 - 2014 A1 - Markus Schedl A1 - Peter Knees A1 - Jialie Shen JF - Proceedings of the 37th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) CY - Gold Coast, Australia ER - TY - CONF T1 - Tempo- and Transposition-invariant Identification of Piece and Score Position T2 - Proceedings of the International Society for Music Information Retrieval Conference (ISMIR) Y1 - 2014 A1 - Andreas Arzt A1 - Widmer, G. A1 - Sonnleitner, R. JF - Proceedings of the International Society for Music Information Retrieval Conference (ISMIR) ER - TY - CONF T1 - To Post or Not to Post: The Effects of Persuasive Cues and Group Targeting Mechanisms on Posting Behavior T2 - Proceedings of the 6th ASE International Conference on Social Computing (SocialCom 2014) Y1 - 2014 A1 - Bruce Ferwerda A1 - Markus Schedl A1 - Marko Tkalčič JF - Proceedings of the 6th ASE International Conference on Social Computing (SocialCom 2014) CY - Stanford, USA ER - TY - CONF T1 - User Geospatial Context for Music Recommendation in Microblogs T2 - Proceedings of the 37th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) Y1 - 2014 A1 - Markus Schedl A1 - Andreu Vall A1 - Katayoun Farrahi JF - Proceedings of the 37th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) CY - Gold Coast, Australia ER - TY - CONF T1 - Using Social Media Mining for Estimating Theory of Planned Behaviour Parameters T2 - Proceedings of the 2nd Workshop on Emotions and Personality in Personalized Services (EMPIRE 2014) Y1 - 2014 A1 - Bruce Ferwerda A1 - Markus Schedl JF - Proceedings of the 2nd Workshop on Emotions and Personality in Personalized Services (EMPIRE 2014) CY - Aalborg, Denmark ER - TY - CONF T1 - Using Social Media Mining for Estimating Theory of Planned Behaviour Parameters T2 - Proceedings of the 2nd Workshop on Emotions and Personality in Personalized Services (EMPIRE 2014) Y1 - 2014 A1 - Marko Tkalčič A1 - Bruce Ferwerda A1 - Markus Schedl A1 - Cynthia C. S. Liem A1 - Mark S. Melenhorst A1 - Ante Odić A1 - Andrej Košir JF - Proceedings of the 2nd Workshop on Emotions and Personality in Personalized Services (EMPIRE 2014) CY - Aalborg, Denmark ER - TY - CONF T1 - Ameliorating Music Recommendation: Integrating Music Content, Music Context, and User Context for Improved Music Retrieval and Recommendation T2 - Proceedings of the 11th International Conference on Advances in Mobile Computing & Multimedia (MoMM 2013) Y1 - 2013 A1 - Markus Schedl JF - Proceedings of the 11th International Conference on Advances in Mobile Computing & Multimedia (MoMM 2013) CY - Vienna, Austria ER - TY - CONF T1 - Context-Aware Gesture Recognition in Classical Music Conducting T2 - ACM Multimedia Y1 - 2013 A1 - Sarasua, Alvaro AB -Body movement has received increasing attention in music technology research during the last years. Some new mu- sical interfaces make use of gestures to control music in a meaningful and intuitive way. A typical approach is to use the orchestra conducting paradigm, in which the computer that generates the music would be a virtual orchestra con- ducted by the user. However, although conductors’ gestures are complex and their meaning can vary depending on the musical context, this context-dependency is still to explore. We propose a method to study context-dependency of body and facial gestures of conductors in orchestral classical mu- sic based on temporal clustering of gestures into actions, followed by an analysis of the evolution of audio features after action occurrences. For this, multi-modal data (audio, video, motion capture) will be recorded in real live concerts and rehearsals situations using unobtrusive techniques.
JF - ACM Multimedia CY - Barcelona ER - TY - CONF T1 - Enhanced peak picking for onset detection with recurrent neural networks T2 - Proceedings of the 6th International Workshop on Machine Learning and Music Y1 - 2013 A1 - Sebastian Böck A1 - Schlüter, Jan A1 - Widmer, Gerhard KW - onset detection KW - peak-picking AB -
We present a new neural network based peak-picking algorithm for common onset detection functions. Compared to existing hand-crafted methods it yields a better performance and leads to a much lower number of false negative detections. The performance is evaluated on basis of a huge dataset with over 25k annotated onsets and shows a significant improvement over existing methods in cases of signals with previously unknown levels.
JF - Proceedings of the 6th International Workshop on Machine Learning and Music CY - Prague, Czech Republic ER - TY - CONF T1 - Hybrid Retrieval Approaches to Geospatial Music Recommendation T2 - Proceedings of the 35th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) Y1 - 2013 A1 - Markus Schedl A1 - Dominik Schnitzer JF - Proceedings of the 35th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) CY - Dublin, Ireland ER - TY - CONF T1 - Innovating the Classical Music Experience in the PHENICX Project: Use Cases and Initial User Feedback T2 - 1st International Workshop on Interactive Content Consumption (WSICC) at EuroITV 2013 Y1 - 2013 A1 - Cynthia C. S. Liem A1 - Ron van der Sterren A1 - Marcel van Tilburg A1 - Álvaro Sarasúa A1 - Juan J. Bosch A1 - Jordi Janer A1 - Mark S. Melenhorst A1 - Emilia Gómez A1 - Alan Hanjalic KW - interactivity KW - multimedia information systems KW - multimodality KW - music information retrieval KW - performing arts KW - social networks KW - user studies AB -The FP7 PHENICX project focuses on creating a new digital classical concert experience, improving the accessibility of classical music concert performances by enhancing and enriching them in novel digital ways, In this paper, we present the project’s foreseen use cases. Subsequently, we summarize initial use case feedback from two different user groups. Despite the early stage of the project, the feedback already gives important insight into real-world considerations to make for interactive music content consumption solutions.
JF - 1st International Workshop on Interactive Content Consumption (WSICC) at EuroITV 2013 CY - Como, Italy ER - TY - CONF T1 - Location-aware Music Recommendation Using Auto-Tagging and Hybrid Matching T2 - Proceedings of the 7th ACM Conference on Recommender Systems (RecSys 2013) Y1 - 2013 A1 - Marius Kaminskas A1 - Francesco Ricci A1 - Markus Schedl JF - Proceedings of the 7th ACM Conference on Recommender Systems (RecSys 2013) CY - Hong Kong, China ER - TY - CONF T1 - The Million Musical Tweets Dataset: What Can We Learn From Microblogs T2 - Proceedings of the 14th International Society for Music Information Retrieval Conference (ISMIR 2013) Y1 - 2013 A1 - David Hauger A1 - Markus Schedl A1 - Andrej Košir A1 - Marko Tkalčič JF - Proceedings of the 14th International Society for Music Information Retrieval Conference (ISMIR 2013) CY - Curitiba, Brazil ER - TY - CONF T1 - Multimedia Information Retrieval: Music and Audio T2 - Proceedings of the 21st ACM International Conference on Multimedia Y1 - 2013 A1 - Markus Schedl A1 - Emilia Gómez A1 - Masataka Goto JF - Proceedings of the 21st ACM International Conference on Multimedia CY - Barcelona, Spain ER - TY - CONF T1 - Musical Onset Detection with Convolutional Neural Networks T2 - Proceedings of the 6th International Workshop on Machine Learning and Music Y1 - 2013 A1 - Schlüter, Jan A1 - Sebastian Böck KW - convolutional neural networks KW - onset detection JF - Proceedings of the 6th International Workshop on Machine Learning and Music CY - Prague, Czech Republic ER - TY - JOUR T1 - The Neglected User in Music Information Retrieval Research JF - Journal of Intelligent Information Systems Y1 - 2013 A1 - Markus Schedl A1 - Arthur Flexer A1 - Julián Urbano VL - 41 ER - TY - CONF T1 - Personalized Music Recommendation in a Mobile Environment T2 - Proceedings of the 11th International Conference on Advances in Mobile Computing & Multimedia (MoMM 2013) Y1 - 2013 A1 - Claus Schabetsberger A1 - Markus Schedl JF - Proceedings of the 11th International Conference on Advances in Mobile Computing & Multimedia (MoMM 2013) CY - Vienna, Austria ER - TY - CONF T1 - PHENICX: Performances as Highly Enriched aNd Interactive Concert Experiences T2 - SMAC Stockholm Music Acoustics Conference 2013 and SMC Sound and Music Computing Conference 2013 Y1 - 2013 A1 - Gómez, E. A1 - Grachten, M. A1 - Hanjalic, A. A1 - Janer, J. A1 - Jordà, S. A1 - Julià, C. F. A1 - Cynthia C. S. Liem A1 - Martorell, A. A1 - Schedl, M. A1 - Widmer, G. AB -Modern digital multimedia and internet technology have radically changed the ways people find entertainment and discover new interests online, seemingly without any phys- ical or social barriers. Such new access paradigms are in sharp contrast with the traditional means of entertainment. An illustrative example of this is live music concert perfor- mances that are largely being attended by dedicated audi- ences only.
This papers introduces the PHENICX project, which aims at enriching traditional concert experiences by using state- of-the-art multimedia and internet technologies. The project focuses on classical music and its main goal is twofold: (a) to make live concerts appealing to potential new au- dience and (b) to maximize the quality of concert experi- ence for everyone. Concerts will then become multimodal, multi-perspective and multilayer digital artifacts that can be easily explored, customized, personalized, (re)enjoyed and shared among the users. The paper presents the main scientific objectives on the project, provides a state of the art review on related research and presents the main chal- lenges to be addressed.