This note presents a system that learns expressive and idiosyncratic gesture variations for gesture-based interaction. The system is used as an interaction technique in a music conducting scenario where gesture variations drive music articulation. A simple model based on Gaussian Mixture Modeling is used to allow the user to configure the system by providing variation examples. The system performance and the influence of user musical expertise is evaluated in a user study, which shows that the model is able to learn idiosyncratic variations that allow users to control articulation, with better performance for users with musical expertise.

The progressive appearance of affordable tabletop technology and devices urges human-computer interaction researchers to provide the necessary methods to make this kind of devices the most useful to their users. Studies show that tabletops have distinctive characteristics that can be specially useful to solve some types of problems, but this potential is arguably not yet translated into real-world applications. We theorize that the important components that can transform those systems into useful tools are application frameworks that take into account the devices affordances, a third party application ecosystem, and multi-application systems supporting concurrent multitasking. In this dissertation we approach these key components: First, we explore the distinctive affordances of tabletops, with two cases: TurTan, a tangible programming language in the education context, and SongExplorer, a music collection browser for large databases. Next, in order to address the difficulty of building such applications in a way that they can exploit these affordances, we focus on software frameworks to support the tabletop application making process, with two different approaches: ofxTableGestures, targeting programmers, and MTCF, designed for music and sound artists. Finally, recognizing that making useful applications is just one part of the problem, we focus on a fundamental issue of multi-application tabletop systems: the difficulty to support multi-user concurrent multitasking with third-party applications. After analyzing the possible approaches, we present GestureAgents, a content-based distributed application-centric disambiguation mechanism and its implementation, which solves this problem in a generic fashion, being also useful to other shareable interfaces, including uncoupled ones.

In the digital domain, music is usually studied from a positivist viewpoint, focusing on general ’objective’ music descriptors. In this work, we strive to put music in a more social and cultural context, looking into ways to unify data analysis methods with thoughts from the humanities on musical meaning and significance. More specifically, we investigate whether information in collaborative web resources on movie plot narratives and folksonomic song tags is capable of revealing common associations between these two. Reported initial findings suggest this is indeed the case, which opens opportunities for further work in this area, cross-disciplinary collaborations, and novel contextually oriented music information retrieval application scenarios.

We present SuperFlux - a new onset detection algorithm with vibrato suppression. It is an enhanced version of the universal spectral flux onset detection algorithm, and reduces the number of false positive detections considerably by tracking spectral trajectories with a maximum filter. Especially for music with heavy use of vibrato (e.g., sung operas or string performances), the number of false positive detections can be reduced by up to 60% without missing any additional events. Algorithm performance was evaluated and compared to state-of-the-art methods on the basis of three different datasets comprising mixed audio material (25,927 onsets), violin recordings (7,677 onsets) and operatic solo voice recordings (1,448 onsets). Due to its causal nature, the algorithm is applicable in both offline and online real-time scenarios.

This work explores the multidimensional, ambiguous and temporal characteristics of tonality from a holistic perspective. The approach relies on interfacing pitch-spaces with time vs. time-scale descriptions. In this combined representation, the spatial and temporal hierarchies of tonality are evidenced simultaneously and in relation to each other. A visual exploration method is proposed for the analysis of tonal context in music works, using a simple model of tonal induction. A geometrical colouring solution, based on the topology of the pitch-space, approaches the perceptual correlation between the tonal properties and the visual representation. A relational taxonomy is proposed for describing tonal ambiguity, which leads to extending the method for the analysis of music based on tonal systems beyond the major-minor paradigm. Two perceptual studies are approached from this descriptive framework. The first study evidences the impact of time-scale in a simple model of tonal induction, and analyses the mathematical artefacts introduced by evaluations in scaled spaces. In the second study, a model of contextual instability is proposed and discussed in relation to the modelling of tonal tension. The analysis and representation methods are then generalised, through a set-class theoretical domain, in order to be applied with any pitch-based music.