It seems like the world of classical music has started to take advantage of the possibilities that mobile devices and related technologies have. Not only within the Phenicx projects, similar efforts are happening elsewhere as well. The Philadelphia Orchestra has been developing their own mobile application (called LiveNote) for live concert following. With Ezra Wiesner, IT department of the Philadelphia Orchestra, we exchanged our experiences on the mobile apps.
User Experience
With regards to the user experience, the LiveNote user interface is divided in two parts, the upper and the lower part. The lower part shows the structure of the piece and there is a bar progressing through it. The upper part shows the content, which is composed of one image and some associated text (see this snapshot ) In the lower part, there is a "drawer" that can be opened at the bottom of the screen to view the structure of the piece, e.g. the form, like ABA'B. Also, there is a main concert page that lists the pieces to be played during the concert.
The app keeps the backlight on all the time so the users can see the content during the concert, but dims the backlight to 50% by default. The content is divided into "tracks", where each track is meant for one type of users (e.g. novice, experts etc.). A user can see one track at a time, although for text and translations, two tracks can be put side by side so viewers can see the source text next to the translated text.
Technical Details
Platform-wise, the LiveNote app comes as a native Android and iPhone application. A great deal of attention has been paid to the quality of the distribution network to ensure a high quality of service (in terms of network bandwidth availability). The app connects to a dedicated network in the concert hall (which has a lot of Wi-Fi routers, well hidden from the audience’s sight and painted with a special paint that does not distract from the aesthetics of the hall) in order to receive the LiveNote multicast stream. This data stream contains the current measure of the piece being played on stage and is updated manually by an expert in the projection booth during the live performance. When the app launches for the first time it downloads a pre-prepared XML document, which contains the content (text and images) with the respective piece-measure-stamps (i.e. at which measure a content is supposed to appear). As the multicast stream sends the current measure being played, the app shows the respective content as defined in the XML document. The developers chose to use multicast because it is much less demanding for the network, as it does not need to keep alive several hundreds of HTTP connections.
The Philadelphia Orchestra has at their disposal the audio tracking software by Drexel University (developed by Matt Prockup, as a coincidence we ran into him at the ACM Multimedia conference a few days later), but has only included it in test performances,as they want to ensure there are no errors during a live performance. Drexel and the Orchestra are planning to use the music tracking algorithm once they are sure the detection is 99% accurate and that there is an override mechanism available to a human operator if the automatic detection loses track of the music.
Content
The content itself is curated. It is prepared through a web user interface where the curator is able to add images and text to a predefined visual layout. The back-end user interface allows also to push messages to the app on demand (it is used for notifying the users to turn the ringer off, for example). The program notes that are available in the app are in the form of a PDF file.
Reception
The LiveNote app was generally well received. There were very few complaints from the audience. An interesting point about the reaction that the concertgoers have towards LiveNote is the following. It turns out that if you use LiveNote you are overwhelmingly in favor of using an in-concert app. If you did not use Livenote during the concert, then you are likely not to recommend it. In other words you like it if you use it. You don't like it if you see someone else using it at the concert but you decided not to try it.
It was also well received by the critics. The Philadelphia Orchestra was especially glad to learn that Peter Dobrin, from The Philadelphia Inquirer, stated that the app didn’t bother him (saying implicitly that you can’t stop the progress; he commented that the change this app brings is similar to the change when the opera subtitles were introduced):
http://articles.philly.com/2014-10-17/news/55112625_1_concert-outside-interference-app
The app was also featured in a Washington Post article:
and elsewhere
http://www.classicfm.com/music-news/latest-news/livenote-music-app/
Comparison with the Phenicx app
After demoing the Phenicx mobile app, Mr. Wiesner said that he liked the design. He valued the visualization of the audio feature. He was very excited about the feedback and the personalization concept (we explained him our vision of it). He said that very interesting program notes were done by them in the past with some living composers who provided more intimate information, such as 'the bass solo in that piece was composed for that particular bass player because …'. It revealed the details of some kind of interaction between the composer, the director and the performer during the creative process that conventional program notes usually do not reveal, making it very attractive. We further showed Mr. Wiesner the youtube video (from the Vilnius’ demonstration) of the live score follower developed within the Phenicx project and he was impressed.
Technology is definitely making its way into classical music. Some people like it, some are against. Mr. Wiesner commented that it’s (relatively) frequent that music students come to the Philadelphia Orchestra concerts with PDF scores on their iPads and swipe throughout the concert. However, they had some complaints on this matter by other attendees of the concerts. There are clearly still open issues about it but the benefits and the potential that the technological support of classical music concerts brings is evident.
