Digital Sheet Music Viewer

Client: Cambridge Consultants

Contact: Andrew Knights <andrew.knights@cambridgeconsultants.com>

A musical score is in effect a single line of n bars. Lines, pages, and to some extent repeat markers appear due to the score being printed on a page. Design software to allow a user to have complete freedom to arrange a musical score on their chosen VDU (be it a laptop, tablet, mobile-phone, or projector) in a format that most suits the display and their needs. Starting with a free digital format of the score, the software should allow users to: • Edit the score layout using drag & drop of bars to arrange the score on-screen • Choose whether they wish for a scrolling or line/page approach to viewing • Manipulate on-screen scaling (that will automatically dictate bars per line, etc. when viewing) • Activate fixed timed scrolling/page turning • Integrate a musical OCR program to allow scanning paper copies into the software • Allow the user to customise the score and/or add comments and reminders. • Implement a detection algorithm that will allow automatic scrolling and/or page turning. (Using maybe: midi keyboard input / audio listening / front-facing camera watching the performer )

Extension tasks: • Extend the detection algorithm to allow real-time tracking of your place in the score. • Extend the detection algorithm to allow multiple voices and be robust to a musician retiring part way through. • Allow multiple users to share, scroll/page turn synchronously, and work together on one score. • Allow recording during a performance which is linked to the score which could be used for a 'musical autopsy'

Doing a quick look online there are numerous musical OCR programs which could be integrated into the software without the need to write it from scratch.

feedback

I'd like to refine the digital sheet music concept into a more distinctive product concept. I like the general idea - in fact, a friend of mine has created an excellent app that incorporates many of the things you suggest: http://www.musicscroll.info/

From past experience, although we usually have a few musicians on the course, Computer Science seems to attract slightly fewer musicians than most Cambridge degrees, so we can't assume that students would immediately recognise the implications of the suggestions you've made (I'm a string player myself, so do have a reasonably clear idea of possible benefits). This means that orchestral support features like score annotation, player coordination, response to a conductor and so on are unlikely to be well understood by a project team.

Based on previous projects, I believe that audio analysis techniques for tracking polyphonic voices or even beat tracking will be beyond the scope of undergraduates. The same probably applies to using cameras for gesture detection from an uncontrolled scene. We've used pitch detection from electric guitar in the past (avoid the problem of background noise from microphone input, which is a big issue on the public demonstration day).

I wonder whether some kind of music tutor application might have some appeal? Thinking of the many YouTube guitar tutorials - "how to play …", it might be interesting to allow those to be segmented, coordinated with a score (or tab) view, and adjusted to follow tempo of the player.

Here's an approach to tutoring that uses the Guitar Hero / Rock Band approach rather than standard music notation: http://www.youtube.com/watch?v=pJUrZsFFrVg