2011-2012-Digital Musical Instruments

Digital Musical Instruments

by Stef Brugmans and Rutger Alders

Wouldn’t it be cool to draw your own instruments and play them? Well, maybe you can!

The idea The idea of this project gives users the ability to draw an instrument, which a computer turns into a playable version. This can be achieved by creating a digital representation of the instrument on the computer. The computer translates the image of the instrument into a corresponding instrument. Next, sounds are added to the instrument. Finally motion of the use of the instrument is tracked, so that sounds can be played.

Functional version Creating an application that can create a digital representation of every viable instrument is a project that can take years. To show that the basic idea is attainable, a prototype is made. This prototype contains functionality to make usable versions of a simple piano or drum. The piano consists of a simple representation of keys, like the first image. This image shows the most basic components needed to play a piano. The second image shows the most basic elements of a drum. With some simple circles, it becomes possible to create a beat.

Implementation To create the application for this program, a couple of Software Development Kits (SDK’s) were used in combination with the programming language C#. To create a digital version of the instrument, AForge was used. With this SDK simple object like circles, triangles en rectangles can be identified in a drawing. The collection of these objects can then be translated to a specific instrument. For instance, a series of rectangles can be identified as a piano. Next, the instrument needs to be playable. With the help of the Touchless SDK, it becomes possible to track small objects (even fingers), which can then be used to play the instrument. The instrument is played by comparing the coordinates of the object used to play the instrument, with the coordinates of the draw objects. Finally a sounds needs to be played for every action made with the instrument. This is achieved by using sound playing functionality, offered by most programming languages. When coordinates of the object used to play and the coordinates of a drawn object collide, a sound is played which corresponds to the instrument drawn.

Future The simple example application, created as a prototype, gives access to create a much better experience. By creating the possibility to map collections of specific drawn objects to instruments, a much larger range of instruments could be supported. In this way, it could be possible to draw an entire orchestra on paper.