StickMusic

Ideas

Second Sensory Channel

Using haptic feedback provides information about the interaction on another sensory channel in addition to hearing and perhaps sight. This engages different parts of the brain allowing the performer to gather a more complete understanding of the interaction. When more sensory channels are utilized, more information about the experience is communicated for any given moment.

Visualization of the music can also provide another sensory channel. If there was a coherent visual representation of the music being produced, this would also provide another layer of feedback, allowing a more complete understanding of the experience.

Pushing Into the Sensory, Away from the Semantic

[Visuals directly mapped to the audio. Perhaps even served as the interface.]

Design

"...though programmable haptic feedback might tempt the instrument designer to experiment with instruments whose feel is continually evolving, players will find that these instruments are hard to learn and control" -O'Modhrain p 81

"...unless simulations of instruments are very close in feel to their real counterparts, they are more likely to confuse players than to promote transfer of skill from the real environment." -O'Modhrain p82

Implementation

StickMusic was implemented using Pd on Linux. I wrote my own mouse object, [linuxmouse], which provides raw relative axes data from the mouse, rather than the absolute pixel data from the OS. This removes the limits of mouse movement that the screen would normally create. It also provides higher resolution data than the OS pixel data.

For the force feedback, I used Gerard van Dongen-Gilcher's ff objects for Pd. These objects give direct control over periodic, spring, friction, constant-force, and auto-center effects. I used the periodic effect to create a vibration in the joystick handle which mimics the sound being generated, both in amplitude and in pitch. I used a constant-force effect to represent how much reverb is being added; as you pull back on the y-axis, the reverb increases, and the strength of the opposing force increases as well.

The mouse and the joystick control a phase vocoder. The mouse Y-axis controls the transposition of the sample, the speed of the mouse X-axis controls the amplitude. The Y-axis of the joystick controls the amount

Bibliography

Instrument Design

• Principles for Designing Computer Music Controllers - Cook
  
• Playing by Feel- Incorporating Haptic Feedback in Computer-based Musical Instruments - Sile O'Modhrain
  

Instruments

• The Vodhran - The Virtual Bodhran (Irish drum) - Marshall, Rath, Moynihan
  
• Cymatic (joystick + mouse + physical model) - Rimmel, Howard, Hunt, Kirk, Tyrrell
  
• KromoZone: A Platform for Networked Multimedia Performance - Moore, Place
  

Parameter Mapping

• The importance of parameter mapping in electronic instrument design - Hunt, Wanderley, Paradis
  
• Bi-manual mapping experimentation, with angular fundamental frequency control and sound color navigation - Kessous
  
• Managing Complexity with Explicit Mapping of Gestures to Sound Control with OSC - Wright, Freed, Lee, Madden, Momeni