Mice

Figure 3: A high-performance gaming mouse.

The mouse has a lot of untapped potential as a musical controller though they are generally viewed as a pointing device relegated to moving a pointer around a screen. But the instrument designer need not use the mouse in its traditional role. Modern mice can be high precision devices that work well in a variety of situations. Optical USB mice can have down to 5 ms of latency with up to 4000 dpi resolution per cycle, making them the most precise of the relatively cheap HIDs. Also, mice come in a wide variety of forms, from standard mice with balls or optical sensors, to trackballs, or trackpads. If it works as a standard mouse it can be used with the standard mouse objects.

The mouse is a little more tricky to use than other HIDs because, like the keyboard, it is usually in constant use by the OS. This means that mouse button clicks can mistakenly land on buttons and other GUI widgets with unintended effects. There are a number of ways to avoid this. The most basic and rudimentary method is to maximize a blank Pd window over the whole screen. Another basic workaround is to avoid using the mouse buttons at all in the instrument. Currently then best solution is to run the patch in -nogui mode, then there is no GUI for the mouse pointer to interact with. Ideally, there would be a way to steal the mouse from the GUI from within Pd so that it can use the mouse exclusively.

There are a number of objects available for getting data from the mouse. [linuxmouse]or [linuxevent] are currently the best choices in general because they provide the most flexible access to the mouse data. [linuxmouse]provides low level access to the mouse data. It outputs the relative change in position in screen pixels from the previous polling cycle. Using the relative data means that the mouse data is not limited by the size of the screen. With other objects, when the mouse pointer hits the edge of the screen, the data in that dimension stops changing, since its outputting absolute pixel coordinates. [linuxmouse]outputs the relative change in position each cycle regardless of where the mouse pointer is. The units are still screen pixels, but the size of the area where the mouse can move is unlimited. Set the mouse at the highest tracking speed for the highest resolution. This will make the pixels per centimeter the mouse moves much higher.

[MouseState] outputs the absolute coordinates of the mouse pointer on the screen in terms of pixels. This object is a clone of the Max/MSP object. Since Max/MSP originally ran on the MacOS only, this object was tailored for that environment, meaning that it only tracks the state of the first mouse button. All of the other mouse buttons are totally ignored by this object. [gemmouse] provides the location of the mouse pointer within the Gem[gem()] window, which can provide a solution for the problem of the OS also using the mouse at the same time. But since the resolution is tied to the resolution of the Gem window, [gemmouse] provides equal or less resolution than [MouseState]. [serialmouse] provides access to the data from serial mice. They have a 25-50Hz refresh rate, which limits it as a serious musical controller.

One example of an instrument built using the [linuxmouse]object is StickMusic, and instrument created using a haptic (meaning related to the sense of touch) joystick and a haptic mouse. In StickMusic, the mouse is used as if bowing a string. The X-axis velocity is mapped to the amplitude of the output with Y-axis position mapped to the pitch. Using veolcity over position works well with the mouse since the mouse provides only relative location information.