Seeing Music - devel

System Architecture

The audio-processor is a black-box that analyzes incoming sounds and generates OSC messages. A basic set of messages and parameters needs to be specified (those can be extended in the future).
- volume / energy
- tempo / rhythm
- pitch / timbre
- frequency spectrum and frequency spread
- etc
- additional msg (initial and periodical updates) configuration (number of channels, samplerate, etc)
Video-generator has an internal pipeline. e.g. shape-generator, texture-overlay, zoom, filter (eg. blur, brightness/contrast), encode.
- every element in the pipeline may have configuration parameters
- these parameters are either fixed or can be modified dynamically (interact with input)
- dynamic parameters can be set by OSC (either from the sound-analyzer or via user-interaction)
- a config file or database (aka preset) describes the pipeline and defines a map of parameters.
User interaction - also sends OSC messages to the video-generator (depending on the config, these are internally forwarded to one or more elements in the pipeline)
- interaction can be via HID (mouse, keyboard, etc)
- Joystick, Wii-mote, etc
- Mixing-desk (faders or knobs)
- advanced systems such as camera → processing → OSC