====== Seeing Music - devel ======

===== System Architecture =====

<ditaa name=seeing_music1>

        +----------------------------------+
        |Microphone(s), Live Stream(s), etc|
        |                              {io}|
        +---------------+------------------+
                        |
                        v 
                    /--------\
                    |Audio-in|
                    \---+----/
                        |
                        v
    +-------------------+---------------------+
    |cYEL Audio Processor                     |
    +-------------------+---------------------+
                        :               +---------*time/sync
                        |OSC            :
                        v               v
    +-------------------+---------------+-----+
    |cYEL Video Generator                     |
    |                   +---------------------+                     
    |                   | Video Generator     |
    |                   +---------------------+
    |                   | Texture engine      |
    |                   +---------------------+
    |                   | Video Filters       |
    | +----------+      +---------------------+
    | |controller|      | Encoder             |
    | +----------+      +---------------------+    
    |                                         |
    +---+---------------+---------------+-----+
        ^               |               ^
        |SQLITE         v               |OSC
        |or TXT    /---------\          |  
        :          |Video-out|          :
    +---+---+      \--+----+-/    +-----+-----+
    |running|         |    :      |user       |
    |config |         |    +----->+interaction|
    |    {d}|         |           |        {d}|
    +-------+         |           +-----------+
                      v
        +--------------------------------+
        | Projector(s), Live Stream, etc |
        |                            {io}|
        +--------------------------------+
                
/---------------------------------------------------\
: Legend                                            |
|cYEL                                               |
|                                                   | 
| +->  control signals                              |
| :                                                 |
|                                                   | 
| -->  A/V data flow                                |
|                                                   | 
\---------------------------------------------------/

</ditaa>

  * The audio-processor is a black-box that analyzes incoming sounds and generates [[http://opensoundcontrol.org|OSC]] messages. A basic set of messages and parameters needs to be specified (those can be extended in the future).
    * volume / energy
    * tempo / rhythm  
    * pitch / timbre
    * frequency spectrum and frequency spread
    * etc
    * additional msg (initial and periodical updates) configuration (number of channels, samplerate, etc)
  * Video-generator has an internal pipeline. e.g. shape-generator, texture-overlay, zoom, filter (eg. blur, brightness/contrast), encode.
    * every element in the pipeline may have configuration parameters
    * these parameters are either fixed or can be modified dynamically (interact with input)
    * dynamic parameters can be set by OSC (either from the sound-analyzer or via user-interaction)
    * a ''config'' file or database (aka preset) describes the pipeline and defines a map of parameters.
  * User interaction - also sends OSC messages to the video-generator (depending on the config, these are internally forwarded to one or more elements in the pipeline)
    * interaction can be via HID (mouse, keyboard, etc)
    * Joystick, Wii-mote, etc
    * Mixing-desk (faders or knobs)
    * advanced systems such as camera -> processing -> OSC

===== Implementation =====

  * Audio-processing: e.g. Max/MSP ; pure-data
  * Video-generation: Max/Jitter, processing, gstreamer, pure-data,...