SoMo - generative music that makes you dance

SoMo is an application that analyzes a video stream coming from the user's webcam and generates music synced to the user's movements.

The app is written in AS3 and uses Flex 4 Beta SDK.

Tech

A quick overview of what the app does:

  • Each video frame is subdivided into a number of regions and compared with the previous frame to get an estimate of the motion per frame. (Overlayed on video)
  • Auto-correlation of the difference signal is calculated to get an estimate of the motion's period. (Plotted in red)
  • The period estimate is used to control the tempo of a sequencer which is pre-programmed with a beat track. (Plotted as a black circle on the red graph)
  • The beats are made using a FFT based synthesizer.
  • Yellow circles shows various parameters (undocumented!).
  • The app could use some heavy tweaking... ;)

    Demo

    You will need:

    • Flash player 10.
    • A webcam
    • Stylish moves
    • MAC: Try Safari
    • WIN: Try Firefox
    • The aim is to produce periodical movement in the left part of the screen to build up a tempo (seen in the red graphs in the bottom). Try waving in the left area of the screen...

    Check out the demo here (warning: will probably crash your browser if no webcam is detected)

    Made by Svante Stadler and Filip Bonnevier

    (svante.stadler at gmail.com) and (filip.bonnevier at gmail.com)

    Please e-mail us if you have any questions! :)