SoMo - generative music that makes you dance
SoMo is an application that analyzes a video stream coming from the user's webcam and generates music synced to the user's movements.
The app is written in AS3 and uses Flex 4 Beta SDK.
Tech
A quick overview of what the app does:
- Each video frame is subdivided into a number of regions and compared with the previous frame to get an estimate of the motion per frame. (Overlayed on video)
- Auto-correlation of the difference signal is calculated to get an estimate of the motion's period. (Plotted in red)
- The period estimate is used to control the tempo of a sequencer which is pre-programmed with a beat track. (Plotted as a black circle on the red graph)
- The beats are made using a FFT based synthesizer.
- Yellow circles shows various parameters (undocumented!).
- The app could use some heavy tweaking... ;)
Demo
You will need:
- Flash player 10.
- A webcam
- Stylish moves
- MAC: Try Safari
- WIN: Try Firefox
- The aim is to produce periodical movement in the left part of the screen to build up a tempo (seen in the red graphs in the bottom). Try waving in the left area of the screen...
Check out the demo here (warning: will probably crash your browser if no webcam is detected)
Made by Svante Stadler and Filip Bonnevier
(svante.stadler at gmail.com) and (filip.bonnevier at gmail.com)
Please e-mail us if you have any questions! :)