Kinect hacked with Mac to allow air guitar jam sessions
updated 02:00 pm EST, Fri December 10, 2010
by MacNN Staff
Air guitar made possible by hacked Xbox Kinect
On his website this month, Chris O'Shea revealed he has hacked Microsoft's Xbox Kinect accessory to allow users to engage in air guitar sessions. There is video proof of the working hack (embedded below), which was ported over to the peripheral from a project O'Shea performed back in 2007 with a stereo camera. The program was written in C++ in openFrameworks and openCV for image processing, with ofxKinect and libfreenect drivers used on the Mac making the hack work.
The software finds a person in the room by a threshold method, before using a histogram to get the most probably depth of a person. Objects in front of the identified person are identified as hands and tracked. One hand holds the neck of the virtual guitar, with a guitar drawn on a virtual line through the center of the body. When the other hand passes through this line, it strums the guitar. Chords are controlled by the neck hand's position.
As a first attempt, the program isn't 100 percent perfect. O'Shea calls the hand tracking inaccurate and flaky and says users need to face the camera in a specific, head-on way. He and other developers will use OpenNI to work on bringing 3D skeletal tracking to the project.
There are no plans to commercialize this idea, as O'Shea is sure others are already developing the technology.