(Photo Credit: Yamaha)

Dancer transformed into an instrument

The Yamaha Corporation developed a new AI system that uses motion to create music. The mission of their new project is to find new hybrid forms of artistic expression. It sounds like something out of a science fiction novel but this technology is quite real and we reveal how it works.

The new AI system uses four types of sensor (located on the back, feet, and hands) that are placed on the body and plays music according to the various motions. Yamaha revealed the new project by attaching the sensors to professional dancer Kaiji Moriyama and the AI uses sound data that is associated with each motion.

The sound data creates a melody based on the movements of the subject and transfers it to a Yamaha Disklavier piano, translating the data into piano keystrokes. The new AI system can turn a dance into a musical instrument.

Accompanied by the Berlin Philharmonic Orchestra, Scharoun Ensemble, Yamaha tested the new system while Moriyama danced for a truly unique artistic experience at the concert hall of the Tokyo University of the Arts. Using AI for the arts is truly inspiring and we’ll be waiting to see Yamaha’s next great development.