Computer technology has been changing the music business for decades. A startup called MuseAmi believes its technology could create significant additional tools for professional musicians–and for people who might otherwise never produce a listenable note.

The Princeton, N.J., company has been focusing on algorithms for analyzing and manipulating music, starting either with written music notation or sounds. CEO Robert Taub, a former concert pianist, says one inspiration was his daughter’s struggles in learning to read sheet music.

What if, he recalls thinking, she could have a device that would allow her to hear what a piece of written music sounded like? In a demonstration using a camera attached to a laptop computer, Taub shows how the company’s technology can do just that–take a picture of a page of music notation and play it back as written. He says that capability could be built into cellphones, which come with cameras that are up the task.

How about generating notation as music is performed? Automatic transcription is not a new trick for commercial software programs, though it typically relies on musical instruments that output signals in the format known as Midi. It’s rarer to get accurate results from more free-form sound sources, but Taub sings and the company’s software accurately records the pitch and timing of the music in notation that appears on his laptop screen. (At the moment, MuseAmi’s technology can only handle one voice or instrument at a time).

MuseAmi also hopes to take the idea of correcting pitch to a new level. The field is closely associated with Auto-Tune, the technology popularized by Antares Audio Technologies that is widely used in recording studios to make singers sound in tune. MuseAmi uses a different technology to correct pitch, while simultaneously generating harmonized vocal lines and other effects.

Taub and colleague Jon Sheldrick demonstrated some possibilities with Simon & Garfunkle’s hit “Mrs. Robinson.” As Sheldrick sang, not very accurately, the introduction–the “deet-dee-dee-dee-dee-dee-deet-dee” part–software on a laptop computer simultaneously corrected his singing on the melody, while adding a close approximation of the higher harmony sung by Art Garfunkle on the original recording.

Aided with a template of a song’s melody–dubbed a Midi map–the technology also corrected singing that was nowhere near the real melody–Sheldrick’s voice in a monotone. And more than one harmony, as well as other effects, can be added in real-time as a performer sings. The company has posted a video demonstration here.

MuseAmi relies largely on technology developed by Yann LeCun, a New York University computer-science professor and musician. Taub says it plans to rely on licensing its algorithms for others to use in products–like music-related game programs–than sell products itself. “We are forging collaborations with a number of major companies,” he says.

Go to Source