Spectrum Visualization with the HTML5 Audio Data API

9 Mar 2010

The HTML5 specification introduces the and media elements, and with them the opportunity to dramatically change the way we integrate media on the web. The current HTML5 media API provides ways to play and get limited information about audio and video, but gives no way to programatically access or create such media. We present a new extension to this API, which allows web developers to read and write raw audio data.

The above quote is from the Audio Data API extension that joins a bunch of juicy developer work in Firefox 3.7.

Thomas Sturm has taken that API and created a spectrum visualization a kin to some of the great work by Scott Schiller (using Flash).

There is a new onaudiowritten attribute:

PLAIN TEXT

HTML:

<audio src=“song.ogg” controls=“true”
onaudiowritten=“audioWritten(event);”>
</audio>

that lets you get access to info such as the spectrum:

PLAIN TEXT

JAVASCRIPT:

function audioWritten(event) {
spectrum = event.mozSpectrum;
var specSize = spectrum.length, magnitude;
// Clear the canvas before drawing spectrum
ctx.clearRect(0,0, canvas.width, canvas.height);
for ( var i = 0; i <specSize; i++ ) {
magnitude = spectrum.item(i) * 4000; // multiply spectrum by a zoom value
// Draw rectangle bars for each frequency bin
ctx.fillRect(i * 4, canvas.height, 3, -magnitude);
}
}

Add to that the ability to write audio….

PLAIN TEXT

JAVASCRIPT:

var audioOutput = new Audio();
audioOutput.mozSetup(2, 44100, 1);
var samples = [0.242, 0.127, 0.0, -0.058, -0.242, …];
audioOutput.mozAudioWrite(samples.length, samples);

Nice work all around.

Go to Source

Programming Blog

Spectrum Visualization with the HTML5 Audio Data API

Comment Form

About this blog