I have created music driven animation since Flash 5 and I must say there has not been much progress until just now. I started out by importing sounds directly to the timeline and made keyframes according to the amplitude shown, a slow and painful method. Soon people created different approaches, some better than others, but the one I have used the most is SwiftMP3 from www.swift-tools.net. It started as a free script that could be used either on a server for real-time conversion or from a dos prompt.
Now, with the release of AS3, we can use existing classes to load external mp3 files and read the spectrum variables in runtime! In this tutorial I will show how it is done.
1: SoundMixer class
The SoundMixer class is a part of the flash.media package. It contains classes for manipulation of both video and sound as well as classes for working with media in Flash Communication Server.
The SoundMixer class inherits from Object and contains properties and methods for controlling embedded streaming sounds but it can not be used together with Sound objects created with AS.
Methods defined in the SoundMixer class:
areSoundsInaccessible():Boolean //Determines whether any sounds are not accessible due to security restrictions.
computeSpectrum(outputArray:ByteArray, FFTMode:Boolean = false, stretchFactor:int = 0):void //Takes a snapshot of the current sound wave and places it into the specified ByteArray object.
stopAll():void //Stops all sounds currently playing.
Methods inherited from the Object class:
hasOwnProperty(name:String):Boolean //Indicates whether an object has a specified property defined.
isPrototypeOf(theClass:Object):Boolean //Indicates whether an instance of the Object class is in the prototype chain of the object specified as the parameter.
propertyIsEnumerable(name:String):Boolean //Indicates whether the specified property exists and is enumerable.
setPropertyIsEnumerable(name:String, isEnum:Boolean = true):void //Sets the availability of a dynamic property for loop operations.
toString():String //Returns the string representation of the specified object.
valueOf():Object //Returns the primitive value of the specified object.
Since this tutorial is about analyzing spectrums and music driven animation we will put our main focus on the computeSpectrum method:
public static function computeSpectrum(outputArray:ByteArray, FFTMode:Boolean = false, stretchFactor:int = 0):void
Every time the computeSpectrum method is called it creates a copy of the current wave and stores the data as normalized floating-points between -1 and 1 in the ByteArray (outputArray). If the sound is not running the floating.points get the value zero. The ByteArray has a fixed size of 512 where the first 256 values represent the left channel and the last 256 represent the right.
The FFTMode variable can by set to either true or false determining whether a Fast Fourier transform is performed before the sound wave data is saved. Setting this to true will create a frequency spectrum where low frequencies are shown to the left and high to the right. This is pretty much standard on music players these days, like Winamp for example.
"The Fourier transform is an integral transform that re-expresses a function in terms of sinusoidal basis functions, i.e. as a sum or integral of sinusoidal functions multiplied by some coefficients ("amplitudes")."http://en.wikipedia.org/wiki/Fourier_transformation
The last variable, stretchFactor, is an integer that determines the sampling rate of the sound. 0 samples the sound at 44 KHz and incrementing the variable with one halves the sampling rate (1 = 22KHz, 2 = 11KHz).
Unfortunately you can not use this method together with sound streamed by the Flash Media Server and its Real Time Messaging Protocol.
3: Load an mp3
Before we can do anything we have to load some music and make it play. Start by opening a new document in Flash 9. In the first frame, we create a Sound object called "mySound":
var mySound:Sound = new Sound();
Next we load an mp3 file into the Sound object and tell it to play:
This is the only code we need to play an mp3 file, simple enough!
4: Read and display the spectrum
Now when we have the music playing we will continue by creating a method to calculate the spectrum of the music. First we to create a ByteArray to hold the raw data we will get from the computeSpectrum method. We will simply call this "myByteArray":
When we call the computeSpectrum method we include our ByteArray created above, this is where the method will save all the data. The Fast Fourier transform is set to false (but you can play with this to see what results you get) and the stretchFactor to 0, meaning that the sounds sampling rate is 44 KHz. After that we delete all graphics from the stage. Well, the first time there will not be any but we will get there.
Then we create a for loop so we can go though all the data we have. The condition will be i < 256 since we have 256 values for each channel in the ByteArray. I want to have 32 bars in my spectrum so I increment i with 8 (256 / 8 = 32). In the loop we will call the next method, drawSpectrum, with myByteArray.readFloat() and the current iteration (i). readFloat is the method used to get floating-point numbers from the byte stream.
Below you can see some traces I made with and without using readFloat and the FFTMode set to true or false:
Tracing myByteArray without readFloat() and FFTMode set to false:
The drawSpectrum method is what will draw our spectrum bars. The floating-point number passed on from the readSpectrum method is multiplied with 500 to get a visible difference and then we use a simple drawRect function to draw the graphics.
function drawSpectrum(myReadFloat:Number, myIteration:Number){
This is everything! The code above should provide you with something like this:
(Click button to start the music)
5: More examples
Here I used the exact same code as above but set the FFTMode to true and added a blur filter:
var filter:BitmapFilter = new BlurFilter(15, 15, BitmapFilterQuality.HIGH);
var myFilters:Array = new Array();
myFilters.push(filter);
filters = myFilters;
(Click button to start the music)
The next one is a little more complex with lines drawn to each peak instead of just using bars:
(Click button to start the music)
The code actually is not very complicated, I only changed the two functions to this:
