Sound in Flash 10 (beta). Generating Waveforms, Timbre and Pitch.
Keywords: generative, music, pitch, Sound, timbre
It’s been quite a while (back in 2002, flash 5) since I did some serious work using oscillators and waveforms. I’m writing this down while trying to get back on track. The stuff described here is definitely no secret. It’s basic, but relevant to create some tunes in flash. Maybe you’ll find some of the stuff useful.
Through the introduction of the SampleDataEvent it is now easy to write data back to the audiostream:
//const AMP_MULTIPLIER:Number = 0.15; //keep this low while testing private function noiseWave(event:SampleDataEvent):void { var sample:Number; for (var i:int=0; i<8192; i++ ) { sample = Math.random() -.5; event.data.writeFloat(sample * AMP_MULTIPLIER); event.data.writeFloat(sample * AMP_MULTIPLIER); } }
This will simply generate some noise. Sounds rather boring, doesn’t it? Don’t neglect the noise though, it’s a great source for percussive sounds, when treated accordingly. Anyway, to get something more exciting we need to introduce pitch, timber (quality of sound) and loudness.
Tones & Pitch
To produce a sound we perceive as a tone at a certain pitch, we need to introduce a bunch of data that is repeated over time. This gives us a repeating pattern. Let’s call a cycle of that pattern the waveform. Aha, so let’s simply write 1,-1,1,-1,1,-1… to the stream, right? Yes, we could do that, but no, you and I wouldn’t hear a bleep. It would simply be out of our hearing range. To get the tones into the hearing range, the pattern needs to be repeated a certain amount of times per second. Remember the note, your music teacher in school was desperately hammering on his piano, to get everyone in tune? That was probably an A above middle C. This note has a frequency of 440 Hz, meaning the waveform (or data in our case) cycles – or oscillates - at 440 times per second. The next lower A is at 220 Hz, the next higher 880 Hz. Basically, the more cycles per sec, the higher the frequency, the higher the pitch.
Flash works at a sampling rate of 44100 Hz. It uses 44100 samples/sec to produce sound. If we want to generate an A (440 Hz) in flash, we need to repeatedly write a bunch of 44100/440 ≈ 100 samples to the audiostream. The 100 samples are actually the wavelength represented in samples.
//const AMP_MULTIPLIER:Number = 0.15; //const BASE_FREQ:int = 440; //const SAMPLING_RATE:int = 44100; //const TWO_PI:Number = 2*Math.PI; //const TWO_PI_OVER_SR:Number = TWO_PI/SAMPLING_RATE; private function sineWave1(event:SampleDataEvent):void { var sample:Number for (var i:int=0; i<8192; i++) { sample = Math.sin((i+event.position) * TWO_PI_OVER_SR * BASE_FREQ); event.data.writeFloat(sample * AMP_MULTIPLIER); event.data.writeFloat(sample * AMP_MULTIPLIER); } }
This will generate a pure sine A440Hz. By layering multiple sines of different frequencies a vast number of waveforms can be created. A square wave could be generated using multiple sines, but that would put heavy loads on the CPU. Luckily we can cheat:
//the square wave implemented via sine, duty cycle for a square is 1:2 private function squareWave1(event:SampleDataEvent):void { //lets be nice to our equipment and keep the amplitude low var amp:Number = 0.075; var sample:Number; for (var i:int=0; i<8192; i++) { sample = Math.sin((i + event.position) * TWO_PI_OVER_SR * BASE_FREQ) > 0 ? amp : -amp; event.data.writeFloat(sample); event.data.writeFloat(sample); } }
This gives us a nice square wave, but there’s quite an expensive Math.sin() in it. Another approach might be better regarding performance:
//keeps track of current phase var phase:Number; private function squareWave2(event:SampleDataEvent):void { var amp:Number = 0.075; var sample:Number; for (var i:int; i < 8192; i++) { sample = phase < Math.PI ? amp : -amp; phase = phase + (TWO_PI_OVER_SR * BASE_FREQ); phase = phase > TWO_PI ? phase-TWO_PI : phase; event.data.writeFloat(sample); event.data.writeFloat(sample); } }
The square wave has the same pitch as the sine wave above but sounds different (different quality of sound or timbre). Why? Overtones.
Timbre (quality of sound)
The perceived timbre of a sound is determined by its spectrum and loudness over time. The spectrum is basically the sum of different frequencies in a sound. Based on the fundamental frequency (in our example 440 Hz) there could be other frequencies (overtones) on top of that. These overtones can either be harmonic, when they are multiple octaves above the fundamental frequency or inharmonic, when somewhere in-between. We don’t need to deal with overtones, let’s just state that the waveform and timbre of a sound are coupled. HOW a waveform sounds is mainly a question of perception and association. The other factor that defines the timbre is the loudness of a sound over time.
Loudness
Loudness of a sound over time is another factor that defines the timbre. If we change
-this: event.data.writeFloat(sample); event.data.writeFloat(sample); -to that: event.data.writeFloat(sample * (8192-c)/8192); event.data.writeFloat(sample * (8192-c)/8192);
we get a sound with a strong attack and a linear decay. However, the sounds we got so far are a little static. To get more interesting results, we need to introduce modulation. Modulation can affect all kinds of parameters (like pitch, loudness, envelope, shifts in overtones, anything) to slightly or drastically change the waveform over time. Here’s an example that modulates the pulse width and the amplitude of a pulse wave:
//modulated pulse, one LFO modulates pulse-width, another one the amplitude private function pulseWaveMod1(event:SampleDataEvent):void { // get those out var amp:Number = 0.075; var pwr:Number = Math.PI/1.05; var dpw:Number; var am:Number; var pos:Number; var sample:Number; for (var i:int=0; i<8192; i++) { pos = i + event.position; dpw = Math.sin (pos/0x4800) * pwr; //LFO -> PW sample = phase < Math.PI - dpw ? amp : -amp; phase = phase + (TWO_PI_OVER_SR * BASE_FREQ); phase = phase > TWO_PI ? phase-TWO_PI : phase; am = Math.sin (pos/0x1000); //LFO -> AM event.data.writeFloat(sample * am); event.data.writeFloat(sample * am); } }
By modulating different parameters the sound gets more interesting. Now, nobody stops you from modulating the modulators, giving more drastic results.
Here’s a little test of basic waveforms up to rather heavily modulated:
Click here for the demo. Turn down volume first. Needs the latest Flash-Player 10 (rc 091508).
Here’s the source. UPDATE: Adobe changed the sound api. Use SampleDataEvent.SAMPLE_DATA instead of Event.SAMPLE_DATA for the listener.
Yeah, sounds still thin, though
You’re right. To get those fat bastard sounds we need more voices, effects (the holy reverb), filters, compressors etc. but that’s a whole different story. In the end, at the basis of it all are the waveforms.
References:
Similar Posts
Frequency Modulation Synthesis in Flash 10 and Gumbo
From cacophony to music. Trying to create bearable evolving music in flash 10 (beta).
Gumbo (Flex 4) plays generated sounds in Style
Sound in Flash gaining momentum
Customize the Spark TextInput Component in Flex 4 (Gumbo). Adding focus and Transitions.
November 4th, 2008 at 6:34 am
[...] http://www.hulstkamp.com/2008/09/12/sound-in-flash-10-beta-generating-waveforms-timbre-and-pitch/175 [...]
November 26th, 2008 at 1:07 am
Hi there. Thank you for publishing this interesting flash demo. I was trying to play around with it in CS4 by creating a .fla file (there were only .as files in the zipped source code) in the same folder as the WaveformTester.as file but when I attempt to instantiate a WaveformTester var like this (it’s the only line of code in the only frame of the only layer):
var wf:WaveformTester = new WaveformTester();
I receive this error:
TypeError: Error #1009: Cannot access a property or method of a null object reference.
at WaveformTester/buildCrapGUI()
at WaveformTester/init()
at WaveformTester()
at waveform_fla::MainTimeline/frame1()
I don’t have a strong background in flash so I guess I’m overlooking something silly. Any insight?
Thanks,
Etan
November 27th, 2008 at 11:00 pm
Etan,
The code has been developed using Flex Builder and the Flash 10 beta api. Adobe changed parts of the api for the final release. Check:
Issues with the Vector-Class (import)
Line #93 stage might be null, since WaveformTester hasn’t been added to the stage when the init is called. Try commenting this out.
Did you use the debugger in Flash and trace through WaveformTester to see where the RTE happens?
December 1st, 2008 at 9:12 am
Yea, I was using the debugger to try and figure it out but the null stage error didn’t mean a whole lot to me since the stage object is sort of new to me (been learning in Actionscript 2.0).
With some web research I managed to get it working by using this in the .fla file:
var mc:MovieClip = new MovieClip();
addChild(mc);
var wf:WaveformTester = new WaveformTester();
mc.addChild(wf);
I also still had to comment out or change line 93 in WaveformTester.as from
btn.y = stage.stageHeight - btn.height - yp;to
btn.y = yp + 200;because the stage null RTE was still being produced, and using no value for btn.y placed the buttons over the plotter.
Thank you for the help,
Etan
December 3rd, 2008 at 2:15 pm
Etan,
you still get the NPE because WaveformTester() has not been added to the stage when it calls the init().
Try removing init() from the constructor WaveformTester() and then call it manually
use
var wf:WaveformTester = new WaveformTester();
//add DisplayObject to stage
mc.addChild(wf);
//now call init()
wf.init();
or try using Stage.stageHeight instead of the stage-member.
hth
andy
December 16th, 2008 at 4:48 pm
My english is a little bad :d. here i go…
is there any way to save those sounds generated with Flash on my computer?
Ex: im working on a virtual dj n am needing to save the mixes in a mp3 file from flash.
Thnks
CT
December 18th, 2008 at 2:14 pm
First of all congratulation for such a great site. I learned a lot reading article here today. I will make sure i visit this site once a day so i can learn more.
December 20th, 2008 at 5:58 pm
thanks Edward, glad you find some stuff usefull.
December 20th, 2008 at 5:59 pm
Claudio,
Better use Adobe AIR for this. Check the FileStream class. This class gives you access to the File-System. You could simply write out the sample data but that won’t be of much help. You need to write the data in a format that you can use in a Sound-Application. MP3 is an compressed format so you’ll need to encode the sample data in mp3-format. You might want to google for an actionscript mp3-encoder that makes all the work for you or check the mp3-format an then create the encoder.
Hope this gets you started.
February 17th, 2009 at 6:31 pm
Really really cool post !
“Sound in Flash for dummies” (like me). I think it’s a must-read
thx for all, can’t wait to read your other stuff !
March 14th, 2009 at 4:39 pm
Andy,
This was exactly the type of blog entry I was searching for. Very well explained and certainly inspires the tech behind the site currently being built by myself.
Claudio’s need also raised some thoughts as I also ran into the same problem of how to manage the persistence of generated audio data in the least obstructive way. Agree however that AIR is probably the way to go, especially when certain more polished scenarios require things like autosave and caching of remotely downloadable sample files for audio production/mixing tools etc.
The other inspiration was the introduction of Abobe Pixel Bender, leveraging the GPU in a non-conventional way and applied to areas where the number crunching does present a performance issue in more intensive applications.
Thanks for this, delightful read, I look forward to your further posts.
Rob
March 16th, 2009 at 5:44 pm
Rob,
Thanks!
I think Claudio wanted to save the generated audio-data locally, so that’s why I suggested to use AIR. I haven’t looked into this but I don’t think there should be any problems of writing the data to a blob from flash.
The use of Pixel Bender to do some number crunching is very tempting, although it is currently not leveraging the GPU in flash in a consistent way.
March 25th, 2009 at 1:00 pm
[...] Player 10 Dynamic Sound Generating Waveforms, Timbre and Pitch Read more | Permalink | Comments | Read more articles in hacks | Digg this! Posted by slashman [...]
March 25th, 2009 at 1:23 pm
[...] Flash Player 10 Dynamic Sound Generating Waveforms, Timbre and Pitch [...]
March 25th, 2009 at 1:43 pm
[...] Flash Player 10 Dynamic Sound Generating Waveforms, Timbre and Pitch [...]
April 10th, 2009 at 9:15 pm
I’ve been looking for this info! Nice work!
June 5th, 2009 at 12:31 am
[...] http://www.hulstkamp.com/2008/09/12/sound-in-flash-10-beta-generating-waveforms-timbre-and-pitch/175 (Info and the reference I used when playing with Flash Player 10 Sound API.) [...]
June 23rd, 2009 at 7:18 pm
Hi, Very nice write up. Is it possible to work out from the sound spectrum what the pitch of a certain sound is. I have been playing around with working out peaks and troughs and allowing for noise by passing in a tolerance. Is there another way?
June 25th, 2009 at 9:05 am
James
thanks.
Maybe this could be of interest.