Introducing HTML5 Audio

Presenter Notes

The HTML5 audio element

Just as we can load an image:

1 <img src="images/myimage.png" alt="My Image"></img>

we can also load audio:

1 <audio src="audio/07-the-forgotten-people.mp3"></audio>

But we don't see anything! So we add some controls:

1 <audio src="audio/07-the-forgotten-people.mp3" controls></audio>

Presenter Notes

Or from JavaScript...

1 var audio = new Audio();
2 audio.src = 'audio/pete.mp3';
3 audio.controls = true;
4 audio.autoplay = false;
5 document.getElementById('audio-element-javascript').appendChild(audio);

Presenter Notes

Rolling your own player

A very simple start:

Presenter Notes

Supporting different browsers

1 <audio controls="controls">
2     <source src="audio/07-the-forgotten-people.mp3" type="audio/mp3">
3     <source src="audio/07-the-forgotten-people.oga" type="audio/oga">
4     <p>Your browser doesn't support the audio element - please update it.</p>
5 </audio>
  • You can of course fallback to Flash (eg SoundManager2) if that's supported

Presenter Notes

Audio Attributes

controls

  • display the browser's built-in auto controller

autoplay

  • please don't :-)

preload

  • a hint to the browser to:
  • auto - start downloading the file
  • none - don't download until user hits 'play'
  • meta - just prefetch metadata (eg duration)

loop

  • loop the audio track

Presenter Notes

Audio and Events

The <audio> element offers a rich set of events. See the HTML5 Media Event Inspector.

HTML5 Media Event Inspector

Presenter Notes

Why use the Audio Element?

  • Easy - as easy as embedding an image
  • Audio is downloaded automatically
  • Browser starts playing audio when enough available
  • Fallbacks supported with <source> element
  • Interactive control with controls attribute
  • Programmatic control with events

Presenter Notes

But more advanced stuff...?

Can I...?

  • have precise control over timing?
  • generate sounds within the browser instead of downloading them?
  • manipulate sounds, filter them, distort them?
  • visualise sounds?

Presenter Notes

The Web Audio API

The Web Audio API gives us:

  1. precise timing control
  2. a powerful and flexible routing system
    • filters and effects can be plugged in
    • custom effects can be written in JavaScript
    • visualisation tools enable visual sound level meters

Presenter Notes

Why Precise Timing?

Digital Clock

  • Users expect immediate response
    • for synchronising to games
    • for sequencing overlapping sounds

But...

  • Playing many <audio> elements at once can slow down the browser
  • Some browsers re-download the sound on each <audio> replay!

Presenter Notes

Timing in the Web Audio API

  • High-performance audio engine
  • Written in C++
  • Audio split out into separate high-priority thread
    • Audio doesn't halt user operations (and vice-versa)
  • Audio buffers separated from audio playback
    • Can efficiently have one buffer and play it many times
  • Hardware scalability
    • Degrades gracefully on lower-spec hardware

Presenter Notes

Routing System using Nodes

Nodes in Blender

Presenter Notes

Why Nodes?

For sophisticated applications (music, games) we need:

  • precise timing
  • filtering, eg to simulate going underwater in a game
  • visualisation, for sound meters and graphic equalisers
  • generate signals programmatically
  • position sound in 3D space
  • provide Doppler shift effects
  • Simulate acoustic environments like concert halls and cathedrals
  • and more...

Presenter Notes

What's audio?

  • Compressions and rarefactions of air
  • Represent with a sound 'wave'
  • Height represents amount of compression or rarefaction reaching ear

Presenter Notes

Representing Digital Audio - 1

We start with a sound wave:

Audio Waveform

Presenter Notes

Representing Digital Audio - 2

Chop it up:

Audio Waveform chopped up

Presenter Notes

Representing Digital Audio - 3

Sample it:

Audio Waveform sampled

Presenter Notes

Representing Digital Audio - 4

Represent the samples with numbers:

Audio Waveform samples

Presenter Notes

Representing Digital Audio - 5

Place these samples in an AudioBuffer:

HTML5 AudioBuffer

In JavaScript, an AudioBuffer has:

  • sampleRate
  • length
  • duration
  • numberOfChannels
  • getChannelData

Presenter Notes

Getting Started with Nodes

An Input Node connected to an Output Node

XXX: JavaScript example showing (a) creating node, and (b) playing sound

Presenter Notes

Adding a Gain Node

image

In JavaScript:

1 var context = new AudioContext();
2 
3 var source = context.createMediaElementSource(audio);
4 var gainNode = context.createGain();
5 source.connect(gainNode);
6 gainNode.connect(context.destination);

Presenter Notes

Kinds of Input

Depending on what we want to do, we can take our input from:

  • an Audio Buffer
  • a Media Element (eg <audio>)
  • a Media Stream
  • an Oscillator

Presenter Notes

Input from an Audio Buffer

Audio Buffer Source Node

Useful for:

  • short segments of preloaded sound
  • replaying segments again and again
  • capturing input
  • manipulating waveforms

Presenter Notes

MediaElementAudioSourceNode

Media Element Audio Source Node

Useful for:

  • taking audio input from an <audio> or <video> element
  • processing longer audio files

Presenter Notes

MediaStreamAudioSourceNode

Media Stream Audio Source Node

Useful for:

  • taking input from live audio or video streams
  • sampling data from a microphone
  • audio or video conferencing

Presenter Notes

Demo - Sampling Sound

Media Stream Audio Source Node with Microphone

  • This demo takes input from the microphone using a MediaStreamAudioSourceNode
  • In Chrome, go into chrome://flags/ and enable "Web Audio Input"

Presenter Notes

Oscillator

Oscillator Node

  • Tone generator
  • Used to generate ('synthesise') sound waves

Presenter Notes

Oscillator

  • Available:
    • sine
    • square
    • sawtooth
    • triangle
    • custom

Presenter Notes

Oscillator Node with Wave Table

Oscillator Node with Wave Table

Useful for:

  • synthesising sounds and music

Presenter Notes

Building a Synthesiser

A basic synthesiser

Presenter Notes

Building a Synthesiser

g200kg

g200kg

XXX: see: http://beausievers.com/synth/synthbasics/

Presenter Notes

Drum Machine

Syd's Kickass Drum Machine

Uses:

  • Samples stored as WAV files
  • Nodes: Gain, Panner, BiQuadFilter, LowPassFilter, Compressor

Presenter Notes

Dynamic Range Compression

image

Compression has many applications

Compression

Presenter Notes

Convolution Reverb

  • Convolution applies one signal to alter another
  • Impulse Response: We sample a space (eg hall or cathedral)
  • Then apply that capture to our signal

Here's a good tutorial: http://audio.tutsplus.com/tutorials/sound-design/how-to-use-convolution-for-reverb-effects/)

Presenter Notes

Analysis and Visualisation

1 var analyser = context.createAnalyser();
2 ...
  • Simple to create an analyser.
  • Then the fun part begins..

Presenter Notes

Demo: Web Audio Playground

webaudioplayground.appspot.com

Web Audio Playground

Written by: Chris Wilson

Presenter Notes

Demo: Web Audio Vocoder

Web Audio Vocoder

Written by: Chris Wilson

Presenter Notes

KievII

kievii.net/k2h.html

KievII Host

Presenter Notes

Presenter Notes

Jam with Chrome

Jam with Chrome

jamwithchrome.com

Presenter Notes

Web Audio Workshop

  • Subject: Build a Synthesiser
  • Date: Saturday 13 July 2013 at 10:00 am
  • Duration: half day (10:00 - 14:00)
  • Location: Merrion Square, Dublin
  • Cost: €25

Presenter Notes

Thank You!

Peter Munro
Web Application Development & Training
Mobile: +353 87 689 1951
Web: petermunro.org
Twitter: @peter_munro
LinkedIn: ie.linkedin.com/in/petemunro/

Presenter Notes