Native Audio in the browser

Until recently, the ability to play any type of audio within a browser involved using Adobe Flash or other browser plugins. Although Adobe’s Flash player is unquestionably the most ubiquitous of these, most developers and designers would agree that it’s better not to rely on a plugin at all.

Enter HTML 5 <audio>

One of the most exciting and long-awaited features in HTML 5 the <audio> element, enabling native audio playback within the browser. We can take advantage of this for browsers that support it — such as Safari 4, Firefox 3.5 and Chrome 3 — while falling back on Flash or other plugins for other browsers.

According to spec

Currently, the HTML 5 spec defines five attributes for the <audio> element:

  1. src — a valid URL specifying the content source
  2. autobuffer — a boolean specifying whether the file is to be buffered in advance
  3. autoplay — a boolean specifying whether the file should play as soon as it can
  4. loop — a boolean specifying whether the file should be repeatedly played.
  5. controls — a boolean specifying whether the browser should display its default media controls

Note that these are the same attributes defined for the <video> element.

Examples

Let’s take a couple of these attributes and create a simple example that will play an audio file:

<audio src="elvis.ogg" controls autobuffer></audio>

(This example will work for Firefox 3.5 and Chrome 3. You’ll need to replace the Ogg file with an MP3 to get it working in Safari 4.)

Of course, the spec is not finalised, and there isn’t yet a consensus on which codecs to support. This table details the codecs supported by today’s browsers:

Codec support in modern browsers
Browser Ogg Vorbis MP3 WAV
FireFox 3.5 ✓ ✓
Safari 4 ✓ ✓
Chrome 3 (beta) ✓ ✓
Opera 10 (beta) ✓

To create our own controls, we can use the API methods defined by the spec:

  • play() — plays the audio
  • pause() — pauses the audio
  • canPlayType() — interrogates the browser to establish whether the given mime type can be played
  • buffered() — attribute that specifies the start and end time of the buffered part of the file

Note that Opera 10 does not support the <audio> HTML element, but it does support the Audio() API in a basic and unique manner: from the above list, it only supports the play() method.

Safari 4, Firefox 3.5, and Chrome 3 beta support both <audio> and Audio(). Internet Explorer 8 has no native audio support whatsoever.

Use the Source

The best way to coerce browsers into playing audio (or video, for that matter) is to use the <source> element. The browser will try to load the first audio source, and if it fails or isn’t supported, it will move on to the next audio source. In addition, we can embed a Flash player if all else fails:

<audio controls autobuffer>
  <source src="elvis.ogg" />
  <source src="elvis.mp3" />
  <!-- now include flash fall back -->
</audio>

One caveat, though: you need to be careful about the order of the <source> elements. Because of a bug in Firefox, if you list the MP3 first (which Firefox doesn’t support), it will silently fail and refuse to render that particular <audio> element. The trick is to list the Ogg Vorbis file first and the other formats after. Webkit (Safari and Chrome) handle unsupported formats just fine.

Opera is another kettle of fish altogether, which we’ll need to solve using JavaScript.

Cross-Browser Implementation

When we created jPlayer, an audio player plugin for jQuery, we were attempting to address some of the limitations of the current crop of Flash-based audio players. Many relied on Flash to implement the player’s graphical interface, effectively isolating the player from the rest of the web design process.

The original jPlayer relied on Flash to play the actual audio while allowing the look and feel to be styled via HTML and CSS. With growing support for HTML 5 in modern browsers, we were inspired to break our Flash dependency and use native audio when it was supported.

The most significant issue is the cross-browser implementation, where lack of a common supported audio format among browsers causes complications. If developers want to take full advantage of all browsers that support HTML 5 audio, they’ll need to create both MP3 and Ogg (and in Opera’s case, WAV) versions of the audio file they want to stream!

Our job was made more difficult by the lack of available information:

Since the HTML 5 standard is still a work in progress, several aspects of the <audio> element vary from browser to browser. For example, there seems to be no way to determine the load progress of an audio file. Although Safari 4 can determine how much of an audio file has downloaded, Firefox 3.5 does not yet implement the buffered DOM attribute. (It is, however, included as part of the Firefox’s documentation, so we presume it will be implemented.)

Firefox 3.5 does appear to enable “seeking” of the file, allowing you to start the download from any point in the media. Other browsers, such as Safari 4, grab the file from the beginning and only allow seeking within the part already downloaded (or wait until the particular part being seeked to has been downloaded).

Although these inconsistencies aren’t showstoppers, in order to compete effectively with plugin-based solutions, we believe any HTML 5 audio implementation should be consistent across all browsers and match current implementations feature for feature.

JavaScript solutions

If we intend to take advantage of each browser’s audio capabilities, we need to create different solutions for different browsers. We could use browser sniffing, but considering the rapidly changing landscape, it’s better to check what capabilities a particular browser supports and adapt accordingly.

To demonstrate this “feature sniffing”, we’ve created a rough and ready HTML 5 audio checker.

Using JavaScript, you can check for audio tag support:

// returns a boolean
var audioTagSupport = !!(document.createElement('audio').canPlayType);

or check for the Audio() object:

try {
  // The 'src' parameter is mandatory in Opera 10, so have used an empty string "".
  // Otherwise, an exception is thrown.
  myAudioObj = new Audio(""); 

  audioObjSupport = !!(myAudioObj.canPlayType);
  basicAudioSupport = !!(!audioObjSupport ? myAudioObj.play : false);
} catch (e) {
  audioObjSupport = false;
  basicAudioSupport = false;
}

or check file type compatibility:

// Need to check the canPlayType first or an exception
// will be thrown for those browsers that don't support it
if (myAudio.canPlayType) {

  // Currently canPlayType(type) returns: "no", "maybe" or "probably"
  canPlayOgg = ("no" != myAudio.canPlayType("audio/ogg")) &#038;& ("" != myAudio.canPlayType("audio/ogg"));
  canPlayMp3 = ("no" != myAudio.canPlayType("audio/mpeg")) &#038;& ("" != myAudio.canPlayType("audio/mpeg"));
}

Note that to change the src attribute of an audio object or element, you’ll need to recreate the object or element with the new value for its src attribute.

So, to create a solution that takes full advantage of HTML 5 audio, you’ll typically need to:

  1. check for HTML 5 audio support, and if not present, fall back on Flash,
  2. check the level of HTML 5 audio support and adapt your code accordingly for each browser, and
  3. check what file types are supported and link to appropriate formats of the files.

The Road Ahead

Although HTML 5 audio is a relatively immature part of the standard, if recent trends continue and users upgrade to the latest versions of Safari and Firefox, browser support will likely rise above the 25% mark in the very near future. This is a significant chunk of the browser market that will no longer need to rely on Adobe’s Flash, Microsoft’s Silverlight, or any other browser plugin for audio support.

And when you consider that mobile and other lower-spec devices — like Apple’s iPod and iPhone (Safari), Nintendo’s Wii (Opera), and Google Android-powered devices (Chrome) — are choosing to support HTML 5 audio rather than Flash, you begin to paint a picture of how important native audio support will soon become.

Further reading:

Share and Save:
  • Twitter
  • Digg
  • Sphinn
  • Reddit
  • del.icio.us
  • StumbleUpon
  • Technorati
  • Netvibes
  • Facebook
  • Google Bookmarks
  • FriendFeed
  • HackerNews
  • LinkedIn
  • NewsVine
  • Tumblr

53 Responses to “ Native Audio in the browser ”

Comment by Garrett Albright at

How about MP4/AAC? I’ll take a guess that only Safari supports it currently…

But all said, plugin-less audio sounds altogether doable, especially if we can use something like jPlayer to abstract the differences away. Is there anything similar to jPlayer for video?

Comment by Mark Boas at

@Andrey agreed – it’s a real pain! To be fair Safari could also support Ogg Vorbis that would give us a bit of choice. From what I can gather it all boils down to licensing and patents sadly.

@Garrett – now if we could abstract all HTML 5 differences away we’d be on to a winner :) Regarding video you may want to check out http://camendesign.com/code/video_for_everybody

Comment by Mark Boas at

@John – You’re absolutely right to pick me up on that – I should have written :
“I think most developers and designers would agree it is better not to rely on a plugin at all.”
Maybe we should do a poll :)

Incidentally I wonder how many sites are using Flash just to play audio.

Comment by Mark Boas at

@Tim It seems to matter very much indeed to the people who make browsers and it’s a shame that things have to turn political at the detriment of the developer community at large.

Let’s say that for the average developer it’s pretty much possible to start using the new tag right now falling back on Flash for Internet Explorer, Opera and older browser support. Then the choice is whether to support Firefox, Safari (or both) depending on which audio formats you provide. Chrome for now is the only given.

Comment by Garrett Albright at

Adam, I strongly against using it if you want me to stick around on your site. Few things will annoy me more about your web site than if it starts making noises without my permission. And I know I’m not alone.

Comment by Mark Boas at

@Adam on the subject of Autoplay the spec http://dev.w3.org/html5/spec/Overview.html does not appear to advise against using it. After all if a feature is in the spec it is expected to be used in some form.

However it does mention:
Note : Authors are urged to use the autoplay attribute rather than using script to trigger automatic playback, as this allows the user to override the automatic playback when it is not desired, e.g. when using a screen reader. Authors are also encouraged to consider not using the automatic playback behavior at all, and instead to let the user agent wait for the user to start playback explicitly.

As to whether you use the autoplay attribute, I think that is down to the individual.

Comment by Ben at

Why are people confusing native playing ability with built-in codecs?!? Flash came about not because the browsers don’t have built in video players (well, they don’t), but to solve the CODEC problem. Now in HTML 5 they’re trying to conquer both at once and the problems are evident even with audio, let alone video. Likely some browsers don’t support MP3 cause it still is licensed to fraunhaffer (sp?) and requires a licensing fee to put it in your application. Instead, HTML5 should just say the browser needs to support video playback without using an embed, BUT you still have to have the CODEC installed on your O/S. This doesn’t mean they need to bundle the CODEC’s with the browser! If they try to dictate a codec that every HTML5 browser has to support, it will likely be obsolete before any actual implementation (this was the whole reason for codec’s in the first place – so media player software didn’t become useless when new formats came out).

Comment by Ben at

sorry, I meant the ubiquitous use of flash for video playback came about to solve the CODEC problem, not that flash itself was created for video playback (it in fact, was not originally meant for that sole purpose).

Comment by Ben Scherrey at

Is HTML5 silent on streaming audio? Most interesting audio services are those that broadcast a continuous stream rather than selecting individual fixed files. I’d be very interested in hearing about any planned support for this.

Comment by stephband at

In all the discussion of the merits of native audio support against Quicktime or Flash plugin alternatives I’ve yet to see any sensible discussion of the merits of the codecs.

For me, the main reason not to use Flash is that it is (still) only capable of playing mp3 files. Some people are bemoaning the lack of native mp3 support in the audio tag, but I can only see this as an advance, because the sound quality of mp3 playback does not compare well with playback of more modern codecs, .m4a (aac) or .oga (vorbis), in files of the same size. For reasons of sound quality alone I have always turned to QuickTime to deliver audio on the web.

mp3 is getting on for 20 years old, and it is high time it is retired. Roll on audio tags with native support for newer, better codecs.

Comment by stephband at

… although I haven’t figured out how to detect the m4a mime type. I’ve tried:

audio/m4a
audio/mp4a
audio/mp4a-latm
audio/quicktime

Any ideas?

Comment by Joeri Kassenaar at

Uggh…

Cool ! A html tag that needs javascript by default to get to work… Hooray!
No thanks, I’ll stick to swfObjects if I need to add javascript, it allows me to create nice pause and rewind features.

O and while I’m at it ( hacking I mean ):
still no ? When is that going to happen? HTML12?
With wmv support in IE, mp4 in Safari and m2v with no audio in FF?…

Useless features hooray.

Comment by Joeri Kassenaar at

( cool site btw, great work here )
Sorry that I used < tags for my little rant… made it unreadable. Ofcourse I read your remark only after my post.

Comment by Mark Boas at

@Victor – the short answer is no. Browsers implementing native audio currently only support file formats such as OGG, MP3, M4A and WAV. I’m no expert on streaming media but I believe the process requires purpose-built formats. I do wonder whether you could stream a file-format such as OGG with the correct server-side technology but I suspect that there are very real problems in achieving this. Would be happy to be proved wrong though :)

@stephband – Did you ever find a satisfactory mime format for M4A? I had a look but could only find the same references as you.

Comment by Hamranhansenhansen at

> It’s worth noting that the spec is not yet finalised and unfortunately there has yet to be
> a consensus on which codecs to support, each browser supporting a different
> combination of codecs

Audio standardization is outside the scope of the HTML5 specification. It’s not up to browser makers or Web developers or anyone at W3C to tell the world how to store and play digital audio. The responsibility for browser makers is to bring their products into compliance with existing audio standardization, which means playback support for ISO MPEG-4 AAC, aka MP4. The responsibility for Web developers is to publish audio and video in the existing standards, which means MP4. You are responsible for this in the same way that Kodak and Apple and even Zune all support MPEG-4.

The thing that’s really important here is to compare the success of Web standardization, which has never, ever succeeded:

- during the HTML4 years we made proprietary Internet Explorer 6 apps with Flash audio video (FAIL)
- during the HTML3 years we made proprietary Netscape apps with proprietary QuickTime audio video (FAIL)

… with the success of audio video standardization, which has never failed:

[1980]
- CD was universal
[1990]
- MPEG (CD-ROM) was universal
- MPEG-2 (DVD) was universal
- MPEG-2 audio layer 3 (MP3) was universal
[2000]
- MPEG-4 (iTunes, YouTube, Flash, QuickTime, iPod, iPhone, Palm, Nokia, Blackberry, Zune, Blu-Ray, NVIDIA, Android, Kodak, Flip, Avid, Final Cut, iMovie, Logic, Pro Tools, Performer, many more) was universal

Me, I’m an audio producer and Web developer, and a huge supporter of HTML5, so I’m embarrassed to see Web developers attempting to open up an audio video standardization debate at this time, as if they are the Kings of the World not just the World Wide Web. Especially when the Ogg vs MPEG-4 debate literally happened 10 years ago and Ogg lost by far, solely on technical merits. Unless there is also a Time Machine in the HTML5 spec, there is no going back to 1999 and redoing the last 10 years of the world’s audio video in Ogg. Had the W3C been thinking about audio video instead of XML at the turn of the century, they might have had some input into MPEG-4. Apple delayed MPEG-4 for 6 months because of issues they had with the licensing (there was a content tax put in, Apple would not support MPEG-4 until the content tax was taken out) but the W3C said nothing. They left audio video to Flash and QuickTime as usual. Well, Flash and QuickTime have both since standardized on MPEG-4. To say now that the Web standard is not MPEG-4 is not practical. The computer time to transcode all the media to Ogg does not exist, the Ogg audio video toolchain does not exist, and we are already into user generated content and camcorders that shoot MPEG-4 and upload via Wi-Fi. How can we say to them “I know you’re making ISO standard media, but we’d like you to find a Linux computer and transcode to Ogg before you upload please so that we can be in compliance with HTML5.” It is not practical.

Also, notice that if the HTML5 spec were to say “Ogg is standard, MPEG-4 is not” that would mean that YouTube could NEVER comply with the HTML5 spec. YouTube is all MPEG-4 in the back end, no matter what you upload, they create an MPEG-4 to use and store your original as a backup. That is why it runs on mobiles, where the ONLY video playback is MPEG-4. Google has already done a study of “YouTube Ogg” and found that there is currently not enough Internet bandwidth to support it. In other words, if YouTube were switched to Ogg overnight tonight by magic, then tomorrow it would need more bandwidth than exists in the world just to serve its current users, even if everyone else stopped using the Internet for the day somehow. YouTube is also growing, so the day after they would need even more.

If YouTube is not Web video then I don’t know what is. And there you have Google being a full supporter of HTML5, they are moving YouTube to it quite steadily, and we’re going to say “sorry, you’re out of spec because you use the same audio video format as the rest of the world”?

It’s also important to understand that MPEG-4 is the ISO standardization of the Apple QuickTime file format that is used almost universally by audio video authoring tools. Therefore, MPEG-4 compatibility in the authoring toolchain came almost for free. What was there was standardized, rather than creating a whole new thing and standardizing that. Instead of replacing QuickTime with something else (which still doesn’t exist), QuickTime was standardized so that now there are “QuickTime Players” from hundreds of manufacturers now. In that sense, MPEG-4 is as practical for audio video people as HTML5 is for Web developers, standardizing what is already happening. So to create a schism between HTML5 and MPEG-4 is terrible sabotage. It would be like HTML5 not supporting UTF-8 text, or JPEG images. It’s so impractical as to make a mockery of the practicality of the whole HTML5 spec. MPEG-4 is how the world’s audio video is stored. Browsers need to be able to (continue to) display it.

If you’re a Web developer or browser maker and you’re not working with ISO MPEG-4 then your audio video is non-standard. The fact that you’re proud of your standardized HTML5 markup in that case is a joke.

> Did you ever find a satisfactory mime format for M4A?

- audio-only MPEG-4 files are audio/mp4
- audio video MPEG-4 files are video/mp4
- MPEG-4 files that contain scripts or Java or other application code are application/mp4

Here is a link to the RFC. This stuff is ancient, even if you’re new to it.

Comment by John-Paul Harold at

re: m4a mimetypes

I asked about this in a related question on the webkit mailing list. I ended up not using the m4a mimetype as Safari ignores it, and I set my source tags so that Chrome uses the vorbis.

hope it helps

jp

Comment by Weston Ruter at

canPlayMp3 = (“no” != myAudio.canPlayType(“audio/mpeg”)) && (“” != myAudio.canPlayType(“audio/mpeg”));

The current version of Chrome can play MP3s via HTML5 audio. However, if you try doing audio.canPlayType("audio/mpeg") it erroneously returns "", meaning it doesn’t support playing MP3s. I developed a workaround which loads up a tiny MP3 via a data: URI and detects for support via error or canplaythrough Media events. See http://gist.github.com/253174

Comment by Wiechu at

It is unuseful (at least: an extra effect, not so needed). Why? It is for audio-blogs, some “short previews” or lo-fi music services for mass-customers. If you have to publish your audio/video stuff use a “RAW” data file (like AIFF, WAV, CAF, etc.) zipped into downloadable “ZIP”. Who is using web browser as Music or Movie Player?

Comment by Pion at

Not everyone has the liberty to choose what audio format to offer. Where I work we have a huge archive of mp3s which we want to offer online to our patrons. They belong to filesets which have mp3 as a required format and converting it all would take both time and space.

Besides the licensing issures, there’s no reason not to support mp3.

Comment by Jonathan at

My requirement needs audio files to be chained together, like a sort of playlist.
Soundfile1,Soundfile2,soundfile3 – works fine in flashplayers, but I can’t work out how to do it in native audio

Comment by Jonathan at

Mark, you are right – brilliant timing, and a brilliant player, too! Looks (and sounds!) REALLY good.

Just tinkering now, I’ll let you know when the site’s ready then I’ll proudly show it off!

Comment by scott at

Hi,

What about generation in the browser? specifically, generating sinewaves from the browser without using flash etc. I need this to work on smartphones…

I recently came across something that does this, but now can’t find it :(

thanks,
s.

Comment by Paul at

I ran into a problem getting the HTML5 audio tag to work with Firefox (just the HTML5 tag, no javascript). The player/controls would show up, but it would not play my ogg vorbis audio file, and a big “X” was shown above the player instead of the “loading…” animation.

After some trial and error, I determined that it was not a problem the HTML or with my audio file. (It worked when the source was set to a file hosted on a different domain. And it would not work with that same file hosted on my domain.) So it turned out to be a problem with my server setup. Apparently my server was not serving the ogg file with the correct MIME type. Here was the answer on the Mozilla site: Configuring servers for Ogg media

“Most servers don’t by default serve Ogg media with the correct MIME types, so you’ll likely need to add the appropriate configuration for this.” I just needed to add the following to my .htaccess file:

AddType audio/ogg .oga
AddType video/ogg .ogv .ogg

That did the trick, and now it works like a charm in Firefox!

Leave a Reply

You can use these tags: <a href="" title=""> <abbr title=""> <b> <blockquote cite=""> <cite> <del datetime=""> <em> <i> <q cite=""> <strong>

You can also use <code>, and remember to use &lt; and &gt; for brackets.