Multimedia Troubleshooting

by .

While I was researching HTML5 multimedia-related topics for my book, HTML5 Multimedia: Develop and Design, I noticed a number people struggling to get HTML5 audio and video working in different scenarios. From Twitter to Stack Overflow, the same questions kept cropping up, so I’ve put together a list of the most common problems (and some not so common) and their solutions (if there is one!).

In most cases, the problems and solutions apply to both audio and video. I’ll note when something is specific to one or the other.

Before I go any further, if you aren’t aware of how you can actually go about adding audio and video to your website via HTML5, you can read a number of other articles on this site to get you up to speed. Tom Leadbetter covered the video element, and Mark Boas covered native audio in the browser and HTML5 Audio – The State of Play.

In addition, I covered the same topics, video and audio, as part of a series over on the Adobe Developer Connection.

Browser Support

This is one of the first things you should check: Does the browser you’re using actually support the type of media file you’re trying to play? It’s simple enough to forget which browsers support which file types, so I’ll refresh your memory here.

Audio

  • Firefox supports Ogg Vorbis and WAV
  • Opera supports Ogg Vorbis and WAV
  • Safari supports MP3, AAC, and MP4
  • Chrome supports Ogg Vorbis, MP3, WAV, AAC, and MP4
  • Internet Explorer 9+ supports MP3, AAC, and MP4
  • iOS supports MP3, AAC, and MP4
  • Android supports AAC and MP3

In order to support all of the browsers mentioned above, it’s advisable to serve your audio file in Ogg Vorbis, and MP3. For example:

<audio controls> 
   <source src="myAudio.ogg" type="audio/ogg">
   <source src="myAudio.mp3" type="audio/mp3">
</audio>

Video

  • Firefox supports Theora Ogg and WebM
  • Opera supports Theora Ogg and WebM
  • Safari supports MP4
  • Chrome supports Theora Ogg, MP4, and WebM
  • Internet Explorer 9+ supports MP4 and WebM (requires a plugin)
  • iOS supports MP4
  • Android supports MP4 and WebM (2.3+)

In order to support all of the browsers mentioned above, you should serve your video file in both WebM and MP4 using the source element. For example:

<video controls>
   <source src="myVideo.mp4" type="video/mp4">
   <source src="myVideo.webm" type="video/webm">
</video>

MIME Types

Even if you have the correct media file for the browser, it’s possible that your server doesn’t have the correct MIME type set up. MIME types tell the server how to handle the different file types. If you’re using something akin to Apache and you know your stuff, head to the .htaccess file and add the following for audio support:

AddType audio/ogg ogg AddType audio/ogg oga AddType audio/wav wav AddType audio/mpeg mp3 AddType audio/mp4 mp4 AddType audio/mp4 mpa

And similarily for video support:

AddType video/webm webm AddType video/mp4 mp4 AddType video/ogg ogg AddType video/ogg ogv

If you don’t have access to your server’s .htaccess file, your control panel should have an option somewhere that allows you to view and add MIME types.

If you have a Windows server, then you may need to add the MIME types to IIS, either using the IIS Manager or in the web.config file.

File Conversion

It’s quite possible that a client has sent you a media file and you’ve determined its MIME type from its file extension. It’s possible, however, that the file wasn’t encoded correctly. It could, for example, be a perfectly valid MP4 file, but for some reason some browsers have trouble playing it. If this happens, you’re better off encoding the file yourself using a tool such as Miro Video Converter or Media Converter so you can be sure of the correct encoding.

Additionally, some files, especially MP4 files, don’t always play in a browser that is supposed to support them. This is due to the fact that MP4 (also known as H.264) files can be encoded according to different “profiles”. If the file in question is encoded in a profile that the browser doesn’t support, then naturally it won’t work. If this happens to you, try to ensure that the file is encoded according to the Baseline profile which is more likely to be supported than any of the more advanced ones. In my experience, using a tool like Miro ensures that the MP4 file will work just fine in supporting browsers.

If you need to support Firefox 3.6 and lower, you’ll also need to convert your audio files to Ogg and add them as an extra <source> for the <audio> element.

MP4 Files That Don’t Start Playing Until Fully Downloaded

Usually HTML5 audio and video players will allow the user to start playing a media file before it has fully downloaded (as long as it actually has something to play!). Sometimes this isn’t the case with an MP4 file, where the browser waits to download the entire file before making it available to play. This is due to an encoding issue.

Sometimes MP4 files are encoded with the file index (which contains information on the file such as it’s duration) placed at the end of the media file rather than at the beginning. This index contains the metadata that the browser requires in order for it to offer progressive download. If this index is at the end, it knows nothing at all about the file and therefore has to wait until it receives the index.

If this happens to you, there’s a simple fix called the QTIndexSwapper by Renaun Erickson that you download to your computer and run on the file that simply moves the index to the start of the file and saves it.

Getting Your Attributes Right

This might seem a little obvious, but it’s amazing how many questions I see on Stack Overflow where people are wondering why a certain function isn’t working when they’re using attributes that no longer exist. A prime example of this is the autobuffer attribute on both the <audio> and <video> elements, which was replaced by the preload attribute as far back as .

People seem to forget that HTML5 is not yet finalised (although large parts of it are quite stable) and that anything you read might be out of date. It’s worth checking the W3C HTML5 specification and also When Can I Use? to view browser support.

Volume Control in Firefox 11+

This is an problem that has cropped up recently. Some of you may have noticed that the volume controls have disappeared from the audio player in Firefox 11+. True, you can mute it, and the volume controls are still controllable via the keyboard’s up and down keys, but the volume slider itself was removed due to two bugs. I assume this will return in the future, but for now, in case you were wondering what happened to them, it’s not your fault. (You can, of course, use the Media API to build your own set of custom controls.)

Plugins Required

One of the biggest selling points of HTML5 audio and video is that no third-party plugins — such as Flash — are required in order to play multimedia files, as long as the browser in question supports it.

Unfortunately, this is not strictly true, as both Internet Explorer (9+) and Safari do require Microsoft Media Player and Apple QuickTime, respectively, to be installed in order for them to be able to play HTML5 audio and video files.

Video and Fullscreen

An oft-requested requirement for HTML5 video is the ability to play in fullscreen mode. The HTML5 specification has nothing to say on this, but a separate specification for a Fullscreen API is currently under work, with experimental versions available in some of the major browsers.

The following browsers go some way to supporting the Fullscreen API, albeit using their own vendor specific versions:

  • Chrome 19+
  • Firefox 12+
  • Safari 5.1+

There is also the screenfull.js JavaScript polyfill that enables this functionality for a limited number of browsers.

Internet Explorer 9+ Ignores Your Poster Image

If you have a HTML5 video defined with a poster image (using the poster attribute), you might be surprised when Internet Explorer 9 completely ignores it unless you have set the preload attribute to be none.

Since Internet Explorer was one of the last browsers to support HTML5, we all grew accustomed to the other browsers’ treatment of the poster attribute: if it’s specified, display that image until the user starts to play the video. Internet Explorer 9 doesn’t do this. It only displays the image contained within the poster attribute if it has nothing else to show, so if the first frame of the video has been loaded (with preload set to auto — the default value — or metadata), then it uses this regardless of what’s in the poster attribute.

This appears to be a case of Internet Explorer interpreting the specification in its own way and doing something different to all other browsers. Internet Explorer 10 also currently exhibits the same (mis)behaviour.

Accessing Webcams and Microphones

The ability to access peripheral devices such as microphones and webcams was always in the minds of the HTML5 specification writers when they added the <device> element. This has since been removed and replaced with the more thorough getUserMedia API, which allows access to such media devices.

The API itself is easy to use, but browser support is currently quite limited. Opera is currently the only browser with an implementation, supporting only video, but you can already see what it can do. Internet Explorer 10 will follow suit, and it will soon appear in Firefox Nightly.

Summary

This ends my round up of gotchas that you may encounter when working with HTML5 audio and video. Of course, it doesn’t cover everything, and I’m sure there are issues that you may have come across that haven’t been mentioned here. If you have, please do add them below in the comments, with their solution if possible, or if not, I or one of the other HTML5 Doctors will try to help.

22 Responses on the article “Multimedia Troubleshooting”

John Drinkwater says

+ Firefox 15 includes audio support for Opus as well.

David Higgins says

Another video convertor is Firefogg (www.firefogg.org). That tool was mentioned in Dive Into HTML5, and I use it all the time.

Personally I was having problems with Miro.

There is also a growing number of online services that automate the conversion process, like vid.ly, but I warn others – that particular tool is quite expensive.

Who knows how many people in Japan are streaming your video from their phone on super-fast connections?

And the MIME-Type thing always goes overlooked for anyone starting out with HTML5 video. Not many people grok it.

+1

Ivan says

I had to figure out what the problem was with the seeking of html5 video files, in the end I found QTindex swapper and it is indeed a priceless tool. For me, only Miro converter messed up the index of the video files, most other converters were fine.

Still, a great article, thanks :)

Federico Panico says

Opera on Linux relies on Gstreamer and will play whatever Gstreamer is capable of, incluiding MP3.

Brian Street says

How did you determine that the minimal video requirements are H.264 and WebM? Each browser that supports WebM also supports either H.264 or Ogg, including “old” Firefox.

Ian Devlin says

Brian, I didn’t say it was explicitly the minimum requirement, but Firefox users tend to have upgraded past 3.6 which is the last one to not support WebM.

On the other hand you are correct, serving Ogg, WebM and MP4 will cover all browsers including older Firefox.

Steven Barnett says

If Android supports AAC and MP4 for audio, how do just Ogg and MP3 support all of the browsers mentioned above?

Ian Devlin says

Steven, that was supposed to read “AAC and MP3″ – fixed now, thanks for pointing it out.

Ron says

Great article, thanks for this one.

This might help you to find the right video player for your website :)

Ravi says

Just a small correction. getUserMedia is available from chrome21. It is there in Firefox nightly and should land in version 17 I guess.

grzes says

android browser does seem to support ogg audio :)

Simon Donaldson says

Excellent article Ian! So many issues covered in one post…

Do you plan to extend it at any point to cover small devices? One topic I often see is getting videos to autoplay on mobiles, and another is how different devices (e.g. iPhones and iPads) handle video – play in-line, or default player.

Rikki says

I’ve tested it and found that Firefox and Opera are supporting MP3 as well. May you test the code from my article (it’s in Russian, but code is standart of course )

Neil Craig (@tdp_org) says

Hi
Nice article, i’ve seen (and asked) those same questions many times too.

Just thought i’d note that Chrome 21 now supports the getUserMedia API too – happy days.

Cheers
Neil

Ian Devlin says

@Simon, there were no plans to extend it to cover mobile devices, but now I suspect that there will be, so watch this space.

Saket Jajodia says

I have a Q. on Browser Support.. Like for IE we have conditional comments is there anything such for other web browsers also? As my website (d2d.co) work almost perfectly in Chrome but when we open in other web browser, few alignment gets messed up.. It can be solved by writing some lines of CSS for each browser but I need don’t know how I can specify different CSS for different browsers. Hope I am able to make myself clear and also hoping to get a reply with a solution.. Thanks.. :)

Marty says

I have an audio control on the page and it plays fine until I connect a button through some javascript to call the .play function. When I step through the script it plays fine, but not if I let it run without debug.

It doesn’t seem to matter if I break execution before the play statement and step through it, or if I break after the play statement. Both play the audio just fine. But if I remove the debugger; statements it doesn’t play.

Any ideas??

Mohamed Alaa says

Chrome 23.0.1271.64 sometimes don’t load the poster image even if you have preload=none

Mike S says

Made a temporary solution to the IE9 poster attribute problem, here:

http://www.artbymikeshaffer.com/vids/finalproject_chetbakerquartet_poster6c.html

It’s a hack, but a workable hack using CSS and some conditional statements.

The best approach, I think, for the future, is to:

a) Include the poster as the first frame of the video, and/or;

b) Overlay Play, Pause, and Rewind buttons (somehow) to deal with the issue that HTML5 video controls are pretty small on my smartphone. In this way, regardless of the browser used, a poster will appear, as well as larger control buttons.

My two cents worth…

Cheers

Asai says

video file only play video, audio is mute html5 code

Asai says

please share

Lakshmana says

ie9 doesnt display the player if you set preload=none

Join the discussion.

Some HTML is ok

You can use these tags:
<a href="" title="">
<abbr title="">
<b>
<blockquote cite="">
<cite>
<del datetime="">
<em>
<i>
<q cite="">
<strong>

You can also use <code>, and remember to use &lt; and &gt; for brackets.