MIME Types #

MIME types (also known as Internet Media Types) are a way of defining file formats so that your system knows how to handle them.

# AddType TYPE/SUBTYPE EXTENSION
AddType audio/mpeg mp3
AddType audio/mp4 m4a
AddType audio/ogg ogg
AddType audio/ogg oga
AddType audio/webm webma
AddType audio/wav wav

<audio>
   <source src="elvis.mp3" type='audio/mpeg; codecs="mp3"'>
   <source src="elvis.oga" type='audio/ogg; codecs="vorbis"'>
   <!-- add your fallback solution here -->
</audio>

Knowing in Advance: canPlayType Can Help, Probably #

Fortunately the audio API provides us with a way to find out whether a certain format is supported by the browser. But first, here’s a quick recap on how we manipulate the audio element via the API.

var audio = document.getElementByTagName('audio')[index];

// or, if you gave it an id attribute
var audio = document.getElementById('my-audio-id');

var audio = new Audio();

Once you have your audio element, you’re ready to access its methods and properties. To test format support, you can use the canPlayType method, which takes a MIME type as a parameter:

audio.canPlayType('audio/ogg');

You can even explicitly include the codec:

audio.canPlayType('audio/ogg; codecs="vorbis"');

canPlayType returns one of three values:

1. probably,
2. maybe, or
3. “” (the empty string).

The reason we have these odd return types is because of the general weirdness surrounding codecs. The browser can only guess at whether a certain codec is playable without actually trying to play it.

So to test for support, you could do this:

var audio = new Audio();
  var canPlayOgg = !!audio.canPlayType && audio.canPlayType('audio/ogg; codecs="vorbis"') != "";

All we’re doing here is checking that canPlayType is supported (!! effectively casts to a boolean) and then checking that canPlayType of our chosen format doesn’t return an empty string.

Current Browser Codec Support #

Let’s check the codec support in the current crop of modern browsers.

† WAV since Chrome 9

The good news is that at the time of writing, it’s estimated that around 80% of browsers now support HTML5 audio.

The bad news is that there is still no consensus on which codec to support, so you’ll need to provide both MP3 and Ogg Vorbis sources in order to take full advantage of HTML5 audio.

var audio = new Audio();
var duration = audio.duration;

Buffering, Seeking, and Time Ranges #

The situation is improving in this area, as browser makers start to implement key parts of the spec.

The API provides attributes called buffered and seekable that can be used in situations where we want to ascertain which part of the media has been buffered, preloaded, or is ready to be played without delay.

Let’s first take a look at the buffered and seekable attributes. Both return a TimeRanges object. The TimeRanges object is a list of time periods containing start and end times that can be referenced by their indexes.

// returns TimeRanges object of buffered media
var buffered = audio.buffered;

// returns time in seconds of the last buffered TimeRange
var bufferedEnd = audio.buffered.end();

------------------------------------------------------
|=============|                    |===========|     |
------------------------------------------------------
0             5                    15          19    21

// Is the player currently seeking?
var isSeeking = audio.seeking;

// Is the media seekable?
var isSeekable = audio.seekable && audio.seekable.length > 0;

// Time in seconds within which the media is seekable.
var seekableEnd = audio.seekable.end();

Well-Played #

It’s worth mentioning the played property in passing. This property tells us which time ranges have been played within the media. For example:

// returns a TimeRanges object
var played = audio.played;

Media Events #

Underpinning both the native audio and video APIs is a comprehensive set of events. A full list can be found at the WhatWG version of the spec.

Attaching handlers to media events allows you to easily react to changes in state. For example, it might be useful to update the time info in a custom-built player each time the timeupdate event occurs.

A quick summary of the more commonly used media events:

Other useful events:

Again, the HTML5 Media Event Inspector is your friend.

Additionally, you can test browser support by using areweplayingyet.org. It’s worth checking out the code behind the tests to understand the goings-on under the hood.

Streaming #

Streaming audio is a common requirement. This is an area that until recently has been dominated by Adobe’s Flash technology. Proprietary server technologies and protocols are well-established. One of the leading examples of this is Adobe’s Real Time Messaging Protocol (RTMP). However, current browsers only support streaming over HTTP, and so something like Flash is required to process RTMP streams.

Currently, audio-enabled browsers only support SHOUTcast and Icecast servers that stream audio over HTTP. SHOUTcast is propriety and allows streaming of MP3 and AAC files only, while Icecast is non-propriety and supports Ogg Vorbis as well as MP3 and AAC.

An Evolving Spec (Or, “Whoa, this thing is moving!”) #

As the audio spec is still evolving, there are a couple of inconsistencies in browser implementations to watch out for. These generally affect older browsers.

var audio = new Audio();
audio.setAttribute("src","mynew.mp3");
audio.load(); // required for 'older' browsers

What’s New? #

There are a few new features and whole new APIs being specified for web-based audio, so let’s take a look at what’s around the corner.

Advanced Audio APIs: The Future Sound of Browsers #

When it comes to more advanced audio functionality, Mozilla was first to enter the fray with an early experimental Audio Data API for Firefox. This was intended to facilitate audio manipulation via JavaScript and created a platform for low-level tinkering.

Not to be outdone, Google released the Web Audio API for Chrome and put it forward as a W3C proposal. The Web Audio API is a higher-level implementation that takes an audio-node based approach and provides many useful functions.

Both implementations address the desire by developers to do more with audio — to create, manipulate, analyse, and mix audio on the fly. In effect, we may one day be able to do in a browser anything we can currently do with native applications.

So what if we want to play with advanced audio in the browser? Well, two separate implementations currently exist. If you want to write applications that use both APIs in any reasonable way, you need third-party bridging libraries to abstract away the differences.

Also fairly new on the scene is Mozilla's MediaStream Processing API, which takes another approach to audio and video streams to allow us to manipulate at both high and low levels.

In fact, using advanced APIs and taking advantage of the speed of modern JavaScript engines, we can even write decoders for unsupported codecs. Currently, we have the JSMad MP3 decoder and the lossless Alac.js decoder. Apparently, FLAC and AAC are in the pipeline. There is hope, then, that in the future, we may not have to worry about which browsers are supporting which formats.

For those curious about advanced web audio, I wrote about the differences in approaches and the different libraries that allow us to write cross-browser solutions in HTML5 Audio APIs: How Low can we Go?

The good news is that all relevant parties are talking, and things seem to be buzzing on the W3C Audio Working Group. It looks like everyone is coming together to put their ideas into a unified Audio Processing API, which is rapidly approaching publication as a Web Standard.

Summary #

Although browser implementations of the current HTML5 audio spec are improving, there are still a few issues to watch out for when creating comprehensive cross-browser solutions. You certainly need to be aware of browser limitations on platforms like iOS. Hopefully, as support matures, these issues will disappear.

Positive news on the "advanced audio" front too: a new standard is being established, and we're close to get a unified spec for browser makers to work from. In the meantime, JavaScript libraries are available to help bridge the gaps between existing implementations.

There are also signs that we may be seeing the end of the requirement for different browser-dependent codecs. Opera Mobile already supports MP3 where the underlying OS supports it, and Mozilla look like they may enable this too. If this option exists for mobile browsers, it's not a huge stretch to imagine that desktop browsers will follow suit.

So lots happening and lots already achieved. The rough edges of existing browser implementations are being smoothed, consensus and standards are being forged, and we can see the foundations of some very exciting new technologies being established.

The future of web-based audio looks bright!

Further Reading #

• "Probably, Maybe, No": The State of HTML5 Audio
• The State of HTML5 Audio
• Mastering the HTML5 <audio> tag
• Unlocking the power of HTML5 <audio>
• Everything you need to know about HTML5 video and audio

Semantics and the Web #

Semantics are the implied meaning of a subject, like a word or sentence. It aids how humans (and these days, machines) interpret subject matter. On the web, HTML serves both humans and machines, suggesting the purpose of the content enclosed within an HTML tag. Since the dawn of HTML, elements have been revised and adapted based on actual usage on the web, ideally so that authors can navigate markup with ease and create carefully structured documents, and so that machines can infer the context of the wonderful collection of data we humans can read.

Until — and perhaps even after — machines can understand language and all its nuances at the same level as a human, we need HTML to help machines understand what we mean. A computer doesn’t care if you had pizza for dinner. It likely just wants to know what on earth it should do with that information.

HTML semantics are a nuanced subject, widely debated and easily open to interpretation. Not everyone agrees on the same thing right away, and this is where problems arise.

What’s the point? #

Discussions over the importance of semantics are happening all the time, and every so often there’s an uproar over specific articles on the subject. Divya Manian caused a stir in her Smashing Magazine article Our Pointless Pursuit Of Semantic Value in November 2011, in which she argued we have become too caught up in trying to use HTML5′s semantics, and that the benefits aren’t worth it:

Allow me to paint a picture:

1. You are busy creating a website.
2. You have a thought, “Oh, now I have to add an element.”
3. Then another thought, “I feel so guilty adding a div. Div-itis is terrible, I hear.”
4. Then, “I should use something else. The aside element might be appropriate.”
5. Three searches and five articles later, you’re fairly confident that aside is not semantically correct.
6. You decide on article, because at least it’s not a div.
7. You’ve wasted 40 minutes, with no tangible benefit to show for it.

This generated a storm of responses, both positive and negative. In Pursuing Semantic Value Jeremy Keith argued that being semantically correct is not fruitless, and he even gave an example of how <section> can be used to adjust a document’s outline. He concludes:

But if you can get past the blustery tone and get to the kernel of the article, it’s a fairly straightforward message: don’t get too hung up on semantics to the detriment of other important facets of web development.

I’ll admit I’ve been in a situation where I’ve dug myself a mind hole, trying to decide which element is “correct” and then depressing myself with thoughts of how irrelevant it all seems. What gives me strength is the thought that I’m not marking this up for me, but for everyone who can benefit from the enhanced meaning. Whether it’s the browser, a search engine spider, an accessibility tool, the person you pass the project on to, or even future you returning to the project in six months time, the markup indicates how the content should be interpreted. As long as your markup makes sense and isn’t over the top (e.g., if you just replace all your HTML 4 <div>s with <section>s, you may be misunderstanding the element or even your content), then don’t worry so much about it all.

To help you choose the most appropriate element, we released a flowchart of HTML5 sectioning elements that you can print off and follow whenever you get stuck. If all else fails, don’t forget about your old buddy <div>. Then again, maybe you’ve stumbled across a need for something new…

Naming Things #

Of all the possible new element names in HTML5, the spec is pretty set on things like <nav> and <footer>. If you’ve used either of those as a class or id in your own markup, it’s no coincidence. Studies of the web from the likes of Google and Opera (amongst others) looked at which names people were using to hint at the purpose of a part of their HTML documents. The authors of the HTML5 spec recognised that developers needed more semantic elements and looked at what classes and IDs were already being used to convey such meaning.

Of course, it isn’t possible to use all of the names researched, and of the millions of words in the English language that could have been used, it’s better to focus on a small subset that meets the demands of the web. Yet some people feel that the spec isn’t yet doing so.

What about adding more elements? #

When I first met fellow HTML5 Doctor Bruce Lawson, I asked him this question: If we have elements like <article>, why don’t we have one for products of a shop? I understand more about HTML now than I did then, but at the time it seemed like a very logical element to add. Why restrict ourselves to documents with <article>s when the web has evolved beyond that with shops, applications, and games? I’m sure many of you have felt the same way. Some have even put cases to the WHATWG suggesting more elements like the often requested <comment>.

Recently, editor of the HTML5 spec Ian Hickson wrote responses to some of these requests for new elements for comments, explaining why <article> within <article> suffices for marking up comments:

<article> isn’t just for articles. That’s the point.

Note that its name is irrelevant here. It could be called <pineapple> — what matters is what it is defined to mean, not what its name is. And its definition is one that covers both articles and comments. They are both self-contained compositions.

As much as I am all for the much needed addition of <pineapple>, Ian makes a good case for why we don’t need <comment> (or similar). Dr Bruce often uses the following analogy when trying to explain <article>:

Don’t think of <article> as a magazine article. Think of it as an article of clothing, an independent entity that can be arranged in conjunction with other articles of clothing, but is a complete thing in itself.

I hear you ask, That’s all well and good, Mike, but what does understanding <article> have to do with adding new elements? To add something to the spec, you need to:

1. document actual use cases,
2. show how developers are working around the lack of this feature now, and
3. make a compelling case for why HTML5 needs it.

Step one is to make sure what you’re asking for can’t be achieved with what already exists. This is exactly the problem with the proposals for <comment>. Any difference from <article> is so minimal that it isn’t worth adding.

If every proposal for a minor deviation of an existing element was accepted, the spec would be bloated with far more elements than necessary. User agents would have to keep up with the vast array of markup possibilities for a given bit of content — as would you, the author. And think about this: at the end of the day, what are you actually going to use that new element for? Really?

That’s not to say you should be discouraged from putting your ideas forward. Take it as advice on what to look for when you want to contribute something to the spec. It may even help you to think differently about your work. Some problems just need a fresh perspective to help solve them.

HTML5 Doctor, or: How I Learned to Stop Worrying and Love HTML #

Once you learn to look past element names and think of their essential meaning, it gets a bit easier to write markup. Think of your chunks of content in terms of how they relate to each other and in which contexts they can be used. In our article archive, we’ve covered a lot of elements with examples of their use. If you’re ever in doubt, I highly recommend our flowchart of HTML5 sectioning elements to help you along.

Try to keep things simple. Overthinking your markup will only cause more problems than it’s worth. And let’s face it: it’s not like you can’t change it later!

You may also be interested in adopting Microformats, something Dr Oli Studholme has written about here on HTML5 Doctor. Extending HTML5 — Microformats is recommended reading for those who want to take their markup a bit further. Dr Oli has also looked at Extending HTML5 with Microdata, which gives us a whole new way to add extra semantic information.

Finally, try utilising HTML5 markup in ways that gain the benefit of these new elements. Create tools, use your own documents to test how well-formed and structured they are, and test how portable your content really is. Who knows, you may even find yourself writing a proposal for the next addition to the spec.

Do you know something the web needs that’s missing from HTML5? Let us know in the comments.

Tasks #

Let's break this process down into specific tasks:

1. Capture the drop event and read its data.
2. Post that binary data to the server.
3. Provide feedback on the progress of the upload.
4. Optionally render a preview of what's being uploaded and its status.

To achieve all of this, we need the following HTML5 and non-HTML5 APIs:

1. Drag and Drop
2. FormData
3. XHR progress event
4. FileReader

Keep in mind that not all browsers support all of this technology today, but they're getting close. I just wanted to create a clear and complete tutorial on how to go the whole hog and upload that dropped file.

The end result: we're able to drop the file anywhere in the browser, we get a preview and progress of the upload, and it's a slick experience.

Drag and Drop #

Just as a forewarning, drag-and-drop has been known to be a bit of a killjoy. In fact, it's a nightmare, but I won't repeat what's been said before.

Our example is going to allow you to drag-and-drop a file anywhere within the browser. To achieve that, we need to attach our event listeners to the body or documentElement (i.e., the root HTML node). By listening on the documentElement, this code could be anywhere in the page, as documentElement will always exist. You can only listen on body once the <body> element has been encountered.

Here's the pattern of code we need to capture the drop event on the entire document:

var doc = document.documentElement;
doc.ondragover = function () { this.className = 'hover'; return false; };
doc.ondragend = function () { this.className = ''; return false; };
doc.ondrop = function (event) {
  event.preventDefault && event.preventDefault();
  this.className = '';

  // now do something with:
  var files = event.dataTransfer.files;

  return false;
};

I'm using the hover class so that I can toggle a tip explaining to the user that the file can be dropped on the page.

Inside the drop event, we can access the dropped files via event.dataTransfer.files.

var dndSupported = function () {
  var div = document.createElement('div');
  return ('draggable' in div) || ('ondragstart' in div && 'ondrop' in div);
};

if (!dndSupported()) {
  // take alternative route
}

document.getElementById('upload').onchange = function (event) {
  // `this` refers to the element the event fired upon
  var files = this.files;
};

Automatically Uploading the File #

This is the neat bit, and it's painfully simple. We use a feature of the XHR2 spec: FormData. Once we create an instance of FormData, we can append items to it:

var formData = new FormData();
for (var i = 0; i < files.length; i++) {
  formData.append('file', files[i]);
}

// now post a new XHR request
var xhr = new XMLHttpRequest();
xhr.open('POST', '/upload');
xhr.onload = function () {
  if (xhr.status === 200) {
    console.log('all done: ' + xhr.status);
  } else {
    console.log('Something went terribly wrong...');
  }
};

xhr.send(formData);

Yeah, that's it.†

Let's look at a couple of key things that are going on the above code.

formData.append('file', files[i]);

We're sending named parameters to our server, specifically an array of values called file. Obviously, you can call it what you want, but the file is the name your server will be looking for when it saves the uploaded file (or files).

xhr.onload = function () {
  if (xhr.status === 200) {
    console.log('all done: ' + xhr.status);
  } else {
    console.log('Something went terribly wrong...');
  }
};

If you're familiar with XHR, you'll notice we aren't listening to onreadystatechange, only onload — which is (in effect) a convenience function to let you know when the readyState is 4 (i.e., loaded!). You should still check and respond appropriately to the status code of the request to ensure it's 200 OK, rather than a 500 (internal server error) or 404 (not found) or anything else.

xhr.send(formData);

The nice trick here is that XHR has automatically set the encoding of the posted data to multipart/form-data. This encoding allows your server to read and save the files. It's like the encoding used when you send an attachment in an email.

Providing Feedback to the User #

XHR2 now (flippin' finally) comes with a progress event. So if you're sending or retrieving a large file via XHR, you can tell the user how far along you are.

It's pretty simple. If you want to track the progress of the XHR request, listen to the progress event. One gotcha that caught me out for some time: I needed to track the progress of the upload, not the download. To do this properly, you need to listen for progress on the XHR upload object, as so:

xhr.upload.onprogress = function (event) {
  if (event.lengthComputable) {
    var complete = (event.loaded / event.total * 100 | 0);
    progress.value = progress.innerHTML = complete;
  }
};

xhr.onload = function () {
  // just in case we get stuck around 99%
  progress.value = progress.innerHTML = 100;
};

Note that instead of xhr.onprogress, we use xhr.upload.onprogess. When this event fires, we check that the event supports calculating the amount of data uploaded (the lengthComputable part), and then calculate the amount completed.

In my HTML, I'm using a <progress> element (similar to the <meter> element, but more semantically appropriate as we're presenting the progress of a task) to show the user how much of their file has been uploaded:

<progress id="progress" min="0" max="100" value="0">0</progress>

Note that I'm using innerHTML for browsers that don't yet support <progress>. Then with a dash of CSS-generated content, both the innerHTML and value of the progress can be the same (perhaps an over optimisation, but I was rather proud of myself at the time!).

And Finally, Rendering a Preview #

As a nice addition, we'll give the user a preview of what we're uploading for them. It requires the File API — specifically the FileReader API.

As the drag-and-drop operation (or the input[type=file]) contains a file object, we can hand this over to the FileReader to get the contents of the file. If it's something like an image, we can get the Base64 data and give the user a preview. Or if it's text, we could render it in a <div>. The MIME type for the file is available as the type property on the file object.

So let's assume we only accept images. Here's the preview part that runs after the file has been received by the browser:

var acceptedTypes = {
  'image/png': true,
  'image/jpeg': true,
  'image/gif': true
};

if (acceptedTypes[file.type] === true) {
  var reader = new FileReader();
  reader.onload = function (event) {
    var image = new Image();
    image.src = event.target.result;
    image.width = 100; // a fake resize
    document.body.appendChild(image);
  };

  reader.readAsDataURL(file);
}

We create the FileReader and give it a file to read. In the above example, I've used readAsDataURL, but you can also read files in plain text and binary formats (as per the spec).

When the reader has read the file and the data is available, it fires the load event, which in turn creates our new image.

The result of the file read action is in the event.target.result property. In the case of the readAsDataURL, the value is along the lines of data:image/png;base64,ABC....

Using All the Tools in the Shed #

This example may be a little contrived for the article that I wanted to write. I've used many different technologies to do something that's a fairly common pattern on the web. In fact, it was because I had trouble finding a definitive source on uploading to the actual server (except, of course, as I wrote this article I found this example from back in late 2010!).

Hopefully, you'll see how each bit of tech can work together to create an application, but equally I hope that you can see where this would work if none or just some of the technology wasn't available. Make sure you detect functionality. Make sure you provide fallbacks. Make sure you develop responsibly.

Here's the final drag-and-drop and upload demo for you to play with.

Getting Started #

First, you need an HTML document:

<!DOCTYPE html>
<html class="no-js" lang="en">
<head>
  <meta charset="utf-8">
  <title>Hello Modernizr</title>
  <script src="modernizr.js"></script>
</head>
<body>
</body>
</html>

You can see in the code above that you need a modernizr.js file. You have to build and download this yourself by choosing the features you want to detect. Do this by choosing the ‘Production’ option in Figure 1 and then ticking the necessary options in Figure 2. This helps keep the file size down by not detecting everything Modernizr is capable of. There is a development version (see Figure 1) that you can link to whilst developing your site, but before you put the site live, you should build the production file you need.

Also notice the second line of the HTML above: there is a no-js class on the <html> element. Modernizr first removes this and replaces it with a js class, which could be useful in your CSS. Along with the js class, it adds classes for all the features the browser supports and for the features it does not support, pre-fixing those with no-.

Here are two examples, one from Chrome 16 and one from IE9:

<html class="js flexbox canvas canvastext webgl no-touch geolocation postmessage websqldatabase indexeddb hashchange history draganddrop websockets rgba hsla multiplebgs backgroundsize borderimage borderradius boxshadow textshadow opacity cssanimations csscolumns cssgradients cssreflections csstransforms csstransforms3d csstransitions fontface generatedcontent video audio localstorage sessionstorage webworkers applicationcache svg inlinesvg smil svgclippaths">

<html class="js no-flexbox canvas canvastext no-webgl no-touch geolocation postmessage no-websqldatabase no-indexeddb hashchange no-history draganddrop no-websockets rgba hsla multiplebgs backgroundsize no-borderimage borderradius boxshadow no-textshadow opacity no-cssanimations no-csscolumns no-cssgradients no-cssreflections csstransforms no-csstransforms3d no-csstransitions fontface generatedcontent video audio localstorage sessionstorage no-webworkers no-applicationcache svg inlinesvg smil svgclippaths">

Do Some Detecting #

Modernizr creates a global Modernizr JavaScript object, which allows us to query different properties of that object to perform feature detection by calling Modernizr.<featurename>. So to test for canvas support, you would write the following:

<script>
  if (Modernizr.canvas) {
    alert("This browser supports HTML5 canvas!");
  }
</script>

So try this page in a modern browser and there will be a nice message for you.

Because you’re all good developers and write unobtrusive, progressive JavaScript so that everyone can use your site, you also want to check if canvas is not supported. You have a couple of options here. You could use the above if statement along with an else statement, like so:

<script>
  if (Modernizr.canvas) {
    alert("This browser supports HTML5 canvas!");
  } else {
    alert("no canvas :(");
  }
</script>

Or if you want to test purely for no canvas support, you can negate the condition like this:

<script>
  if (!Modernizr.canvas) {
    alert("No canvas here");
  }
</script>

Feature Detection and Polyfilling with YepNope #

In the examples above, you’ve seen the simplest way to detect a browser feature. What if you wanted to detect a feature and use a polyfill to make the browser perform better? You can do this with YepNope.

YepNope is a conditional loader, which means it will only load the scripts that are needed by the browser. And it’s built into Modernizr, so you don’t have to worry about downloading and linking to another JavaScript file.

So how do you use it?

Using canvas as an example again, you’ll generally want a fallback for non-supporting IE8 and below. The usual way of doing this would be to link to a JavaScript polyfill, such as FlashCanvas in your HTML:

<script src="http://flashcanvas.net/bin/flashcanvas.js"></script>

The problem with this approach is that all browsers will download this script. That’s unnecessary and should be avoided where possible. You could arguably wrap the <script> element in conditional comments, but if you can keep the files out of the markup altogether, then you should. You can do this using Modernizr.load(). Modernizr has YepNope built into it, so you can easily test for a feature and then supply a polyfill.

So let’s take a look.

The basic use of the .load() function allows you to test for a feature and ask whether it’s true (yep) or false (nope). In this example, Modernizr tests for canvas support, and if the feature doesn’t exist, then it loads FlashCanvas:

Modernizr.load({
  test: Modernizr.canvas,
  nope: 'http://flashcanvas.net/bin/flashcanvas.js'
});

So open up IE8, take a look at the network tab (Figure 5) in the developer tools. With the above code, you’ll see that it downloaded and initialised flashcanvas.js. Pretty nifty, yes?

Here’s a more practical example that detects the support of <input type="date">. If it isn’t supported, it loads in two jQuery files and a CSS file to generate a styled date picker:

<script src="modernizr.js"></script>
<script>Modernizr.load({
  test: Modernizr.inputtypes.date,
  nope: ['http://ajax.googleapis.com/ajax/libs/jquery/1.4.4/jquery.min.js', 'http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.7/jquery-ui.min.js', 'jquery-ui.css'],
  complete: function () {
    $('input[type=date]').datepicker({
      dateFormat: 'yy-mm-dd'
    });
  }
});
</script>

This test looks for <input type="date"> support. When it fails, it loads in the two external jQuery JavaScript files and a local CSS file. (Our tests suggest that the CSS file needs to be local.) Once it’s done that (i.e., on complete), it then calls the plugin for each <input type="date"> in the DOM. In most browsers, the jQuery will be loaded (Figure 6), but in Opera, the files aren’t loaded because it has a native calendar control (Figure 7):

The two figures below show the difference in which resources the browsers downloads:

Notice how the two jQuery JavaScript files are loaded twice. That’s a YepNope behaviour, first loading the resource and then executing it. So don’t worry, that’s normal.

You can do much more with YepNope. Here’s an example taken from the YepNope web site of all the possible properties, all of which are optional:

yepnope([{
    test : /* boolean(ish) - Something truthy that you want to test */,
    yep : /* array (of strings) | string - The things to load if test is true */,
    nope : /* array (of strings) | string - The things to load if test is false */,
    both : /* array (of strings) | string - Load everytime (sugar) */,
    load : /* array (of strings) | string - Load everytime (sugar) */,
    callback : /* function ( testResult, key ) | object { key : fn } */,
    complete : /* function */ }, /* ... */ ]);

Conclusion #

Modernizr is a powerful feature detection library. It allows you to check whether a browser supports various features and, depending on what you want to achieve, lets you use a polyfill. This article looked at how to generate a Modernizr JavaScript file and examined two different ways of using Modernizr: directly using the Modernizr object (Modernizr.<featurename>) and then using the built-in YepNope.

So, what do you think? Let us know in the comments below!

A Recap of Pseudo-Classes #

A pseudo-class is information about an element that’s in the document tree but not available through any specified attributes. For example, the :first-child pseudo-class applies to elements which are the first child of their parents. You can get that information from the DOM tree, but there’s no first-child attribute.

In CSS2.1, there were a handful of pseudo-classes available, notably the link states (:link, :visited) and those of user actions (:active, :hover). In CSS3, that handful becomes a basketful, with pseudo-classes of structure (:nth-child, :nth-of-type), UI element (:enabled, :checked), and the ones we’ll look at now, of form validation.

Form Validation without JavaScript #

So as I mentioned at the beginning, HTML5 introduces client-side form validation without the need for JavaScript. When you attempt to submit a form, your browser will validate all of the fields and return an error if any of the fields is incorrect. This will happen mostly automagically — provided your browser has the capability — and will look something like what’s shown in Figure 1:

Those error messages are browser and OS specific, and hard to modify (which I documented on my blog), but you can change the way the errors appear on the elements themselves with the new validation pseudo-classes, which are part of the CSS3 Basic UI module.

I’ll cover current browser support as we go through the article, but in a nutshell:

• Firefox and IE10 support some of it.
• Opera and Chrome support all of it.
• Safari supports all of it too, but form validation is disabled by default.

That being the case, I’d recommend using Chrome, Opera, or Safari to view the examples I’ll be showing you.

If you want to keep up to date with browser support for form validation, I suggest keeping an eye on When can I use.

So with all of the boring technical guff out of the way, let’s get on with the interesting technical guff and take a look at some examples of form validation.

Required and Optional Elements #

One of the most common patterns of validation is that of mandatory (that is, required) values — the fields that the user must complete in order to progress. To mark a form element as mandatory, you need only add the required attribute to it:

<input type="text" required>

If you want to apply styles to this in order to show that requirement, you can use the new :required pseudo-class, which applies rules to any matched element which has the required attribute. For example, you may want to put an asterisk after the text of the label:

input:required + label::after { content: "*"; }

The exact rules you use depends on how you’ve marked up your document. The rule in this example is based on the markup having this structure:

<input type="text" required id="foo">
<label for="foo">Foo</label>

The opposite of required is, of course, optional. And wouldn’t you know it, we have a pseudo-class for that too, logically named :optional. It affects any form element that doesn’t have a required attribute. For example, you may want to make those have a lighter border:

input:optional { border-color: silver; }

You can see this in action in Figure 2, or take a look at a live example.

Valid and Invalid Elements #

There are other types of validation besides simply “required” or “optional”. You can use pattern matching, such as email address validation:

<input type="email">

If the user enters anything into the field, it has to match the pattern of an email address, or else it will be marked as invalid. (Of course, if it’s not required, then it can be left blank.)

You can style valid or invalid form elements using pseudo-classes called — wait for it — :valid and :invalid. Maybe you want add a symbol depending on the validation status:

input:invalid + label::after { content: ' ⨉'; }
input:valid + label::after { content: ' ✓'; }

Note that these will take effect as soon as the page loads, so you’ll probably want to use JavaScript to apply them only after submission, perhaps by adding a class to the form with JavaScript:

document.forms[0].addEventListener('submit', function(e) {
  e.currentTarget.classList.add('submitted');
  // Your fallback form validation function
});

The exact script you use will vary, but however you do it, you can use that class to style the form elements, like so:

.submitted input:invalid + label::after { content: ' ⨉'; }

Figure 3 shows the output of this, and you can also see the live example.

Two extra things to note about this example: first, in order to use my own JavaScript, I’ve prevented the form from automatically validating by using the novalidate attribute on the submit button; and second, Firefox puts a coloured glow around elements depending on their validation state, which I’ve over-ruled by using their proprietary :-moz-ui-invalid selector. View the source of the demo to see how these are used.

Number Ranges #

Some of the new input types, such as number, allow a range of values using the min and max attributes, like so:

<input type="number" max="10" min="1">

Although form controls will usually prevent the user from entering a value outside of this range, there may be occasions (such as when JavaScript is used to provide values) where the value provided to the input does exceed the range:

<input type="number" max="10" min="1" value="11">

When this happens, you can use the :out-of-range pseudo-class for your styling:

input[type='number']:out-of-range { border-color: red; }

And, as you’d expect, there’s a counterpart known by the name of :in-range:

input[type='number']:in-range { border-color: green; }

In Figure 4 you can see these two pseudo-classes at work, and once more there’s also a live example to view.

Reading and Writing #

You may have a form field that’s pre-filled with a value that you don’t want the user to be able to edit, such as terms and conditions in a textarea. If that’s the case, you can add the readonly attribute:

<textarea readonly>Lorem ipsum</textarea>

You can style elements which have the readonly attribute applied, using…can you guess? You are correct, imagined reader: the :read-only pseudo-class. You could combine it with the user-select CSS property to prevent the user from being able to select the text (in the example, I’ve also changed the border style to make it more obvious):

textarea:read-only { user-select: none; }

And if you should need it, there is of course a :read-write pseudo-class for elements which don’t have the attribute applied:

textarea:read-write { user-select: text; }

You can see the results in Figure 5, and as before there’s a live example too.

Firefox and IE10 have actually implemented the readonly attribute, but don’t yet have support for these pseudo-classes.

Browser Support #

This table lists support for HTML5 Form Validation in browsers at the time of writing:

That’s All from Me #

I hope I’ve given you enough to whet your appetite for finding out more about HTML5 Form Validation. There’s plenty more to it, including a full JavaScript API (you can read about this in the HTML5 spec, but MozDev has better documentation).

Thanks for your time, and thanks to the Doctors for this opportunity.

Presentational and semantic? #

As we all know, presentational stuff belongs in CSS, not HTML. In HTML 4.01, the type attribute was replaced by list-style-type, and the start and value attributes were dropped, with only the potential (although fiddly) replacement in some cases of CSS generated content-based counters. So why would we want to specify “presentational” stuff like a list’s style in our HTML?

<!-- Continuing a previous list with value="" -->
<ol type="I">
  <li value="7">seventh item</li>
  <li>eighth item</li>
  <li>ninth item</li>
</ol>
<!-- Continuing a previous list with start="" -->
<ol type="I" start="7">
  <li>seventh item</li>
  <li>eighth item</li>
  <li>ninth item</li>
</ol>

Counting It Down with reverse #

The new attribute reverse allows us to make ordered lists that count down rather than up. If you’re coding a top ten countdown or you’re a space junkie, it’s the attribute you’ve always wanted.

By default, <ol reversed> starts from the total number of list items and counts down to one, so there’s no need to also specify start unless you want something different to happen. Unfortunately, as we’ll see in the browser support table in just a moment, none of them do yet, so in the meantime you can make a reversed list by manually specifying each list item’s number with value (as you can see in the following example) or via CSS counters.

<!-- default list -->
<ol>
  <li>three</li>
  <li>two</li>
  <li>one</li>
</ol>
<!-- using the reversed attribute (will look the same as the third list in a supporting browser) -->
<ol reversed>
  <li>three</li>
  <li>two</li>
  <li>one</li>
</ol>
<!-- using value attributes on li -->
<ol>
  <li value="3">three</li>
  <li value="2">two</li>
  <li value="1">one</li>
</ol>

Browser support table #

Browsers support the start, type, and value attributes as part of supporting legacy content (HTML 3.2 represent!), so we can use them now.

1. We can use a JavaScript polyfill to get around the lack of native support for the reversed attribute, such as the polyfill (and accompanying article) from Louis Lazaris, or Titani’s polyfills, which both use the value attribute.
2. It’s bug 601912, but there’s no progress as yet
3. Chrome 18 (in the dev channel at the time of writing) has support for reversed (thanks Philip Tellis), as does Safari 5.2 Developer Release (thanks Sam Rayner)

<!-- default (unsupported) reversed attribute -->
<ol reversed>
  <li>three</li>
  <li>two</li>
  <li>one</li>
</ol>
<!-- using a reversed attribute polyfill -->
<ol class="polyfill-me" reversed>
  <li>three</li>
  <li>two</li>
  <li>one</li>
</ol>

You can detect for browser support using Modernizr, via the Community add-on “lists-reversed”. If your polyfill was called reversed.js you could load it using the following code:

yepnope({
  test : Modernizr.olreversed,
  nope : ['reversed.js']
});

The custom Modernizr build for doing this would be:

• Extra > Modernizr.load
• Extensibility > Modernizr.addTest
• Community add-ons > lists-reversed

Conclusion #

These attributes aren’t ones you’ll use often, but sometimes they’ll be just the ticket. They’ll help you avoid adding list item numbering as part of your content (leading to double numbering if your CSS with list-style-type: none; is disabled), or ugly hacks like height:0; on a bunch of filler <li> elements to get the right start value. Even better, type, start, and value are supported and ready to use now.

As usual, just make sure you are only using them where appropriate:

• Only use type if the list style for counters is semantic, and the document’s meaning will change if your CSS (containing the equivalent list-style-type values) didn’t load.
• For start, consider whether your lists could be combined.
• Avoid value if at all possible, as it’s less fragile and error-prone to let the browser number list items for you.
• Unfortunately, the current lack of browser support means you’ll need to polyfill reversed if you want to use it for now.

So what do you think of these new attributes? Have you needed them in the past? Have you actually used some of them back in the day to rock some old-skool HTML 3.2? Will you use them in the future? Let us know in the comments!

Updates #

1. 2012-02-22 I’ve added more browser support information and corrections from the comments. Thanks!

getUserMedia #

getUserMedia is an API that gives a web page access to a user’s camera and microphone via JavaScript. It’s supported in Opera Labs (latest Android build, latest desktop builds) and WebKit in Chrome Canary builds (instructions).

Like many other APIs, it’s not part of the “real” HTML5 spec. It started life as the HTML5 <device> element, then got moved into the W3C as part of the webRTC specifications. But let’s not taxonomise when we could be having fun.

The first thing we need to do when using super-cool new UIs is detect whether it’s supported:

 if (navigator.getUserMedia) {
  // do something cool
} else {
  // fallback code
}

For this article, our “do something cool” will be to grab the dominant colour of the current input stream from the camera. As a fallback, we have several options.

One option, if we’re using the default Android browser, is to show a form that automatically starts the camera when focussed:

<form enctype="multipart/form-data" method="post">
  <input type="file" accept="video/*;capture=camera" />
</form>

There’s also the capture attribute, a proposed extension from the W3C Device API Media Capture API that tells a user agent where to get the input data. Android extends the accept attribute. See more in David Calhoun’s Device API article. Alternative capture arguments are camcorder (for video), microphone, and filesystem.

Browsers that don’t understand the still-draft Media Capture API will simply ignore the capture attrbute and prompt the user to browse to a file on their file system for uploading.

Users on iOS are out of luck, as their devices don’t allow access to the file system.

getUserMedia API #

Let’s assume (because we’re optimists) that your user has a getUserMedia-enabled device. The API is nice and simple, as all Web APIs should be.

navigator.getUserMedia accepts two required arguments and an optional third.

The first argument tells the device which media you require access to, and it’s passed as a JavaScript object. So, if you only require access to the microphone, the first argument would be {audio: true}; for video-chatting, you would use {audio: true, video: true}.

The device decides which camera to use: User agents are encouraged to default to using the user’s primary or system default camera and/or microphone (as appropriate) to generate the media stream.

A previous version of the specification allowed hints to user agents about which camera to use. The API could specify “user” (the front camera on a phone) or “environment” (the rear camera on a phone). Debate continues about whether to reinstate this (read the ongoing conversation). One argument against is that letting sites know what cameras a device has could help Dr Malice of Evil Corp. fingerprint users after luring them to his site (which I find to be overly-cautious). Harald Alvestrand poses a conundrum: conferencing units may have a room camera, a document camera and a lectern camera, neither of which is attached to the physical box; which one of these is front?

But let’s not worry at the moment. Devices with more than one camera generally have a mechanism that allow the user to choose which one is used, so this gives the user control. Note also that the spec says User agents may allow users to use any media source, including pre-recorded media files.

The second argument is the success callback, the function to be executed on success, assuming that the user allows access and the device supports your request.

There is an optional third argument. This is the failure callback, the function to be executed if something went wrong. It’s optional, but only optional in the sense that washing your hands before you eat is optional: if you don’t do it, you leave yourself open to all sorts of bugs and your mum will shout at you.

Of course, it’s important to do feature detection before making an API call to ensure that the browser and device are capable, but even if feature detection succeeds, there is still much that can cause failure. For example, the user could deny permission or turn off the camera via a hardware switch, or the device itself could disable the camera.

So here’s how we recommend you use the API:

navigator.getUserMedia({audio: true, video: true}, success, error);

function success(stream) { 
  // ... use 'stream' ... 
} 

function error(){ 
  // display fallback content 
}

To check whether it’s working correctly, we can attach the output of the camera’s stream to the input of an HTML5 video on the page, like this:

<video autoplay></video>
<script>
if (navigator.getUserMedia) {
  navigator.getUserMedia({audio: true, video: true}, success, error);
  function success(stream)
  { 
    // ... use 'stream' ...
    var video = document.getElementsByTagName('video')[0];
    video.src = stream; 
  }
  // ...
</script>

video = document.getElementById('video');

Possible Applications #

A few simple examples of applications that could make use of Server-Sent Events:

• A real-time chart of streaming stock prices
• Real-time news coverage of an important event (posting links, tweets, and images)
• A live Twitter wall fed by Twitter’s streaming API
• A monitor for server statistics like uptime, health, and running processes

We’ll use the server monitor for this article’s examples. If this application were to be used in the wild, we could also check the EventSource‘s connection state to indicate when there’s a potential problem connecting to the server.

Overview of the API #

The client-side API is rather simple, and it hands-down beats the insane hacks required to get real-time events to the browser back in the bad old days.

The main points of interest:

• new EventSource(url) — this creates our EventSource object, which immediately starts listening for events on the given URL.
• readyState — as with XHR, we have a readyState for the EventSource that tells us if we’re connecting (0), open (1), or closed (2).
• onopen, onmessage — two events that we can listen for on the new EventSource object. By default, the message event will fire when new messages are received, unless the server explicitly sets the event type.
• addEventListener — not only can we listen for the default message event, but we can also listen for custom messages using the addEventListener on the EventSource object, just as if we were listening for a click event.
• event.data — as with most messaging APIs, the contents of the message reside in the data property of the event object. This is a string, so if we want to pass around an object, we need to encode and decode it with JSON.
• close — closes the connection from the client side.

In the future, EventSource will also support CORS using an options argument to the EventSource object: { withCredentials: true }. But at the time of writing, no stable release includes this property.

Simple Example #

Our simple web app will notify us of server status messages — things like the load average, number of currently connected users, and most CPU-intensive processes. If I were using this application in anger, I’d probably build server modules that emit specific event types when they cross specific thresholds, so that I’m only notified when something gets to warning level.

This snippet of JavaScript connects to our server, listens for messages, and handles the data that comes with the messages:

var source = new EventSource('/stats');
source.onopen = function () {
  connectionOpen(true);
};

source.onerror = function () {
  connectionOpen(false);
};

source.addEventListener('connections', updateConnections, false);
source.addEventListener('requests', updateRequests, false);
source.addEventListener('uptime', updateUptime, false);

source.onmessage = function (event) {
  // a message without a type was fired
};

Properties of Server-Sent Events #

Server-Sent Events are more than just a one-way web socket. They have some unique features:

• The connection stream is from the server and read-only. This suits lots of applications, some examples of which I listed above.
• They use regular HTTP requests for the persistent connection, not a special protocol, which means we can polyfill using vanilla JavaScript.
• If the connection drops, the EventSource fires an error event and automatically tries to reconnect. The server can also control the timeout before the client tries to reconnect.
• Clients can send a unique ID with messages. When a client tries to reconnect after a dropped connection, it will send the last known ID. Then the server can see that the client missed n messages and send the backlog of missed messages on reconnect.

Message Format #

A simple message doesn’t require much:

data: this is a simple message
<blank line>

Note that the end of a message is indicated by a blank line (obviously not the literal characters <blank line>).

For a message with multiple lines:

data: this is line one
data: and this is line two

You can send message IDs to be used if the connection is dropped:

id: 33
data: this is line one
data: this is line two

You can even send multiple messages in a single response so long as you separate the messages by blank lines:

id: 34
data: Remy is awesome

id: 35
data: Bruce is stinky

And you can specify your own event types (the above messages will all trigger the message event):

id: 36
event: price
data: 103.34

id: 37
event: news
data: Bruce sells his collection of replica bananas

You don’t have to worry about this structure on the client side. It only applies to the server, which I’ll touch on next.

Typical Server #

I’m not going to give a full walkthrough of the server-side code, since this is an HTML5 web site :) But there are a few important and simple features that you need to know to build the server (you’ll need this part anyway if you’re going to use EventSource).

I’ve included all the files for this demo on GitHub for you to peruse at your own leisure, and I’ve also deployed a live example of this code.

Ideally, you should use a server that has an event loop. This means you should not use Apache, but instead use a platform such as Node.js (which I’ve used) or Twisted for Python.

Key properties:

1. You can only accept EventSource requests if the HTTP request says it can accept the event-stream MIME type.
2. You need to maintain a list of all the connected users in order to emit new events.
3. You should listen for dropped connections and remove them from the list of connected users.
4. You should optionally maintain a history of messages so that reconnecting clients can catch up on missed messages.

Here’s a sample of my Node.js based server. It’s using Connect (a simple webby framework for Node). When it receives a request for /stats, it calls the following function. I’ve commented the code so you can follow along:

function stats(request, response) {
  // only response to an event-stream if the request 
  // actually accepts an event-stream
  if (request.headers.accept == 'text/event-stream') {

    // send the header to tell the client that we're going
    // to be streaming content down the connection
    response.writeHead(200, {
      'content-type': 'text/event-stream',
      'cache-control': 'no-cache',
      'connection': 'keep-alive'
    });

    // support the polyfill - we'll come on to this later
    if (request.headers['x-requested-with'] == 'XMLHttpRequest') {
      response.xhr = null;
    }

    // if there was a lastEventId sent in the header, send
    // the history of messages we've already stored up
    if (request.headers['last-event-id']) {
      var id = parseInt(request.headers['last-event-id']);
      for (var i = 0; i < history.length; i++) {
        if (history[i].id >= id) {
          sendSSE(response, history[i].id, history[i].event, history[i].message);
        }
      }
    } else {
      // if the client didn't send a lastEventId, it's the
      // first time they've come to our service, so send an
      // initial empty message with no id - this will reset
      // their id counter too.
      response.write('id\n\n');
    }

    // cache their connection - the response is where we write
    // to send messages
    connections.push(response);

    // send a broadcast message to all connected clients with
    // the total number of connections we have.
    broadcast('connections', connections.length);

    // if the connection closes to the client, remove them
    // from the connections array.
    request.on('close', function () {
      removeConnection(response);
    });
  } else {
    // if the client doesn't accept event-stream mime type,
    // send them the regular index.html page - you could do
    // anything here, including sending the client an error.
    response.writeHead(302, { location: "/index.html" });
    response.end();
  }
}

The important trick on the server is to ensure you don’t close the connection to the EventSource object. If you do, it will of course handle the closed connection, fire an error event, and try to reconnect. So you’ll just want to maintain the persistent connection.

Polyfills and Tweaks to the Server #

There are two good polyfills I know of, and though I prefer the one I wrote, I’ll still lay them both out for you.

// send the data (event type, id, message, etc)
response.write(data);

// if our response contains our custom xhr property, then...
if (response.hasOwnProperty('xhr')) {
  // clear any previous timers using the xhr prop as the value
  clearTimeout(response.xhr);

  // now set a timer for 1/4 second (abritrary number) that
  // then closes the connection and removes itself from the
  // connection array.
  // The delay is in place so that a burst of messages can 
  // go out on the same connection *before* it's closed.
  response.xhr = setTimeout(function () {
    response.end();
    removeConnection(response);
  }, 250);
}

Bugs? #

The error event should always fire when the readyState changes, assuming you didn’t explicitly call the close method. This works nearly all the time, but in writing this article, I found a few edge cases where it doesn’t fire. In Chrome, if I put my machine to sleep and then woke it back up, it would close the connection but not fire the event, therefore never triggering a reconnection. As I said, this is an edge case and I’ll file a bug against it, so I don’t expect it to hang around for long.

Why Not Use WebSockets? #

There are two reasons I’d advocate using EventSource over WebSockets, as they’re currently the two contenders for sending real-time events to the browser.

The first is that EventSource (as we saw earlier) works over regular HTTP and can therefore be replicated entirely using JavaScript if it’s not available natively. That means that we can polyfill browsers without support, like IE9.

The second is probably more important: you should always use the right technology for the job. If your real-time data is sourced from your web site, and the user doesn’t interact in real-time, it’s likely you need Server-Sent Events.

I recently saw a cool demo of snowflakes drifting down a web site. Each snowflake is a tweet based around the Christmas theme — like if someone mentions a particular Christmas-y term, it’s sucked in to the snowflake. Don’t get me wrong, I know this is a demo, and it’s very cool (if you wrote it, this is me sending you hugs), but it’s based on WebSockets. I’d suggest this demo should be based on EventSource since all the data is read-only and the user doesn’t interact with it at all.

The point: evaluate the technology against your problem, and aim to get good fit.