This post is about the Web Storage API. Technically it’s been shifted out of the HTML5 specification and can now be found in it’s very own dedicated spec. But if it counts at all – it used to be part of the Web Applications spec.

Web Storage is a very, very simple way to store data in the client – i.e. the browser. What’s more, the support is fabulous: IE8 and upwards has support natively, and there’s lots of good polyfills in the wild already.

This post however will just focus on the features of Web Storage and hopefully show you a trick or two you may not have known about.

What Web Storage Does

Using JavaScript, Web Storage makes it possible to easily store arbitrary values in the browser. Storage is different from cookies in that it’s not shared with the server. It’s also different from cookies in that it’s dead simple to work with.

There are two versions of Web Storage: local and session. Local means that it’s persistent, and session is simply that the data is lost when the session ends (i.e. when you close you browser window or tab). It’s also worth noting that a session is tied to a single browser window or tab. The session doesn’t leak out in to other open windows on that same domain.

Any data stored is tied to the document origin, in that it’s tied to the specific protocol (http or https, etc), the host (html5doctor.com) and the port (usually port 80).

Get Storing

The API for localStorage and sessionStorage is exactly the same, and distills down to the following methods:

• .setItem(key, value);
• .getItem(key)
• .removeItem(key)
• .clear()
• .key(index)
• .length

You can probably guess what most of those methods do, maybe not the last two, but ignore all that syntax for a moment.

Due to the way that the storage API has been defined, the set/get/remove methods all are getters, setters and deleters. What that means to you is you can use localStorage as follows:

localStorage.name = 'Remy';

If you tried using the link above, this will save a property called “name” against localStorage. Completely closing your browser, and going back to http://jsconsole.com and test the name property again:

console.log(localStorage.name);

Still holds the value doesn’t it? Pretty easy eh? And no faffing around with cookies, for that, we can be thankful!

Equally deleting data is very easy:

delete localStorage.name;
console.log(localStorage.name);

You’ll see the value is undefined.

What’s happening under the hood is that when you set a property on localStorage (or sessionStorage) it’s calling the .setItem method. When you get a property, it’s calling .getItem and when you delete, it’s calling removeItem. That way you can treat the Web Storage interface as any other regular object in JavaScript, except you’ll know it’ll hang around after the page has unloaded.

Hey everyone! I’m storing some data!

That’s what your browser does when you store some data. It announces it via events.

When you set some data on localStorage or sessionStorage, an event, called “storage”, fires on all the other windows on the same origin.

To listen for events the following code is used:

function storageEvent(event) {
  event = event || window.event; // give IE8 some love
  alert('Yo people! Something just got stored!');
}

if (window.attachEvent) { // ::sigh:: IE8 support
   window.attachEvent('onstorage', storageEvent);
} else {
    window.addEventListener('storage', storageEvent, false);
}

Note that as IE8 has support, if you don’t want to use onstorage you’ll want to double up your event listeners with attachEvent.

Clearly a massive alert box isn’t useful, but what is useful is what you capture inside of the event object:

• key – the property name of the value stored
• newValue – the newly set value (duh!)
• oldValue – the previous value before being overwritten by .newValue
• url – the full url path of the origin of the storage event
• storageArea – the storage object, either localStorage or sessionStorage

With this information, you can do a lot with the storage event, like perhaps keep tabs that are open synchronised. Here’s a simple, contrived example of using localStorage to echo a value across different windows on html5demos.com.

Or for a real world example, Font Deck recently implemented this very feature, if you change the sample text in one window, all other open tabs (or windows) mirror that same sample text.

Gotchas to watch out for

The main gotcha with Web Storage is that although the specification used to say^† that any object type can be stored, in fact all browsers currently coerce to strings. That means if you want to store a JavaScript object (or an array perhaps), you’ll need to use JSON to encode and decode:

var doctors = [
  'rem', 
  'rich_clark', 
  'brucel', 
  'jackosborne', 
  'leads', 
  'akamike', 
  'boblet'];
localStorage.doctors = JSON.stringify(doctors);

// later that evening…
var html5docs = JSON.parse(localStorage.doctors);
alert('There be ' + html5docs.length + ' doctors in the house');

† Edited thanks to feedback from zcorpan

When working with storage events, a couple of things to also watch out for – this is on purpose of course, but still just about qualify for gotcha status:

1. The event only fires on the other windows – it won’t fire on the window that did the storing.
2. The event won’t fire if the data doesn’t change, i.e. if you store .name = ‘Remy’ and set it to ‘Remy’ again it won’t fire the storage event (obviously, since nothing was stored).

and finally…

If you want to use web storage in browsers like IE6 and IE7 (or rather than want: need to in your particular case) – then there’s no shortage of polyfills for Web Storage. However be wary that you’ll need to stick to the regular setItem syntax rather than being able to use the sexy setter/getter syntax – and I’ve yet to see a polyfill that supports the storage events.

Maybe that isn’t a problem for you though – maybe the sheer simplicity and spiffingness of Web Storage is enough to just support IE8 and above with this enhanced client storage model.

Storing Data the Simple HTML5 Way (and a few tricks you might not have known) originally appeared on HTML5 Doctor on August 23, 2011.

In the last year, riding the popularity of HTML5, a great number of JavaScript polyfills/shim/shams/what-have-you have been released, and the Modernizr project even maintains a list of these scripts.

But we've gone one step further.

What if you could include a single JavaScript library, and it automatically detected the state of the browser support for the latest technology, and where the browser is lacking, the JavaScript library plugs the missing gaps.

An über polyfill library if you will!

That's exactly what we've been working on for the last few months. Here today, we give you a new service from the HTML5 Doctors that will give your old and busted browser a kick up the backside and finally bring it up to scratch.

It goes without saying that this is a fairly mammoth task, so this is just the beta of the service, but we felt that it was in a good enough state to release to our loyal readers.

The Über Polyfill

So enough jibber-jabber, right? Let's get down and dirty with how to get using this library today.

<script src="http://html5doctor.com/uber.js?all"></script>

That's it. By including this script in to a browser like, say IE6, it will have access to Web Storage, WebSockets, Web Forms and more.

We've also configured the service so that you can include just specific pieces of technology - so if you just want Web Forms, you would do:

<script src="http://html5doctor.com/uber.js?webforms"></script>

And if you wanted to mix Web Forms with full CSS3 support:

<script src="http://html5doctor.com/uber.js?webforms,css3"></script>

And so on.

This is a hosted service by HTML5 Doctor, so you'll need to hotlink the script in your site. It needs to be included before you try to access the "native" API in your JavaScript. So you can include this service in the head element, or just before your own JavaScript - as you like.

Support

Currently the service supports the following specifications:

• All of CSS3 (including selectors)
• PostMessage and related messaging APIs
• RDFa
• Includes all the new HTML5 elements - article, section, etc
• Live connections via: XHR2, EventSource and WebSockets
• File API
• Offline support
• Offline events
• Loading web pages APIs, most importantly, the History API (ref)
• Storage API, including the events (which most browsers currently lack)

Obviously sexy HTML5 things like native video and audio are supported, and specifically, we've been able to ensure that all browsers support WebM (yep, even gave Safari a bit of kick to support WebM) that way you only need to encode your videos once (finally). There's lots more we're planning to do in the future to make developing for the web with HTML5 so much easier.

The Future

There's still a lot that can be done with this service, and we're already working on the next exciting feature will be the full DAP implementation - yes that does mean you'll be able to access your camera from an input element:

<input type="file" accept="image/*;capture=camera">

Pretty exciting stuff, eh? Let us know what you'd like to be included that isn't in this and we'll look at including it in the full release.

How to get all the browsers playing ball originally appeared on HTML5 Doctor on April 1, 2011.

This post will be an overview of the technologies available, how well they’re currently supported, and, where possible, live demos. I’m going to touch on the following technologies:

• XHR & XHR2 with CORS
• Web Messaging
• Web Sockets
• Server Sent Events
• Web Workers

Before I get on to the APIs, I want to outline a fairly common communication model that several of these APIs use.

A common communication event model

All event handlers (with the exception of XHR) receive an event object containing a data property. This property includes the data sent as part of the message.

The event model (again with the exception of XHR) is mostly based around onmessage and postMessage or send. For example:

// in the recipient code
recipient.onmessage = function (event) {
  console.log('received message: ' + event.data);
};

// from the sender code
recipient.postMessage('hi there'); // or recipient.send('hi there');

This is just a common model and isn’t the exactly the same among all these technologies. The two key similarities are that they use:

• a sending method (postMessage or send) on the recipient object, and
• an event handler that listens for the message event and receives an event object containing a data property.

Very importantly, most browsers only support sending strings from sender to recipient, so we often need to JSON stringify and parse if we want to send anything other than a string.

XHR & XHR2 with CORS

XHR can be both synchronous and asynchronous. XHR is the only API that (purposely) supports synchronous requests, meaning the execution of code will block until the callback fires.

There’s nothing particularly new about XHR, but in XHR2 we can handle uploads, and there’s a progress event to tell you how the upload or download is getting on.

The super shiny new toy in XHR2 is its support for Cross-Origin Resource Sharing (CORS). This means you can make an XHR request across domains, but only if the server you’re connecting to allows it.

The request is as you’d expect from XHR:

var client = new XMLHttpRequest();
client.onreadystatechange = function () {
  if (this.readyState == 4 && this.status == 200) {
    alert('The most awesome-est person to follow: ' + this.responseText);
  }
};
client.open('GET', '/no-cors');
client.send();

If our server responds with a CORS header, however, we can put our XHR responder on another server. So on the URL http://remysharp.com/demo/cors.php, I have the following PHP script:

<?php
header('Access-Control-Allow-Origin: *');
?>
@rem

This says that anyone can make an XHR request to this particular script. Now when I run the following code in a browser that supports XHR2, the cross domain request succeeds!

var client = new XMLHttpRequest();
client.onreadystatechange = function () {
  if (this.readyState == 4 && this.status == 200) {
    alert('The most awesome-est person to follow: ' + this.responseText);
  }
};
client.open('GET', 'http://remysharp.com/demo/cors.php');
client.send();

Here’s a live example of CORS. (You can also edit it here.)

Note that IE8 supports CORS, but not XHR2 (no surprise there then). You need to use their proprietary (booo!) XDomainRequest object. Nicholas C. Zakas has an excellent article explaining how to handle these differences.

XHR usage is pretty common already, but XHR2 with CORS is a winner over JSON-P, particularly as you have finer control over the request, can handle timeouts, and can handle errors correctly.

Support for XHR & XHR2 with CORS

• XHR support is pretty solid nowadays (even though IE6 uses ActiveXObject to get it going)
• XHR2 with CORS: Safari & MobileSafari, Firefox 3.5, Chrome and IE8 (via XDomainRequest)

postMessage

This API is older, but it’s very useful if you want to get around the XHR same-origin rules. If you have an <iframe> document that can accept onmessage events from your origin (i.e., your site), then you can communicate across domains (and origins).

For example, a page that accepts an onmessage event might contain code such as this:

window.onmessage = function (event) {
  if (event.origin == 'mytrustedsite.com') {
    alert('my trusted site said: ' + event.data);
  }
};

Now you can include an <iframe> that contains that code, and using the <iframe> DOM node, you can post to the <iframe>:

// where iframe is the actual iframe DOM node
iframe.contentWindow.postMessage("hello there", location.host);

This gives you the ability to send strings across two mutually trusted domains. (Remember that you can use JSON.stringify and JSON.parse to convert to an object to and from string format.)

Support for postMessage

• Chrome
• Safari
• Opera
• Firefox
• IE8

Here’s a demo using postMessage.

There’s also a project called EasyXDM, which adds cross domain messaging to IE6 and upwards (along with all the other browsers) through the library’s abstraction. Definitely worth a look if this is a route you need to take.

Web Sockets

It’s my opinion that Web Sockets replaces Comet. Comet is a way of hacking the browser to giving us real-time server messages. The Web Sockets API provides that natively.

Web Sockets are used to send messages to and from the server — i.e., a bi-directional socket. In contrast to other similar technologies, with Web Sockets, you can go across domains, and you’re not bound by the same-origin policy. This means you can host your normal “apps” server while another server is for streaming content. Or you could host your own pages and connect to a live Twitter stream if your users turn on Web Socket support.

You can only send messages once the socket is open (duh). The communication model looks like this:

var ws = new WebSocket('ws://somesite.com/updates');

ws.onmessage = function (event) {
  alert(event.data);
};

ws.onopen = function () {
  ws.send('yay! we connected!');
};

Once the socket is closed, you can’t reuse it. Similarly, there’s no explicit method for opening a socket. That just happens when you create the WebSocket object.

This API is extremely simple. I most often get asked, “What do you put on the server side?” I personally use Node, but you could use an Nginx server or something like Jetty. I’m no expert on the latter servers, but I can vouch that a Node-based server is very, very simple to get going. The live demo below also includes a link to the code that I used to run the server.

Check out this demo of Web Sockets.

Support for Web Sockets

• Chrome
• Safari & MobileSafari

There’s also an excellent Flash poly-fill called web-socket-js. Drop this into an application and it provides Web Sockets support as if it were native. I’ve used this on a few projects of my own, and it works very well.

In early December 2010, there was a security notice posted about Web Sockets, and both Firefox and Opera pulled it from their upcoming releases. Mozilla have said that they expect Web Sockets to be back in Firefox by version 4.0.1.

Server-Sent Events

The Server-Sent Events API is something that originated from Opera back in 2006 and is used for pushing events from the server to the client. Note that the client cannot send messages to the server through an EventSource (SSE) — it can only listen for messages.

This API uses the onmessage model. It’s constructed using the EventSource object and is limited by the same origin rules:

var es = new EventSource('/sse');
es.onopen = function () {
  console.log('opened stream');
};

es.onmessage = function (event) {
  console.log('new message: ' + event.data);
};

The SSE automatically connects when you create the object (similar to Web Sockets), and once open will trigger the onopen event.

Here’s a live demo of Server-Sent Events

Here’s how this is to work: when a new message is pushed from the server to the client, it triggers the onmessage callback.

The key to your server is ensuring it doesn't close the connection on the client - the browser. Most of the examples around the web are doing this: closing the connection which tells the API to switch in to polling mode (note that this is the exact problem I hit when I first published this article).

When the API is in polling mode, it's no more different from an XHR poll, and the onopen will continually fire.

Support for Server-Sent Events

• Opera 11
• Safari & MobileSafari
• Chrome

Web Workers

Web Workers are a way of creating a new thread of execution inside the browser. I’m including this because you still need to communicate with your Web Workers, and the method of communication is similar to some of those techniques discussed above. Do note, however, that this is not a method communicating from a client (browser) to a server. It’s more like there’s another browser window executing a particular block of JavaScript.

As an example of when to use a Web Worker, say you’re running a lot of JavaScript and the UI becomes unresponsive. The browser UI hangs because, in a way, it’s a “single-threaded application”. (Under the hood it isn’t really, but from a rendering and JavaScript perspective it is). This JavaScript task could be given to a Web Worker so that the UI can continue functioning.

It’s vital to understand that a Web Worker lives in a sand-boxed environment that doesn’t have access to things like the DOM. What’s more, you can only communicate with it using the onmessage and postMessage functions.

Our application can create and send messages to a worker using the following code:

var worker = new Worker('bigjob.js');
worker.onmessage = function (event) {
  alert('Message from worker: ' + event.data); // remember event.data is a string!
};

worker.postMessage('task=job1');

In the JavaScript file bigjob.js, we run some computationally intensive task and listen for messages in a way similar to what we’ve done in the previous examples. We can also post messages back to the invoking application:

this.onmessage = function (event) {
  var job = event.data;
  if (job == 'task=job1') {
    job1();
  } else if (job == 'task=job2') {
    job2();
  }
};

// just a pseudo example
function job1() {
  // do some big task
  while (working) {
    // continue task
    this.postMessage('job1 ' + amountComplete + '% complete');
  }
  this.postMessage('job1 100% complete');
}

There’s a lot more to Web Workers than just running a couple of small tasks, and no doubt we HTML5 Doctors will be posting a detailed article soon. This example just demonstrates how to communicate with Web Workers and how similar that is to the other technologies we’ve discussed here.

Support for Web Workers

• Chrome
• Safari
• Opera
• Firefox

A final word

Hopefully you agree that this is just the tip of the iceberg of communicating between client and server in HTML5. We’re no longer stuck with the same origin policy we had with vanilla Ajax. In fact, when you think about it, since IE8, we’ve actually had decent cross-domain messaging.

I’m personally most excited about Web Sockets and the support of CORS in services that have APIs, like Flickr, Twitter, and URL shorteners. What could you build with this?

Methods of communication originally appeared on HTML5 Doctor on January 18, 2011.

Instead of elements, these name-value pairs are defined via attributes:

• itemscope — defines a group of name-value pair(s), called an item
• itemprop="property-name" — adds a property to a microdata item. The name of the property can be a word or URL, and the value is the ‘content’ of the element with this attribute:
  • For most elements, the value is the element’s text content (not including any HTML tags)
  • For elements with a URL attribute, the value is the URL (<img src="">, <a href="">, <object data="">, etc.)
  • For the <time> element, the value is the datetime="" attribute
  • For <meta itemprop="" content="">, the value is the content="" attribute
• itemref="" — allows a microdata item to include non-descendent properties by referring to the ids of element(s) containing them
• itemtype="" — defines the item’s type when used on the same element as itemscope. The itemtype value is a URL that acts as an identifying vocabulary name.
• itemid="" — allows a vocabulary to define a global identifier for a microdata item, for example an ISBN number on a book. Use itemid on the same element as the item’s itemscope and itemtype attributes.

Let’s go through these new attributes and see how to use them in practice with everyone’s favourite example band, Salter Cane.

<p itemscope>I’m going to the <span itemprop="name">Salter Cane</span> gig next week.
Excited!</p>

<p itemscope>I’m going to the <a itemprop="url"
  href="http://www.saltercane.com/">Salter Cane</a> gig 
  <time itemprop="date" datetime="2010-07-18">next week</time>. Excited!</p>

<p itemscope>I’m going to the <a itemprop="url" href="http://www.saltercane.com/"><span itemprop="name">Salter Cane</span></a> gig <time itemprop="date" datetime="2010-02-18">next week</time>. They’re gonna rawk!</p>

<p itemscope>The <span itemprop="name">Salter 
  Cane</span> drummer is <span itemprop="members"
  itemscope><span itemprop="name">Jamie
  Freeman</span>.</span></p>

<span itemprop="members" itemscope>The band members are
  <span itemprop="name">Chris Askew</span>,
  <span itemprop="name">Jeremy Keith</span>,
  <span itemprop="name">Jessica Spengler</span> and
  <span itemprop="name">Jamie Freeman</span>.</span>

<p itemscope><span itemprop="guitar vocals">Chris Askew</span>
  is so dreamy.</p>

<p itemscope itemref="band-members">I’m going to the
  <a itemprop="url" href="http://www.saltercane.com/">
  <span itemprop="name">Salter Cane</span></a> gig
  <time itemprop="date" datetime="2010-07-18">next 
  week</time>. Excited!</p>
  …
  <p>Salter Cane are <span id="band-members"
  itemprop="members" itemscope>
  <span itemprop="name">Chris Askew</span>, 
  <span itemprop="name">Jeremy Keith</span>, 
  <span itemprop="name">Jessica Spengler</span>
  and <span itemprop="name">Jamie Freeman</span>.</span></p>

<p itemscope><span itemprop="name" itemscope>Jessica Spengler
  <meta itemprop="likes" content="Mameshiba"></span>’s fans are always
  really raucous.</p>

<p itemscope><span itemprop="name" itemscope itemref="meta-likes">
  Jessica Spengler<meta id="meta-likes" itemprop="likes" content="Mameshiba">
  </span>’s fans are always really raucous.</p>

<p itemscope itemtype="http://schema.org/MusicGroup"> I went to
  hear <a itemprop="url" href="http://saltercane.com/"><span itemprop="name">
  Salter Cane</span></a> last night. They were great!</p>

<p itemscope>I went to hear <a
  itemprop="http://schema.org/MusicGroup/url"
  href="http://saltercane.com/"><span
  itemprop="http://schema.org/MusicGroup/name">Salter Cane</span>
  </a> last night. They were great!</p>

<p itemscope itemtype="http://vocab.example.com/book"
  itemid="urn:isbn:0321687299">
  <!-- book info… -->
</p>

Microdata in action #

So now that we know how, why would we want to use microdata?

One use is adding extra semantics or data that we can manipulate via JavaScript in a similar way to custom data attributes (data-*). But if we use a vocabulary via itemtype or URL-based itemprop names, microdata becomes considerably more powerful.

While microdata is machine-readable without needing to know the vocabulary, using a vocabulary means others can know what our properties mean. This allows the data to take on a life of its own. Say what? Well, in effect, using a vocabulary makes microdata a lightweight API for your content.

If you visited someone’s homepage, wouldn’t it be great if you could add their contact information to your address book automatically? The same is true for adding an event you’re attending to your calendar. As the syntax examples were a bit example-y, let’s see how to do that using a real-world example — an upcoming event I’m organising (well, it was upcoming!):

<section>
  <h3><a href="http://atnd.org/events/5181" title="WDE-ex Vol11『iPad
  のウェブデザイン：私たちがみつけたこと 』 : ATND">WDE-ex Vol.11 — Designing
  for iPad: Our experience so far</a></h3>
  <p>On <time datetime="2010-07-21T19:00:00+09:00">July 21st 19:00
  </time>-<time datetime="2010-07-21T20:00:00+09:00">20:00</time>
  at <a href="http://www.apple.com/jp/retail/ginza/map/">Apple Ginza</a>,
  <a href="http://informationarchitects.jp/" title="iA">Oliver Reichenstein,
  CEO of iA</a>, will share the lessons they’ve learned while creating
  three iPad apps and one iPad website.</p>
</section>

WDE-ex Vol.11 — Designing for iPad: Our experience so far

On July 21st 19:00-20:00 at Apple Ginza, Oliver Reichenstein, CEO of iA, will share the lessons they’ve learned while creating three iPad apps and one iPad website.

Now we could start making up our own itemprop names on an ad-hoc basis, but this effectively prevents anyone else from using our data. By using a vocabulary and following its rules, others can also use our data. It’s a good idea to use a vocabulary, so where do we find one?

Using schema.org vocabularies #

2011-08-16 The schema.org vocabularies have now superseded the old Google vocabularies, so I’ve updated the article to reflect this.

Bing, Google and Yahoo have collaborated on a set of microdata vocabularies under the name schema.org. By using these vocabularies we can convey semantic information in our content in a way these search engines can understand. While adding semantics using these vocabularies won’t affect your search ranking, the included data may be shown in search results. The main vocabularies schema.org offers are:

• Creative works: CreativeWork, Book, Movie, MusicRecording, Recipe, TVSeries…
• Embedded non-text objects: AudioObject, ImageObject, VideoObject
• Event
• Organization
• Person
• Place, LocalBusiness, Restaurant…
• Product, Offer, AggregateOffer
• Review, AggregateRating

They’re the cross-search engine successors to Google’s earlier Rich Snippets vocabularies. Unlike Rich Snippets, which came in microformats and RDFa versions, schema.org vocabularies controversially only support microdata at the moment.

Taking our fairly standard HTML5 code for Oliver’s iPad event above, let’s add some microdata pixie dust using the schema.org vocabularies. First, the speaker (using the http://schema.org/Person vocabulary) and the venue (using the http://schema.org/Organization vocabulary):

<section>
  <h3><a href="http://atnd.org/events/5181" title="WDE-ex Vol11『iPad
  のウェブデザイン：私たちがみつけたこと 』 : ATND">WDE-ex Vol.11 — Designing
  for iPad: Our experience so far</a></h3>
  <p>On <time datetime="2010-07-21T19:00:00+09:00">July 21st 19:00
  </time>-<time datetime="2010-07-21T20:00:00+09:00">20:00</time> at
  <span itemscope itemtype="http://schema.org/Organization">
  <a itemprop="url" href="http://www.apple.com/jp/retail/ginza/map/">
  <span itemprop="name">Apple Ginza</span></a></span>,
  <span itemscope itemtype="http://schema.org/Person">
  <a itemprop="url" href="http://informationarchitects.jp/" title="iA">
  <span itemprop="name">Oliver Reichenstein</span>, CEO of iA</a>
  </span>, will share the lessons they’ve learned while creating three
  iPad apps and one iPad website.</p>
</section>

While this content will still look the same, adding these microdata items means the following information is now machine readable:

• A company’s name (name)
• That company’s URL (url on <a>)
• A person’s name (name)
• A URL associated with that person (url on <a>)

In JSON (ref: Live Microdata) this would be:

{
  "items": [
    {
      "type": "http://schema.org/Organization",
      "properties": {
        "url": [
          "http://www.apple.com/jp/retail/ginza/map/"
        ],
        "name": [
          "Apple Ginza"
        ]
      }
    },
    {
      "type": "http://schema.org/Person",
      "properties": {
        "url": [
          "http://informationarchitects.jp/"
        ],
        "name": [
          "Oliver Reichenstein"
        ]
      }
    }
  ]
}

So we now have a semantic association between a company and a URL and between a person and a URL. With the right tool, we could add this information to an address book automatically.

Next, let’s add microdata attributes to the event data, using the http://schema.org/Event vocabulary:

<section itemscope itemtype="http://schema.org/Event">
  <h3><a itemprop="url" href="http://atnd.org/events/5181" title="WDE-ex
  Vol11『iPad のウェブデザイン：私たちがみつけたこと 』 : ATND"><span
  itemprop="summary">WDE-ex Vol.11 — Designing for iPad: Our experience so
  far</span></a></h3>
  <p itemprop="description">On <time itemprop="startDate"
  datetime="2010-07-21T19:00:00+09:00">July 21st 19:00</time>-
  <time itemprop="endDate" datetime="2010-07-21T20:00:00+09:00">20:00
  </time> at <span itemprop="location" itemscope
  itemtype="http://schema.org/Organization"><a itemprop="url"
  href="http://www.apple.com/jp/retail/ginza/map/"><span itemprop="name">
  Apple Ginza</span></a></span>, <span itemscope
  itemtype="http://schema.org/Person"><a itemprop="url"
  href="http://informationarchitects.jp/" title="iA"><span itemprop="name">
  Oliver Reichenstein</span>, CEO of iA</a></span>, will share
  the lessons they’ve learned while creating three iPad apps and
  one iPad website.</p>
</section>

WDE-ex Vol.11 — Designing for iPad: Our experience so far

On July 21st 19:00-20:00 at Apple Ginza, Oliver Reichenstein, CEO of iA, will share the lessons they’ve learned while creating three iPad apps and one iPad website.

Now things are a lot more interesting! We can extract the following data:

• Event details
  • Event name (summary)
  • Event start and end time (startDate and endDate)
  • Event summary (summary)
  • Event URL (url on <a>)
  • Event location (location), which is represented by:
    • Company name (name)
    • Company URL (url on <a>)
• Speaker details (well, someone connected with the event, although this connection isn’t explicit)
  • A person’s name (name)
  • That person’s associated URL (url)

Google provides a Rich Snippet testing tool, which shows what data it can extract from the schema.org microdata:

Google could use this additional data in search results as follows:

The summary (linked to the event URL) and the date and venue are included. Nice! Just by using a vocabulary, Google (or a script supporting microdata) can discover lots of useful data about our event without needing one of those pesky natural language interpreters (otherwise known as humans).

Anyone who has used microformats before will also notice these vocabularies look very similar to hCard and hCalendar, although there are a couple of name changes — e.g., hCalendar’s class="dtstart" becomes itemprop="startDate".

While the schema.org vocabularies are all the search engines are promising to support, you can extend these vocabularies yourself. The safest ways to do this would be via:

• the Product Types Ontology for products or services, based on Wikipedia
• the GoodRelations vocabulary terms for e-commerce, which is supported (along with hProduct) by Rich Snippets
• making your own schema.org vocabulary extension collaboratively using the schema.org email list (although read Making your own vocabulary later in this article first)

Using Google’s Rich Snippets vocabularies #

Google also has some basic vocabularies (precursors of schema.org vocabularies) for the following kinds of data, under the moniker of Rich Snippets:

• people
• businesses and organizations
• events
• products
• reviews
• recipes

These vocabularies support microformats and RDFa, two other ways to add extra semantics to our content, in addition to microdata. Apart from this difference, they’re basically identical to the matching schema.org vocabularies, except they use www.data-vocabulary.org instead of schema.org in the itemtype. While Google still supports them, the newer schema.org offers more vocabularies that are also supported by Bing and Yahoo, so choose schema.org vocabularies as long as you’re happy with microdata. You might still want to check out the Rich Snippets documentation, as it includes code samples and is generally better than schema.org’s at the time of writing.

Using WHATWG/microformats.org vocabularies #

If you’re familiar with microformats or want more properties than Google’s vocabularies, the WHATWG HTML5 specification actually contains microdata vocabularies for both the vCard and vEvent specifications that hCard and hCalendar are based on, plus a licensing vocabulary.

Let’s take our earlier example and rewrite it using these vocabularies instead:

<section itemscope
  itemtype="http://microformats.org/profile/hcalendar#vevent">
  <h3><a itemprop="url" href="http://atnd.org/events/5181" title="WDE-ex
  Vol11『iPad のウェブデザイン：私たちがみつけたこと 』 : ATND"><span
  itemprop="summary">WDE-ex Vol.11 — Designing for iPad: Our experience so
  far</span></a></h3>
  <p itemprop="description">On <time itemprop="dtstart"
  datetime="2010-07-21T19:00:00+09:00">July 21st 19:00</time>-
  <time itemprop="dtend" datetime="2010-07-21T20:00:00+09:00">20:00</time>
  at <span itemprop="location" itemscope
  itemtype="http://microformats.org/profile/hcard"><a itemprop="url"
  href="http://www.apple.com/jp/retail/ginza/map/"><span itemprop="fn org">
  Apple Ginza</span></a></span>, <span itemscope
  itemtype="http://microformats.org/profile/hcard"><a itemprop="url"
  href="http://informationarchitects.jp/" title="iA"><span itemprop="fn">
  Oliver Reichenstein</span>, CEO of iA</a></span>, will share
  the lessons they’ve learned while creating three iPad apps and one
  iPad website.</p>
</section>

Currently, search engines don’t map these vocabularies to schema.org ones. It’s possible they will at some stage, so decide which vocabularies to use based on what information you want to mark up, as the data is accessible regardless.

Criticism on microformats.org #

Despite these vocabularies being based on vCard and vEvent and using microformats.org as their URLs, the microformats wiki actually warns against using the vCard and vEvent microdata vocabularies, stating:

For common semantics on the web … microformats are still simpler and easier than microdata, and are already well implemented across numerous services and tools.

Personally, I think the difference is marginal. If you use the recommended microformat profile links, I’d say it’s a wash. (But of course no one does . Microdata is actually simpler to use for date/time data than the microformat equivalents (although it is less permissive for fuzzy or ancient antiquity times), and it's more explicit, for example, avoiding the internationalisation issues of the “implied fn optimisation”. Tool support is a valid concern, but again I expect this to change over time — microdata is relatively new after all.

Browser support #

The microdata specification describes the microdata DOM API, which allows us to access top-level items and their properties. Unfortunately, no browser supports this yet. Opera has experimental support in their latest snapshot, with support expected in Opera 12. It’s also being implemented by Mozilla.

Browser support for microdata (as of 16th August, 2011)

Browser	Support
Chrome	No
Safari	No
Firefox	Work in progress
Opera	In snapshot 12.00-1033
Internet Explorer	No

But that’s okay because this data is still useful for search engine robots and third party tools. For example, Bing, Google, and Yahoo are using microdata with the schema.org vocabularies in search results.

Tools for making and using microdata #

With the right tools, we could use this data complete with its explicit semantics to, for example, add a microdata-annotated event directly into a calendar — very handy if you were planning to attend. With the release of schema.org, tools for microdata are starting to appear, but are still somewhat thin on the ground. (No Firefox plugin?! Inconceivable!). However we have libraries for three languages now:

• Live Microdata and the associated microdatajs by Philip Jägenstedt
• the PHP Microdata Parser by Alex Holt
• Mida, a Microdata parser/extractor library for Ruby and command line tool by Lawrence Woodman

Live Microdata converts microdata into JSON. If you’re using WHATWG/microformats.org vocabularies for vCard or vEvent, it also produces vCard and iCal output. The PHP Microdata library allows you to parse microdata in an HTML file, returning JSON or a PHP array. Mida allows you to extract microdata as JSON, and search for or inspect items. It supports defining vocabularies, and includes schema.org vocabularies. You can even use it from the command line.

Until there is native API support in browsers, we can use these libraries to access microdata. For example, you could put microdatajs on your own server to provide vCard and iCal file downloads, to allow adding the data to an address book or calendar app.

Validator.nu will validate your use of microdata, but not whether it conforms to a vocabulary. Google’s Rich Snippets testing tool also validates microdata. In addition, if you’re using the schema.org or Rich Snippets vocabularies it should display how that data could be incorporated into search results, as we saw in the Using schema.org vocabularies examples above. At the time of writing it isn’t, possibly due to this tool being updated for schema.org vocabularies.

If you have valid microdata in these vocabularies search engines will understand that data, but currently you have to “register your interest” in having your rich snippets actually be displayed (more information).

If you’d like a simple way to create microdata, with schema.org there are now several web-based form options:

• Schema Creator supports the schema.org vocabularies for Person, Product, Event, Organization, Movie, Book, and Review
• Schemafied supports the schema.org vocabularies for aggregate rating, article, contact point, creative work, event, local business, media object, nutrition information, offer, organization, person, place, postal address, product, rating, recipe, and thing, and also includes relevant items from other schemas
• Microdata Generator supports the schema.org vocabularies for Attorney, Auto Dealer, Dentist, HVAC, Local Business Schema (NAP), Locksmith, Physician, Plumber, Real Estate, and Restaurant
• HTML5 Microdata Templates supports the WHATWG and Rich Snippets vocabularies for events, organisations, people, reviews, and content

For CMSs, there’s a Schema for WordPress plugin, and initial work on adding microdata to Drupal too.

Other vocabularies #

Here's a short list of microdata vocabularies and their itemtypes:

Person

• schema.org Person — http://schema.org/Person
• vCard — http://microformats.org/profile/hcard
• Rich Snippets Person — http://data-vocabulary.org/Person

Organisation or business

• schema.org Organization — http://schema.org/Organization
• vCard (using fn org) — http://microformats.org/profile/hcard
• Rich Snippets Organization — http://data-vocabulary.org/Organization

Calendar

• schema.org Event — http://schema.org/Event
• vEvent — http://microformats.org/profile/hcalendar#vevent
• Rich Snippets Event — http://data-vocabulary.org/Event

Review

• schema.org Review — http://schema.org/Review
• schema.org AggregateRating — http://www.schema.org/AggregateRating (different to Rich Snippets itemtype)
• hReview — http://microformats.org/wiki/hreview
• Rich Snippets Review — http://data-vocabulary.org/Review
• Rich Snippets Review-aggregate — http://www.data-vocabulary.org/Review-aggregate

License

• Licensing works — http://n.whatwg.org/work

Products and services

• schema.org Product — http://schema.org/Product
  • This can be extended with productontology.org descriptions (example)
• hProduct — http://microformats.org/wiki/hproduct
• GoodRelations Product — http://purl.org/goodrelations/ (e.g. <a itemprop="http://purl.org/goodrelations/v1#availableDeliveryMethods" href="hhttp://purl.org/goodrelations/v1#UPS">via UPS</a>)
• Rich Snippets Product — http://data-vocabulary.org/Product

Atom feed

• hAtom — http://microformats.org/wiki/hatom

Recipes

• schema.org Recipe — http://schema.org/Recipe
• hRecipe — http://microformats.org/wiki/hrecipe
• Rich Snippets Recipe — http://data-vocabulary.org/Recipe

It’s also possible to use RDFa vocabularies by both specifying the itemtype and using URLs for itemprop names. Refer to schema.rdfs.org for Linked Data versions of schema.org vocabularies, and the RDF vocabulary clearing-house (“namespace lookup”) http://prefix.cc.

Making your own vocabulary #

If you don’t see a suitable vocabulary, you could make your own. The microdata vocabularies in the HTML5 spec are included as examples of how to do it. Basically:

1. Work out your vocabulary’s rules. This is a little like setting up a database — work out names for each type of data, then think what kind of data each name’s value should/must contain (URL, datetime, free text, text with restrictions…), and whether something needs to be the child of something else.
2. Make up a URL on a domain you control, and ideally put your vocabulary specification there.
3. Use the URL in itemtype="" to reference your vocabulary.

There are, however, very good reasons not to make your own vocabulary. They can be quite hard to create, as evidenced by the work that goes into microformats vocabularies. For truly site-specific data, you’re fine with HTML5 custom data-* attributes, or using microdata the same way. But to really get the quasi-API benefits of microdata, you need to use a vocabulary that’s on more than just your site. To make a vocabulary like that, you need to cover not just your own needs, but 80% of the needs of everyone else in the same subject area.

First, check out microformats.org to see if there’s anything in roughly the same area you can just microdata-ify. After that, try RDFa vocabularies. If you still have no luck, try collaborating on a vocabulary with other people in your subject domain. If you’re going to write your own microdata vocabulary from scratch, I’d recommend trying to write a microformat first, as you’ll get a lot of good feedback and they have good info on how to write one. It’s easy to then convert the resulting microformat vocabulary into a microdata vocabulary.

Extending HTML5 — Microdata originally appeared on HTML5 Doctor on November 23, 2010.

What’s in the box?

If you haven’t guessed from the overly verbose specification title, Web SQL Databases is a spec that brings SQL to the client side. If you have a back-end developer’s background, then you’ll probably be familiar with SQL and happy as a pig in muck. If not, you might want to learn some SQL before you start hacking around, Google’s your friend here.

The specification is based around SQLite (3.1.19), but having come from MySQL myself, it’s all pretty much the same (sorry for the sweeping statement!).

For an example of Web SQL Databases working, have a look at the Twitter HTML5 chatter demo I put together. It uses SQL and the WHERE clause to narrow down the recent chat about HTML5 on Twitter (it will work in Safari, Chrome and Opera 10.50).

There are three core methods in the spec that I’m going to cover in this article:

1. openDatabase
2. transaction
3. executeSql

Support is a little patchy at the moment. Only Webkit (Safari, SafariMobile and Chrome) and Opera 10.50 (ATOW alpha on Mac) support web databases. Fellow Doctor Bruce Lawson has told me that Firefox are holding off as they feel there’s a better implementation than SQLite (though I hope it’s similar, whatever they pick). Either way, I’d definitely recommend checking out the SQLite documentation for the functions that are available.

Because of this patchy support and the simple fact that Webkit had implemented the database spec some time ago, the spec on the W3C is now slightly ahead of the implementations in Safari, while Webkit is still catching up. On the other hand, since Opera has only just added support, it’s closer to the spec (I’ll mention the differences as we go along).

Nonetheless, it’s fun to play with, so let’s get playing!

Creating and Opening Databases

If you try to open a database that doesn’t exist, the API will create it on the fly for you. You also don’t have to worry about closing databases.

To create and open a database, use the following code:

var db = openDatabase('mydb', '1.0', 'my first database', 2 * 1024 * 1024);

I’ve passed four arguments to the openDatabase method. These are:

1. Database name
2. Version number
3. Text description
4. Estimated size of database

The missing feature of openDatabase (I’m not sure when it was added) is the fifth argument:

5. Creation callback

The creation callback will be called if the database is being created. Without this feature, however, the databases are still being created on the fly and correctly versioned.

The return value from openDatabase contains the transaction methods, so we’ll need to capture this to be able to perform SQL queries.

Estimated database size

From the tests I’ve run, only Safari prompts the user if you try to create a database over the size of the default database size, 5MB. The prompt is shown the image below, asking whether you want to grant the database permission to scale up to the next size of database — 5, 10, 50, 100 and 500MB. Opera, on the other hand, builds the database without complaining, which I expect might change later as it’s still in alpha.

Versions

I could be wrong, but everything I’ve tested so far says that versioning in SQL databases is borked. The problem is this:

If you upgrade your database to version 2.0 (e.g., there are some important schema changes since version 1.0), how do you know which visitors are on version 1.0 and which are on version 2.0?

The version number is a required argument to openDatabase, so you must know the version number before you try to open it. Otherwise, an exception is thrown.

Also, changeVersion, the method to change the database version, is not fully supported in Webkit. It works in Chrome and Opera, but not in Safari or Webkit. Regardless, if I can’t determine which version of database the user is on, then I can’t upgrade the user.

A possible workaround is to maintain a state database, something like the ‘mysql’ database in MySQL. This way, you would only have one version of this state database, and within this you would record the current version of any databases that control your application. It’s a hack, but it works.

Transactions

Now that we’ve opened our database, we can create transactions. Why bother with transactions instead of just running our SQL? Transactions give us the ability to rollback. This means that if a transaction — which could contain one or more SQL statements — fails (either the SQL or the code in the transaction), the updates to the database are never committed — i.e. it’s as if the transaction never happened.

There are also error and success callbacks on the transaction, so you can manage errors, but it’s important to understand that transactions have the ability to rollback changes.

The transaction is simply a function that contains some code:

var db = openDatabase('mydb', '1.0', 'my first database', 2 * 1024 * 1024);
db.transaction(function (tx) {
  // here be the transaction
  // do SQL magic here using the tx object
});

I recently uploaded a demo to html5demos.com that demonstrates a transaction rollback in action: Web SQL database rollback demo

In the nightly builds of the browsers, we also have db.readTransaction, which allows only read statements to run on the database. I assume there are performance benefits to using a read-only readTransaction instead of a read/write transaction, most probably to do with table locking.

Now that we’ve got our transaction object (named tx in my example) we’re ready to run some SQL!

executeSql

This is the funnel of love for all your SQL goodness. executeSql is used for both read and write statements, includes SQL injection projection, and provides a callback method to process the results of any queries you may have written.

Once we have a transaction object, we can call executeSql:

var db = openDatabase('mydb', '1.0', 'my first database', 2 * 1024 * 1024);
db.transaction(function (tx) {
  tx.executeSql('CREATE TABLE foo (id unique, text)');
});

This will now create a simple table called “foo” in our database called “mydb”. Note that if the database already exists the transaction will fail, so any successive SQL wouldn’t run. So we can either use another transaction, or we can only create the table if it doesn’t exist, which I’ll do now so I can insert a new row in the same transaction:

var db = openDatabase('mydb', '1.0', 'my first database', 2 * 1024 * 1024);
db.transaction(function (tx) {
  tx.executeSql('CREATE TABLE IF NOT EXISTS foo (id unique, text)');
  tx.executeSql('INSERT INTO foo (id, text) VALUES (1, "synergies")');
});

Now our table has a single row inside it. What if we want to capture the text from the user or some external source? We’d want to ensure it can’t compromise the security of our database (using something nasty like SQL injection). The second argument to executeSql maps field data to the query, like so:

tx.executeSql('INSERT INTO foo (id, text) VALUES (?, ?)', [id, userValue]);

id and userValue are external variables, and executeSql maps each item in the array argument to the “?”s.

Finally, if we want to select values from the table, we use a callback to capture the results:

tx.executeSql('SELECT * FROM foo', [], function (tx, results) {
  var len = results.rows.length, i;
  for (i = 0; i < len; i++) {
    alert(results.rows.item(i).text);
  }
});

(Notice that in this query, there are no fields being mapped, but in order to use the third argument, I need to pass in an empty array for the second argument.)

The callback receives the transaction object (again) and the results object. The results object contains a rows object, which is array-like but isn’t an array. It has a length, but to get to the individual rows, you need to use results.rows.item(i), where i is the index of the row. This will return an object representation of the row. For example, if your database has a name and an age field, the row will contain a name and an age property. The value of the age field could be accessed using results.rows.item(i).age.

That’s all you should need to get started with Web SQL Databases. I’m certain that mini JavaScript libraries are going to emerge to help support working with databases. If you want to find out more about SQL databases (shameless self promotion begins) I just finished the storage chapter for Introducing HTML5, which I’m writing with fellow Doc Bruce, so check that bad boy out too!

Demos

• HTML5 demo showing simple database usage
• HTML5 demonstration of a transaction rolling back
• Demo showing time range selection using SQLite

Introducing Web SQL Databases originally appeared on HTML5 Doctor on February 24, 2010.

Multiple use of tags

Daniel Davis asked:

Dear Doctor,

Just a quick confirmation please.

Am I right in thinking that nav, like header and footer, can be used more than once in a single page? For example, surrounding a menu at the top of the page and surrounding next/previous buttons at the bottom.
Come to think of it, is it fair to say that all tags except html, head and body can be used more than once?
Thanks in advance, doctor.

I’ll be back soon about my dodgy knee.

You’re correct all the elements you mention can be used more than once a page. If you see this article about the header and this about the footer – they show that you can use them multiple times on a page. Also you don’t really need html, head or body tags, as all browsers will assume them anyway but it’s also safe to leave them included.

— Hope that clears things up Rich and Bruce

HTML5 + Javascript Client Web App

Girish asked:

Is it possible to create a complete client side application using HTML 5 (it will embed chat clients, stock ticker) + JavaScript, and then bundle is with underlining Firefox 3.5 engine, so that user can install it as a desktop app on Linux, and then it runs in its own window using firefox 3.5 engine. The app will store everything in local storage – like url, usernames etc, and will not interact with any local server.

It would pure HTML5 + JavaScript based client web app which would be packaged, and can be installed or launched from startup scripts.

Or instead of using firefox 3.5 engine, we can use Mozilla Prism to convert it to web app, and then bundle it, is it possible?

Technically it should be possible. Certainly the HTML5 offline applications API (via the manifest) gives you all that ability to create a client side app and it run locally without a web connection.

However, the browser is the problem. Prism might be a good way to deploy this as a standalone app, but I don’t know if Prism is running the latest Gecko engine (or whether that comes with the JS engine, etc) – and even if it does – Firefox 3.5 currently has a pretty severe bug with the offline applications: it doesn’t work!

I have logged the bug with Mozilla and I know they’re working (in fact, I believe they’ve fixed it and are now testing).

The alternative is to use Fluid (a Webkit based app) – but again, that’s only if they have updated to the latest Webkit and thus including the offline applications API (that said, quickly looking at the Fluid site, their in-progress version may have the latest Webkit).

Regarding launch from start up scripts – you could achieve this though the custom scheme handlers (but this is just Firefox IIRC): http://www.whatwg.org/specs/web-apps/current-work/multipage/browsers.html#custom-handlers

Equally, there may be an API that you could exploit via the standalone browser, either Prism or Fluid.

— Good luck! Remy

MIME or DOCTYPE, which one prevails

Pedro Estébanez asked:

Hello doctor,

I have found this site a very good resource for web developers and I thank you for your effort.

Now, my question:

There is a lot of discussion about hot to tell the browser we are using HTML5 or XHTML5 by setting the Content-Type HTTP header. But then I don’t know the true role the DOCTYPE plays (apart from keeping quirks mode from being activated and validation).

For example, if I compose my documents as XHTML with an according DOCTYPE and my server serves them as text/html, why XML mode is not triggered in the browser?

Thanks in advance.

Did you see this article on the site from Bruce? HTML5, XML and XHTML5 and this one from Remy HTML5 Boilerplates.

Also see this (linked in Bruce’s article) for some advice on using the right mime type. http://www.webstandards.org/learn/articles/askw3c/sep2003/.

— Cheers Rich

Outline Algorithm

Mike Taylor asked:

In Section 4.4.11.1 of the current HTML5 spec, the outline algorithm is described–yet it’s not super clear in what ways this is useful to users or developers. It would be great if some more light could be shed on the topic!

Thanks for the awesome blog, btw.

The outliner basically allows you to easily see the heading structure of a site. The outline can then figure out from the heading structure where in the outline it should sit, should you want to only parse a section of the code into another site or page for example. (I realise this isn’t totally clear). Also check out the HTML5 Outliner for checking your work.

Bruce covered it a little bit in his section article – “With very few exceptions, section should not be used if there is no natural heading for it. Check your work in the HTML 5 outiner tool. If you see any instances of “untitled section” that corresponds to a section, you’re probably doing it wrong. (It’s fine for a nav or aside element to be untitled, however).”

We’ll try and cover it in more detail in a future post but I hope this helps for now, Rich.

HTML5 and xmlns

Ad Taylor asked:

I hope this isn’t a stupid question BUT …

If I am using the html5 DOCTYPE and I am still using xhtml syntax do I need to define the xmlns in the html tag (i.e. xmlns=”http://www.w3.org/1999/xhtml”)?

Thank you for taking the time to do this.

We said, Not a stupid question, but a simple answer: No!

— Ta Bruce (HTML5 Shrink)

Developing to a changing spec

Sam Rayner asked:

I read the Super Friends Guide to HTML5 Hiccups – http://www.zeldman.com/superfriends/guide/ and it’s worried me a little.

I’ve recently been developing personal projects using HTML5 as I find I learn best ‘on the job’ and wish to become familiar with the new spec by putting it in to practice. However, it sounds like I’ve misinterpreted bits and often used new elements such as footer in the way the Super Friends describe rather than in line with the spec.

I really want to continue enjoying the benefits of HTML5 in my own non-critical web work but it seems I run the risk of having to return to it all at a later date and amend stuff due to changes in the spec.

What do you think the best course of action is? Continue with 5 but keep going back and fixing old projects? – Could become a real hassle.

Continue with 5 and leave non-visual discrepancies as they are (the code may be slightly off but it’s styled fine)? – Goes against all of my natural instincts to do things right.

Return to 4 and use class names as apposed to the new elements for the time being?

I realise this is the risk I take developing with a developing spec but I’m worried if I wait until everything is set in stone I’ll still be using HTML4 years from now!

Thanks a lot,

I understand your concerns with the spec changing from day to day. It’s hard to keep up with (there’s 6 of us and that’s tough enough).

Although I’m sure you’re aware in footers case, its content model has now been changed to allow it to act more like header.

I think essentially you have three choices for your development or a combination of the below currently:

1. Continue working as you are with new HTML5 elements and change them as the spec becomes more defined
2. Use HTML4, with HTML5 class names (as you suggested)
3. Use only the simplified doctype and those elements that will only be benificial in the future or add progressive enchancement (e.g. inline elements such as time.)

Its going to be a while until we can use HTML5 in development projects but in my opinion there isn’t any reason to not use it on personal projects now. You have to judge it on a case by case basis, but whichever parts you do you will be futureproofed to some extent.

— Cheers Rich

HTML5 and Browser Compatibility

A lot of people ask this type of question regularly so here’s a few links that we think are useful.

• Can I use
• Mollys HTML5 Table
• Comparison of Layout Engines Wiki

That about wraps up this round of questions, we’ll be publishing more soon, but in the meantime don’t be afraid to keep asking the doctor about HTML5.

Your Questions Answered #3 originally appeared on HTML5 Doctor on October 22, 2009.

I then promised to look at implementing accessibility with native drag and drop, and here’s my findings.

ARIA Support

This was quite easy to add, though only by following Gez’s article. Since I already had events for drag start, I just needed to add drag end to clean up and remove the ARIA effects.

I added the following to the dragstart event:

addEvent(dragItems, 'dragstart', function (event) {
  // via ARIA say the element has been grabbed
  this.setAttribute('aria-grabbed', 'true');
  // code continues...

The added a new dropend event:

addEvent(dragItems, 'dragend', function (event) {
  // remove the ARIA grab attibute when we're not dragging
  this.setAttribute('aria-grabbed', 'false');
});

The effectAllowed and dropEffect combo was difficult to work out at first, though probably because I was playing with link and reference, whereas I should have been playing with move or copy. That said, a note about move and copy, move is a type of copy, with a clean up function to delete afterwards. Once I had this combo correct, the drop effects worked.

The changes required were to:

1. On drag: on all the drop targets add: setAttribute('aria-dropeffect', 'copy')
2. On drag: set the allowedEffect on the drag event: event.dataTransfer.effectAllowed = 'copy';
3. On drop: say what we’ll accept: event.dataTransfer.dropEffect = 'move';
4. On dragend: remove the drop effect from the drop targets: removeAttribute('aria-dropeffect')

The big problem for me was that I couldn’t truely test this.

Update: between the time I had finished my code and writing this article, NVDA (the Windows open source screenreader) announced that they recieved support from Yahoo!, and that ARIA drag and drop support is in their development roadmap.

The full working version of native drag and drop with ARIA support is available here: native drag and drop with ARIA support (source code)

Keyboard Support

Keyboard, sadly, is a whole different story.

I was able to replicate a lot of the keyboard features that Gez included in his demo. This included setting the tabIndex = 0 to allow the elements to tab to (superb trick by the way), included highlighting the drop targets as we drag.

However, the problem was: I wanted the user to be able to start the drag operation using the keyboard. Initially looking at the spec I couldn’t see that it was supposed to be supported (which it turns out, it is).

I started going down the path of using the keyboard to trigger drag events. I was able to replicate the drag event being triggered from the keyboard (like space starts the event), however, and this is a biggie, I couldn’t replicate the dataTransfer object. I tried faking it. I tried giving it nothing. I tried all kinds of random attempts to trick the browser in to giving me the object. It’s a no goer.

As it turns out, there is keyboard support written in to the spec (though in my personally opinion it’s not completely obvious to an author reading). The drag and drop operations should work like a copy and paste operation.

This is very clever, and I like it. Tab round to the dragable element, copy using the keyboard, which triggers the dragstart event, which I can then add classes to the drop targets, and make them tabbable. The user then tabs round to the drop targets, and pastes. This triggers the drop event and we’re all good.

No browsers support this keyboard event model.

I’ve raised bug for Mozilla and Webkit. They’ve both had comments on, and in both cases I’m not 100% sure what the comments mean, but speaking to the people involved via IRC, they’ve both said if it’s in the spec, it should be in the browser.

Let’s hope they do something about it. Now, how do I report a bug in IE…?!

Accessibility & Native Drag and Drop originally appeared on HTML5 Doctor on July 23, 2009.

HTML 5 DnD is based on Microsoft’s original implementation which was available as early as Internet Explorer 5! Now currently supported in IE, Firefox 3.5 and Safari 4.

We’re going to look how to implement DnD and review some of the issues (or hoops we have to jump through) with the current implementations now.

My First Drag and Drop

DnD requires only a few things to work:

• Something to drag
• A drop target
• JavaScript event handlers on the target to tell the browser it can drop

The following elements are draggable by default: <img> elements and <a> elements (with an href).

If you want to drag a different element, you need to set the draggable attribute to true. That’s according to the spec, but to get it to work in Safari and Firefox there’s a little more that you need to do, which we’ll come on to.

If you want to skip the code walk through, here’s the simple drag and drop (source code)

Here’s our example markup:

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset=utf-8 />
<title>Basic Drag and Drop</title>
<style>
#drop {
  min-height: 100px;
  width: 200px;
  border: 3px dashed #ccc;
  margin: 10px;
  padding: 10px;
}
p {
  margin: 3px 0;
}
</style>
<script src="http://html5demos.com/h5utils.js"></script>
</head>
<body>
  <img src="http://twitter.com/api/users/profile_image/rem" alt="Remy Sharp" />
  <img src="http://twitter.com/api/users/profile_image/brucel" alt="Bruce Lawson" />
  <img src="http://twitter.com/api/users/profile_image/Rich_Clark" alt="Rich Clark" />
  <div id="drop"></div>
</body>
</html>

Note: the included h5utils.js script is a small library to trigger HTML 5 elements and to give me cross browser event binding.

Currently the <img> elements can be dragged, but they can’t be dropped (this is the browser’s default).

To drop elements we need to:

1. Tell the browser that the element can be dragged over a specific point
2. Handle the drop event

To tell the browser we can drop in this element, all we have to do is cancel the dragover event. However, since IE behaves differently, we need to do the same thing for the dragenter event.

Here’s the code for basic drop handling:

var drop = document.querySelector('#drop');

// Tells the browser that we *can* drop on this target
addEvent(drop, 'dragover', cancel);
addEvent(drop, 'dragenter', cancel);

addEvent(drop, 'drop', function (event) {
  // stops the browser from redirecting off to the text.
  if (event.preventDefault) {
    event.preventDefault();
  }

  this.innerHTML += '<p>' + event.dataTransfer.getData('Text') + '</p>';

  return false;
});

function cancel(event) {
  if (event.preventDefault) {
    event.preventDefault();
  }
  return false;
}

Note for this code, I’m using addEvent where you’d normally see element.addEventListener, it’s from the h5utils.js library to support IE.

In the JavaScript we are:

1. Searching for the drop target in the DOM using document.querySelector (which returns the first result only)
2. When dragover event is fired (when the user drags the element over another), it will trigger the function called ‘cancel’ (at the bottom of the code above) which will prevent the default browser action
3. Do the same for dragenter to support IE
4. Bind the drop event, and within there grab some data about what was dropped.

dragover & dragenter

By cancelling this event, we’re telling the browser this element that we’re over is the one you can release and drop upon.

I’m still not entirely sure why there’s a difference here, but Firefox and friends needs preventDefault on the event, and IE requires the return false. Note that the examples out in the wild that use inline JavaScript (yuk, bad), the preventDefault is supposed to be implicit, so you won’t see it in the code.

drop

To show what was dropped, we need to do two things:

1. Cancel the default browser action. This is to ensure that if we drop a link, the browser doesn’t go off and surf to that location
2. Get the contents out of the dataTransfer object

The cancelling of the browser default is, again, done using the preventDefault and return false.

If we don’t set the dataTransfer data manually (which I’ll show you later), the default key is set to Text. So this might be the value of an href of an <a> element, or it might be <img> src address.

Doing More with Drag’n Drop

There’s a number of extra bits in the spec other than just dragging images.

Here’s a few bits that I can see being useful:

1. Being able to drag any element
2. More complex data types, other than text
3. Set a different drag icon/image when dragging the element

Dragging Anything

If we change the <img> to <div> tags with background images, I still want to be able to drag them around the page, and drop them in to my container.

The HTML 5 spec says it should be as simple as adding the following attributes to the markup of the elements in question:

draggable="true"

However, this doesn’t work completely for Safari or Firefox.

For Safari you need to add the following style to the element:

[draggable=true] {
  -khtml-user-drag: element;
}

This will start working in Safari, and as you drag it will set a default, empty value with the dataTransfer object. However, Firefox won’t allow you to drag the element unless you manually set some data to go with it.

To solve this, we need a dragstart event handler, and we’ll give it some data to be dragged around with:

var dragItems = document.querySelectorAll('[draggable=true]');

for (var i = 0; i < dragItems.length; i++) {
  addEvent(dragItems[i], 'dragstart', function (event) {
    // store the ID of the element, and collect it on the drop later on
    event.dataTransfer.setData('Text', this.id);
  });
}

Here’s a working demo of drag and drop anything (source code)

More Complex Data Types

You’ve already seen how we can use dataTransfer.setData(format, string) to associate some data in the example above.

You can also use this key/value pair to store more data, but you’re limited to storing strings only.

One way I would get around this is to have a data lookup, where the key of the lookup may be the ID of the element, and then I can de-reference the data on the drop event.

For example:

var people = {
  rem : {
    name : "Remy Sharp",
    blog : "http://remysharp.com"
  },
  brucel : {
    name : "Bruce Lawson",
    blog : "http://brucelawson.co.uk"
  }
  // etc...
}

var dragItems = document.querySelectorAll('[draggable=true]');

for (var i = 0; i < dragItems.length; i++) {
  addEvent(dragItems[i], 'dragstart', function (event) {
    // store the ID of the element, and collect it on the drop later on
    event.dataTransfer.setData('Text', this.id);
  });
}

addEvent(drop, 'drop', function (event) {
  // stops the browser from redirecting off to the text.
  if (event.preventDefault) {
    event.preventDefault();
  }

  var person = people[event.dataTransfer.getData('Text')];

  this.innerHTML += '<p><a href="' + person.blog + ">' + person.name + '</a></p>';

  return false;
});

Here’s a working demo of drag and drop with associated data (source code)

Obviously this isn’t the most ideal situation, but there’s also the possibility to set other content types in the drag data, which should produce some interesting apps in the future.

Drag Icon

Along with several other options with the dragstart event, you can also set a drag image, i.e. what you see under your cursor.

You can create a DOM fragment, and then associate the fragment with the dataTransfer using setDragImage(element, x, y).

So we can add the following to our example to use a custom drag image:

var dragIcon = document.createElement('img');
dragIcon.src = 'http://twitter.com/api/users/profile_image/twitter?size=mini';

Which creates an image element, and how within the dragstart event, we’ll set the drag image:

addEvent(dragItems[i], 'dragstart', function (event) {
  // store the ID of the element, and collect it on the drop later on
  event.dataTransfer.setData('Text', this.id);
  event.dataTransfer.setDragImage(dragIcon, -10, -10);
  return false;
});

This sets the custom drag icon 10 pixels below the cursor, as seen in this example: drag and drop with custom image (source code)

Native Drag

There’s lots of Drag and Drop JavaScript libraries available, but what I’d like to see, is native DnD support falling back to library based. However, I know for a fact that some libraries, including jQuery, construct a custom event when passed in to the event handler which means, currently, you can’t use the dataTransfer object, so you’ll have to rely on binding the events yourself. I’m sure this will change soon.

There’s lots more in Drag and Drop but this should be enough to go and play with now!

As for me, I'll be updating the Drag and Drop demos on HTML5demos.com to see if I can add ARIA support and capture a screencast of it running. Watch this space!

Further Reader

• HTML5 Drag and Drop spec
• Alternative demo
• Demo showing different content types in drag and drop
• Inline event handles
• Accessible Drag and Drop

Native Drag and Drop originally appeared on HTML5 Doctor on July 9, 2009.

The problematic A-grade browsers include IE 8 and below, Firefox 2, and Camino 1 (these last two browsers both use the Gecko rendering engine, which is why they’re both affected).

Let’s start with Internet Explorer.

IE doesn’t believe in HTML 5 elements

Quite simply, IE doesn’t even see HTML 5 elements, much less style them.

This is actually the same issue that we had before HTML 5, where the <abbr> element couldn’t be styled in IE 6, resulting in all manner of workarounds. (Let me add that we’ll also fix the <abbr> element while we convince IE to recognise HTML 5 elements).

The fix

There is hope! The trick, discovered by Sjoerd Visscher, is simply to create the new element using JavaScript, and voilà, IE is able to style it:

document.createElement('header');

John Resig has also written about this HTML 5 shiv.

For example, say you wanted to style the <time> element in italics:

<!DOCTYPE html>
<html>
<head>
  <title>Header test</title>
  <style>
  time { font-style: italic; }
  </style>
</head>
<body>
  <time datetime="2009-09-13">my birthday</time>
</body>
</html>

This screenshot shows the rendering in IE before we apply the fix:

To apply the fix, add the indicated line of code:

<!DOCTYPE html>
<html>
<head>
  <title>Header test</title>
  <style>
  time { font-style: italic; }
  </style>
  
  <!-- Add this line -->
  <script>document.createElement('time');</script>
</head>
<body>
  <time datetime="2009-09-13">my birthday</time>
</body>
</html>

Now after we’ve applied the fix, it’s correctly styled in IE:

One hit solution

For everyone’s convenience, I wrote a single JavaScript file that can be included to create all the HTML 5 elements (and the <abbr> element) for IE.

Download the IE HTML 5 enabling script

Include the script in your <head> tag, and you’ll be able to style the elements appropriately in IE:

<!--[if lte IE 8]>
<script src="html5.js" type="text/javascript"></script>
<![endif]-->

Note that I’ve used a conditional comment to only apply this to IE 8 and below. It’s my hope that IE 9 and onwards will support HTML 5 elements, but when that day comes, make sure to double check the conditional!

Conditions & Gotchas

There are a couple of things to be aware of when using the HTML 5 shiv.

JavaScript required

This obviously means that your design now depends on JavaScript. Personally, I feel that if you’ve used semantic markup for your site and the elements can’t be styled, the content is still completely readable.

Here’s a screenshot of the Full Frontal web site, written using HTML 5 elements, rendered in IE with and without JavaScript enabled:

You can see in the second screenshot that the content isn’t perfect, but it’s still readable — it cascades down correctly, much as if CSS were disabled.

A little head is always good

If you create the new element and don’t use a <body> tag (which is perfectly valid HTML 5), IE will put all those created elements inside the <head> tag. Pretty crazy, but you can easily avoid this by always using both the <head> and <body> tags in your markup. Leif Halvard explains further with demos.

Firefox 2 and Camino 1 rendering bug

Both Firefox 2 and Camino 1 have a bug in the Gecko rendering engine (specifically versions prior to 1.9b5):

Firefox 2 (or any other Gecko-based browser with a Gecko version pre 1.9b5) has a parsing bug where it will close an unknown element when it sees the start tag of a “block” element p, h1, div, and so forth.

According to the the stats (note that Firefox 2 doesn’t even factor), Firefox 2 only has around 3% of the market — perhaps low enough to justify ignoring it. It’s safe to assume that Camino 1 commands an even smaller percentage of the market.

By ignoring this issue, however, a site can look quite bad in these browsers. So how can we fix it?

The bug surfaces when Gecko doesn’t recognise an element. Explained roughly, when Gecko parses an unrecognised element, it removes the element’s contents and puts them next to the element.

Take, for example, the following code:

<body>
  <header><h1>Title</h1></header>
  <article><p>...</p></article>
</body>

This will be parsed in Gecko (prior to version 1.9b5) as if the markup were actually as follows:

<body>
  <header></header>
  <h1>Title</h1>
  <article></article>
  <p>...</p>
</body>

The visual result is similar to the above screenshot of IE running without JavaScript (though subtly different, as the DOM tree is actually in a different order than you as the author intended).

The fix

There are two approaches to fixing this issue, and so far I've only successfully used the non-JavaScript approach.

The JavaScript solution

The first approach is to use JavaScript to traverse the DOM tree, rearranging elements as issues are encountered. Simon Pieters has a small working example of how this can be done (towards the bottom of the page). In practise, however, I personally found it didn't work for my markup. The problem is definitely solvable using JavaScript, but this solution still needs work to handle all permutations of markup.

The XHTML solution

The second approach is to serve Gecko XHTML. I've found this to be the easier approach if you're either generating a page dynamically (using something like PHP) or if you can create your own .htaccess file to use Apache's mod_rewrite.

The first change to your markup is to add the xmlns attribute to your <html> tag:

<html lang="en" xmlns="http://www.w3.org/1999/xhtml">

Next, we need to sniff the user agent string (typically a bad approach, but justifiable when targeting such a specific group of users). If the Gecko build is less than 1.9, then we need to set the Content-type header to application/xhtml+xml.

If you want to use mod_rewrite in an .htaccess file (or the httpd.conf file), you need the following rules:

RewriteCond %{REQUEST_URI} \.html$
RewriteCond %{HTTP_USER_AGENT} rv:1\.(([1-8]|9pre|9a|9b[0-4])\.[0-9.]+).*Gecko
RewriteRule .* - [T=application/xhtml+xml]

This rule sends the proper Content-type header to all Gecko based browsers where version is less than 1.9, or "rv:1.9pre" or "rv:1.9a" or "rv:1.9bx" where x is less than 5.

If you don't want to use the mod_rewrite approach, you'll need to manually send the header in your server side scripts. Here's a solution for a PHP script:

if (preg_match('/rv:1\.(([1-8]|9pre|9a|9b[0-4])\.[0-9.]+).*Gecko/', $_SERVER['HTTP_USER_AGENT'])) {
  header('Content-type: application/xhtml+xml');
}

This snippet needs to be included before anything has been printed by your script — i.e., as early as possible.

Gotcha: Yellow Screen of Death

The Yellow Screen of Death shows up whenever there's an XML error on the page. If we're serving markup as XML and telling the browser to interpret it strictly as XML, then we can't serve any characters it doesn't recognise or else it's going to bork:

Below are a few ways to avoid XML parsing errors.

Create strict markup

You need to ensure your markup is squeaky clean — but that's easy, because you're already a Markup Jedi, right?

Use XML entities

HTML entities are a no-no. Sorry, but • isn't going to fly anymore. You need to use XML entities, the numerical representation of these characters.

I've built an entity lookup tool that shows the HTML entity and the XML value of that entity. For example, • is 8226, so the XML entity is •.

Sanitise user generated content

If your site relies on any user generated content (e.g., blog comments), then you need to sanitise the output to ensure there are no validation issues to trigger the Yellow Screen of Death.

This issue alone may justify further investigation of a JavaScript solution.

Worth the trouble?

All that said, Firefox has a very good automated upgrade path. Looking at the stats, it's safe to say that the number of users with this Gecko bug is rapidly diminishing.

Further reading

• Firefox 2 rendering issue on WHATWG Blog
• IE / missing head tag issue
• HTML 5 shiv
• HTML 5 enabling script

How to get HTML5 working in IE and Firefox 2 originally appeared on HTML5 Doctor on June 20, 2009.