<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>HTML5 Doctor &#187; HTML 5</title>
	<atom:link href="http://html5doctor.com/tag/html-5/feed/" rel="self" type="application/rss+xml" />
	<link>http://html5doctor.com</link>
	<description>helping you implement HTML5 today</description>
	<lastBuildDate>Wed, 16 May 2012 11:31:21 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=</generator>
		<item>
		<title>HTML5 Audio — The State of Play</title>
		<link>http://html5doctor.com/html5-audio-the-state-of-play/</link>
		<comments>http://html5doctor.com/html5-audio-the-state-of-play/#comments</comments>
		<pubDate>Tue, 08 May 2012 14:00:24 +0000</pubDate>
		<dc:creator>Mark Boas</dc:creator>
				<category><![CDATA[Elements]]></category>
		<category><![CDATA[JavaScript APIs]]></category>
		<category><![CDATA[multimedia]]></category>
		<category><![CDATA[api]]></category>
		<category><![CDATA[audio]]></category>
		<category><![CDATA[HTML 5]]></category>
		<category><![CDATA[html5]]></category>
		<category><![CDATA[jplayer]]></category>

		<guid isPermaLink="false">http://html5doctor.com/?p=4628</guid>
		<description><![CDATA[<p>Guest doctor Mark Boas returns with a follow up to his 2009 article <cite>Native Audio in the Browser</cite>, which covers the basics of HTML5 audio. Read the original if you want to get a feel for the <code>&#60;audio&#62;</code> element and associated API. If not, get comfortable and dive deep to learn about the current state of play for HTML5 audio.</p>]]></description>
			<content:encoded><![CDATA[<p>This is a follow up to my 2009 article <a href="http://html5doctor.com/native-audio-in-the-browser/"><cite>Native Audio in the Browser</cite></a>, which covers the basics of HTML5 audio. It may well be worth reading if you want to get a feel for the <code>&lt;audio&gt;</code> element and associated API.</p>
<p>Now, two and a half years later, it&#8217;s time to see how things are progressing. With many new advanced audio APIs being actively worked on and plenty of improvements to the existing native audio we all know and love, it&#8217;s certainly an exciting time to revisit the heady world of <code>&lt;audio&gt;</code>.</p>
<p>A good way of understanding how the land lies is by going through a few use cases. That&#8217;s what I&#8217;ll attempt to do in this post.</p>
<p>So how do we get started? Well, there are a few things we need to do to prepare the ground. Let&#8217;s tackle MIME types first.</p>
<section id="mime-types">
<h2>MIME Types <a href="#mime-types" class="permalink">#</a></h2>
<p>MIME types (also known as <a href="http://en.wikipedia.org/wiki/Internet_media_type">Internet Media Types</a>) are a way of defining file formats so that your system knows how to handle them.</p>
<section id="mime-types-server-side">
<h3>Server Side <a href="#mime-types-server-side" class="permalink">#</a></h3>
<p>First things first: your media server should be configured to serve correct MIME types. In the case of the Apache web server, this means adding the following lines to your <code>.htaccess</code> file:</p>
<aside class="sidenote">
<p><strong>Tip:</strong> Do not apply gzip compression to your media files on the server. Most formats are already compressed, and there&#8217;s limited support for those that aren&#8217;t. Further, in the case of a fallback solution, Flash does not support gzipped media.</p>
</aside>
<pre><code># AddType TYPE/SUBTYPE EXTENSION
AddType audio/mpeg mp3
AddType audio/mp4 m4a
AddType audio/ogg ogg
AddType audio/ogg oga
AddType audio/webm webma
AddType audio/wav wav</code></pre>
</section>
<section id="mime-types-client-side">
<h3>Client Side <a href="#mime-types-client-side" class="permalink">#</a></h3>
<p>When defining sources in your code or markup, you can also specify the MIME type, which will help the browser identify the media correctly.</p>
<p>To set up HTML5 audio in the most robust manner, you could write something like this:</p>
<pre><code>&lt;audio&gt;
   &lt;source src="elvis.mp3" type='audio/mpeg; codecs="mp3"'&gt;
   &lt;source src="elvis.oga" type='audio/ogg; codecs="vorbis"'&gt;
   &lt;!-- add your fallback solution here --&gt;
&lt;/audio&gt;</code></pre>
<p>Here we define the element and the sources to use. The browser will only pick one. It won&#8217;t play both. In this code, we also place a fallback solution after the &lt;<code>source</code>&gt; elements.</p>
<aside class="sidenote">
<p>Note that you can omit the codecs portion of the <code>type</code> attribute, but for robustness and efficiency, I recommend you supply the browser with as much media information as possible.</p>
</aside>
<p>Along with the source, we specify a <code>type</code> attribute. Although not strictly necessary, this attribute allows the browser to know the MIME type and the codecs of the supplied media before it downloads it. If not, supplied the browser will guess and take a trial-and-error approach to detecting the media type.</p>
<p>Cool. So now we know how to define our audio sources, and the browser will happily pick the first source that it supports. But what if we want to supply the correct source in a more dynamic manner?</p>
</section>
</section>
<section id="canPlayType">
<h2>Knowing in Advance: <code>canPlayType</code> Can Help, Probably <a href="#canPlayType" class="permalink">#</a></h2>
<p>Fortunately the audio API provides us with a way to find out whether a certain format is supported by the browser. But first, here&#8217;s a quick recap on how we manipulate the audio element via the API.</p>
<div class="callout highlight-block">
<p>If you mark up your element in HTML as we did in our previous example, you can grab the <code>&lt;audio&gt;</code> element by doing the following:</p>
<pre><code>var audio = document.getElementByTagName('audio')[index];

// or, if you gave it an id attribute
var audio = document.getElementById('my-audio-id');</code></pre>
<p>Alternatively, you can also create your element entirely in JavaScript:</p>
<pre><code>var audio = new Audio();</code></pre>
</p></div>
<p>Once you have your audio element, you&#8217;re ready to access its methods and properties. To test format support, you can use the <code>canPlayType</code> method, which takes a MIME type as a parameter:</p>
<pre><code>audio.canPlayType('audio/ogg');</code></pre>
<p>You can even explicitly include the codec:</p>
<pre><code>audio.canPlayType('audio/ogg; codecs="vorbis"');</code></pre>
<p><code>canPlayType</code> returns one of three values:</p>
<ol>
<li><code>probably</code>,</li>
<li><code>maybe</code>, or</li>
<li>&#8220;&#8221; (the empty string).</li>
</ol>
<aside class="sidenote">
<p>Previously, <code>canPlayType</code> returned &#8220;no&#8221; instead of the empty string. Worth knowing for use with early audio enabled browsers such as Firefox 3.5, Chrome 4, and Safari 4, although these make up a very small percentage of active browsers.</p>
</aside>
<p>The reason we have these odd return types is because of the general weirdness surrounding codecs. The browser can only guess at whether a certain codec is playable without actually trying to play it.</p>
<p>So to test for support, you could do this:</p>
<pre><code>var audio = new Audio();
  var canPlayOgg = !!audio.canPlayType &amp;&amp; audio.canPlayType('audio/ogg; codecs="vorbis"') != "";</code></pre>
<p>All we&#8217;re doing here is checking that <code>canPlayType</code> is supported (<code>!!</code> effectively casts to a boolean) and then checking that <code>canPlayType</code> of our chosen format doesn&#8217;t return an empty string.</p>
</section>
<section id="support">
<h2>Current Browser Codec Support <a href="#support" class="permalink">#</a></h2>
<p>Let&#8217;s check the codec support in the current crop of modern browsers.</p>
<table>
<caption>Desktop browser audio codec support</caption>
<thead>
<tr>
<th>Desktop Browser</th>
<th>Version</th>
<th>Codec Support</th>
</tr>
</thead>
<tbody>
<tr>
<td>Internet Explorer</td>
<td>9.0+</td>
<td>MP3, AAC</td>
</tr>
<tr>
<td>Chrome</td>
<td>6.0+</td>
<td>Ogg Vorbis, MP3, WAV†</td>
</tr>
<tr>
<td>Firefox</td>
<td>3.6+</td>
<td>Ogg Vorbis, WAV</td>
</tr>
<tr>
<td>Safari</td>
<td>5.0+</td>
<td>MP3, AAC, WAV</td>
</tr>
<tr>
<td>Opera</td>
<td>10.0+</td>
<td>Ogg Vorbis, WAV</td>
</tr>
</tbody>
</table>
<p>  <small>† WAV since Chrome 9</small></p>
<table>
<caption>Mobile browser audio codec support</caption>
<thead>
<tr>
<th>Mobile Browser</th>
<th>Version</th>
<th>Codec Support</th>
</tr>
</thead>
<tbody>
<tr>
<td>Opera Mobile</td>
<td>11.0+</td>
<td>Device-dependent</td>
</tr>
<tr>
<td>Android</td>
<td>2.3+</td>
<td>Device-dependent</td>
</tr>
<tr>
<td>Mobile Safari (iPhone, iPad, iPod Touch)</td>
<td>iOS 3.0+</td>
<td>MP3, AAC</td>
</tr>
<tr>
<td>Blackberry</td>
<td>6.0+</td>
<td>MP3, AAC</td>
</tr>
</tbody>
</table>
<aside class="sidenote">
<p><strong>Fun fact:</strong> Android 2.2 supports the <code>&lt;video&gt;</code> element but not its <code>&lt;audio&gt;</code> counterpart. In order to play audio, you need to use the video element.</p>
</aside>
<p>The good news is that at the time of writing, it&#8217;s estimated that around 80% of browsers now support HTML5 audio.</p>
<p>The bad news is that there is still no consensus on which codec to support, so you&#8217;ll need to provide both MP3 <em>and</em> Ogg Vorbis sources in order to take full advantage of HTML5 audio.</p>
<section id="containers">
<h2>Containers, Formats, and File Extensions (oh, and MIME types again) <a href="#containers" class="permalink">#</a></h2>
<p>Above, I&#8217;ve referred to the audio formats as they&#8217;re commonly known, but technically we should refer to their container format. (A container can contain more than one format — e.g., MP4 can contain AAC or AAC+.)</p>
<aside class="sidenote">
<p><strong>Tip:</strong> To check the codec support of your browser, you can visit <a href="http://jplayer.org/HTML5.Audio.Support/">the jPlayer audio support tester</a> or, more comprehensively (for audio and video), <a href="http://www.baccano.com/video/support.htm">baccano.com</a>.</p>
</aside>
<table>
<thead>
<tr>
<th>Container</th>
<th>Format(s)</th>
<th>File Extensions</th>
<th>MIME Type</th>
<th>Codec String</th>
</tr>
</thead>
<tbody>
<tr>
<td>MP3</td>
<td>MP3</td>
<td>.mp3</td>
<td>audio/mpeg</td>
<td>mp3</td>
</tr>
<tr>
<td>MP4</td>
<td>AAC, AAC+</td>
<td>.mp4, .m4a, .aac</td>
<td>audio/mp4</td>
<td>mp4a.40.5</td>
</tr>
<tr>
<td>OGA/OGG</td>
<td>Ogg Vorbis</td>
<td>.oga, .ogg</td>
<td>audio/ogg</td>
<td>vorbis</td>
</tr>
<tr>
<td>WAV</td>
<td>PCM</td>
<td>.wav</td>
<td>audio/wav</td>
<td>1</td>
</tr>
</tbody>
</table>
</section>
<section id="properties">
<h2>We have <code>&lt;audio&gt;</code> and we&#8217;re not afraid to use it! <a href="#properties" class="permalink">#</a></h2>
<p>Okay, we&#8217;ve done the minimum amount of work to get our audio element set up and playable. What else can we do? At the moment, we&#8217;re relying on the browsers&#8217; default audio players, each of which looks and works a little bit differently than the rest. Perhaps we&#8217;d like to customise the experience and create our own. To help us do that, the <code>&lt;audio&gt;</code> element supports several different properties exposing its current state.</p>
<p>Some of the more commonly used properties:</p>
<table>
<thead>
<tr>
<th>Property</th>
<th>Description</th>
<th>Return Value</th>
</tr>
</thead>
<tbody>
<tr>
<td>currentTime</td>
<td>playhead position</td>
<td>double (seconds)</td>
</tr>
<tr>
<td>duration</td>
<td>media duration</td>
<td>double (seconds); read-only</td>
</tr>
<tr>
<td>muted</td>
<td>is volume muted?</td>
<td>boolean</td>
</tr>
<tr>
<td>paused</td>
<td>is media paused?</td>
<td>boolean</td>
</tr>
<tr>
<td>volume</td>
<td>volume level</td>
<td>double (between 0 and 1)</td>
</tr>
</tbody>
</table>
<aside class="sidenote">
<p>You may need to check the <code>durationchange</code> event as some durations could change while media downloads. Also, depending on whether metadata is available, you might need to wait until the audio starts playing to check its duration. In short, keep an eye on the <code>durationchange</code> event, and watch out for <code>NaN</code> values when the duration isn&#8217;t yet known!</p>
</aside>
<p>Using these properties is pretty straightforward. For example:</p>
<pre><code>var audio = new Audio();
var duration = audio.duration;</code></pre>
<p>The variable <code>duration</code> now holds the duration (in seconds) of the audio clip.</p>
</section>
</section>
<section id="buffering-seeking-and-time-ranges">
<h2>Buffering, Seeking, and Time Ranges <a href="#buffering-seeking-and-time-ranges" class="permalink">#</a></h2>
<p>The situation is improving in this area, as browser makers start to implement key parts of the spec.</p>
<p>The API provides attributes called <code>buffered</code> and <code>seekable</code> that can be used in situations where we want to ascertain which part of the media has been buffered, preloaded, or is ready to be played without delay.</p>
<p>Let&#8217;s first take a look at the <code>buffered</code> and <code>seekable</code> attributes. Both return a <a href="http://www.w3.org/TR/html5/video.html#timeranges">TimeRanges</a> object. The <code>TimeRanges</code> object is a list of time periods containing start and end times that can be referenced by their indexes.</p>
<section id="buffered-attribute">
<h3>The <code>buffered</code> Attribute <a href="#buffered-attribute" class="permalink">#</a></h3>
<p>The <code>buffered</code> attribute will return the time ranges that have been completely downloaded. A little bit of a code:</p>
<pre><code>// returns TimeRanges object of buffered media
var buffered = audio.buffered;

// returns time in seconds of the last buffered TimeRange
var bufferedEnd = audio.buffered.end();</code></pre>
</section>
<section id="time-ranges">
<h3>The <code>TimeRanges</code> Object <a href="#time-ranges" class="permalink">#</a></h3>
<p>The <code>TimeRanges</code> object contains data on the parts on buffered media in the form of one or more — you guessed it — time ranges. A <code>TimeRanges</code> object consists of these properties:</p>
<ol>
<li><code>length</code> — number of time ranges</li>
<li><code>start(index)</code> — start time in seconds of a particular time range</li>
<li><code>end(index)</code> — end time in seconds of a particular time range</li>
</ol>
<aside class="sidenote" style="clear: left; margin-top: 0.5em">
<p><strong>Fun fact:</strong> The default unit of time used by the JavaScript Audio API is seconds, while many traditional JavaScript functions, such as <code>setInterval</code>, use milliseconds.</p>
</aside>
<p>You may be wondering in what situation the <code>TimeRanges</code> object would contain more than one time range. Imagine the user clicks forward to a portion of unbuffered media. The idea is that the media would then start buffering from that point, and you&#8217;d have two time ranges:</p>
<pre>------------------------------------------------------
|=============|                    |===========|     |
------------------------------------------------------
0             5                    15          19    21</pre>
<p>So in this case:</p>
<ul>
<li><code>audio.buffered.length</code> returns 2</li>
<li><code>audio.buffered.start(0)</code> returns 0</li>
<li><code>audio.buffered.end(0)</code> returns 5</li>
<li><code>audio.buffered.start(1)</code> returns 15</li>
<li><code>audio.buffered.end(1)</code> returns 19</li>
<li><code>audio.buffered.end()</code> returns 19</li>
</ul>
<aside class="sidenote">
<p><strong>Tip:</strong> Most audio-capable browsers enable seeking to new file positions during a download. To allow this, you must enable range requests on your server. Although enabled by default on web servers such as Apache, you can verify by checking that your server responds with the <code>Accept-Ranges</code> header.</p>
</aside>
<p>Note that if the user is actively seeking through the media throughout the buffering process, a contiguous buffered progress bar actually makes little sense. Also consider that some browsers will read part of the end of the file to establish duration and so create two time ranges almost immediately. Now you&#8217;re starting to appreciate why making an accurate buffered progress bar is a little tricky!</p>
<p>You can check out <code>TimeRanges</code> in real-time using this handy <a href="http://jplayer.org/HTML5.Media.Event.Inspector/">HTML5 Media Event Inspector</a>.</p>
</section>
<section id="seeking-and-seekable">
<h3>Seeking and Seekable <a href="#seeking-and-seekable" class="permalink">#</a></h3>
<p>Seeking is the act of looking forward (or backward) in a media file. This usually happens when a section of media is requested before it&#8217;s finished loading.</p>
<p>The <code>seeking</code> attribute can be used to determine whether that part of the media is being actively &#8220;seeked&#8221;. When it returns true, the portion of the media the user requested is still being loaded.</p>
<aside class="sidenote">
<p><strong>Fun fact:</strong> All modern browsers, apart from Safari on Windows, enable the <code>seekable</code> attribute, which means the <code>seeking</code> event never seems to fire as the user can jump directly to the requested part. The <code>seeked</code> event does fire though, which annoys some people who think that <code>seeking</code> should always fire before <code>seeked</code>.</p>
</aside>
<p>To recap a little, the <code>buffered</code> property tells us what&#8217;s been downloaded and is often used as an indication of what part of the media can be directly played. If the browser supports it, however, it may make more sense to use the <code>seekable</code> attribute to determine which parts of the media can be jumped to and played immediately.</p>
<p><code>seekable</code> returns a <code>TimeRanges</code> object of time ranges that can be played immediately. This uses a technology known as byte-range requests, which allows part of the content to be requested over HTTP. In short, we don&#8217;t have to load all the data prior to the desired part in order to play it.</p>
<p>An example:</p>
<pre><code>// Is the player currently seeking?
var isSeeking = audio.seeking;

// Is the media seekable?
var isSeekable = audio.seekable &amp;&amp; audio.seekable.length &gt; 0;

// Time in seconds within which the media is seekable.
var seekableEnd = audio.seekable.end();</code></pre>
<div class="callout highlight-block">
<p>Time ranges can be confusing. <code>audio.seekable.end()</code> actually tells us the end point of the last time range (not the end point of all seekable media). In practice, though, this is good enough, as the browser either enables range requests or doesn&#8217;t. If it doesn&#8217;t, then <code>audio.seekable</code> will be equivalent to <code>audio.buffered</code>, which will give a valid indication of the end of seekable media.</p>
</p></div>
<p>Note that media being in a &#8220;seekable&#8221; state is different than media being in a &#8220;buffered&#8221; state. Media doesn&#8217;t have to be buffered to be seekable.</p>
<aside class="sidenote">
<p><strong>Fun fact:</strong> In practice only one seekable TimeRange object is created. Where browsers support range requests the TimeRange object spans zero to the media duration, where they don&#8217;t, the TimeRange object starts at zero and its end grows as the media downloads.</p>
</aside>
<p>Buffered and seekable data can be useful information, but it would be too easy if there weren&#8217;t a few gotchas.</p>
<ol>
<li>Preloading is not supported in all older audio-capable browsers (Opera 10/Firefox 3.6).</li>
<li>The <code>buffered</code> attribute is not always supported (Blackberry PlayBook).</li>
<li>Not all HTML5 browsers allow byte-range seeking — for example, Safari on Windows. In this case, you can still seek within the downloaded content. It downloads from the start until the end, as does Flash.</li>
</ol>
<p>If you want to deliver the best possible solution to your users, you&#8217;ll need to feature test.</p>
</section>
<section id="preloading">
<h3>A Note about Preloading <a href="#preloading" class="permalink">#</a></h3>
<p>I mentioned the <code>preload</code> attribute in the previous article. This attribute accepts three possible values:</p>
<ol>
<li><code>none</code> — Do not preload any media. Wait for a play event before downloading anything.</li>
<li><code>metadata</code> — Preload just the metadata. Grab the start and the end of the file via range-request and determine the duration.</li>
<li><code>auto</code> — Preload the whole file. Grab the start and the end of the file to determine duration, then seek back to the start again for the preload proper.</li>
</ol>
<aside class="sidenote">
<p><strong>Fun fact:</strong> Mobile Safari ignores the <code>preload</code> attribute, effectively always using <code>preload="none"</code>.</p>
</aside>
<p>When it comes to preloading, remember that it&#8217;s a request or hint to tell the browser how you&#8217;d like to preload the media. The browser is not under any obligation to fulfill that request. For this reason, certain browsers may handle these requests differently.</p>
</section>
</section>
<section id="played">
<h2>Well-Played <a href="#played" class="permalink">#</a></h2>
<aside class="sidenote">
<p><strong>Tip:</strong> If you loop through <code>audio.played</code>, sum the time ranges between starts and ends, and then compare it with the duration, you can determine the percentage of the audio that the user has listened to. This may be a useful metric.</p>
</aside>
<p>It&#8217;s worth mentioning the <code>played</code> property in passing. This property tells us which time ranges have been played within the media. For example:</p>
<pre><code>// returns a TimeRanges object
var played = audio.played;</code></pre>
</section>
<section id="events">
<h2>Media Events <a href="#events" class="permalink">#</a></h2>
<p>Underpinning both the native audio and video APIs is a comprehensive set of events. A full list can be found at the <a href="http://www.whatwg.org/specs/web-apps/current-work/multipage/the-video-element.html#mediaevents">WhatWG version of the spec</a>.</p>
<p>Attaching handlers to media events allows you to easily react to changes in state. For example, it might be useful to update the time info in a custom-built player each time the <code>timeupdate</code> event occurs.</p>
<p>A quick summary of the more commonly used media events:</p>
<table>
<thead>
<tr>
<th>Event</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>durationchange</td>
<td>The <code>duration</code> attribute has been updated.</td>
</tr>
<tr>
<td>ended</td>
<td>Playback has stopped as the end of the media was reached.</td>
</tr>
<tr>
<td>pause</td>
<td>The media playback has been paused. Note there is no <code>stop</code> event.</td>
</tr>
<tr>
<td>play</td>
<td>The media has started playing.</td>
</tr>
<tr>
<td>timeupdate</td>
<td>The current playback position changed (usually every 250ms).</td>
</tr>
<tr>
<td>volumechange</td>
<td>The volume changed.</td>
</tr>
</tbody>
</table>
<aside class="sidenote">
<p><strong>Tip:</strong> Android (at least 2.3) does not always fire the <code>ended</code> event. Well, it sometimes throws one out for good measure, but generally you need to create your own by capturing the <code>pause</code> event that is fired upon reaching the end of the media and comparing <code>audio.currentTime</code> with <code>audio.duration</code>. Alternatively, you can listen for the <code>pause</code> event and check that <code>audio.currentTime</code> is zero. To be absolutely certain, use a combination of both techniques.</p>
</aside>
<p>Other useful events:</p>
<table>
<thead>
<tr>
<th>Event</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>canplay</td>
<td>The media can be played but may need to pause while the file is downloaded.</td>
</tr>
<tr>
<td>canplaythrough</td>
<td>At current download rates, it is estimated that the media can be played from start to finish without pause.</td>
</tr>
<tr>
<td>progress</td>
<td>The browser is fetching the media data (usually every 250ms).</td>
</tr>
</tbody>
</table>
<p>Again, the <a href="http://jplayer.org/HTML5.Media.Event.Inspector/">HTML5 Media Event Inspector</a> is your friend.</p>
<p>Additionally, you can test browser support by using <a href="http://areweplayingyet.org/">areweplayingyet.org</a>. It&#8217;s worth checking out the code behind the tests to understand the goings-on under the hood.</p>
</section>
<section id="streaming">
<h2>Streaming <a href="#streaming" class="permalink">#</a></h2>
<p>Streaming audio is a common requirement. This is an area that until recently has been dominated by Adobe&#8217;s Flash technology. Proprietary server technologies and protocols are well-established. One of the leading examples of this is Adobe&#8217;s <a href="http://en.wikipedia.org/wiki/Real_Time_Messaging_Protocol">Real Time Messaging Protocol</a> (RTMP). However, current browsers only support streaming over HTTP, and so something like Flash is required to process RTMP streams.</p>
<p>Currently, audio-enabled browsers only support <a href="http://en.wikipedia.org/wiki/SHOUTcast">SHOUTcast</a> and <a href="http://en.wikipedia.org/wiki/Icecast">Icecast</a> servers that stream audio over HTTP. SHOUTcast is propriety and allows streaming of MP3 and AAC files only, while Icecast is non-propriety and supports Ogg Vorbis as well as MP3 and AAC.</p>
<table>
<thead>
<tr>
<th>Stream</th>
<th>AAC</th>
<th>MP3</th>
<th>Ogg Vorbis</th>
<th>Native Support</th>
</tr>
</thead>
<tbody>
<tr>
<td>Icecast</td>
<td>Yes</td>
<td>Yes</td>
<td>Yes</td>
<td>Yes</td>
</tr>
<tr>
<td>SHOUTCast</td>
<td>Yes</td>
<td>Yes</td>
<td>No</td>
<td>Yes</td>
</tr>
<tr>
<td>RTMP</td>
<td>Yes</td>
<td>Yes</td>
<td>No</td>
<td>No (requires Flash)</td>
</tr>
</thead>
</table>
</section>
<section id="spec">
<h2>An Evolving Spec (Or, &#8220;Whoa, this thing is moving!&#8221;) <a href="#spec" class="permalink">#</a></h2>
<p>As the audio spec is still evolving, there are a couple of inconsistencies in browser implementations to watch out for. These generally affect older browsers.</p>
<section id="load">
<h3>The <code>load</code> Method <a href="#load" class="permalink">#</a></h3>
<p>Until fairly recently, the <code>load</code> method was required in order to tell the browser about a change in the media source and to get it to update appropriately. Now, <code>media.load()</code> causes the element to reset and to start selecting and loading a new media resource from scratch.</p>
<p>So for older browsers, such as Firefox 3.6, in order to change the media source, you&#8217;ll not only need to change the source value but also issue a load command:</p>
<pre><code>var audio = new Audio();
audio.setAttribute("src","mynew.mp3");
audio.load(); // required for 'older' browsers</code></pre>
</section>
<section id="hacks">
<h3>When Browsers Go Off-Spec <a href="#hacks" class="permalink">#</a></h3>
<p>When creating cross-browser solutions, it&#8217;s useful to know which browsers follow <a href="http://dev.w3.org/html5/spec/video.html#audio">the W3C spec</a> to what extent and how they deviate.</p>
<section id="autoplay-and-volume">
<h4>Autoplay and Volume <a href="#autoplay-and-volume" class="permalink">#</a></h4>
<p>iOS and Blackberry devices ignore the <code>autoplay</code> attribute and media element volume changes, whereas Android supports <code>autoplay</code> but not volume changes.</p>
<p>Effectively, on Blackberry and iOS devices, <code>autoplay</code> functionality is disabled as both require a user-initiated event to &#8220;kick it off&#8221;. This has the unfortunate side-effect that any audio suffers a delay between user interaction and playback.</p>
<p>To mitigate against these issues, you may want to take a look at <a href="http://remysharp.com/2010/12/23/audio-sprites/">Doctor Remy Sharp&#8217;s excellent article on Audio Sprites and fixes for iOS</a>.</p>
</section>
<section id="multiaudio">
<h4>Simultaneous Playback of Multiple Audio Elements <a href="#multiaudio" class="permalink">#</a></h4>
<p>A particular annoyance for developers of anything but the most simple audio web apps: not all browsers will play multiple audio elements simultaneously. This is a particular problem for games developers. Again, we see this issue with iOS and Blackberry devices.</p>
</section>
<section id="oses">
<h4>Operating System Dependence <a href="#oses" class="permalink">#</a></h4>
<aside class="sidenote">
<p>Although of limited use to developers, you could install other codecs such as Ogg Vorbis on the underlying operating system, and browsers that use OS codecs should be able to play them back.</p>
</aside>
<p><a href="http://happyworm.com/blog/2010/08/17/safari-requires-quicktime-for-html5-based-media/">Safari (5+) relies on Quicktime being installed.</a> This is rarely a problem unless you are running it on Windows.</p>
<p>Internet Explorer (9+) relies on the codecs being present at the operating system level. As it only runs on Windows, this is fortunately almost always the case.</p>
</section>
</section>
</section>
<section id="whats-new">
<h2>What&#8217;s New? <a href="#whats-new" class="permalink">#</a></h2>
<p>There are a few new features and whole new APIs being specified for web-based audio, so let&#8217;s take a look at what&#8217;s around the corner.</p>
<section id="playback-rate">
<h3>A Change of Pace <a href="#playback-rate" class="permalink">#</a></h3>
<p>A couple of noteworthy upcoming features are <code>playbackRate</code> and <code>defaultPlaybackRate</code>. As you can probably imagine, these fellas let us alter the speed and direction of playback. This functionality could be used for fast-forward and rewind functions or perhaps to allow users to tweak the playback speed so they can fit more podcasts into their day.</p>
<ul>
<li><code>audio.playbackRate</code> returns 1 at normal speed and acts as a multiple that is applied to the rate of playback. For example, setting <code>playbackRate</code> to 2 would double the speed, while setting it to -1 would play the media backwards.</li>
<li><code>audio.defaultPlaybackRate</code> is the rate at which the audio will play after you pause and restart the media (or issue any event for that matter).</code></li>
</ul>
</section>
<section id="media-fragments">
<h3>Media Fragments <a href="#media-fragments" class="permalink">#</a></h3>
<p>Currently, if we wish to reference some part of a media file, we often first have to download at least some of the media we don't actually need. <a href="http://www.w3.org/TR/media-frags/">The W3C Media Fragments proposal</a> was created to address this issue <a href="http://www.w3.org/TR/2009/WD-media-frags-reqs-20091217/">and others</a>, such as the retrieval of an area of video or just the associated text track. In the case of audio, it will allow us to specify which parts we want to download using parameters on the source URI.</p>
</section>
</section>
<section id="advanced-apis">
<h2>Advanced Audio APIs: The Future Sound of Browsers <a href="#advanced-apis" class="permalink">#</a></h2>
<p>When it comes to more advanced audio functionality, Mozilla was first to enter the fray with <a href="https://wiki.mozilla.org/Audio_Data_API">an early experimental Audio Data API for Firefox</a>. This was intended to facilitate audio manipulation via JavaScript and created a platform for low-level tinkering.</p>
<p>Not to be outdone, Google released <a href="https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html">the Web Audio API for Chrome</a> and put it forward as a W3C proposal. The Web Audio API is a higher-level implementation that takes an audio-node based approach and provides many useful functions.</p>
<p>Both implementations address the desire by developers to do more with audio — to create, manipulate, analyse, and mix audio on the fly. In effect, we may one day be able to do in a browser anything we can currently do with native applications.</p>
<p>So what if we want to play with advanced audio in the browser? Well, two separate implementations currently exist. If you want to write applications that use both APIs in any reasonable way, you need third-party bridging libraries to abstract away the differences.</p>
<p>Also fairly new on the scene is <a href="http://www.w3.org/TR/2011/WD-streamproc-20111215/">Mozilla's MediaStream Processing API</a>, which takes another approach to audio and video streams to allow us to manipulate at both high and low levels.</p>
<p>In fact, using advanced APIs and taking advantage of the speed of modern JavaScript engines, we can even write decoders for unsupported codecs. Currently, we have <a href="http://codecs.ofmlabs.org/">the JSMad MP3 decoder and the lossless Alac.js decoder</a>. Apparently, FLAC and AAC are in the pipeline. There is hope, then, that in the future, we may not have to worry about which browsers are supporting which formats.</p>
<p>For those curious about advanced web audio, I wrote about the differences in approaches and the different libraries that allow us to write cross-browser solutions in <a href="http://happyworm.com/blog/2011/11/15/html5-audio-apis-how-low-can-we-go/"><cite>HTML5 Audio APIs: How Low can we Go?</cite></a></p>
<p>The good news is that all relevant parties are talking, and things seem to be buzzing on <a href="http://www.w3.org/2011/audio/">the W3C Audio Working Group</a>. It looks like everyone is coming together to put their ideas into <a href="http://www.w3.org/TR/audioproc/">a unified Audio Processing API</a>, which is rapidly approaching publication as a Web Standard.</p>
</section>
<section id="summary">
<h2>Summary <a href="#summary" class="permalink">#</a></h2>
<p>Although browser implementations of the current HTML5 audio spec are improving, there are still a few issues to watch out for when creating comprehensive cross-browser solutions. You certainly need to be aware of browser limitations on platforms like iOS. Hopefully, as support matures, these issues will disappear.</p>
<p>Positive news on the "advanced audio" front too: <a href="http://www.w3.org/TR/audioproc/">a new standard is being established</a>, and we're close to get a unified spec for browser makers to work from. In the meantime, JavaScript libraries are available to help bridge the gaps between existing implementations.</p>
<p>There are also signs that we may be seeing the end of the requirement for different browser-dependent codecs. Opera Mobile already supports MP3 where the underlying OS supports it, and <a href="http://groups.google.com/group/mozilla.dev.platform/browse_thread/thread/fb14de8b9ad84e15/4cfccdde2cb064d1?#4cfccdde2cb064d1">Mozilla look like they may enable this too</a>. If this option exists for mobile browsers, it's not a huge stretch to imagine that desktop browsers will follow suit.</p>
<p>So lots happening and lots already achieved. The rough edges of existing browser implementations are being smoothed, consensus and standards are being forged, and we can see the foundations of some very exciting new technologies being established.</p>
<p>The future of web-based audio looks bright!</p>
</section>
<section id="links">
<h2>Further Reading <a href="#links" class="permalink">#</a></h2>
<ul>
<li><a href="http://24ways.org/2010/the-state-of-html5-audio">"Probably, Maybe, No": The State of HTML5 Audio</a></li>
<li><a href="http://www.phoboslab.org/log/2011/03/the-state-of-html5-audio">The State of HTML5 Audio</a></li>
<li><a href="http://www.catswhocode.com/blog/mastering-the-html5-audio-property">Mastering the HTML5 &lt;audio&gt; tag</a></li>
<li><a href="http://blogs.msdn.com/b/ie/archive/2011/05/13/unlocking-the-power-of-html5-lt-audio-gt.aspx">Unlocking the power of HTML5 &lt;audio&gt;</a></li>
<li><a href="http://my.opera.com/core/blog/2010/03/03/everything-you-need-to-know-about-html5-video-and-audio-2">Everything you need to know about HTML5 video and audio</a></li>
</ul>
</section>
<div id="crp_related">
<h3>Related Posts:</h3>
<ul class="related">
<li><a href="http://html5doctor.com/native-audio-in-the-browser/" rel="bookmark" class="crp_title">Native Audio in the browser</a></li>
<li><a href="http://html5doctor.com/video-the-track-element-and-webm-codec/" rel="bookmark" class="crp_title">Video: the track element and webM codec</a></li>
<li><a href="http://html5doctor.com/the-video-element/" rel="bookmark" class="crp_title">The video element</a></li>
<li><a href="http://html5doctor.com/video-subtitling-and-webvtt/" rel="bookmark" class="crp_title">Video Subtitling and WebVTT</a></li>
<li><a href="http://html5doctor.com/getusermedia/" rel="bookmark" class="crp_title">It&#8217;s Curtains for Marital Strife Thanks to getUserMedia</a></li>
</ul>
</div>
<p><a href="http://html5doctor.com/html5-audio-the-state-of-play/" rel="bookmark">HTML5 Audio — The State of Play</a> originally appeared on <a href="http://html5doctor.com">HTML5 Doctor</a> on May 8, 2012.</p>
]]></content:encoded>
			<wfw:commentRss>http://html5doctor.com/html5-audio-the-state-of-play/feed/</wfw:commentRss>
		<slash:comments>6</slash:comments>
		</item>
		<item>
		<title>Drag and Drop and Automatically Send to the Server</title>
		<link>http://html5doctor.com/drag-and-drop-to-server/</link>
		<comments>http://html5doctor.com/drag-and-drop-to-server/#comments</comments>
		<pubDate>Tue, 03 Apr 2012 13:30:17 +0000</pubDate>
		<dc:creator>Remy Sharp</dc:creator>
				<category><![CDATA[Events]]></category>
		<category><![CDATA[forms]]></category>
		<category><![CDATA[JavaScript APIs]]></category>
		<category><![CDATA[api]]></category>
		<category><![CDATA[drag and drop]]></category>
		<category><![CDATA[file]]></category>
		<category><![CDATA[filereader]]></category>
		<category><![CDATA[HTML 5]]></category>
		<category><![CDATA[html5]]></category>
		<category><![CDATA[javascript]]></category>
		<category><![CDATA[meter]]></category>
		<category><![CDATA[xhr2]]></category>

		<guid isPermaLink="false">http://html5doctor.com/?p=4500</guid>
		<description><![CDATA[<p>I realised (when looking myself) that there are a lot of demos and tutorials that show you how to drag-and-drop a file into the browser and then render it on the page. They're often labelled as "drag-and-drop and upload", but they actually <em>don't upload</em>. This tutorial will take you that final step.</p>]]></description>
			<content:encoded><![CDATA[<p>I realised (when looking myself) that there are a lot of demos and tutorials that show you how to drag-and-drop a file into the browser and then render it on the page. They're often labelled as "drag-and-drop and upload", but they actually <em>don't upload</em>. This tutorial will take you that final step.</p>

<p>I'll walk you through <a href="http://html5demos.com/dnd-upload">an example that you can play with</a>: drag a file into the web page, and it'll be uploaded to the server instantly.</p>

<section id="tasks">
  <h2>Tasks <a href="#tasks" class="permalink">#</a></h2>

  <p>Let's break this process down into specific tasks:</p>

  <ol>
    <li>Capture the drop event and read its data.</li>
    <li>Post that binary data to the server.</li>
    <li>Provide feedback on the progress of the upload.</li>
    <li>Optionally render a preview of what's being uploaded and its status.</li>
  </ol>

  <p>To achieve all of this, we need the following HTML5 and non-HTML5 APIs:</p>

  <ol>
    <li><a href="http://www.w3.org/TR/html5/dnd.html">Drag and Drop</a></li>
    <li><a href="http://www.w3.org/TR/2010/WD-XMLHttpRequest2-20100907/#the-formdata-interface">FormData</a></li>
    <li><a href="http://www.w3.org/TR/2010/WD-XMLHttpRequest2-20100907/#handler-xhr-onprogress">XHR progress event</a></li>
    <li><a href="http://www.w3.org/TR/FileAPI/#dfn-filereader">FileReader</a></li>
  </ol>

  <p>Keep in mind that not all browsers support all of this technology today, but they're getting close. I just wanted to create a clear and complete tutorial on how to go the whole hog and upload that dropped file.</p>

  <p>The end result: we're able to drop the file anywhere in the browser, we get a preview and progress of the upload, and it's a slick experience.</p>

  <figure>
    <img src="http://html5doctor.com/wp-content/uploads/2012/03/dnd-upload.jpg" alt="Drag and drop upload" title="" />
    <figcaption>Our drag-and-drop example page</figcaption>
  </figure>
</section>

<section id="dnd">
  <h2>Drag and Drop <a href="#dnd" class="permalink">#</a></h2>

  <p>Just as a forewarning, drag-and-drop has been known to be a bit of a killjoy. In fact, it's a nightmare, but I won't repeat what's been <a href="http://www.quirksmode.org/blog/archives/2009/09/the_html5_drag.html">said</a> <a href="http://html5doctor.com/native-drag-and-drop/">before</a>.</p>

  <p>Our example is going to allow you to drag-and-drop a file anywhere within the browser. To achieve that, we need to attach our event listeners to the <code>body</code> or <code>documentElement</code> (i.e., the root HTML node). By listening on the <code>documentElement</code>, this code <em>could</em> be anywhere in the page, as <code>documentElement</code> will always exist. You can only listen on <code>body</code> once the <code>&lt;body&gt;</code> element has been encountered.</p>

  <p>Here's the pattern of code we need to capture the drop event on the entire document:</p>

  <figure>
    <pre><code>var doc = document.documentElement;
doc.ondragover = function () { this.className = 'hover'; return false; };
doc.ondragend = function () { this.className = ''; return false; };
doc.ondrop = function (event) {
  event.preventDefault &amp;&amp; event.preventDefault();
  this.className = '';

  // now do something with:
  var files = event.dataTransfer.files;

  return false;
};</code></pre>
    <figcaption>Basic pattern for capturing drop events</figcaption>
  </figure>

  <p>I'm using the <code>hover</code> class so that I can toggle a tip explaining to the user that the file can be dropped on the page.</p>

  <p>Inside the <code>drop</code> event, we can access the dropped files via <code>event.dataTransfer.files</code>.</p>

  <section id="alternative">
    <h3>An Alternative to Drag and Drop <a href="#alternative" class="permalink">#</a></h3>

    <p>Annoyingly, as I write this, I realise this combination of API requirements is currently only met by Chrome and Firefox. So, taking the test from <a href="http://modernizr.com">Modernizr</a>, we can test for support and provide our alternative:</p>

    <figure>
      <pre><code>var dndSupported = function () {
  var div = document.createElement('div');
  return ('draggable' in div) || ('ondragstart' in div &amp;&amp; 'ondrop' in div);
};

if (!dndSupported()) {
  // take alternative route
}</code></pre>
      <figcaption>Testing for drag-and-drop support</figcaption>
    </figure>

    <p>Instead of drag-and-drop, we can insert a file input element (I've given it an <code>id</code> of "upload"), and when its value is changed, the file can be scooped in:</p>

    <figure>
      <pre><code>document.getElementById('upload').onchange = function (event) {
  // `this` refers to the element the event fired upon
  var files = this.files;
};</code></pre>
      <figcaption>An alternative to native drag-and-drop</figcaption>
    </figure>

    <p>You can see the equivalent is simply the <code>files</code> property of the <code>HTMLInputElement</code> object. This will give us access to the same file as the <code>event.dataTransfer.files</code> from the drop event.</p>
  </section>
</section>

<section id="uploading">
  <h2>Automatically Uploading the File <a href="#uploading" class="permalink">#</a></h2>

  <p>This is the neat bit, and it's painfully simple. We use a feature of the XHR2 spec: <code>FormData</code>. Once we create an instance of <code>FormData</code>, we can append items to it:</p>

  <figure>
    <pre><code>var formData = new FormData();
for (var i = 0; i &lt; files.length; i++) {
  formData.append('file', files[i]);
}

// now post a new XHR request
var xhr = new XMLHttpRequest();
xhr.open('POST', '/upload');
xhr.onload = function () {
  if (xhr.status === 200) {
    console.log('all done: ' + xhr.status);
  } else {
    console.log('Something went terribly wrong...');
  }
};

xhr.send(formData);</code></pre>
    <figcaption>Creating and posting <code>FormData</code></figcaption>
  </figure>

  <p>Yeah, that's it.†</p>

  <aside class="sidenote">† As with everything and anything, a caveat: <code>FormData</code> isn't in Opera (nor .next) at time of writing, but I'm sure they'll catch up.</aside>

  <p>Let's look at a couple of key things that are going on the above code.</p>

  <pre><code>formData.append('file', files[i]);</code></pre>

  <p>We're sending named parameters to our server, specifically an array of values called <code>file</code>. Obviously, you can call it what you want, but the <code>file</code> is the name your server will be looking for when it saves the uploaded file (or files).</p>

  <pre><code>xhr.onload = function () {
  if (xhr.status === 200) {
    console.log('all done: ' + xhr.status);
  } else {
    console.log('Something went terribly wrong...');
  }
};</code></pre>

  <p>If you're familiar with XHR, you'll notice we aren't listening to <code>onreadystatechange</code>, only <code>onload</code> — which is (in effect) a convenience function to let you know when the <code>readyState</code> is 4 (i.e., loaded!). You should still check and respond appropriately to the status code of the request to ensure it's 200 OK, rather than a 500 (internal server error) or 404 (not found) or anything else.</p>

  <pre><code>xhr.send(formData);</code></pre>

  <p>The nice trick here is that XHR has automatically set the encoding of the posted data to <code>multipart/form-data</code>. This encoding allows your server to read and save the files. It's like the encoding used when you send an attachment in an email.</p>
</section>

<section id="feedback">
  <h2>Providing Feedback to the User <a href="#feedback" class="permalink">#</a></h2>

  <p>XHR2 now (flippin' finally) comes with a <code>progress</code> event. So if you're sending or retrieving a large file via XHR, you can tell the user how far along you are.</p>

  <p>It's pretty simple. If you want to track the progress of the XHR request, listen to the <code>progress</code> event. One gotcha that caught me out for some time: I needed to track the progress of the <em>upload</em>, not the download. To do this properly, you need to listen for progress on the <strong>XHR upload</strong> object, as so:</p>

  <figure>
    <pre><code>xhr.upload.onprogress = function (event) {
  if (event.lengthComputable) {
    var complete = (event.loaded / event.total * 100 | 0);
    progress.value = progress.innerHTML = complete;
  }
};

xhr.onload = function () {
  // just in case we get stuck around 99%
  progress.value = progress.innerHTML = 100;
};</code></pre>
    <figcaption>Providing feedback with XHR2's <code>progress</code> event</figcaption>
  </figure>

  <aside class="sidenote">The funky <code>| 0</code> syntax is bitwise shift to floor the value — for example, 2.69 becomes 2.</aside>

  <p>Note that instead of <code>xhr.onprogress</code>, we use <code>xhr.upload.onprogess</code>. When this event fires, we check that the event supports calculating the amount of data uploaded (the <code>lengthComputable</code> part), and then calculate the amount completed.</p>

  <p>In my HTML, I'm using a <code>&lt;progress&gt;</code> element (similar to the <code>&lt;meter&gt;</code> element, but more semantically appropriate as we're presenting the progress of a task) to show the user how much of their file has been uploaded:</p>

  <pre><code>&lt;progress id="progress" min="0" max="100" value="0"&gt;0&lt;/progress&gt;</code></pre>

  <p>Note that I'm using <code>innerHTML</code> for browsers that don't yet support <code>&lt;progress&gt;</code>. Then with a dash of CSS-generated content, both the <code>innerHTML</code> and <code>value</code> of the progress can be the same (perhaps an over optimisation, but I was rather proud of myself at the time!).</p>
</section>

<section id="preview">
  <h2>And Finally, Rendering a Preview <a href="#preview" class="permalink">#</a></h2>

  <p>As a nice addition, we'll give the user a preview of what we're uploading for them. It requires the <a href="http://dev.w3.org/2006/webapi/FileAPI/">File API</a> — specifically the <code>FileReader</code> API.</p>

  <p>As the drag-and-drop operation (or the <code>input[type=file]</code>) contains a file object, we can hand this over to the <code>FileReader</code> to get the contents of the file. If it's something like an image, we can get the Base64 data and give the user a preview. Or if it's text, we could render it in a <code>&lt;div&gt;</code>. The MIME type for the file is available as the <code>type</code> property on the file object.</p>

  <p>So let's assume we only accept images. Here's the preview part that runs after the file has been received by the browser:</p>

  <figure>
    <pre><code>var acceptedTypes = {
  'image/png': true,
  'image/jpeg': true,
  'image/gif': true
};

if (acceptedTypes[file.type] === true) {
  var reader = new FileReader();
  reader.onload = function (event) {
    var image = new Image();
    image.src = event.target.result;
    image.width = 100; // a fake resize
    document.body.appendChild(image);
  };

  reader.readAsDataURL(file);
}</code></pre>
    <figcaption>Rendering a preview of an uploaded image</figcaption>
  </figure>

  <p>We create the <code>FileReader</code> and give it a file to read. In the above example, I've used <code>readAsDataURL</code>, but you can also read files in plain text and binary formats (<a href="http://www.w3.org/TR/FileAPI/#FileReader-interface">as per the spec</a>).</p>

  <p>When the reader has read the file and the data is available, it fires the <code>load</code> event, which in turn creates our new image.</p>

  <p>The <em>result</em> of the file read action is in the <code>event.target.result</code> property. In the case of the <code>readAsDataURL</code>, the value is along the lines of <code>data:image/png;base64,ABC...</code>.</p>
</section>

<section id="finale">
  <h2>Using All the Tools in the Shed <a href="#finale" class="permalink">#</a></h2>

  <p>This example may be a little contrived for the article that I wanted to write. I've used many different technologies to do something that's a fairly common pattern on the web. In fact, it was because I had trouble finding a definitive source on uploading to the <em>actual</em> server (except, of course, as I wrote this article I found <a href="http://www.profilepicture.co.uk/tutorials/ajax-file-upload-xmlhttprequest-level-2/">this example</a> from back in late 2010!).</p>

  <p>Hopefully, you'll see how each bit of tech can work together to create an application, but equally I hope that you can see where this would work if none or just some of the technology wasn't available. Make sure you detect functionality. Make sure you provide fallbacks. Make sure you develop responsibly.</p>

  <p>Here's the final <a href="http://html5demos.com/dnd-upload">drag-and-drop and upload demo</a> for you to play with.</p>
</section><div id="crp_related"><h3>Related Posts:</h3><ul class="related"><li><a href="http://html5doctor.com/native-drag-and-drop/" rel="bookmark" class="crp_title">Native Drag and Drop</a></li><li><a href="http://html5doctor.com/accessibility-native-drag-and-drop/" rel="bookmark" class="crp_title">Accessibility &#038; Native Drag and Drop</a></li><li><a href="http://html5doctor.com/methods-of-communication/" rel="bookmark" class="crp_title">Methods of communication</a></li><li><a href="http://html5doctor.com/your-questions-answered-9/" rel="bookmark" class="crp_title">Your Questions Answered 9</a></li><li><a href="http://html5doctor.com/server-sent-events/" rel="bookmark" class="crp_title">Server-Sent Events</a></li></ul></div><p><a href="http://html5doctor.com/drag-and-drop-to-server/" rel="bookmark">Drag and Drop and Automatically Send to the Server</a> originally appeared on <a href="http://html5doctor.com">HTML5 Doctor</a> on April 3, 2012.</p>
]]></content:encoded>
			<wfw:commentRss>http://html5doctor.com/drag-and-drop-to-server/feed/</wfw:commentRss>
		<slash:comments>18</slash:comments>
		</item>
		<item>
		<title>CSS3 Pseudo-Classes and HTML5 Forms</title>
		<link>http://html5doctor.com/css3-pseudo-classes-and-html5-forms/</link>
		<comments>http://html5doctor.com/css3-pseudo-classes-and-html5-forms/#comments</comments>
		<pubDate>Tue, 28 Feb 2012 14:30:47 +0000</pubDate>
		<dc:creator>Peter Gasston</dc:creator>
				<category><![CDATA[CSS]]></category>
		<category><![CDATA[forms]]></category>
		<category><![CDATA[css3]]></category>
		<category><![CDATA[HTML 5]]></category>
		<category><![CDATA[html5]]></category>
		<category><![CDATA[readonly]]></category>
		<category><![CDATA[required]]></category>

		<guid isPermaLink="false">http://html5doctor.com/?p=4483</guid>
		<description><![CDATA[<p>Guest Doctor Peter Gasston takes a look at how new CSS3 Pseudo-classes align perfectly with HTML5 Forms.</p>]]></description>
			<content:encoded><![CDATA[<p>Hello! I’m Doctor Peter and I’m here to treat you with a dose of complementary CSS3. Don’t worry, this won’t hurt a bit.</p>

<p>Contrary to what <a href="http://html5please.com/">HTML5 Please</a> and the W3C would have you believe, CSS3 is not part of HTML5. “But this is HTML5 Doctor,” I imagine you saying. “Why are you talking about CSS3 here?” Well, I want to talk about a very specific part of CSS3, one that works in perfect tandem with HTML5, specifically with the new form functions that are available.</p>

<p>One of the killer features that HTML5 introduces is client-side form validation without using JavaScript. Now, I know the other doctors haven’t discussed this yet, and as I’m just a locum, you might feel as if I’m putting the cart before the horse (#mixedmetaphors), but I’m hoping to explain just enough to let you know how it works, and then your regular doctors can provide the details at a later date.</p>

<section id="pseudo-classes">
  <h2>A Recap of Pseudo-Classes <a href="#pseudo-classes" class="permalink">#</a></h2>

  <p>A <dfn>pseudo-class</dfn> is information about an element that’s in the document tree but not available through any specified attributes. For example, the <code>:first-child</code> pseudo-class applies to elements which are the first child of their parents. You can get that information from the DOM tree, but there’s no <code>first-child</code> attribute.</p>

  <p>In CSS2.1, there were a handful of pseudo-classes available, notably the link states (<code>:link</code>, <code>:visited</code>) and those of user actions (<code>:active</code>, <code>:hover</code>). In CSS3, that handful becomes a basketful, with pseudo-classes of structure (<code>:nth-child</code>, <code>:nth-of-type</code>), UI element (<code>:enabled</code>, <code>:checked</code>), and the ones we’ll look at now, of form validation.</p>
</section>

<section id="form-validation">
  <h2>Form Validation without JavaScript <a href="#form-validation" class="permalink">#</a></h2>
  
  <p>So as I mentioned at the beginning, HTML5 introduces client-side form validation without the need for JavaScript. When you attempt to submit a form, your browser will validate all of the fields and return an error if any of the fields is incorrect. This will happen mostly automagically — provided your browser has the capability — and will look something like what’s shown in Figure 1:</p>

  <figure>
    <img src="http://html5doctor.com/demos/pseudo-classes/figure1.png" alt="Form with an invalid field and associated error message">
    <figcaption>Figure 1: A form validation error message</figcaption>
  </figure>

  <p>Those error messages are browser and OS specific, and hard to modify (<a href="http://www.broken-links.com/2011/06/16/styling-html5-form-validation-errors/" title="Styling HTML5 Form Validation Errors">which I documented on my blog</a>), but you can change the way the errors appear on the elements themselves with the new validation pseudo-classes, which are part of the <a href="http://www.w3.org/TR/css3-ui/">CSS3 Basic UI module</a>.</p>
  
  <p>I’ll cover current browser support as we go through the article, but in a nutshell:</p>
  
  <ul>
    <li>Firefox and IE10 support some of it.</li>
    <li>Opera and Chrome support all of it.</li>
    <li>Safari supports all of it too, but form validation is disabled by default.</li>
  </ul>
  
  <p>That being the case, I’d recommend using Chrome, Opera, or Safari to view the examples I’ll be showing you.</p>
  
  <p>If you want to keep up to date with browser support for form validation, I suggest keeping an eye on <a href="http://caniuse.com/#feat=form-validation">When can I use</a>.</p>
  
  <p>So with all of the boring technical guff out of the way, let’s get on with the interesting technical guff and take a look at some examples of form validation.</p>
  
  <section id="required-and-optional-elements">
    <h3>Required and Optional Elements <a href="#required-and-optional-elements" class="permalink">#</a></h3>
    
    <p>One of the most common patterns of validation is that of mandatory (that is, required) values — the fields that the user must complete in order to progress. To mark a form element as mandatory, you need only add the <code>required</code> attribute to it:</p>
    
    <pre><code>&lt;input type="text" required&gt;</code></pre>
    
    <p>If you want to apply styles to this in order to show that requirement, you can use the new <code>:required</code> pseudo-class, which applies rules to any matched element which has the <code>required</code> attribute. For example, you may want to put an asterisk after the text of the label:</p>
    
    <pre><code>input:required + label::after { content: "*"; }</code></pre>
    
    <p>The exact rules you use depends on how you’ve marked up your document. The rule in this example is based on the markup having this structure:</p>
    
    <pre><code>&lt;input type="text" required id="foo"&gt;
&lt;label for="foo"&gt;Foo&lt;/label&gt;</code></pre>
    
    <p>The opposite of required is, of course, optional. And wouldn’t you know it, we have a pseudo-class for that too, logically named <code>:optional</code>. It affects any form element that doesn’t have a <code>required</code> attribute. For example, you may want to make those have a lighter border:</p>
    
    <pre><code>input:optional { border-color: silver; }</code></pre>
    
    <p>You can see this in action in Figure 2, or <a href="http://html5doctor.com/demos/pseudo-classes/example1.html" title="Live Example #1">take a look at a live example</a>.</p>

    <figure>
      <img src="http://html5doctor.com/demos/pseudo-classes/figure2.png" alt="Required and optional form fields">
      <figcaption>Figure 2: Input fields styled with <code>:required</code> (top) and <code>:optional</code> (bottom)</figcaption>
    </figure>
  </section>

  <section id="valid-and-invalid-elements">
    <h3>Valid and Invalid Elements <a href="#valid-and-invalid-elements" class="permalink">#</a></h3>

    <p>There are other types of validation besides simply “required” or “optional”. You can use pattern matching, such as email address validation:</p>
    
    <pre><code>&lt;input type="email"&gt;</code></pre>

    <p>If the user enters anything into the field, it has to match the pattern of an email address, or else it will be marked as invalid. (Of course, if it’s not required, then it can be left blank.)</p>
    
    <p>You can style valid or invalid form elements using pseudo-classes called — wait for it — <code>:valid</code> and <code>:invalid</code>. Maybe you want add a symbol depending on the validation status:</p>
    
    <pre><code>input:invalid + label::after { content: ' ⨉'; }
input:valid + label::after { content: ' ✓'; }</code></pre>

    <p>Note that these will take effect as soon as the page loads, so you’ll probably want to use JavaScript to apply them only after submission, perhaps by adding a class to the form with JavaScript:</p>
    
    <pre><code>document.forms[0].addEventListener('submit', function(e) {
  e.currentTarget.classList.add('submitted');
  // Your fallback form validation function
});</code></pre>

    <p>The exact script you use will vary, but however you do it, you can use that class to style the form elements, like so:</p>
    
    <pre><code>.submitted input:invalid + label::after { content: ' ⨉'; }</code></pre>
    
    <p>Figure 3 shows the output of this, and you can also <a href="http://html5doctor.com/demos/pseudo-classes/example2.html" title="Live Example #2">see the live example</a>.</p>

    <figure>
      <img src="http://html5doctor.com/demos/pseudo-classes/figure3.png" alt="Valid and invalid form fields">
      <figcaption>Figure 3: Showing the use of the <code>:valid</code> (top) and <code>:invalid</code> (bottom) pseudo-classes</figcaption>
    </figure>
    
    <p>Two extra things to note about this example: first, in order to use my own JavaScript, I’ve prevented the form from automatically validating by using the <code>novalidate</code> attribute on the submit button; and second, Firefox puts a coloured glow around elements depending on their validation state, which I’ve over-ruled by using their proprietary <code>:-moz-ui-invalid</code> selector. View the source of the demo to see how these are used.</p>
  </section>
  
  <section id="number-ranges">
    <h3>Number Ranges <a href="#number-ranges" class="permalink">#</a></h3>

    <p>Some of the new input types, such as <code>number</code>, allow a range of values using the <code>min</code> and <code>max</code> attributes, like so:</p>
    
    <pre><code>&lt;input type="number" max="10" min="1"&gt;</code></pre>
    
    <p>Although form controls will usually prevent the user from entering a value outside of this range, there may be occasions (such as when JavaScript is used to provide values) where the value provided to the input does exceed the range:</p>
    
    <pre><code>&lt;input type="number" max="10" min="1" value="11"&gt;</code></pre>
    
    <p>When this happens, you can use the <code>:out-of-range</code> pseudo-class for your styling:</p>
    
    <pre><code>input[type='number']:out-of-range { border-color: red; }</code></pre>

    <p>And, as you’d expect, there’s a counterpart known by the name of <code>:in-range</code>:</p>
    
    <pre><code>input[type='number']:in-range { border-color: green; }</code></pre>
    
    <p>In Figure 4 you can see these two pseudo-classes at work, and once more there’s also <a href="http://html5doctor.com/demos/pseudo-classes/example3.html" title="Live Example #3">a live example</a> to view.</p>

    <figure>
      <img src="http://html5doctor.com/demos/pseudo-classes/figure4.png" alt="Out-of-range and in-range form fields">
      <figcaption>Figure 4: The <code>:out-of-range</code> (top) and <code>:in-range</code> (bottom) pseudo-classes applied to a <code>range</code> input</figcaption>
    </figure>
  </section>
  
  <section id="reading-and-writing">
    <h3>Reading and Writing <a href="#reading-and-writing" class="permalink">#</a></h3>
    
    <p>You may have a form field that’s pre-filled with a value that you don’t want the user to be able to edit, such as terms and conditions in a <code>textarea</code>. If that’s the case, you can add the <code>readonly</code> attribute:</p>
    
    <pre><code>&lt;textarea readonly&gt;Lorem ipsum&lt;/textarea&gt;</code></pre>
    
    <p>You can style elements which have the <code>readonly</code> attribute applied, using…can you guess? You are correct, imagined reader: the <code>:read-only</code> pseudo-class. You could combine it with the <code>user-select</code> CSS property to prevent the user from being able to select the text (in the example, I’ve also changed the border style to make it more obvious):</p>
    
    <pre><code>textarea:read-only { user-select: none; }</code></pre>
    
    <p>And if you should need it, there is of course a <code>:read-write</code> pseudo-class for elements which don’t have the attribute applied:</p>
    
    <pre><code>textarea:read-write { user-select: text; }</code></pre>
    
    <p>You can see the results in Figure 5, and as before there’s <a href="http://html5doctor.com/demos/pseudo-classes/example4.html" title="Live Example #4">a live example</a> too.</p>

    <figure>
      <img src="http://html5doctor.com/demos/pseudo-classes/figure5.png" alt="Read-only and read-write form fields">
      <figcaption>Figure 5: The textarea on the left is styled using <code>:read-write</code>, and the one on the right, with <code>:read-only</code></figcaption>
    </figure>

    <p>Firefox and IE10 have actually implemented the <code>readonly</code> attribute, but don’t yet have support for these pseudo-classes.</p>
  </section>
</section>

<section id="browser-support">
  <h2>Browser Support <a href="#browser-support" class="permalink">#</a></h2>
  
  <p>This table lists support for HTML5 Form Validation in browsers at the time of writing:</p>
  
  <table>
    <caption>Browser support for HTML5 Form Validation</caption>
    <thead>
      <tr>
        <th>Browser/Attribute</th>
        <th scope="col">Chrome</th>
        <th scope="col">Firefox</th>
        <th scope="col">IE</th>
        <th scope="col">Opera</th>
        <th scope="col">Safari</th>
      </tr>
    </thead>
    <tbody>
      <tr>
        <th scope="row"><code>:optional</code> / <code>:required</code></th>
        <td>10.0</td>
        <td>4.0</td>
        <td>10.0</td>
        <td>10.0</td>
        <td>5.0</td>
      </tr>

      <tr>
        <th scope="row"><code>:invalid</code> / <code>:valid</code></th>
        <td>10.0</td>
        <td>4.0</td>
        <td>n/a</td>
        <td>10.0</td>
        <td>5.0</td>
      </tr>

      <tr>
        <th scope="row"><code>:in-range</code> / <code>:out-of-range</code></th>
        <td>10.0</td>
        <td>n/a</td>
        <td>n/a</td>
        <td>10.0</td>
        <td>5.0</td>
      </tr>

      <tr>
        <th scope="row"><code>:read-only</code> / <code>:read-write</code></th>
        <td>10.0</td>
        <td>n/a</td>
        <td>n/a</td>
        <td>10.0</td>
        <td>5.0</td>
      </tr>
    </tbody>
  </table>
</section>

<section id="goodbye">
  <h2>That’s All from Me <a href="#goodbye" class="permalink">#</a></h2>
  
  <p>I hope I’ve given you enough to whet your appetite for finding out more about HTML5 Form Validation. There’s plenty more to it, including a full JavaScript API (you can <a href="http://dev.w3.org/html5/spec-author-view/constraints.html#the-constraint-validation-api" title="4.10.21 Constraints — HTML5: Edition for Web Authors">read about this in the HTML5 spec</a>, but <a href="https://developer.mozilla.org/en/HTML/HTML5/Constraint_validation" title="Constraint Validation">MozDev has better documentation</a>).</p>
  
  <p>Thanks for your time, and thanks to the Doctors for this opportunity.</p>
</section><div id="crp_related"><h3>Related Posts:</h3><ul class="related"><li><a href="http://html5doctor.com/review-html5-designing-rich-internet-applications/" rel="bookmark" class="crp_title">Review: HTML5 Designing Rich Internet Applications</a></li><li><a href="http://html5doctor.com/the-output-element/" rel="bookmark" class="crp_title">The output element</a></li><li><a href="http://html5doctor.com/the-details-and-summary-elements/" rel="bookmark" class="crp_title">The details and summary elements</a></li><li><a href="http://html5doctor.com/html-5-reset-stylesheet/" rel="bookmark" class="crp_title">HTML5 Reset Stylesheet</a></li><li><a href="http://html5doctor.com/the-scoped-attribute/" rel="bookmark" class="crp_title">The scoped attribute</a></li></ul></div><p><a href="http://html5doctor.com/css3-pseudo-classes-and-html5-forms/" rel="bookmark">CSS3 Pseudo-Classes and HTML5 Forms</a> originally appeared on <a href="http://html5doctor.com">HTML5 Doctor</a> on February 28, 2012.</p>
]]></content:encoded>
			<wfw:commentRss>http://html5doctor.com/css3-pseudo-classes-and-html5-forms/feed/</wfw:commentRss>
		<slash:comments>33</slash:comments>
		</item>
		<item>
		<title>HTML5 Doctor Drop-In Clinic</title>
		<link>http://html5doctor.com/html5-doctor-drop-in-clinic/</link>
		<comments>http://html5doctor.com/html5-doctor-drop-in-clinic/#comments</comments>
		<pubDate>Thu, 16 Feb 2012 14:41:39 +0000</pubDate>
		<dc:creator>Mike Robinson</dc:creator>
				<category><![CDATA[Misc]]></category>
		<category><![CDATA[freenode]]></category>
		<category><![CDATA[HTML 5]]></category>
		<category><![CDATA[html5]]></category>
		<category><![CDATA[irc]]></category>
		<category><![CDATA[WHATWG]]></category>

		<guid isPermaLink="false">http://html5doctor.com/?p=4430</guid>
		<description><![CDATA[<p>Got an itch? A problem that's slowing you down? Need someone to answer your question? If email just doesn't quite patch you up, you should stop by the HTML5 Doctor <abbr title="Internet Relay Chat">IRC</abbr> channel.</p>]]></description>
			<content:encoded><![CDATA[<p>Got an itch? A problem that&#8217;s slowing you down? Need someone to answer your question? If email just doesn&#8217;t quite patch you up, you should stop by the HTML5 Doctor <abbr title="Internet Relay Chat">IRC</abbr> channel.</p>
<p>Originally set up by <a href="http://twitter.com/rem">Remy</a>, the <a href="http://en.wikipedia.org/wiki/Internet_Relay_Chat">IRC</a> channel is there for brief questions about all things HTML5. If one of the HTML5 Doctors is around, we can try to help you, or if you don&#8217;t have a question, you&#8217;re more than welcome to idle and talk with the <abbr title="HTML5 Doctor">H5D</abbr> community.</p>
<p>For those who don&#8217;t know what IRC is, it&#8217;s a protocol for chat that&#8217;s been around almost as long as I have! No matter what <abbr title="Operating System">OS</abbr> you use, you should be able to find an IRC client to get connected. There are even web-based services like <a href="http://webchat.freenode.net/?channels=html5doctor">Freenode&#8217;s own web IRC</a> and <a href="http://irccloud.com/">IRC Cloud</a>, so you can jump on no matter where you are with nothing to install. The Mibbit wiki has an <a href="http://wiki.mibbit.com/index.php/IRC_tutorial">IRC tutorial</a> to help get you using IRC, and Freenode hosts a guide to <a href="http://freenode.net/using_the_network.shtml">using the network</a>.</p>
<p>When you&#8217;re ready, the channel can be found at <a href="irc://irc.freenode.net/html5doctor">#html5doctor</a> on <a href="http://freenode.net/">Freenode</a> (irc.freenode.net). Once you&#8217;re connected and you&#8217;ve joined the channel, say hi and see who&#8217;s around. If no one is there, stick around, and maybe check out other channels on Freenode such as the WHATWG channel, <a href="irc://irc.freenode.org/whatwg">#whatwg</a>.</p>
<p>The only thing to keep in mind is that we HTML5 Doctors do this voluntarily with jobs and lives to run. We can&#8217;t be there 24/7. When we are available to answer questions, we&#8217;ll try to address whomever we can, and maybe another idling patient will be able to lend a hand.</p>
<p>For more extensive problems, or if you also don&#8217;t have time to hang around on IRC, check our <a href="http://html5doctor.com/article-archive/">article archive</a> or drop us an email through <a href="http://html5doctor.com/ask-the-doctor/">Ask the HTML5 Doctor</a> if you can&#8217;t find your answer there.</p>
<p>Drop on by, hang out, and feel free to talk all about HTML5 and the wonderful web.</p>
<p>And please, no medical questions!
<div id="crp_related">
<h3>Related Posts:</h3>
<ul class="related">
<li><a href="http://html5doctor.com/2022-or-when-will-html-5-be-ready/" rel="bookmark" class="crp_title">2022, or when will HTML 5 be ready?</a></li>
<li><a href="http://html5doctor.com/dd-details-wrong-again/" rel="bookmark" class="crp_title">dd-details wrong again</a></li>
<li><a href="http://html5doctor.com/your-questions-answered-2/" rel="bookmark" class="crp_title">Your questions answered #2</a></li>
<li><a href="http://html5doctor.com/u-element/" rel="bookmark" class="crp_title">The return of the u element</a></li>
<li><a href="http://html5doctor.com/your-questions-answered-1/" rel="bookmark" class="crp_title">Your questions answered #1</a></li>
</ul>
</div>
<p><a href="http://html5doctor.com/html5-doctor-drop-in-clinic/" rel="bookmark">HTML5 Doctor Drop-In Clinic</a> originally appeared on <a href="http://html5doctor.com">HTML5 Doctor</a> on February 16, 2012.</p>
]]></content:encoded>
			<wfw:commentRss>http://html5doctor.com/html5-doctor-drop-in-clinic/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>It&#8217;s Curtains for Marital Strife Thanks to getUserMedia</title>
		<link>http://html5doctor.com/getusermedia/</link>
		<comments>http://html5doctor.com/getusermedia/#comments</comments>
		<pubDate>Tue, 07 Feb 2012 14:56:03 +0000</pubDate>
		<dc:creator>Bruce Lawson</dc:creator>
				<category><![CDATA[Elements]]></category>
		<category><![CDATA[JavaScript APIs]]></category>
		<category><![CDATA[multimedia]]></category>
		<category><![CDATA[getusermedia]]></category>
		<category><![CDATA[HTML 5]]></category>
		<category><![CDATA[html5]]></category>
		<category><![CDATA[video]]></category>
		<category><![CDATA[webrtc]]></category>

		<guid isPermaLink="false">http://html5doctor.com/?p=4335</guid>
		<description><![CDATA[<p>HTML5 (or now, the WebRTC spec) gives us getUserMedia, a way for JavaScript to access streams from a device's camera and microphone. Find out how to use it and normalise the syntax differences between Opera and Chrome with the gUMshield.</p>]]></description>
			<content:encoded><![CDATA[<p>True story: I was tasked by the lovely Mrs Lawson to buy some curtains that match our carpet during the January sales. I dutifully did so — and had to return to the shop straight away because they didn&#8217;t match at all. Mrs Lawson accompanied me, and with a withering glance at her incompetent mate, immediately found some correctly hued fabric, and all was well.</p>

<p>But what&#8217;s a middle-aged colour-blind bloke to do? I had early in the curtain procurement process decided against cutting a hole in the carpet in order that I may take a sample with me. (All other mistakes aside, this was a correct decision.)</p>

<p>So, in order to ensure that I would never again repeat the mistake, I set out to make an app that would allow me to capture the colour of an image straight from my camera. Of course, it had to be a web app rather than a native app, because we&#8217;re web angels, not proprietary devils.</p>

<section id="intro">
  <h2><code>getUserMedia</code> <a href="#intro" class="permalink">#</a></h2>

  <p><a href="http://dev.w3.org/2011/webrtc/editor/getusermedia.html"><code>getUserMedia</code></a> is an API that gives a web page access to a user&#8217;s camera and microphone via JavaScript. It&#8217;s supported in Opera Labs (<a href="http://dev.opera.com/articles/view/labs-more-fun-using-the-web-with-getusermedia-and-native-pages/">latest Android build</a>, <a href="http://dev.opera.com/articles/view/getusermedia-access-camera-privacy-ui/">latest desktop builds</a>) and WebKit in <a href="http://tools.google.com/dlpage/chromesxs?platform=win">Chrome Canary</a> builds (<a href="https://sites.google.com/site/webrtc/running-the-demos">instructions</a>).</p>

  <p>Like many other APIs, it&#8217;s not part of the &#8220;real&#8221; HTML5 spec. It started life as the HTML5 <code>&lt;device&gt;</code> element, then got moved into the W3C as part of the <a href="http://www.w3.org/TR/webrtc/">webRTC</a> specifications. But let&#8217;s not taxonomise when we could be having fun.</p>

  <p>The first thing we need to do when using super-cool new UIs is detect whether it&#8217;s supported:</p>

  <figure>
    <pre><code> if (navigator.getUserMedia) {
  // do something cool
} else {
  // fallback code
}</code></pre>
    <figcaption>Basic usage of <code>getUserMedia</code></figcaption>
  </figure>

  <p>For this article, our &#8220;do something cool&#8221; will be to grab the dominant colour of the current input stream from the camera. As a fallback, we have several options.</p>

  <p>One option, if we&#8217;re using the default Android browser, is to show a form that automatically starts the camera when focussed:</p>

  <figure>
    <pre><code>&lt;form enctype="multipart/form-data" method="post"&gt;
  &lt;input type="file" accept="video/*;capture=camera" /&gt;
&lt;/form&gt;</code></pre>
    <figcaption>Fallback code for Android</figcaption>
  </figure>

  <p>There&#8217;s also the <a href="http://www.w3.org/TR/2011/WD-html-media-capture-20110414/#captureparam"><code>capture</code> attribute</a>, a proposed extension from the <a href="http://www.w3.org/TR/html-media-capture/">W3C Device API Media Capture API</a> that tells a user agent where to get the input data. Android extends the <a href="http://dev.w3.org/html5/spec/states-of-the-type-attribute.html#attr-input-accept"><code>accept</code> attribute</a>. See more in <a href="http://davidbcalhoun.com/2011/android-3-0-honeycomb-is-first-to-implement-the-device-api">David Calhoun&#8217;s Device API article</a>. Alternative capture arguments are <code>camcorder</code> (for video), <code>microphone</code>, and <code>filesystem</code>.</p>

  <p>Browsers that don&#8217;t understand the still-draft Media Capture API will simply ignore the <code>capture</code> attrbute and prompt the user to browse to a file on their file system for uploading.</p>

  <p>Users on iOS are out of luck, as their devices don&#8217;t allow access to the file system.</p>

  <section id="differences-from-media-capture">
    <h3>Differences Between the Media Capture API and <code>getUserMedia</code> <a href="#differences-from-media-capture" class="permalink">#</a></h3>

    <p>Robin Berjon, co-chair of the Device API (DAP) Working Group explains: The markup-only Media Capture API is a simpler, low-hanging-fruit version that can work easily for a lot of use cases. It also has a very simple security model (just a file upload).</p>

    <p>In fact, it works exactly like <code>&lt;input type=file&gt;</code>, except that it knows to allow the user to grab a pic from the camera by default instead of hitting the filesystem. But just like with <code>&lt;input type=file&gt;</code>, once a picture has been taken (i.e., a file selected), you can access it and read from it using the File API, which means you can stuff it in an <code>&lt;img&gt;</code> or a <code>&lt;canvas&gt;</code> and do nasty things to it without annoying any faraway server.</p>

    <p>But of course, if it&#8217;s part of a form and you submit it, it gets submitted just like a regular file upload.</p>

    <p>The thing is, though, it&#8217;s always a snapshot. Even if you use it to take a video (which you can), the page only gets it after you&#8217;ve finished shooting. With <code>getUserMedia</code> (<b>gUM</b>), you get the live stream, which you can modify, record, or stream elsewhere. The gUM approach is more powerful but more complex, and it has more severe security issues.</p>
  </section>
</section>

<section id="api">
  <h2><code>getUserMedia</code> API <a href="#api" class="permalink">#</a></h2>

  <p>Let&#8217;s assume (because we&#8217;re optimists) that your user has a <code>getUserMedia</code>-enabled device. The API is nice and simple, as all Web APIs should be.</p>

  <p><code>navigator.getUserMedia</code> accepts two required arguments and an optional third.</p>

  <p>The first argument tells the device <b>which media you require access to</b>, and it&#8217;s passed as a JavaScript object. So, if you only require access to the microphone, the first argument would be <code>{audio: true}</code>; for video-chatting, you would use <code>{audio: true, video: true}</code>.</p>

  <p>The device decides which camera to use: <q>User agents are encouraged to default to using the user&#8217;s primary or system default camera and/or microphone (as appropriate) to generate the media stream.</q></p>

  <p>A previous version of the specification allowed hints to user agents about which camera to use. The API could specify &#8220;user&#8221; (the front camera on a phone) or &#8220;environment&#8221; (the rear camera on a phone). Debate continues about whether to reinstate this (<a href="http://lists.w3.org/Archives/Public/public-media-capture/2012Jan/0014.html">read the ongoing conversation</a>). One argument against is that letting sites know what cameras a device has could help Dr Malice of Evil Corp. fingerprint users after luring them to his site (which I find to be overly-cautious). <a href="http://lists.w3.org/Archives/Public/public-media-capture/2012Jan/0018.html">Harald Alvestrand poses a conundrum</a>: <q>conferencing units may have a room camera, a document camera and a lectern camera, neither of which is attached to the physical box; which one of these is <q>front</q>?</q></p>

  <p>But let&#8217;s not worry at the moment. Devices with more than one camera generally have a mechanism that allow the user to choose which one is used, so this gives the user control. Note also that the spec says <q>User agents may allow users to use any media source, including pre-recorded media files.</q></p>

  <p>The second argument is the <b>success callback</b>, the function to be executed on success, assuming that the user allows access and the device supports your request.</p>

  <p>There is an optional third argument. This is the <b>failure callback</b>, the function to be executed if something went wrong. It&#8217;s optional, but only optional in the sense that washing your hands before you eat is optional: if you don&#8217;t do it, you leave yourself open to all sorts of bugs and your mum will shout at you.</p>

  <p>Of course, it&#8217;s important to do feature detection before making an API call to ensure that the browser and device are capable, but even if feature detection succeeds, there is still much that can cause failure. For example, the user could deny permission or turn off the camera via a hardware switch, or the device itself could disable the camera.</p>

  <p>So here&#8217;s how we recommend you use the API:</p>

  <figure>
    <pre><code>navigator.getUserMedia({audio: true, video: true}, success, error);

function success(stream) { 
  // ... use 'stream' ... 
} 

function error(){ 
  // display fallback content 
}</code></pre>
    <figcaption>Using the gUM API</figcaption>
  </figure>

  <p>To check whether it&#8217;s working correctly, we can attach the output of the camera&#8217;s stream to the input of an HTML5 video on the page, like this:</p>

  <figure>
    <pre><code>&lt;video autoplay&gt;&lt;/video&gt;
&lt;script&gt;
if (navigator.getUserMedia) {
  navigator.getUserMedia({audio: true, video: true}, success, error);
  function success(stream)
  { 
    // ... use 'stream' ...
    var video = document.getElementsByTagName('video')[0];
    video.src = stream; 
  }
  // ...
&lt;/script&gt;</code></pre>
    <figcaption>Streaming captured video to an HTML5 <code>&lt;video&gt;</code> element</figcaption>
  </figure>

  <section id="differences-between-webkit-and-opera">
    <h3>Differences Between Webkit and Opera <a href="#differences-between-webkit-and-opera" class="permalink">#</a></h3>

    <p>The two rendering engines that implement getUserMedia do so in two different ways. Hopefully, they&#8217;ll harmonise when the features reach the released browsers.</p>

    <p>The differences:</p>

    <ul>
      <li>WebKit uses a prefix — <code>navigator.webkitGetUserMedia</code> — while Opera doesn&#8217;t.</li>
      <li>
        WebKit implements the options according to an old version of the spec, in which audio and video were passed as strings:
        <pre><code>navigator.webkitGetUserMedia("video, audio", /* ... */)</code></pre>
        whereas Opera implements the most recent version of the specification which uses JavaScript objects:
        <pre><code>navigator.getUserMedia({video: true, audio: true}, /* ... */)</code></pre>
      </li>
      <li>
        WebKit attaches the resulting stream like this:
        <pre><code>video.src = window.webkitURL.createObjectURL(stream);</code></pre>
        which is according to the current version of the spec. Opera uses:
        <pre><code>video.src = stream;</code></pre>
        and has proposed simplifying the spec (but discussion continues).
      </li>
    </ul>

    <p>But this fragmentation, even though it&#8217;s only in experimental browser releases is, like, a total drag, man.</p>
  </section>

  <section id="gUM-shield">
    <h3>Introducing The gUM Shield <a href="#gUM-shield" class="permalink">#</a></h3>

    <p>Fortunately, we&#8217;ve got your back. Two of the finest minds I know — actually, two of the finest <em>Mikes</em>, Doctor <a href="http://twitter.com/akamike">Mike Robinson</a> and Opera&#8217;s <a href="http://twitter.com/miketaylr">Mike Taylor</a> — have created a snippet of script that you can copy to shield you from these annoying differences. We&#8217;ve named this <code>getUserMedia</code> syntax normalisation script snippet <a href="https://gist.github.com/f2ac64ed7fc467ccdfe3">The gUM Shield</a>.

    <p>It assumes that there&#8217;s a variable called <var>video</var> to which you&#8217;ve assigned a reference to the <code>&lt;video&gt;</code> element you&#8217;ll stream to. Do this with some mechanism like:</p>

    <pre><code>video = document.getElementById('video');</code></pre>

    <p>Simply <a href="https://gist.github.com/f2ac64ed7fc467ccdfe3">hit the &#8216;Hub</a>, download it, plug it into your script, and <a href="http://jsbin.com/avaxaq/edit#preview">check it out</a>!</p>

    <p>Note that it&#8217;s <em>experimental</em>. Who knows what&#8217;ll happen when Mozilla, Microsoft, and Apple implement it, but please, have a play. Fork it and improve it!</p>
  </section>
</section>

<sectino id="exciting">
  <h2>Doing Exciting Things with the Video Stream <a href="#exciting" class="permalink">#</a></h2>

  <p>So far, we&#8217;re only echoing the video stream. But, as you&#8217;ll know from Tab Atkins&#8217; guest article <a href="http://html5doctor.com/video-canvas-magic/">video + canvas = magic</a>, if we copy the video into a canvas, we can manipulate it:</p>

  <figure>
    <pre><code>var video = document.querySelector('video');
var canvas = document.querySelector('canvas');
var ctx = canvas.getContext('2d');

ctx.drawImage(video, 0, 0, video.width, video.height,
  0, 0, canvas.width, canvas.height);</code></pre>
    <figcaption>Canvas and video magic</figcaption>
  </figure>

  <p>A common mistake is assuming that this somehow establishes a &#8220;live connection&#8221; between the video stream and the canvas. It doesn&#8217;t. It only grabs the current frame. So you need to grab the current frame repeatedly in order to replicate a video. The magical number is 15 times a second (or every 67 milliseconds), but a slower frame rate would probably be okay for this demo:</p>

  <figure>
    <pre><code>setInterval(function () { 
  ctx.drawImage(video, 0, 0, video.width, video.height,
  0, 0, canvas.width, canvas.height);
}, 1000 / 67);</code></pre>
    <figcaption>Live updating of <code>&lt;canvas&gt;</code> from a <code>&lt;video&gt;</code></figcaption>
  </figure>

  <p>See the <a href="http://jsbin.com/ahozok/edit#html,live">working demo</a>.</p>

  <p>What I want to do now is to grab the dominant colour from the canvas. I&#8217;m stealing <a href="http://experimenting.in/exp/real-life-color-picker/">a demo by Shwetank Dixit</a> (who borrowed from <a href="http://www.lokeshdhakar.com/">Lokesh Dhakar</a>&#8216;s <a href="http://www.lokeshdhakar.com/2011/11/03/color-thief/">Color Thief demo</a>). We won&#8217;t look at how it determines the dominant colour (because that&#8217;s nothing to do with getUserMedia).</p>

  <p><a href="http://html5doctor.com/demos/getusermedia/dominantcolor/">Try this in Opera Labs and Chrome Canary</a>!</p>

  <p>Unfortunately, this works splendidly in Opera but it occasionally fails in Chrome, and we&#8217;re not sure why. (We&#8217;ve emailed the Chrome team to ask them.) Apart from the rather anti-climactic end to an article, you shouldn&#8217;t be dismayed by this. <code>getUserMedia</code> is only available in experimental builds that have plenty of work to be done before they hit production.</p>

  <p>Back to our demo. Once you&#8217;ve got the dominant colour, you could prompt the user for a name for that colour (&#8220;carpet&#8221;, for example) and store that in local storage using its name as a key. Then, when you find a pair of curtains in the shop, you can find their dominant colour and easily compare it with that stored against &#8220;carpet&#8221; from earlier.</p>

  <p>And you can return home, confident of delighting the love of your life with your chromatically appropriate choice of soft furnishings. That&#8217;s HTML5: enriching marriages since 2004.</p>

  <img src="http://html5doctor.com/wp-content/uploads/2012/01/getUserMedia1.jpg" alt="woman says &#039;hey daddy-o, the curtains really razz my berries, ya dig?&#039;. Man replies &#039;Baby, I totally dig. Early night?&#039;. A thought bubble shows him thinking &#039;Thank you, getUserMedia&#039;" width="570" height="390" class="aligncenter size-full wp-image-4336" />
</section>

<section id="harshing-your-mellow">
  <h2>Harshing Your Mellow <a href="#harshing-your-mellow" class="permalink">#</a></h2>

  <p>Things need refining before web apps with <code>getUserMedia</code> are on a par with native applications. At the moment, there is no way of controlling which camera is to be used (and no agreement that a developer should be able to). The camera&#8217;s <a href="http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2011-July/032471.html">flash mechanism isn&#8217;t programmatically controllable</a>, which could harm the user experience. Most importantly are the privacy implications of web pages being able to access your camera. Opera has an <a href="http://dev.opera.com/articles/view/getusermedia-access-camera-privacy-ui/">experimental privacy UI</a>, but Chrome has no UI yet.</p>
</section>

<section id="demos">
  <h2>More Demos <a href="#demos" class="permalink">#</a></h2>

  <p>See Paul Neave&#8217;s <a href="http://neave.com/webcam/html5/">HTML5 Webcam Toy</a> and the links at the foot of my <a href="http://dev.opera.com/articles/view/getusermedia-access-camera-privacy-ui/">Opera Labs article</a>.</p>
</section>

<section id="thanks">
  <h2>Thanks <a href="#thanks" class="permalink">#</a></h2>

  <p>Thank for help and code are due to <a href="http://twitter.com/robinberjon">Robin Berjon</a>, <a href="http://twitter.com/shwetank">Shwetank Dixit</a>, <a href="http://twitter.com/ourmaninjapan">Daniel Davis</a>, <a href="http://www.lokeshdhakar.com/">Lokesh Dhakar</a>, <a href="http://twitter.com/miketaylr">Mike Taylor</a>, <a href="http://twitter.com/akamike">Mike Robinson</a>, and <a href="http://twitter.com/richtibbett">Rich Tibbett</a>.</p>
</section>
<section>
<h2>Addy Osmani&#8217;s Polyfill</h2>
<p>Added 13 March 2012: Addy Osmani has published a <a href="https://github.com/addyosmani/getUserMedia.js">getUserMedia polyfill</a> called getUserMedia.js, which uses Flash if gUM isn&#8217;t available. It&#8217;s not ready for production yet, as IE is still being tested. Forkers may hit the &#8216;hub.
</section><div id="crp_related"><h3>Related Posts:</h3><ul class="related"><li><a href="http://html5doctor.com/how-to-get-all-the-browsers-playing-ball/" rel="bookmark" class="crp_title">How to get all the browsers playing ball</a></li><li><a href="http://html5doctor.com/using-modernizr-to-detect-html5-features-and-provide-fallbacks/" rel="bookmark" class="crp_title">Using Modernizr to detect HTML5 features and provide fallbacks</a></li><li><a href="http://html5doctor.com/introducing-web-sql-databases/" rel="bookmark" class="crp_title">Introducing Web SQL Databases</a></li><li><a href="http://html5doctor.com/html5-briefing-notes-journalists-analysts/" rel="bookmark" class="crp_title">HTML5: briefing notes for journalists and analysts</a></li><li><a href="http://html5doctor.com/video-canvas-magic/" rel="bookmark" class="crp_title">video + canvas = magic</a></li></ul></div><p><a href="http://html5doctor.com/getusermedia/" rel="bookmark">It&#8217;s Curtains for Marital Strife Thanks to getUserMedia</a> originally appeared on <a href="http://html5doctor.com">HTML5 Doctor</a> on February 7, 2012.</p>
]]></content:encoded>
			<wfw:commentRss>http://html5doctor.com/getusermedia/feed/</wfw:commentRss>
		<slash:comments>6</slash:comments>
		</item>
		<item>
		<title>Server-Sent Events</title>
		<link>http://html5doctor.com/server-sent-events/</link>
		<comments>http://html5doctor.com/server-sent-events/#comments</comments>
		<pubDate>Tue, 24 Jan 2012 14:30:33 +0000</pubDate>
		<dc:creator>Remy Sharp</dc:creator>
				<category><![CDATA[Events]]></category>
		<category><![CDATA[JavaScript APIs]]></category>
		<category><![CDATA[api]]></category>
		<category><![CDATA[eventsource]]></category>
		<category><![CDATA[HTML 5]]></category>
		<category><![CDATA[html5]]></category>
		<category><![CDATA[realtime]]></category>
		<category><![CDATA[serversentevents]]></category>
		<category><![CDATA[websockets]]></category>

		<guid isPermaLink="false">http://html5doctor.com/?p=4181</guid>
		<description><![CDATA[We've already had a glimpse at Server-Sent Events (also known as EventSource, and I'll switch between the two to keep you on your toes) in my Methods of Communication article from last year. In this article, I want to delve in to more detail about the SSE API, demonstrate its features, and even show you how to polyfill browsers that lack EventSource support.]]></description>
			<content:encoded><![CDATA[<p>We&#8217;ve already had a glimpse at <a href="http://dev.w3.org/html5/eventsource/">Server-Sent Events</a> (also known as EventSource<sup>†</sup>, and I&#8217;ll switch between the two to keep you on your toes) in my <a href="http://html5doctor.com/methods-of-communication/">Methods of Communication</a> article from last year. In this article, I want to delve in to more detail about the SSE API, demonstrate its features, and even show you how to polyfill browsers that lack EventSource support.</p>

<p>Server-Sent Events are real-time events emitted by the server and received by the browser. They&#8217;re similar to WebSockets in that they happen in real time, but they&#8217;re very much a one-way communication method <em>from</em> the server.</p>

<p>These events are similar to ordinary JavaScript events that occur in the browser — like <em>click</em> events — except we can control the name of the event and the data associated with it.</p>

<p>All the code for this article is <a href="https://github.com/remy/eventsource-h5d">available on github</a>, and a <a href="http://node.remysharp.com:8004/">live demo is available online</a>.
<span id="more-4181"></span></p>

<p><small>† Is it Server-Sent Events or EventSource? Well, they both work. Server-Sent Events is the name of the API and specification. EventSource is the name of the JavaScript object you&#8217;re instantiating. It&#8217;s a bit like Ajax and XHR, where XHR refers to the XMLHttpRequest object…kinda.</small></p>

<p><small>Two notes: a) the uptime for this example is, I&#8217;m afraid, usually rather low — good for my server, bad for you. If you test the demo locally it will give you more interesting figures. b) IE6 isn&#8217;t supported in any of this article.</small></p>

<p><section id="applications"></p>

<h2>Possible Applications <a href="#applications" class="permalink">#</a></h2>

<p>A few simple examples of applications that could make use of Server-Sent Events:</p>

<ul>
<li>A real-time chart of streaming stock prices</li>
<li>Real-time news coverage of an important event (posting links, tweets, and images)</li>
<li>A live Twitter wall fed by Twitter&#8217;s streaming API</li>
<li>A monitor for server statistics like uptime, health, and running processes</li>
</ul>

<p>We&#8217;ll use the server monitor for this article&#8217;s examples. If this application were to be used in the wild, we could also check the <code>EventSource</code>&#8216;s connection state to indicate when there&#8217;s a potential problem connecting to the server.</p>

<p></section></p>

<p><section id="api"></p>

<h2>Overview of the API <a href="#api" class="permalink">#</a></h2>

<p>The client-side API is rather simple, and it hands-down beats the insane hacks required to get real-time events to the browser back in the bad old days.</p>

<p>The main points of interest:</p>

<ul>
<li><code>new EventSource(url)</code> — this creates our <code>EventSource</code> object, which immediately starts listening for events on the given URL.</li>
<li><code>readyState</code> — as with XHR, we have a <code>readyState</code> for the <code>EventSource</code> that tells us if we&#8217;re connecting (0), open (1), or closed (2).</li>
<li><code>onopen</code>, <code>onmessage</code> — two events that we can listen for on the new <code>EventSource</code> object. By default, the <code>message</code> event will fire when new messages are received, <em>unless</em> the server explicitly sets the event type.</li>
<li><code>addEventListener</code> — not only can we listen for the default <code>message</code> event, but we can also listen for custom messages using the <code>addEventListener</code> on the <code>EventSource</code> object, just as if we were listening for a <code>click</code> event.</li>
<li><code>event.data</code> — as with most messaging APIs, the contents of the message reside in the <code>data</code> property of the <code>event</code> object. This is a string, so if we want to pass around an object, we need to encode and decode it with JSON.</li>
<li><code>close</code> — closes the connection from the client side.</li>
</ul>

<p>In the future, EventSource will also support <a href="http://www.w3.org/TR/cors/">CORS</a> using an <code>options</code> argument to the <code>EventSource</code> object: <code>{ withCredentials: true }</code>. But at the time of writing, no stable release includes this property.</p>

<p></section></p>

<p><section id="example"></p>

<h2>Simple Example <a href="#example" class="permalink">#</a></h2>

<p>Our simple web app will notify us of server status messages — things like the load average, number of currently connected users, and most CPU-intensive processes. If I were using this application in anger, I&#8217;d probably build server modules that emit specific event types when they cross specific thresholds, so that I&#8217;m only notified when something gets to warning level.</p>

<p>This snippet of JavaScript connects to our server, listens for messages, and handles the data that comes with the messages:</p>

<pre><code>var source = new EventSource('/stats');
source.onopen = function () {
  connectionOpen(true);
};

source.onerror = function () {
  connectionOpen(false);
};

source.addEventListener('connections', updateConnections, false);
source.addEventListener('requests', updateRequests, false);
source.addEventListener('uptime', updateUptime, false);

source.onmessage = function (event) {
  // a message without a type was fired
};
</code></pre>

<p></section></p>

<p><section id="properties"></p>

<h2>Properties of Server-Sent Events <a href="#properties" class="permalink">#</a></h2>

<p>Server-Sent Events are more than just a one-way web socket. They have some unique features:</p>

<ul>
<li>The connection stream is <em>from</em> the server and read-only. This suits lots of applications, some examples of which I listed above.</li>
<li>They use regular HTTP requests for the persistent connection, not a special protocol, which means we can polyfill using vanilla JavaScript.</li>
<li>If the connection drops, the <code>EventSource</code> fires an error event and automatically tries to reconnect. The server can also control the timeout before the client tries to reconnect.</li>
<li>Clients can send a unique ID with messages. When a client tries to reconnect after a dropped connection, it will send the last known ID. Then the server can see that the client missed <em>n</em> messages and send the backlog of missed messages on reconnect.</li>
</ul>

<p></section></p>

<p><section id="message-format"></p>

<h2>Message Format <a href="#message-format" class="permalink">#</a></h2>

<p>A simple message doesn&#8217;t require much:</p>

<pre><code>data: this is a simple message
&lt;blank line&gt;
</code></pre>

<p>Note that the end of a message is indicated by a blank line (obviously not the literal characters <code>&lt;blank line&gt;</code>).</p>

<p>For a message with multiple lines:</p>

<pre><code>data: this is line one
data: and this is line two
</code></pre>

<p>You can send message IDs to be used if the connection is dropped:</p>

<pre><code>id: 33
data: this is line one
data: this is line two
</code></pre>

<p>You can even send multiple messages in a single response so long as you separate the messages by blank lines:</p>

<pre><code>id: 34
data: Remy is awesome

id: 35
data: Bruce is stinky
</code></pre>

<p>And you can specify your own event types (the above messages will all trigger the <code>message</code> event):</p>

<pre><code>id: 36
event: price
data: 103.34

id: 37
event: news
data: Bruce sells his collection of replica bananas
</code></pre>

<p>You don&#8217;t have to worry about this structure on the client side. It only applies to the server, which I&#8217;ll touch on next.</p>

<p></section></p>

<p><section id="server"></p>

<h2>Typical Server <a href="#server" class="permalink">#</a></h2>

<p>I&#8217;m not going to give a full walkthrough of the server-side code, since this is an HTML5 web site :) But there are a few important and simple features that you need to know to build the server (you&#8217;ll need this part anyway if you&#8217;re going to use EventSource).</p>

<p>I&#8217;ve included <a href="https://github.com/remy/eventsource-h5d">all the files for this demo on GitHub</a> for you to peruse at your own leisure, and I&#8217;ve also deployed a <a href="http://node.remysharp.com:8004">live example of this code</a>.</p>

<p>Ideally, you should use a server that has an <a href="http://en.wikipedia.org/wiki/Event_loop">event loop</a>. This means you should <em>not</em> use Apache, but instead use a platform such as <a href="http://nodejs.org">Node.js</a> (which I&#8217;ve used) or <a href="http://twistedmatrix.com/trac/">Twisted</a> for Python.</p>

<p>Key properties:</p>

<ol>
<li>You can only accept EventSource requests if the HTTP request says it can accept the <code>event-stream</code> MIME type.</li>
<li>You need to maintain a list of all the connected users in order to emit new events.</li>
<li>You should listen for dropped connections and remove them from the list of connected users.</li>
<li>You should optionally maintain a history of messages so that reconnecting clients can catch up on missed messages.</li>
</ol>

<p>Here&#8217;s a sample of my Node.js based server.  It&#8217;s using <a href="http://senchalabs.github.com/connect/">Connect</a> (a simple webby framework for Node). When it receives a request for <code>/stats</code>, it calls the following function. I&#8217;ve commented the code so you can follow along:</p>

<pre><code>function stats(request, response) {
  // only response to an event-stream if the request 
  // actually accepts an event-stream
  if (request.headers.accept == 'text/event-stream') {

    // send the header to tell the client that we're going
    // to be streaming content down the connection
    response.writeHead(200, {
      'content-type': 'text/event-stream',
      'cache-control': 'no-cache',
      'connection': 'keep-alive'
    });

    // support the polyfill - we'll come on to this later
    if (request.headers['x-requested-with'] == 'XMLHttpRequest') {
      response.xhr = null;
    }

    // if there was a lastEventId sent in the header, send
    // the history of messages we've already stored up
    if (request.headers['last-event-id']) {
      var id = parseInt(request.headers['last-event-id']);
      for (var i = 0; i &lt; history.length; i++) {
        if (history[i].id &gt;= id) {
          sendSSE(response, history[i].id, history[i].event, history[i].message);
        }
      }
    } else {
      // if the client didn't send a lastEventId, it's the
      // first time they've come to our service, so send an
      // initial empty message with no id - this will reset
      // their id counter too.
      response.write('id\n\n');
    }

    // cache their connection - the response is where we write
    // to send messages
    connections.push(response);

    // send a broadcast message to all connected clients with
    // the total number of connections we have.
    broadcast('connections', connections.length);

    // if the connection closes to the client, remove them
    // from the connections array.
    request.on('close', function () {
      removeConnection(response);
    });
  } else {
    // if the client doesn't accept event-stream mime type,
    // send them the regular index.html page - you could do
    // anything here, including sending the client an error.
    response.writeHead(302, { location: "/index.html" });
    response.end();
  }
}
</code></pre>

<p>The important trick on the server is to ensure you <em>don&#8217;t</em> close the connection to the <code>EventSource</code> object. If you do, it will of course handle the closed connection, fire an error event, and try to reconnect. So you&#8217;ll just want to maintain the persistent connection.</p>

<p></section></p>

<p><section id="polyfills"></p>

<h2>Polyfills and Tweaks to the Server <a href="#polyfills" class="permalink">#</a></h2>

<p>There are two good polyfills I know of, and though I prefer the one I wrote, I&#8217;ll still lay them both out for you.</p>

<p><section id="yaffle"></p>

<h3>Yaffle&#8217;s Polyfill <a href="#yaffle" class="permalink">#</a></h3>

<p>The first is one by <a href="https://github.com/Yaffle">Yaffle</a> (on github), available on <a href="https://github.com/Yaffle/EventSource">github/Yaffle/eventsource</a>. </p>

<p>The cons:</p>

<ul>
<li>It doesn&#8217;t send the <code>accepts</code> header, and</li>
<li>It completely replaces the <code>EventSource</code> object, so even if your browser supports <code>EventSource</code> natively, this script will replace it. But there&#8217;s a good reason for that.</li>
</ul>

<p>The pros:</p>

<ul>
<li>It maintains a persistent connection (whereas the one we&#8217;re using doesn&#8217;t), and</li>
<li>More interestingly, it supports CORS (which I imagine is why it replaces the native <code>EventSource</code>).</li>
</ul>

<p>These two pros are quite compelling. But when I was testing, I couldn&#8217;t get it working in IE7 (which was my minimum browser target), so that might be a blocker for you…or not.</p>

<p></section></p>

<p><section id="remy"></p>

<h3>Remy&#8217;s Polyfill <a href="#remy" class="permalink">#</a></h3>

<p>The second is my own, available on <a href="https://github.com/remy/polyfills/blob/master/EventSource.js">github.com/remy/polyfills</a>.</p>

<p>The cons:</p>

<ul>
<li>It uses polling, so once a small group of messages come down, it re-establishes the connection, which could lead to significant overhead (though less so on a Node-based server). You have to add about 4 extra lines to your server code.</li>
</ul>

<p>The pros:</p>

<ul>
<li>It doesn&#8217;t replace the native <code>EventSource</code> object (but that also implies that, for now, it won&#8217;t support CORS), and</li>
<li>It supports IE7.</li>
</ul>

<p><small>Retrospectively, I might choose Yaffle&#8217;s polyfill over mine in the future if I wasn&#8217;t bothered about IE7 support.</small></p>

<p></section></p>

<p><section id="using-the-polyfill"></p>

<h3>Using the Polyfill <a href="#using-the-polyfill" class="permalink">#</a></h3>

<p>By including the <code>EventSource.js</code> JavaScript library before my client-side code, I just need a couple of small changes to my server-side code. Other than that, everything on the client side works without any changes (as a polyfill <em>should</em> work).</p>

<p>When posting the server&#8217;s reply to the client, instead of keeping the connection open, I include the following in my server when it&#8217;s finished writing the response:</p>

<pre><code>// send the data (event type, id, message, etc)
response.write(data);

// if our response contains our custom xhr property, then...
if (response.hasOwnProperty('xhr')) {
  // clear any previous timers using the xhr prop as the value
  clearTimeout(response.xhr);

  // now set a timer for 1/4 second (abritrary number) that
  // then closes the connection and removes itself from the
  // connection array.
  // The delay is in place so that a burst of messages can 
  // go out on the same connection *before* it's closed.
  response.xhr = setTimeout(function () {
    response.end();
    removeConnection(response);
  }, 250);
}
</code></pre>

<p></section>
</section></p>

<p><section id="bugs"></p>

<h2>Bugs? <a href="#bugs" class="permalink">#</a></h2>

<p>The <code>error</code> event should always fire when the <code>readyState</code> changes, assuming you didn&#8217;t explicitly call the <code>close</code> method. This works nearly all the time, but in writing this article, I found a few edge cases where it doesn&#8217;t fire. In Chrome, if I put my machine to sleep and then woke it back up, it would close the connection but not fire the event, therefore never triggering a reconnection. As I said, this is an edge case and I&#8217;ll file a bug against it, so I don&#8217;t expect it to hang around for long.</p>

<p></section></p>

<p><section id="why-not-websockets"></p>

<h2>Why Not Use WebSockets? <a href="#why-not-websockets" class="permalink">#</a></h2>

<p>There are two reasons I&#8217;d advocate using EventSource over WebSockets, as they&#8217;re currently the two contenders for sending real-time events to the browser.</p>

<p>The first is that EventSource (as we saw earlier) works over regular HTTP and can therefore be replicated entirely using JavaScript if it&#8217;s not available natively. That means that we can polyfill browsers without support, like IE9.</p>

<p>The second is probably more important: you should always use the right technology for the job. If your real-time data is sourced from your web site, <em>and</em> the user doesn&#8217;t interact in real-time, it&#8217;s likely you need Server-Sent Events. </p>

<p>I recently saw a cool <a href="http://paynedigital.com/2011/12/nodeflakes">demo of snowflakes</a> drifting down a web site. Each snowflake is a tweet based around the Christmas theme — like if someone mentions a particular Christmas-y term, it&#8217;s sucked in to the snowflake. Don&#8217;t get me wrong, I know this is a demo, and it&#8217;s very cool (if you wrote it, this is me sending you hugs), but it&#8217;s based on WebSockets. I&#8217;d suggest this demo should be based on EventSource since all the data is read-only and the user doesn&#8217;t interact with it at all.</p>

<p>The point: evaluate the technology against your problem, and aim to get good fit.</p>

<p></section>
<div id="crp_related"><h3>Related Posts:</h3><ul class="related"><li><a href="http://html5doctor.com/methods-of-communication/" rel="bookmark" class="crp_title">Methods of communication</a></li><li><a href="http://html5doctor.com/storing-data-the-simple-html5-way-and-a-few-tricks-you-might-not-have-known/" rel="bookmark" class="crp_title">Storing Data the Simple HTML5 Way (and a few tricks you might not have known)</a></li><li><a href="http://html5doctor.com/drag-and-drop-to-server/" rel="bookmark" class="crp_title">Drag and Drop and Automatically Send to the Server</a></li><li><a href="http://html5doctor.com/history-api/" rel="bookmark" class="crp_title">Pushing and Popping with the History API</a></li><li><a href="http://html5doctor.com/how-to-get-all-the-browsers-playing-ball/" rel="bookmark" class="crp_title">How to get all the browsers playing ball</a></li></ul></div></p>
<p><a href="http://html5doctor.com/server-sent-events/" rel="bookmark">Server-Sent Events</a> originally appeared on <a href="http://html5doctor.com">HTML5 Doctor</a> on January 24, 2012.</p>
]]></content:encoded>
			<wfw:commentRss>http://html5doctor.com/server-sent-events/feed/</wfw:commentRss>
		<slash:comments>26</slash:comments>
		</item>
		<item>
		<title>The contenteditable attribute</title>
		<link>http://html5doctor.com/the-contenteditable-attribute/</link>
		<comments>http://html5doctor.com/the-contenteditable-attribute/#comments</comments>
		<pubDate>Tue, 10 Jan 2012 15:10:16 +0000</pubDate>
		<dc:creator>Jack Osborne</dc:creator>
				<category><![CDATA[Attributes]]></category>
		<category><![CDATA[contenteditable]]></category>
		<category><![CDATA[HTML 5]]></category>
		<category><![CDATA[html5]]></category>

		<guid isPermaLink="false">http://html5doctor.com/?p=3796</guid>
		<description><![CDATA[<p>For some time now, we’ve been using various technologies to edit and store text within a web browser. Now with the <code>contenteditable</code> attribute, things have got a whole lot easier. In this article, I’ll tell you what this attribute is for, how it works, and how we can take things further.</p>
]]></description>
			<content:encoded><![CDATA[<p>For some time now, we’ve been using various technologies to edit and store text within a web browser. Now with the <code>contenteditable</code> attribute, things have got a whole lot easier. In this article, I’ll tell you what this attribute is for, how it works, and how we can take things further.</p>

<section id="basics">
  <h2>The Basics <a href="#basics" class="permalink">#</a></h2>

  <p>First, let’s check out the spec:</p>

  <blockquote>
    <p>The <code>contenteditable</code> attribute is an enumerated attribute whose keywords are the empty string, true, and false. The empty string and the true keyword map to the true state. The false keyword maps to the false state. In addition, there is a third state, the inherit state, which is the missing value default (and the invalid value default).</p>
    <footer>— <cite><a href="http://www.whatwg.org/specs/web-apps/current-work/multipage/editing.html#contenteditable" title="contenteditable attribute — HTML5">WHATWG</a></cite></footer>
  </blockquote>

  <p>The <code>contenteditable</code> attribute was mainly intended for providing an in-browser rich-text or WYSIWYG experience. You’ve likely seen this sort of thing in blog-based authoring tools like Symphony or sites like Flickr where you can begin editing page content simply by clicking in a given area.</p>

  <p>As mentioned above, <code>contenteditable</code> has three possible states:</p>

  <dl>
    <dt><code>contenteditable=""</code> or <code>contenteditable="true"</code></dt>
    <dd>Indicates that the element is editable.</dd>

    <dt><code>contenteditable="false"</code></dt>
    <dd>Indicates that the element is not editable.</dd>

    <dt><code>contenteditable="inherit"</code></dt>
    <dd>Indicates that the element is editable if its immediate parent element is editable. This is the default value.</dd>
  </dl>

  <p>When you add <code>contenteditable</code> to an element, the browser will make that element editable. In addition, any children of that element will also become editable unless the child elements are explicitly <code>contenteditable="false"</code>.</p>
</section>

<section id="code-example">
  <h2>Code Example <a href="#code-example" class="permalink">#</a></h2>

  <p>Here’s some example code to get us started:</p>

  <figure>
    <pre><code>&#60;div id="example-one" contenteditable="true"&#62;
&#60;style scoped&#62;
  #example-one { margin-bottom: 10px; }
  [contenteditable="true"] { padding: 10px; outline: 2px dashed #CCC; }
  [contenteditable="true"]:hover { outline: 2px dashed #0090D2; }
&#60;/style&#62;
&#60;p&#62;Everything contained within this div is editable in browsers that support &#60;code&#62;HTML5&#60;/code&#62;. Go on, give it a try: click it and start typing.&#60;/p&#62;
&#60;/div&#62;
    </code></pre>
    <figcaption>Putting it together</figcaption>
  </figure>
</section>

<section id="live-examples">
  <h2>Live examples <a href="#live-examples" class="permalink">#</a></h2>
  
  <p>Here are two basic-but-live examples showing what you can do with <code>contenteditable</code>.</p>
  
  <section id="first-example">
    <h3>Example One <a href="#first-example" class="permalink">#</a></h3>
  
    <div id="example-one" contenteditable="true">
      <figure>
        <style scoped>
          #example-one { margin: 1.5em 0; }
          [contenteditable="true"] { padding: 10px; outline: 2px dashed #CCC; }
          [contenteditable="true"]:hover { outline: 2px dashed #0090D2; }
        </style>
        <p>Everything contained within this div is editable in browsers that support <code>HTML5</code>. Go on, give it a try: click it and start typing.</p>
        <figcaption>Live text editing</figcaption>
      </figure>
    </div>

    <p>I’ve used CSS to create an obvious visual cue that this area of text is different from the rest. Note that I’ve future-proofed this with <code>&#60;style scoped&#62;</code>, which <a href="http://html5doctor.com/the-scoped-attribute/">I covered in my previous article</a>.</p>
  </section>
  
  <section id="second-example">
    <h3>Example Two <a href="#second-example" class="permalink">#</a></h3>
  
    <p><a href="http://twitter.com/chriscoyier">Chris Coyier</a> of CSS-Tricks pointed out that you can let your users <a href="http://css-tricks.com/show-and-edit-style-element/">edit the CSS</a> in real-time. Because the <code>&lt;style&gt;</code> element is set to <code>display: none</code> by the user agent, we need to set it to <code>block</code> for this code to work.</p>

    <p>Try editing the <code>CSS</code> below:</p>

    <div id="example-two" contenteditable="true">
      <figure>
        <div id="style-block">
          <style contenteditable>
          #example-two {
          background: #fff;
          color: #444;
          }
          [contenteditable="true"]{
          padding: 10px;
          outline: 3px dashed #CCC;
          }
          [contenteditable="true"]:hover{
          background: rgba(255, 255, 153, 1);
          outline: 3px dashed #0090D2;
          }
          </style>
        </div>
        <figcaption>Live CSS editing</figcaption>
      </figure>
    </div>
  </section>
</section>

<section id="browser-support">
  <h2>Browser Support <a href="#browser-support" class="permalink">#</a></h2>
  
  <p>Browser support for <code>contenteditable</code> is surprisingly good:</p>
  
  <table class="wide">
    <caption>Browser Support for <code>contenteditable</code></caption>
    <thead>
      <tr>
        <th scope="col">Browser</th>
        <th scope="col">Version</th>
      </tr>
    </thead>
    <tbody>
      <tr>
        <th scope="row">Chrome</th>
        <td>4.0+</td>
      </tr>
      <tr>
        <th scope="row">Safari</th>
        <td>3.1+</td>
      </tr>
      <tr>
        <th scope="row">Mobile Safari</th>
        <td>5.0+</td>
      </tr>
      <tr>
        <th scope="row">Firefox</th>
        <td>3.5+</td>
      </tr>
      <tr>
        <th scope="row">Opera</th>
        <td>9.0+</td>
      </tr>
      <tr>
        <th scope="row">Opera Mini/Mobile</th>
        <td>N/A</td>
      </tr>
      <tr>
        <th scope="row">Internet Explorer</th>
        <td>5.5+</td>
      </tr>
      <tr>
        <th scope="row">Android</th>
        <td>3.0+</td>
      </tr>
    </tbody>
  </table>

  <p>We have to give credit where it’s due and point out that Internet Explorer has supported this attribute since IE5.5. In fact, a very early iteration of <code>contenteditable</code> was <a href="http://msdn.microsoft.com/en-us/library/ms537837(VS.85).aspx">designed and implemented by Microsoft in July 2000</a>.</p>

  <p>For a more in-depth compatibility table, visit <a href="http://caniuse.com/contenteditable">When Can I Use…</a>.</p>
</section>

<section id="storing-changes">
  <h2>Storing the Changes <a href="#storing-changes" class="permalink">#</a></h2>

  <p>For this section, I went straight to Doctor Remy for help, as he is much more knowledgeable than me when it comes to <del>storing your data</del> everything.</p>

  <blockquote>
    <p>Depending on the complexity of your editable block, your code could be listening for the enter key (keyCode 13) to save the changes, and the escape key (keyCode 27) to undo the changes.</p>

    <p>When the user hits enter (assuming it’s a single line of editable content), it would grab the innerHTML of the editable field and send an Ajax post to your server with the change.</p>

    <p>A simple example of this can be seen on JS Bin: <a href="http://jsbin.com/owavu3">contenteditable saving to Ajax</a></p>

    <footer>— <cite><a href="http://html5doctor.com/author/remys/" title="Remy Sharp on storing content changes with the contenteditable attribute">Remy Sharp</a></cite></footer>
  </blockquote>
</section>

<section id="conclusion">
  <h2>Conclusion <a href="#conclusion" class="permalink">#</a></h2>
  
  <p>We’ve repeated the phrase “paving the cowpaths” all over this site, and I’m mentioning it again with the introduction of the <code>contenteditable</code> attribute. The spec finally makes official something that’s been implemented by browser makers for years.</p>

  <p>Although this is one of the lesser-known new attributes, I bet you’ll find yourself using it more often than you would think.</p>

  <p>Imagine being able to simply click a block of content and start making changes instantly: making quick corrections to an article in-place, allowing users to edit their comments, or even building spreadsheets within company applications that aren’t hooked up to any sort of back-end user interface.</p>

  <p>If you can think of other uses for this attribute, then head on down to the comments section and tell us where else you think this could be implemented.</p>
</section>

<section id="related-reading">
  <h2>Related Reading <a href="#related-reading" class="permalink">#</a></h2>
  
  <ul>
    <li><a href="http://blog.whatwg.org/the-road-to-html-5-contenteditable#what">What is <code>contenteditable</code>?</a></li>
    <li><a href="http://css-tricks.com/expanding-images-html5/">Expanding Images Using HTML5’s <code>contenteditable</code></a></li>
  </ul>
</section><div id="crp_related"><h3>Related Posts:</h3><ul class="related"><li><a href="http://html5doctor.com/native-drag-and-drop/" rel="bookmark" class="crp_title">Native Drag and Drop</a></li><li><a href="http://html5doctor.com/css3-pseudo-classes-and-html5-forms/" rel="bookmark" class="crp_title">CSS3 Pseudo-Classes and HTML5 Forms</a></li><li><a href="http://html5doctor.com/the-output-element/" rel="bookmark" class="crp_title">The output element</a></li><li><a href="http://html5doctor.com/ol-element-attributes/" rel="bookmark" class="crp_title">The <code>ol</code> Element and Related Attributes: <code>type</code>, <code>start</code>, <code>value</code>, and <code>reversed</code></a></li><li><a href="http://html5doctor.com/why-designers-should-care-about-html5/" rel="bookmark" class="crp_title">Why designers should care about HTML5</a></li></ul></div><p><a href="http://html5doctor.com/the-contenteditable-attribute/" rel="bookmark">The contenteditable attribute</a> originally appeared on <a href="http://html5doctor.com">HTML5 Doctor</a> on January 10, 2012.</p>
]]></content:encoded>
			<wfw:commentRss>http://html5doctor.com/the-contenteditable-attribute/feed/</wfw:commentRss>
		<slash:comments>22</slash:comments>
		</item>
		<item>
		<title>The output element</title>
		<link>http://html5doctor.com/the-output-element/</link>
		<comments>http://html5doctor.com/the-output-element/#comments</comments>
		<pubDate>Tue, 20 Dec 2011 14:30:20 +0000</pubDate>
		<dc:creator>Richard Clark</dc:creator>
				<category><![CDATA[Elements]]></category>
		<category><![CDATA[HTML 5]]></category>
		<category><![CDATA[html5]]></category>
		<category><![CDATA[output]]></category>

		<guid isPermaLink="false">http://html5doctor.com/?p=3976</guid>
		<description><![CDATA[<p>Across the web, you’ll see a range of sites that feature calculators for working out things like loan repayments, mortgage rates, tax, insurance, and more. Until now, we’ve had no way of semantically marking up the result of those calculations. Enter: the <code>&#60;output&#62;</code> element! In this article, we’ll show you <code>&#60;output&#62;</code> and some related JavaScript tricks. Let’s get cracking.</p> ]]></description>
			<content:encoded><![CDATA[<p>Across the web, you’ll see a range of sites that feature calculators for working out things like loan repayments, mortgage rates, tax, insurance, and more. Until now, we’ve had no way of semantically marking up the result of those calculations. Enter: the <code>&lt;output&gt;</code> element! In this article, we’ll show you <code>&lt;output&gt;</code> and some related JavaScript tricks. Let’s get cracking.</p> 

<section id="definition">
  <h2>The Definition <a href="#definition" class="permalink">#</a></h2>
  
  <p>The <code>&lt;output&gt;</code> element, new in HTML5, is used in forms. The WHATWG HTML specification describes <code>&lt;output&gt;</code> very simply:</p>
  
  <blockquote>
    <p>The output element represents the result of a calculation.</p>
<footer>&mdash; <cite><a href="http://www.whatwg.org/specs/web-apps/current-work/multipage/the-button-element.html#the-output-element">WHATWG HTML specification</a></cite></footer>
</blockquote>

  <p>Along with the standard <a href="http://www.whatwg.org/specs/web-apps/current-work/multipage/elements.html#global-attributes">global attributes</a>, <code>&lt;output&gt;</code> also accepts the following:</p>
  
  <dl> 
    <dt><code>for</code></dt>
    <dd>A space-separated list containing the IDs of the elements whose values went into the calculation.</dd>
    <dt><code>form</code></dt>
    <dd>Associates the <code>&lt;output&gt;</code> element with its <a href="http://www.whatwg.org/specs/web-apps/current-work/multipage/association-of-controls-and-forms.html#form-owner">form owner</a>. The value must be the ID of a form in the same document. This allows you to place an <code>&lt;output&gt;</code> element outside of the <code>&lt;form&gt;</code> with which it is associated.</dd>
    <dt><code>name</code></dt>
    <dd>The name of the element.</dd>
  </dl>
</section>

<section id="implementation">
  <h2>Implementing <code>&lt;output&gt;</code> <a href="#implementation" class="permalink">#</a></h2>
  
  <p>We’ll start by creating a simple example that adds two whole numbers together (Figure 1). We’ll use the new-in-HTML5 <code>number</code> input type and the <code>parseInt</code> JavaScript function to convert the input strings to integers:</p>
  
  <figure>
    <pre><code>&lt;form onsubmit="return false" oninput="o.value = parseInt(a.value) + parseInt(b.value)"&gt;
  &lt;input name="a" type="number" step="any"&gt; +
  &lt;input name="b" type="number" step="any"&gt; =
  <mark>&lt;output name="o"&gt;&lt;/output&gt;</mark>
&lt;/form&gt;</code></pre>
  
    <img src="http://html5doctor.com/wp-content/uploads/2011/10/output-construction.png" alt="Construction of a calculation using the output element" />
  
    <figcaption>Figure 1: A simple calculator, rendered in Chrome</figcaption>
  </figure>
  
  <aside class="sidenote">
    <p><a href="http://www.whatwg.org/specs/web-apps/current-work/multipage/common-input-element-attributes.html#the-step-attribute">The <code>step</code> attribute</a> is new to HTML5. It defines the increment by which a <code>number</code> input increases and decreases.</p>
  </aside>    
  
  <div class="callout highlight-block">
    <p>Notice that we’re using the now-standard <code>oninput</code> event, which has replaced the <a href="http://www.w3.org/Bugs/Public/show_bug.cgi?id=11129">deprecated <code>onforminput</code> event</a>. Daniel Friesen has written a detailed article on <a href="http://blog.danielfriesen.name/2010/02/16/html5-browser-maze-oninput-support/">the background and current support of <code>oninput</code></a>. <code>oninput</code> isn’t supported in IE8 and below, and its IE9 support is a little flaky, but you can work around these problems using <a href="http://www.useragentman.com/blog/2011/05/12/fixing-oninput-in-ie9-using-html5widgets/">the html5Widgets polyfill</a>.</p>
  </div>
  
  <p><a href="http://html5doctor.com/demos/output/1.html">See a live example of the code in Figure 1.</a></p>
  
  <p>As you’d expect, if you only enter a single value, the function returns <abbr title="Not a Number">NaN</abbr>. It’s effectively trying to add a number and an undefined value, and 1 + undefined = undefined.</p>
  
  <section id="for-attribute">
    <h3>Using the <code>for</code> Attribute <a href="#for-attribute" class="permalink">#</a></h3>
    
    <p>Let’s build on the previous example and add the <code>for</code> attribute to the <code>&lt;output&gt;</code> (Figure 2). We need to add IDs to each of the associated <code>&lt;input&gt;</code>s, just as we would with the <code>for</code> attribute on a <code>&lt;label&gt;</code>:</p>
    
    <figure>
      <pre><code>&lt;form onsubmit="return false" oninput="o.value = parseInt(a.value) + parseInt(b.value)"&gt;
  &lt;input name="a" id="a" type="number" step="any"&gt; +
  &lt;input name="b" id="b" type="number" step="any"&gt; =
  &lt;output name="o" <mark>for="a b"</mark>&gt;&lt;/output&gt;
&lt;/form&gt;</code></pre>

      <figcaption>Figure 2: Using the <code>for</code> attribute on the <code>&lt;output&gt;</code> element</figcaption>
    </figure>

    <p><a href="http://html5doctor.com/demos/output/2.html">See a live example of the code in Figure 2.</a></p>
  </section>
  
  <section id="valueAsNumber">
    <h3>The <code>valueAsNumber</code> Property <a href="#valueAsNumber" class="permalink">#</a></h3>
    
    <p>HTML5 has also introduced the <code>valueAsNumber</code> property of JavaScript input objects (specifically those of type number, date, and range). This returns the value as a number rather than as a string, meaning we no longer need to use <code>parseInt</code> or <code>parseFloat</code>, and the <code>+</code> operator adds rather than concatenates:</p>
    
    <figure>
      <pre><code>&lt;form onsubmit="return false" <mark>oninput="o.value = a.valueAsNumber + b.valueAsNumber"</mark>&gt;
  &lt;input name="a" id="a" type="number" step="any"&gt; +
  &lt;input name="b" id="b" type="number" step="any"&gt; =
  &lt;output name="o" for="a b"&gt;&lt;/output&gt;
&lt;/form&gt;</code></pre>

      <figcaption>Figure 3: Using the <code>valueAsNumber</code> property for retrieving a numeric value from an input</figcaption>
    </figure>

    <p><a href="http://html5doctor.com/demos/output/3.html">See a live example of the code in Figure 3.</a></p>
  </section>
</section>

<section id="invoicing-calculator">
  <h2>Invoicing Calculator: An In-depth Example with Fallbacks <a href="#invoicing-calculator" class="permalink">#</a></h2>
  
  <p>For a more realistic example, let’s create an invoicing calculator that multiplies the number of hours by the hourly rate and adds tax to give an overall total (Figure 4):</p>
  
  <figure>
    <pre><code>&lt;form onsubmit="return false" oninput="amount.value = (hours.valueAsNumber * rate.valueAsNumber) + ((hours.valueAsNumber * rate.valueAsNumber) * vat.valueAsNumber)"&gt;
  &lt;legend&gt;Invoice&lt;/legend&gt;

  &lt;p&gt;&lt;label for="hours"&gt;Number of hours&lt;/label&gt;
  &lt;input type="number" min="0" id="hours" name="hours"&gt;&lt;/p&gt;

  &lt;p&gt;&lt;label for="rate"&gt;Rate&lt;/label&gt;
  &lt;span&gt;&pound;&lt;/span&gt;&lt;input type="number" min="0" id="rate" name="rate"&gt;&lt;/p&gt;

  &lt;p&gt;&lt;label for="vat"&gt;VAT&lt;/label&gt;
  &lt;input type="number" min="0" id="vat" value="0.20" name="vat"&gt;&lt;/p&gt;

  &lt;p&gt;Total: &lt;strong&gt;&pound;&lt;output name="amount" for="hours rate vat"&gt;0&lt;/output&gt;&lt;/strong&gt;&lt;/p&gt;
&lt;/form&gt;</code></pre>

    <img src="http://html5doctor.com/wp-content/uploads/2011/10/invoice-calc.png" alt="Invoice tax calculator using the output element" />

    <figcaption>Figure 4: An invoicing calculator that renders its result in an <code>output</code> element</figcaption>
  </figure>

  <p><a href="http://html5doctor.com/demos/output/4.html">See a live example of the code in Figure 4.</a></p>
  
  <p>Nothing too complicated going on there. In fact, the scripting is so simple that even I can do it ;)</p>
</section>

<section id="input-range">
  <h2>A Mildly Controversial Example Using <code>&lt;input type="range"&gt;</code> <a href="#input-range" class="permalink">#</a></h2>
  
  <p>HTML5 also introduces the <code>range</code> input type, which renders as a slider control in supporting browsers. With this input type, a user can enter an approximate value within a given range without having to be completely precise or directly type a numeric value. For cross browser compatibility, see <a href="http://remysharp.com/2011/07/18/input-range-polyfill/">Remy’s input range polyfill</a>.</p>
  
  <p>While researching this article, I found a number of examples using the <code>&lt;output&gt;</code> element in conjunction with <code>&lt;input type="range"&gt;</code> as shown in Figure 5:</p>
  
  <figure>
    <pre><code>&lt;form onsubmit="return false" oninput="level.value = flevel.valueAsNumber"&gt;
  &lt;label for="flying"&gt;Flying Skill Level&lt;/label&gt;
  &lt;input name="flevel" id="flying" type="range" min="0" max="100" value="0"&gt; 
  &lt;output for="flying" name="level"&gt;0&lt;/output&gt;/100
&lt;/form&gt;</code></pre>

    <img src="http://html5doctor.com/wp-content/uploads/2011/10/output-range.png" alt="Output example using type=range" />

    <figcaption>Figure 5: Using <code>&lt;input type="range"&gt;</code> with the <code>&lt;output&gt;</code> element</figcaption>
  </figure>
  
  <p><a href="http://html5doctor.com/demos/output/5.html">See a live example of the code in Figure 5.</a></p>
  
  <p>Using <code>&lt;output&gt;</code> to show the current value to the user seems like a perfectly reasonable use case to me, but it isn’t <q cite="http://www.whatwg.org/specs/web-apps/current-work/multipage/the-button-element.html#the-output-element">the result of a calculation</q> as the spec describes. I haven’t spoken directly to Hixie about this, but a couple of people on the IRC channel seemed to agree with, so I filed a <a href="http://www.w3.org/Bugs/Public/show_bug.cgi?id=14379">bug report</a> to request the definition be amended. I’ll update this article if something changes.</p>
</section>  

<section id="support">
  <h2>Browser Support and Polyfills <a href="#support" class="permalink">#</a></h2>
  
  <p>The good news is that all modern browsers support the <code>&lt;output&gt;</code> element to some extent. The bad news is that there are still a few gotchas.</p>
  
  <table class="wide">
    <caption>Browser support for the <code>&lt;output&gt;</code> element</caption>
    <thead>
      <tr>
        <th scope="col">Browser</th>
        <th scope="col">Support</th>
      </tr>
    </thead>
    <tbody>
      <tr>
        <th scope="row">Chrome</th>
        <td>13+</td>
      </tr>
      <tr>
        <th scope="row">Safari</th>
        <td>5.1+</td>
      </tr>
      <tr>
        <th scope="row">Firefox*</th>
        <td>6+</td>
      </tr>
      <tr>
        <th scope="row">Opera</th>
        <td>9.20+</td>
      </tr>
      <tr>
        <th scope="row">Internet Explorer</th>
        <td>10+</td>
      </tr>
    </tbody>
  </table>
  
  <p><small>* Firefox does support the <code>&lt;output&gt;</code> element and the <code>oninput</code> event, but <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=636737">it doesn’t support the <code>valueAsNumber</code> property</a>.</small></p>
  
  <p>All the examples we’ve seen so far should work perfectly in Opera 11.5+, Safari 5.1+, and Chrome 13+. IE, as you might expect, is lagging somewhat, but support for <code>&lt;output&gt;</code> is in IE10 Platform Preview 2, and support for <code>oninput</code> is already in IE9.</p>
  
  <p>In order to work around this in our simple examples, we’ll need to revert to the tried-and-trusted <code>getElementByID</code> and <code>parseFloat</code> (Figure 6):</p>
  
  <figure>
    <pre><code>&lt;form onsubmit="return false" <mark>oninput="document.getElementById('o').innerHTML = parseFloat(document.getElementById('a').value) + parseFloat(document.getElementById('b').value)"</mark>&gt;
  &lt;input name="a" id="a" type="number" step="any"&gt; +
  &lt;input name="b" id="b" type="number" step="any"&gt; =
  &lt;output name="o" id="o" for="a b"&gt;&lt;/output&gt;
&lt;/form&gt;</code></pre>

    <figcaption>Figure 6: Using <code>getElementById</code> to support older browsers</figcaption>
  </figure>

  <p><a href="http://html5doctor.com/demos/output/6.html">See a live example of the code in Figure 6.</a></p>

  <p>While this isn’t ideal, it’s workable until those less-capable browsers die a slow and painful death. Alternatively, you can use polyfills to plug the gaps.</p>
  
  <p>To check support in your current browser, <a href="http://miketaylr.com/code/html5-output-element-support.html">open up Mike Taylor’s support test page</a>.
  
  <section id="polyfills">
    <h3>Polyfills <a href="#polyfills" class="permalink">#</a></h3>
    
    <p>To support less-capable browsers, there are polyfills such as <a href="https://github.com/zoltan-dulac/html5Widgets">HTML5 Widgets</a> created by <cite><a href="http://www.useragentman.com">Zoltan &#8220;Du Lac&#8221; Hawryluk</a></cite>. Zoltan talks you through the process in <a href="http://www.useragentman.com/blog/2010/07/27/cross-browser-html5-forms-using-modernizr-webforms2-and-html5widgets/">this detailed article on creating cross browser forms using html5Widgets</a>.</p>
  </section>
</section>

<section id="summary">
  <h2>Summary <a href="#summary" class="permalink">#</a></h2>
  
  <p>You probably won&#8217;t find yourself using the <code>&lt;output&gt;</code> element all the time, but it’s useful in a whole host of situations. Calculating values on financial sites spring to mind, or outputting the current mouse position, or perhaps the goal difference in a table of sports teams. What other use cases can you think of for <code>&lt;output&gt;</code>? Let us know in the comments!</p>
</section>

<section id="links">
  <h2>Related reading <a href="#links" class="permalink">#</a></h2>
  
  <ul>
    <li><a href="http://www.whatwg.org/specs/web-apps/current-work/multipage/the-button-element.html#the-output-element">WHATWG HTML Specification for <code>&lt;output&gt;</code></a></li>
    <li><a href="http://www.w3.org/wiki/HTML/Elements/output">W3C Elements Wiki Page for <code>&lt;output&gt;</code></a></li>
    <li><a href="http://wufoo.com/html5/elements/3-output.html">Wufoo HTML5 Forms Tests for <code>&lt;output&gt;</code></a></li>
    <li><a href="https://developer.mozilla.org/en/HTML/Element/output">Mozilla Developer Network docs on <code>&lt;output&gt;</code></a></li>
    <li><a href="http://www.useragentman.com/blog/2011/05/10/is-onforminput-deprecated-in-html5-forms-and-why-should-i-care-anyways/">Is <code>onforminput</code> Deprecated in HTML5 Forms? (And Why Should I Care Anyways?) by <cite>Zoltan Hawryluk</cite></a></li>
    <li><a href="http://blog.danielfriesen.name/2010/02/16/html5-browser-maze-oninput-support/">A HTML5 Browser Maze: <code>oninput</code> Support by <cite>Daniel Friessen</cite></a></li>
    <li><a href="http://www.useragentman.com/blog/2011/05/12/fixing-oninput-in-ie9-using-html5widgets/">Fixing <code>oninput</code> in IE Using html5Widgets by <cite>Zoltan Hawryluk</cite></a></li>
  </ul>
</section><div id="crp_related"><h3>Related Posts:</h3><ul class="related"><li><a href="http://html5doctor.com/css3-pseudo-classes-and-html5-forms/" rel="bookmark" class="crp_title">CSS3 Pseudo-Classes and HTML5 Forms</a></li><li><a href="http://html5doctor.com/ol-element-attributes/" rel="bookmark" class="crp_title">The <code>ol</code> Element and Related Attributes: <code>type</code>, <code>start</code>, <code>value</code>, and <code>reversed</code></a></li><li><a href="http://html5doctor.com/getusermedia/" rel="bookmark" class="crp_title">It&#8217;s Curtains for Marital Strife Thanks to getUserMedia</a></li><li><a href="http://html5doctor.com/using-modernizr-to-detect-html5-features-and-provide-fallbacks/" rel="bookmark" class="crp_title">Using Modernizr to detect HTML5 features and provide fallbacks</a></li><li><a href="http://html5doctor.com/drag-and-drop-to-server/" rel="bookmark" class="crp_title">Drag and Drop and Automatically Send to the Server</a></li></ul></div><p><a href="http://html5doctor.com/the-output-element/" rel="bookmark">The output element</a> originally appeared on <a href="http://html5doctor.com">HTML5 Doctor</a> on December 20, 2011.</p>
]]></content:encoded>
			<wfw:commentRss>http://html5doctor.com/the-output-element/feed/</wfw:commentRss>
		<slash:comments>20</slash:comments>
		</item>
		<item>
		<title>Video Subtitling and WebVTT</title>
		<link>http://html5doctor.com/video-subtitling-and-webvtt/</link>
		<comments>http://html5doctor.com/video-subtitling-and-webvtt/#comments</comments>
		<pubDate>Tue, 29 Nov 2011 15:33:59 +0000</pubDate>
		<dc:creator>Tom Leadbetter</dc:creator>
				<category><![CDATA[Accessibility]]></category>
		<category><![CDATA[Elements]]></category>
		<category><![CDATA[multimedia]]></category>
		<category><![CDATA[HTML 5]]></category>
		<category><![CDATA[html5]]></category>
		<category><![CDATA[subtitles]]></category>
		<category><![CDATA[video]]></category>
		<category><![CDATA[webvtt]]></category>

		<guid isPermaLink="false">http://html5doctor.com/?p=4084</guid>
		<description><![CDATA[<p>We’ve been able to play video in the browser without a plugin for a couple of years now, and whilst there are still some codec annoyances, things appear to have settled down on the video front. The next step is adding resources to the video to make it more accessible and provide more options to the viewer.</p>]]></description>
			<content:encoded><![CDATA[<p>We’ve been able to play video in the browser without a plugin for a couple of years now, and whilst there are still some codec annoyances, things appear to have settled down on the video front. The next step is adding resources to the video to make it more accessible and provide more options to the viewer.</p>
<p>We currently have no means to provide information about what’s happening or being said in the video, which means the video isn’t very accessible and the user can’t easily navigate to a particular section of the video. Thankfully, there’s a new format specification in the works called <a href="http://www.whatwg.org/specs/web-apps/current-work/webvtt.html">WebVTT (Web Video Text Tracks)</a>. As of now, it’s only in the WHATWG spec, but the recently established <a href="http://www.w3.org/community/texttracks/">W3C Web Media Text Tracks Community Group</a> should introduce a WebVTT spec to the W3C soon.</p>
<p class="callout">You may recall a similar format called WebSRT (Web Subtitle Resource Tracks) that was recently under discussion. WebSRT was renamed to, and has been replaced by, WebVTT.</p>
<p>A WebVTT (.vtt) file is simply plain text containing several types of information about the video:</p>
<dl>
<dt>Subtitles</dt>
<dd>The transcription or translation of the dialogue.</dd>
<dt>Captions</dt>
<dd>Similar to subtitles, but may also include sound effects and other audio information.</dd>
<dt>Descriptions</dt>
<dd>Intended to be a separate text file that describes the video through a screenreader.</dd>
<dt>Chapters</dt>
<dd>Intended to help the user navigate through the video.</dd>
<dt>Metadata</dt>
<dd>Information and content about the video, which isn’t intended to be displayed to the viewer by default, though you may wish to do so using JavaScript.</dd>
</dl>
<p>This article will mostly be talking about <a href="#contents">subtitles and captions</a>, but it will briefly touch on <a href="#chapters">chapters</a> too.</p>
<p class="callout">Beyond the scope of this article but worth mentioning is the <a href="http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#text-track-api">text track API</a>, which, amongst other things, denotes how many text tracks there are and which ones have loaded and are ready for use. If you have used this API, <a href="http://html5doctor.com/ask-the-doctor/">let us know</a>.</p>
<section id="how">
<h2>How to Make and Link to a WebVTT file <a href="#how" class="permalink">#</a></h2>
<p>All you need to make a WebVTT file is a simple text editor. Type <code>WEBVTT</code> as the first line of the file and save it as a .vtt file. In the future, we expect existing captioning tools such as <a href="http://www.universalsubtitles.org/">Universal Subtitles</a> to export to WebVTT format.</p>
<figure>
<pre><code>WEBVTT</code></pre>
<figcaption>The simplest possible valid WebVTT file</figcaption>
</figure>
<p>That’s all you need to get started. Next, we have to link to the file in the HTML document. We do this via the <a href="http://html5doctor.com/video-the-track-element-and-webm-codec/"><code>&lt;track&gt;</code> element</a>, which is a child of the video element. The <code>&lt;track&gt;</code> element has several optional attributes:</p>
<ul>
<li>the source WebVTT file (<code>src</code>),</li>
<li>the language of the track (<code>srclang</code>),</li>
<li>a user-readable <code>label</code>, and</li>
<li>what <code>kind</code> of track it is. The values of the <code>kind</code> attribute come from the list above (i.e., <code>subtitles</code>, <code>captions</code>, etc.).</li>
</ul>
<p>In the following example, we’re using a <code>&lt;track&gt;</code> element for subtitles:</p>
<pre><code>&lt;video width=&quot;640&quot; height=&quot;480&quot; controls&gt;
  &lt;source src=&quot;video.mp4&quot; type=&quot;video/mp4&quot; /&gt;
  &lt;source src=&quot;video.webm&quot; type=&quot;video/webm&quot; /&gt;
  &lt;track src=&quot;subtitles.vtt&quot; kind=&quot;subtitles&quot; srclang=&quot;en&quot; label=&quot;English&quot; /&gt;
  &lt;!-- fallback for rubbish browsers --&gt;
&lt;/video&gt;</code></pre>
<p>A few notes about the attributes:</p>
<ul>
<li>If no <code>kind</code> is specified, the default is <code>subtitles</code>.</li>
<li>If the <code>kind</code> is <code>subtitles</code>, then <code>srclang</code> is required.</li>
<li>There should not be two tracks of the same <code>kind</code> with the same <code>label</code>.</li>
</ul>
<p>In the above example, we use a <code>&lt;video&gt;</code> element with two different <code>&lt;src&gt;</code> elements (for cross-browser loveliness). After the sources comes the <code>&lt;track&gt;</code> element. You can have several <code>&lt;track&gt;</code> elements as you might have subtitles, captions, and descriptions all in different languages.</p>
<p class="callout"><code>&lt;track&gt;</code> doesn’t presuppose a particular file format. <a href="http://html5labs.interoperabilitybridges.com/prototypes/video-captioning/video-captioning/info">Microsoft supports</a> the <a href="http://www.w3.org/TR/ttaf1-dfxp/"><abbr title="Timed Text Markup Language">TTML</abbr></a> format, but this format will not be implemented by other browser vendors.</p>
</section>
<section id="contents">
<h2>WebVTT Contents <a href="#contents" class="permalink">#</a></h2>
<p>We now know how to make a WebVTT file and how to reference it in an HTML document, but what goes inside it? Within the file, we list what are known as “cues”. The WebVTT file might only have one cue, but it can contain as many as you want. Each cue starts with an ID, followed by the time settings, followed by the text. Each cue is separated by a blank line. Here’s an example of captions:</p>
<figure>
<pre><code>WEBVTT

1
00:00:01.000 --&gt; 00:00:10.000
This is the first line of text, displaying from 1-10 seconds

2
00:00:15.000 --&gt; 00:00:20.000
And the second line of text
separated over two lines</code></pre>
<figcaption>WebVTT example content</figcaption>
</figure>
<p>The above example has two cues. Times must be written in <code>hh:mm:ss.mmm</code> format, so the timings in this example occur in the first twenty seconds. The second cue will split the text over two lines automatically.</p>
<p>If you have a segment of text that needs to appear in a karaoke/paint-on caption style, then you can place timers inline with text:</p>
<figure>
<pre><code>1
00:00:01.000 --&gt; 00:00:10.000
Never gonna give you up &lt;00:00:01.000&gt; Never gonna let you down &lt;00:00:05.000&gt; Never gonna run around and desert you</code></pre>
<figcaption>Karaoke-style captioning</figcaption>
</figure>
</section>
<section id="styling">
<h2>Styling Options <a href="#styling" class="permalink">#</a></h2>
<p>The previous examples specify the minimum configuration you need for subtitling and captioning, but you can style your captions too. Let’s start with the cue settings, which are done on the same line as the time settings:</p>
<dl>
<dt><code>D:vertical / D:vertical-lr</code></dt>
<dd>Display the text vertically rather than horizontally. This also specifies whether the text grows to the left (<code>vertical</code>) or to the right (<code>vertical-lr</code>).</dd>
<dt><code>L:X / L:X%</code></dt>
<dd>Either a number or a percentage. If a percentage, then it is the position from the top of the frame. If a number, this represents what line number it will be.</dd>
<dt><code>T:X%</code></dt>
<dd>The position of the text horizontally on the video. <code>T:100%</code> would place the text on the right side of the video.</dd>
<dt><code>A:start / A:middle / A:end</code></dt>
<dd>The alignment of the text within its box – <code>start</code> is left-aligned, <code>middle</code> is centre-aligned, and <code>end</code> is right-aligned.</dd>
<dt><code>S:X%</code></dt>
<dd>The width of the text box as a percentage of the video width.</dd>
</dl>
<p>To make use of these settings, put them alongside the time settings, like this:</p>
<pre><code>00:00:01.000 --&gt; 00:00:10.000 A:middle T:50%
00:00:01.000 --&gt; 00:00:10.000 A:end D:vertical
00:00:01.000 --&gt; 00:00:10.000 A:start T:100% L:0%</code></pre>
<p>which would result in something like the following:</p>
<figure>
    <img src="http://html5doctor.com/wp-content/uploads/2011/11/video-subtitles-demo1.png" alt="Examples of subtitle display" width="354" height="109" /></p>
<figcaption>Examples of subtitle display and alignment</figcaption>
</figure>
<p>Along with the above cue settings, you can use inline styles for text:</p>
<dl>
<dt>Bold text</dt>
<dd><code>&lt;b&gt;Lorem ipsum&lt;/b&gt;</code></dd>
<dt>Italic text</dt>
<dd><code>&lt;i&gt;dolor sit amet&lt;/i&gt;</code></dd>
<dt>Underlined text</dt>
<dd><code>&lt;u&gt;consectetuer adipiscing&lt;/u&gt;</code></dd>
<dt>Ruby text</dt>
<dd><code>&lt;ruby&gt;見&lt;rt&gt;み&lt;/rt&gt;&lt;/ruby&gt;</code></dd>
</dl>
<p>You can even apply a CSS class to a section of text using <code>&lt;c.myClass&gt;Lorem ipsum&lt;/c&gt;</code>, giving us many more styling options.</p>
<p>Finally, you can add a declaration representing the name of the voice: <code>&lt;v Tom&gt;Hello world&lt;/v&gt;</code>. This declaration accomplishes three things:</p>
<ol>
<li>The caption will display the voice (Tom) in addition to the caption text.</li>
<li>The name of the voice can be read by a screenreader, possibly event using a different voice for male or female names.</li>
<li>It offers a hook for styling so that, for example, all captions for Tom could be in blue.</li>
</ol>
<section id="chapters">
<h3>Chapters</h3>
<p>You can provide a chapter list for the video the same way you would provide subtitles or captions. Start with the same <code>WEBVTT</code> declaration, and then for each cue, declare the chapter number, the start and stop times, and the chapter title:</p>
<figure>
<pre><code>&lt;track src=&quot;chapters.vtt&quot; kind=&quot;chapters&quot; srclang=&quot;en&quot; /&gt;</code></pre>
<figcaption>HTML <code>&lt;track&gt;</code> element for providing chapters to a video</figcaption>
</figure>
<figure>
<pre><code>WEBVTT

Chapter 1
00:00:01.000 --&gt; 00:00:10.000>
Introduction to HTML5</code></pre>
<figcaption>WebVTT file containing video chapter markers</figcaption>
</figure>
</section>
</section>
<section id="browsers">
<h2>Browser Support <a href="#browsers" class="permalink">#</a></h2>
<p>One slight glitch with WebVTT: not a single browser currently supports it. All major browsers have started working on implementations, so we should see some results soon. Thankfully, in the meantime, there are several JavaScript polyfills available:</p>
<ul>
<li><a href="http://www.storiesinflight.com/js_videosub/">js_videosub</a></li>
<li><a href="http://www.delphiki.com/html5/playr/">Playr</a></li>
<li><a href="http://mediaelementjs.com/">MediaElementJS</a></li>
<li><a href="http://dev.mennerich.name/showroom/html5_video/">LeanBack player</a> (and upcoming <a href="http://leanbackplayer.com/test/webvtt.html">new version</a>)</li>
<li><a href="https://github.com/cgiffard/Captionator">Captionator</a>&nbsp;</li>
</ul>
</section>
<section id="demo">
<h2>Demo <a href="#demo" class="permalink">#</a></h2>
<p>We&#8217;ve put together a <a href="http://html5doctor.com/demos/webvtt/">quick demo</a> which uses the <a href="http://www.delphiki.com/html5/playr/">Playr</a> polyfill. We started using <a href="http://mediaelementjs.com/">MediaElementJS</a>, but it doesn’t sport as many features as <a href="http://www.delphiki.com/html5/playr/">Playr</a>, such as separate lines of text and CSS classes. In the <a href="http://html5doctor.com/demos/webvtt/">demo</a>, the subtitles start at 2 seconds and 15 seconds and use bold, underline, and custom styles. Here’s the <a href="http://html5doctor.com/demos/webvtt/subtitles.vtt">associated WebVTT file</a>.</p>
</section>
<section id="conclusion">
<h2>Conclusion <a href="#conclusion" class="permalink">#</a></h2>
<p>This article covers the basics of creating a WebVTT file suitable for subtitles or captions for a video. We know how to add cues and chapters and how to add styles and change how the text appears on the video. Although no browser yet supports it, there’s a lot more to come for accessible video, so stay tuned to the <a href="http://www.w3.org/community/texttracks/">W3C Web Media Text Tracks Community Group</a>.</p>
<p>What are your thoughts on WebVTT? Are any of you using it now? How can it be improved?</p>
<p>Finally, let’s thank <a href="http://twitter.com/#!/silviapfeiffer">@silviapfeiffer</a> for taking the time to answer some questions about WebVTT and for her tremendous work in this field.</p>
</section>
<section id="reading">
<h2>Reading <a href="#reading" class="permalink">#</a></h2>
<ul>
<li>Follow <a href="http://twitter.com/#!/silviapfeiffer">@silviapfeiffer</a></li>
<li><a href="http://www.w3.org/community/texttracks/">W3C Web Media Text Tracks Community Group</a></li>
<li><a href="http://blog.gingertech.net/2011/06/27/recent-developments-around-webvtt/">Recent developments around WebVTT</a></li>
<li><a href="http://html5videoguide.net/presentations/WebVTT/">Presentation: HTML5 video accessibility and the WebVTT file format</a></li>
<li><a href="http://www.youtube.com/watch?v=gK72pcu3cpk">HTML5 video accessibility and the WebVTT file format &#8211; Audio Described</a></li>
<li><a href="http://leanbackplayer.com/other/webvtt.html">A review with notes and thoughts for LeanBack Player</a></li>
<li><a href="http://quuz.org/webvtt/">WebVTT validator</a></li>
<li><a href="http://www.iandevlin.com/blog/2011/05/html5/webvtt-and-video-subtitles">WebVTT and Video Subtitles</a></li>
<li><a href="http://www.openmediadevelopers.org/pmwiki.php/Main/OVC2011VidA11y">The Open Video Alliance</a></li>
<li><a href="http://www.delphiki.com/webvtt/">Understanding WebVTT file format (draft)</a></li>
<li><a href="http://scottbw.wordpress.com/2011/06/28/creating-subtitles-and-audio-descriptions-with-html5-video/">Creating subtitles and audio descriptions with HTML5 video</a></li>
</ul>
</section>
<div id="crp_related">
<h3>Related Posts:</h3>
<ul class="related">
<li><a href="http://html5doctor.com/video-the-track-element-and-webm-codec/" rel="bookmark" class="crp_title">Video: the track element and webM codec</a></li>
<li><a href="http://html5doctor.com/the-video-element/" rel="bookmark" class="crp_title">The video element</a></li>
<li><a href="http://html5doctor.com/html5-simplequiz-4-figures-captions-and-alt-text/" rel="bookmark" class="crp_title">HTML5 Simplequiz #4: figures, captions and alt text</a></li>
<li><a href="http://html5doctor.com/your-questions-answered-8/" rel="bookmark" class="crp_title">Your Questions Answered #8</a></li>
<li><a href="http://html5doctor.com/html5-for-web-developers/" rel="bookmark" class="crp_title">HTML5 for Web Developers</a></li>
</ul>
</div>
<p><a href="http://html5doctor.com/video-subtitling-and-webvtt/" rel="bookmark">Video Subtitling and WebVTT</a> originally appeared on <a href="http://html5doctor.com">HTML5 Doctor</a> on November 29, 2011.</p>
]]></content:encoded>
			<wfw:commentRss>http://html5doctor.com/video-subtitling-and-webvtt/feed/</wfw:commentRss>
		<slash:comments>12</slash:comments>
		</item>
		<item>
		<title>Pushing and Popping with the History API</title>
		<link>http://html5doctor.com/history-api/</link>
		<comments>http://html5doctor.com/history-api/#comments</comments>
		<pubDate>Tue, 15 Nov 2011 13:09:47 +0000</pubDate>
		<dc:creator>Mike Robinson</dc:creator>
				<category><![CDATA[JavaScript APIs]]></category>
		<category><![CDATA[api]]></category>
		<category><![CDATA[history]]></category>
		<category><![CDATA[HTML 5]]></category>
		<category><![CDATA[html5]]></category>

		<guid isPermaLink="false">http://html5doctor.com/?p=3897</guid>
		<description><![CDATA[Until recently, we developers couldn’t to do much with the state and history of the browser. We could check the number of items in the history and push users forwards and backwards, but this provides little benefit to the user. With the rise of more dynamic web pages, we need more control. Thankfully, HTML5 gives [...]]]></description>
			<content:encoded><![CDATA[<p>Until recently, we developers couldn’t to do much with the state and history of the browser. We could check the number of items in the history and push users forwards and backwards, but this provides little benefit to the user. With the rise of more dynamic web pages, we need more control. Thankfully, HTML5 gives us that control by extending the JavaScript History API.</p>
<section id="the-point">
<h2>What’s the point? <a class="permalink" href="#the-point">#</a></h2>
<p>It goes without saying that URLs are important. They’re <em>the</em> method of accessing the vast collections of information and resources on the web, and more recently, they’ve begun representing the intended state of a web application. You can copy these URLs and share them with your friends or use them to create links from any HTML document. They’re the veins of the web, and they need to be looked after.</p>
<p>Previously, the JavaScript History API offered some very simple functionality:</p>
<pre><code>// Check the length of the history stack
console.log(history.length);

// Send the user agent forward
console.log(history.forward());

// Send the user agent back
console.log(history.back());

// Send the user agent back (negative) or forward (positive)
// by a given number of items
console.log(history.go(-3));
</code></pre>
<p>With dynamic Ajax web applications, where the browser updates the page in parts instead of changing location entirely, it’s difficult to give the user a URL to bookmark or share the current application state. <a href="http://www.whatwg.org/specs/web-apps/current-work/multipage/elements.html#the-id-attribute">Fragment identifiers</a>, like those used on this article’s headings via the <code>id</code> attribute, provide some state information, but they’re entirely dependent on client-side scripts.</p>
<p>The changes to the <a href="http://www.whatwg.org/specs/web-apps/current-work/multipage/history.html">History API</a> are intended to give developers ways to push history items to the browser so the native back and forward actions can cycle through those items. These history items can also hold data that you can later extract to restore the page state.</p>
<blockquote>
<p>Pages can <a href="http://www.whatwg.org/specs/web-apps/current-work/multipage/history.html#dom-history-pushstate" title="dom-history-pushState">add</a> <a href="http://www.whatwg.org/specs/web-apps/current-work/multipage/history.html#state-object" title="state object">state objects</a> between their entry in the session history and the next (“forward”) entry. These are then <a href="http://www.whatwg.org/specs/web-apps/current-work/multipage/history.html#event-popstate" title="event-popstate">returned to the script</a> when the user (or script) goes back in the history, thus enabling authors to use the “navigation” metaphor even in one-page applications.</p>
<footer>- <cite><a href="http://www.whatwg.org/specs/web-apps/current-work/multipage/history.html" title="6.4 Session history and navigation">WHATWG</a></cite></footer>
</blockquote>
<p>If the user copies or bookmarks a stateful URL and visits it later, your back-end can be configured to interpret such a URL and jump the user right to the correct page and/or state.</p>
<p>In this article, I’ll cover the client-side use of the History API, so make sure you set up your server to work with the new URLs. If you’ve already built an accessible website that provide these entry points, you’re laughing!</p>
<section id="those-fking-hashbangs">
<h4>Those <em>f#!king</em> hashbangs… <a class="permalink" href="#those-fking-hashbangs">#</a></h4>
<p>You may have already seen articles fussing over the adoption of the “hashbang” (#!) pattern on sites like <a href="http://twitter.com">Twitter</a>. This technique updates the address bar with a fragment identifier that can then be used by JavaScript to determine which page and state should be displayed.</p>
<p>This works as a method of creating a bookmarkable, shareable URL for a page’s state in the absense of a standard API. While the Twitter implementation accepts both <code>http://twitter.com/#!/akamike</code> and <code>http://twitter.com/akamike</code>, it has some disadvantages:</p>
<ul>
<li>The fragment identifier is only accessible on the client side. This means that only JavaScript can utilise it, so browsers without JavaScript enabled are out of luck.</li>
<li>As the server does not receive the path following the hashbang, removing that JavaScript drops support for all those URLs. That’s a lot of broken links, so you’re stuck with that JavaScript <em>forever</em>.</li>
<li>It’s ugly. It’s a hack and it looks like one.</li>
</ul>
<p>The hashbang was never intended to be a long-term solution, so don’t rely on it. If you do use hashbangs, be prepared to deal with the consequences (and possible backlash from web purists).</p>
</section>
</section>
<section id="making-history">
<h2>Making History <a class="permalink" href="#making-history">#</a></h2>
<p>These examples will build on top of each other. We’ll start with a <a href="http://html5doctor.com/wp-content/uploads/2011/10/history_base.html">basic HTML document</a> with some inline styles and scripts for your convenience.</p>
<aside class="sidenote">
<p>For a simple HTTP server, open the command line, <code>cd</code> to the directory you would like to serve, run <code>python -m SimpleHTTPServer 8080</code>, then open <code>localhost:8080</code> in your browser. Alternatively, try a bundled setup like <a href="http://www.apachefriends.org/en/xampp.html">XAMPP</a> or <a href="http://www.mamp.info/en/index.html">MAMP</a>.</p>
</aside>
<p>Save this file and open it in your favourite editor. It must be accessed via HTTP, so that means you need either a local server (e.g. <code>http://localhost/</code>) or an online web server you can upload to. <strong>Viewing the file directly using your browser’s Open File function will not work</strong>, since it uses the <code>file://</code> protocol and not HTTP. Also be sure to update the <code>href</code> attributes on each of the navigation links to ensure the correct directory structure is used. Personally, I’m viewing the demo locally at <code>http://localhost/history</code>.</p>
<p>We’ll be working exclusively within the <code>&lt;script&gt;</code> element at the end of the <code>&lt;body&gt;</code>. The code includes some simple styles and dynamically changes the content as you click the links. In reality, this could be loaded from your server via <code>XMLHttpRequest</code>, but for the purposes of this demonstration I’ve bundled it up into a self-contained file. The important part is that we have a quick-and-dirty dynamic page to work with, so let the fun begin!</p>
<p>At the moment there, is no bookmarkable URL for the different states of this page. If you click around the navigation items, then click Back in your browser, you won’t be taken back to the previous state and may even be taken away from the page to whatever you viewed before (depending on your browser). It would be nice if you could share “Socks” with your friends, right? We can do that via <code>history.pushState()</code>.</p>
<p>The <code>history.pushState()</code> method takes three parameters:</p>
<dl>
<dt><code>data</code></dt>
<dd>Some structured data, such as settings or content, assigned to the history item.</dd>
<dt><code>title</code></dt>
<dd>The name of the item in the history drop-down shown by the browser’s back and forward buttons. (Note: this is not currently supported by any major browsers.)</dd>
<dt><code>url</code> <em>(optional)</em></dt>
<dd>The URL to this state that should be displayed in the address bar.</dd>
</dl>
<p>With these parameters, you can define the state of the page, give that state a name, and even provide a bookmarkable address, as if the page had reloaded entirely. Let’s dive right in and add this to the <code>clickHandler</code> function, right above the <code>return</code> statement:</p>
<figure>
<pre><code>function clickHandler(e) {
  /* Snip... */

  // Add an item to the history log
  history.pushState(data, event.target.textContent, event.target.href);

  return event.preventDefault();
}
</code></pre>
</figure>
<p>The single line of code we added informs the <code>history</code> object that:</p>
<ul>
<li>we want to add an item to the log,</li>
<li>it should remember the data that we’ve already loaded,</li>
<li>it should assign a name to this state based on the text of the link we clicked (even though this isn’t used — it’s good to get into the habit of recording a name for the state), and</li>
<li>it should update the address bar with the <code>href</code> attribute of that link.</li>
</ul>
<p>Reload the page in your browser and click a few of the links, keeping an eye on the address bar. Notice how it changes on each click, despite the fact that you aren’t actually navigating away from this page. If you also have a look at your history log, you’ll see a long list of page titles (in this case ”Kittens!” over and over). Provided your server is set up to serve the correct page upon access, the user could copy that URL and paste it into a new browser window to jump straight to that kitten.</p>
<p>At the moment, clicking the back button will pop you through the history items, but the page won’t react to these changes. That&#8217;s because so far, we’ve only created the history records. How can we allow active users to return to a previous state? We listen to the <code>popstate</code> event.</p>
</section>
<section id="historical-events">
<h2>Historical Events in Navigation <a class="permalink" href="#historical-events">#</a></h2>
<p>The user agent fires a <code>popstate</code> event when the user navigates through their history, whether backwards or forwards, provided it isn’t taking the user away from the current page. That is, all those <code>pushState</code>s we called will keep the user on the current page, so the <code>popstate</code> event will fire for each history item they pop through.</p>
<p>Before the closing <code>&lt;/script&gt;</code> tag, add a new listener for the <code>popstate</code> event:</p>
<figure>
<pre><code>// Revert to a previously saved state
window.addEventListener('popstate', function(event) {
  console.log('popstate fired!');

  updateContent(event.state);
});
</code></pre>
</figure>
<p>We attach the event listener to the <code>window</code>, which is responsible for firing the event, and pass this event into our handler. We log a message (so we can see when this event is firing), and then we update the content using the state we saved previously. The state is attached to the <code>event</code> object via the <code>state</code> property.</p>
<p>Open up the page fresh in your browser, click around like before, and then click back. As before, the URL in the address bar changes as you cycle through states, but now the content is also restored back to what it should be. Click forward, and the content is likewise correctly restored.</p>
<div class="callout warning-block">
<p>If you look at the developer console in Chrome when you load the page for the first time, you’ll see the <code>popstate</code> event fired immediately, before you’ve even clicked a link. This is because Chrome considers the initial page load to be a change in state, and so it fires the event. In this instance, the <code>state</code> property is <code>null</code>, but thankfully the <code>updateContent</code> function deals with this. Keep this in mind when developing as it could catch you out, especially if other browsers assume this behavior.</p>
</p></div>
</section>
<section id="rewriting-history">
<h2>Rewriting history <a class="permalink" href="#rewriting-history">#</a></h2>
<p>Unfortunately, as fantastic as HTML5 is, it doesn’t allow us actual time travel. If it did, I would be going back to my childhood and telling a younger me, “Yes, you should have a slice of cake”. Take that as you will.</p>
<p>The History API does, however, allow us to make amends to our history log items. For example, we could update the current state in response to fresh user input in a form. We can do this with <code>history.replaceState</code>.</p>
<p><code>replaceState</code> works just as <code>pushState</code> does, with the exact same parameters, except that it updates the current entry instead of adding a new one. I can think of one situation in our demo where this could be used: the initial page load. If you click back for long enough, you’ll find that going back to the original URL doesn’t provide you the original content. Let’s fix that by adding the following to the bottom of our script:</p>
<figure>
<pre><code>// Store the initial content so we can revisit it later
history.replaceState({
  content: contentEl.textContent,
  photo: photoEl.src
}, document.title, document.location.href);
</code></pre>
</figure>
<p>As this runs when the page loads, it saves the initial page state. We can later load this state when the user browses back to this point via the event listener we set up previously. You can try it out by loading up the page, clicking a few links, and then hitting back until you return to the original URL. The initial content has returned!</p>
</section>
<section id="demo">
<h2>Demo <a class="permalink" href="#demo">#</a></h2>
<p>I’ve set up a demo of our completed code. I’ve also added a little back-end magic to make our <code>history.pushState</code> URLs work like a real site. Remember that the URLs you push should be live URLs that the user can bookmark and share as real entry points to your site.</p>
<p class="btn half"><a href="http://html5doctor.com/demos/history/">View the History API demo</a></p>
</section>
<section id="browser-support">
<h2>Browser support <a class="permalink" href="#browser-support">#</a></h2>
<p>Up-to-date copies of Chrome (5+), Safari (5.0+), Firefox (4.0+), and Opera (11.50+) have support for the new History API. Even some mobile browsers work just fine, like Mobile Safari on iOS 4+. Unfortunately, IE 9 and below lack support, but it <a href="http://msdn.microsoft.com/en-us/ie/hh272905.aspx#_HTML5History">should work in IE 10</a> when it arrives.</p>
<p>Safari 5.0 sometimes exhibits one oddity: navigating between states causes the loading spinner to appear and stay even when the state has been loaded. This stops when you navigate away using a link or action that does not involve a state saved by the History API.</p>
<section id="polyfill">
<h3>Polyfill <a class="permalink" href="#polyfill">#</a></h3>
<p>A polyfill does exist for the History API. The aptly named <a href="https://github.com/balupton/history.js">History.js</a> uses HTML4’s <code>hashchange</code> event with document fragment identifiers to mimic the history API in older browsers. If one of the hash URLs is used by a modern browser, it uses <code>replaceState</code> to quietly correct the URL.</p>
<p>It sounds like magic, but make sure you’re aware of the consequences of using fragment identifiers, as mentioned previously in this article. As such, the author of History.js has put together a guide titled <a href="https://github.com/balupton/history.js/wiki/Intelligent-State-Handling">Intelligent State Handling</a>.</p>
</section>
</section>
<section id="closing">
<h2>Closing thoughts <a class="permalink" href="#closing">#</a></h2>
<p>URLs go beyond just the browsing session of a user. They’re historically important markers for resources that could very well remain in use for many years to come. Before you embark on developing your site’s JavaScript, you should give thought to the <a href="http://warpspire.com/posts/url-design/">design of your URLs</a>. Make them meaningful and organised. Make sure you can directly access them without JavaScript. Only then should you add your JavaScript to enhance the browsing experience.</p>
</section>
<div id="crp_related">
<h3>Related Posts:</h3>
<ul class="related">
<li><a href="http://html5doctor.com/server-sent-events/" rel="bookmark" class="crp_title">Server-Sent Events</a></li>
<li><a href="http://html5doctor.com/drag-and-drop-to-server/" rel="bookmark" class="crp_title">Drag and Drop and Automatically Send to the Server</a></li>
<li><a href="http://html5doctor.com/how-to-get-all-the-browsers-playing-ball/" rel="bookmark" class="crp_title">How to get all the browsers playing ball</a></li>
<li><a href="http://html5doctor.com/methods-of-communication/" rel="bookmark" class="crp_title">Methods of communication</a></li>
<li><a href="http://html5doctor.com/storing-data-the-simple-html5-way-and-a-few-tricks-you-might-not-have-known/" rel="bookmark" class="crp_title">Storing Data the Simple HTML5 Way (and a few tricks you might not have known)</a></li>
</ul>
</div>
<p><a href="http://html5doctor.com/history-api/" rel="bookmark">Pushing and Popping with the History API</a> originally appeared on <a href="http://html5doctor.com">HTML5 Doctor</a> on November 15, 2011.</p>
]]></content:encoded>
			<wfw:commentRss>http://html5doctor.com/history-api/feed/</wfw:commentRss>
		<slash:comments>11</slash:comments>
		</item>
	</channel>
</rss>

