But what’s a middle-aged colour-blind bloke to do? I had early in the curtain procurement process decided against cutting a hole in the carpet in order that I may take a sample with me. (All other mistakes aside, this was a correct decision.)

So, in order to ensure that I would never again repeat the mistake, I set out to make an app that would allow me to capture the colour of an image straight from my camera. Of course, it had to be a web app rather than a native app, because we’re web angels, not proprietary devils.

 if (navigator.getUserMedia) {
  // do something cool
} else {
  // fallback code
}

<form enctype="multipart/form-data" method="post">
  <input type="file" accept="video/*;capture=camera" />
</form>

navigator.getUserMedia({audio: true, video: true}, success, error);

function success(stream) { 
  // ... use 'stream' ... 
} 

function error(){ 
  // display fallback content 
}

<video autoplay></video>
<script>
if (navigator.getUserMedia) {
  navigator.getUserMedia({audio: true, video: true}, success, error);
  function success(stream)
  { 
    // ... use 'stream' ...
    var video = document.getElementsByTagName('video')[0];
    video.src = stream; 
  }
  // ...
</script>

video = document.getElementById('video');

Doing Exciting Things with the Video Stream #

So far, we’re only echoing the video stream. But, as you’ll know from Tab Atkins’ guest article video + canvas = magic, if we copy the video into a canvas, we can manipulate it:

var video = document.querySelector('video');
var canvas = document.querySelector('canvas');
var ctx = canvas.getContext('2d');

ctx.drawImage(video, 0, 0, video.width, video.height,
  0, 0, canvas.width, canvas.height);

A common mistake is assuming that this somehow establishes a “live connection” between the video stream and the canvas. It doesn’t. It only grabs the current frame. So you need to grab the current frame repeatedly in order to replicate a video. The magical number is 15 times a second (or every 67 milliseconds), but a slower frame rate would probably be okay for this demo:

setInterval(function () { 
  ctx.drawImage(video, 0, 0, video.width, video.height,
  0, 0, canvas.width, canvas.height);
}, 1000 / 67);

See the working demo.

What I want to do now is to grab the dominant colour from the canvas. I’m stealing a demo by Shwetank Dixit (who borrowed from Lokesh Dhakar‘s Color Thief demo). We won’t look at how it determines the dominant colour (because that’s nothing to do with getUserMedia).

Try this in Opera Labs and Chrome Canary!

Unfortunately, this works splendidly in Opera but it occasionally fails in Chrome, and we’re not sure why. (We’ve emailed the Chrome team to ask them.) Apart from the rather anti-climactic end to an article, you shouldn’t be dismayed by this. getUserMedia is only available in experimental builds that have plenty of work to be done before they hit production.

Back to our demo. Once you’ve got the dominant colour, you could prompt the user for a name for that colour (“carpet”, for example) and store that in local storage using its name as a key. Then, when you find a pair of curtains in the shop, you can find their dominant colour and easily compare it with that stored against “carpet” from earlier.

And you can return home, confident of delighting the love of your life with your chromatically appropriate choice of soft furnishings. That’s HTML5: enriching marriages since 2004.

Harshing Your Mellow #

Things need refining before web apps with getUserMedia are on a par with native applications. At the moment, there is no way of controlling which camera is to be used (and no agreement that a developer should be able to). The camera’s flash mechanism isn’t programmatically controllable, which could harm the user experience. Most importantly are the privacy implications of web pages being able to access your camera. Opera has an experimental privacy UI, but Chrome has no UI yet.

More Demos #

See Paul Neave’s HTML5 Webcam Toy and the links at the foot of my Opera Labs article.

Thanks #

Thank for help and code are due to Robin Berjon, Shwetank Dixit, Daniel Davis, Lokesh Dhakar, Mike Taylor, Mike Robinson, and Rich Tibbett.

Addy Osmani’s Polyfill

Added 13 March 2012: Addy Osmani has published a getUserMedia polyfill called getUserMedia.js, which uses Flash if gUM isn’t available. It’s not ready for production yet, as IE is still being tested. Forkers may hit the ‘hub.

Related Posts:

• How to get all the browsers playing ball
• Using Modernizr to detect HTML5 features and provide fallbacks
• Introducing Web SQL Databases
• HTML5: briefing notes for journalists and analysts
• video + canvas = magic

It’s Curtains for Marital Strife Thanks to getUserMedia originally appeared on HTML5 Doctor on February 7, 2012.

]]>

We currently have no means to provide information about what’s happening or being said in the video, which means the video isn’t very accessible and the user can’t easily navigate to a particular section of the video. Thankfully, there’s a new format specification in the works called WebVTT (Web Video Text Tracks). As of now, it’s only in the WHATWG spec, but the recently established W3C Web Media Text Tracks Community Group should introduce a WebVTT spec to the W3C soon.

You may recall a similar format called WebSRT (Web Subtitle Resource Tracks) that was recently under discussion. WebSRT was renamed to, and has been replaced by, WebVTT.

A WebVTT (.vtt) file is simply plain text containing several types of information about the video:

Subtitles

The transcription or translation of the dialogue.

Captions

Similar to subtitles, but may also include sound effects and other audio information.

Descriptions

Intended to be a separate text file that describes the video through a screenreader.

Chapters

Intended to help the user navigate through the video.

Metadata

Information and content about the video, which isn’t intended to be displayed to the viewer by default, though you may wish to do so using JavaScript.

This article will mostly be talking about subtitles and captions, but it will briefly touch on chapters too.

Beyond the scope of this article but worth mentioning is the text track API, which, amongst other things, denotes how many text tracks there are and which ones have loaded and are ready for use. If you have used this API, let us know.

WEBVTT

<video width="640" height="480" controls>
  <source src="video.mp4" type="video/mp4" />
  <source src="video.webm" type="video/webm" />
  <track src="subtitles.vtt" kind="subtitles" srclang="en" label="English" />
  <!-- fallback for rubbish browsers -->
</video>

WEBVTT

1
00:00:01.000 --> 00:00:10.000
This is the first line of text, displaying from 1-10 seconds

2
00:00:15.000 --> 00:00:20.000
And the second line of text
separated over two lines

1
00:00:01.000 --> 00:00:10.000
Never gonna give you up <00:00:01.000> Never gonna let you down <00:00:05.000> Never gonna run around and desert you

Styling Options #

The previous examples specify the minimum configuration you need for subtitling and captioning, but you can style your captions too. Let’s start with the cue settings, which are done on the same line as the time settings:

D:vertical / D:vertical-lr

Display the text vertically rather than horizontally. This also specifies whether the text grows to the left (vertical) or to the right (vertical-lr).

L:X / L:X%

Either a number or a percentage. If a percentage, then it is the position from the top of the frame. If a number, this represents what line number it will be.

T:X%

The position of the text horizontally on the video. T:100% would place the text on the right side of the video.

A:start / A:middle / A:end

The alignment of the text within its box – start is left-aligned, middle is centre-aligned, and end is right-aligned.

S:X%

The width of the text box as a percentage of the video width.

To make use of these settings, put them alongside the time settings, like this:

00:00:01.000 --> 00:00:10.000 A:middle T:50%
00:00:01.000 --> 00:00:10.000 A:end D:vertical
00:00:01.000 --> 00:00:10.000 A:start T:100% L:0%

which would result in something like the following:

Along with the above cue settings, you can use inline styles for text:

Bold text

<b>Lorem ipsum</b>

Italic text

<i>dolor sit amet</i>

Underlined text

<u>consectetuer adipiscing</u>

Ruby text

<ruby>見<rt>み</rt></ruby>

You can even apply a CSS class to a section of text using <c.myClass>Lorem ipsum</c>, giving us many more styling options.

Finally, you can add a declaration representing the name of the voice: <v Tom>Hello world</v>. This declaration accomplishes three things:

1. The caption will display the voice (Tom) in addition to the caption text.
2. The name of the voice can be read by a screenreader, possibly event using a different voice for male or female names.
3. It offers a hook for styling so that, for example, all captions for Tom could be in blue.

Chapters

You can provide a chapter list for the video the same way you would provide subtitles or captions. Start with the same WEBVTT declaration, and then for each cue, declare the chapter number, the start and stop times, and the chapter title:

<track src="chapters.vtt" kind="chapters" srclang="en" />

WEBVTT

Chapter 1
00:00:01.000 --> 00:00:10.000>
Introduction to HTML5

Video Subtitling and WebVTT originally appeared on HTML5 Doctor on November 29, 2011.

First, the basics

If you’re just starting with HTML5, you may not be familiar with the <video> element and how to use it. Here’s a simple example that we’ll be using in the later demos:

<video controls loop>
    <source src=video.webm type=video/webm>
    <source src=video.ogg type=video/ogg>
    <source src=video.mp4 type=video/mp4>
</video>

The <video> element contains two attributes: @controls and @loop. @controls tells the browser to give the video the standard set of video controls: play/pause, scrubber, volume, etc. @loop tells the browser to start the video over again from the beginning once it ends.

Then, inside the <video> element, we have three child <source> elements, each pointing to a different encoding of the same video. The browser will try each source in order and play the first one that it understands.

See this code in action, playing the intro to one of the greatest cartoon series of all time.

(A note about fallback: all of these demos assume that your browser has <video> support, which isn’t true in IE8 or earlier. Normally, it’s good practice to specify a Flash fallback or similar for those browsers, but that wouldn’t accomplish much here — all of the techniques I demonstrate rely on basic integration between the <video> element and the <canvas> element, which you can’t achieve with a Flash player. So I’ve omitted any non-<video> fallback content in these examples. I’ve still provided multiple sources, though, so all current browsers that do support <video> will be able to play it.)

Now, a simple example

Now that we know how to play a video, let’s mix in some <canvas> shenanigans. First, check out the demo, then come back here for a code walkthrough. I’ll wait.

…

Done? Cool! Now, how does this work? Surely it requires a few hundred lines of JavaScript, right? If you’ve cheated and already looked at the source code of the demo page, you’ll know how easy it is.

We start with this HTML:

<!DOCTYPE html>
<title>Video/Canvas Demo 1</title>

<canvas id=c></canvas>
<video id=v controls loop>
    <source src=video.webm type=video/webm>
    <source src=video.ogg type=video/ogg>

    <source src=video.mp4 type=video/mp4>
</video>

Same video code as before, but now with a <canvas> element thrown into the mix. Kinda empty and useless at the moment, though. We’ll script it into action later.

Now, let’s pair that with some CSS to get things positioned right:

<style>

body {
    background: black;
}

#c {
    position: absolute;
    top: 0;
    bottom: 0;
    left: 0;
    right: 0;
    width: 100%;
    height: 100%;
}

#v {
    position: absolute;
    top: 50%;
    left: 50%;
    margin: -180px 0 0 -240px;
}
</style>

This just centers the video in the screen and stretches the canvas to the full width and height of the browser window. Since the canvas comes first in the document, it’ll be behind the video, exactly where we want it.

Now, here comes the magic!

<script>
document.addEventListener('DOMContentLoaded', function(){
    var v = document.getElementById('v');
    var canvas = document.getElementById('c');
    var context = canvas.getContext('2d');

    var cw = Math.floor(canvas.clientWidth / 100);
    var ch = Math.floor(canvas.clientHeight / 100);
    canvas.width = cw;
    canvas.height = ch;

    v.addEventListener('play', function(){
        draw(this,context,cw,ch);
    },false);

},false);

function draw(v,c,w,h) {
    if(v.paused || v.ended) return false;
    c.drawImage(v,0,0,w,h);
    setTimeout(draw,20,v,c,w,h);
}
</script>

Take that in for a moment. Just breathe deeply and absorb it. So short, so sweet, so pretty…

Now, let’s step through it.

    var v = document.getElementById('v');
    var canvas = document.getElementById('c');
    var context = canvas.getContext('2d');

    var cw = Math.floor(canvas.clientWidth / 100);
    var ch = Math.floor(canvas.clientHeight / 100);
    canvas.width = cw;
    canvas.height = ch;

This part is simple. I grab hold of the video and canvas elements on the page, and I grab the canvas’s 2D-context so I can draw on it. Then I do some quick calculating to find out how wide and tall I want the canvas’s drawing surface to be. The <canvas> element itself is already stretched to the size of the screen via CSS, so this’ll make each pixel of the drawing surface equal to about 100×100 pixels on the screen.

That last bit may need some explanation if you’re new to canvas. Normally, the visual size and the drawing-surface size of a <canvas> element will be the same. In that case, drawing a line 50px long will display a line 50px long. But that doesn’t have to be true — you can set the drawing surface’s size through the @width and @height properties on the <canvas> element itself, and then change the visual size of the canvas with CSS to be something different. The browser will then automatically upscale or downscale the drawing appropriately to make the drawing surface fill the visual size. In this case, I’m setting the drawing surface of the canvas to be very small — on most screens, it’ll be about 10px wide and 7px tall — and then stretching the visual size with CSS so that each pixel I draw gets blown up 100-fold by the browser. That’s what causes the cool visual effect in the demo.

    v.addEventListener('play', function(){
        draw(v,context,cw,ch);
    },false);

Another simple part. Here I attach some code to the “play” event on the video element. This event gets fired whenever the user hits the “play” button to start watching the video. All I do is call the draw() function with the appropriate parameters: the video itself, the canvas’s drawing context, and the canvas’s width and height.

function draw(v,c,w,h) {
    if(v.paused || v.ended) return false;
    c.drawImage(v,0,0,w,h);
    setTimeout(draw,20,v,c,w,h);
}

The first line just makes the function stop immediately if the user pauses or stops the video, so it’s not burning CPU when nothing’s changing. The third line calls the draw() function again, allowing the browser a little breathing space to do other things like update the video itself. I’m putting in a 20ms delay, so we’ll get roughly 50fps, which is more than enough.

The second line is where the magic happens — it draws the current frame of the video directly onto the canvas. Yes, it’s exactly as simple as it looks. Just pass the video element and the x, y, width, and height of the rectangle on the canvas you want it to draw into. In this case, it’s filling up the entire canvas, but you could do less (or more!) if you wanted.

I’m using another trick here. Remember how the canvas is really tiny? The video will be at least 20 times bigger than the canvas on most screens, so how do we draw it onto such a tiny canvas? The drawImage() function handles that for us — it automatically scales whatever you hand it in the first argument so that it fills the rectangle you specify. That means we authors don’t have to worry about averaging the pixel colors (or extrapolating, if you’re drawing a small video into a big rectangle) because the browser does it all for us. I’ll use this trick more in the future, so watch out for it.

And…that’s it! The entire demo is done in 20 lines of easy-to-read JavaScript code, instantly producing a nifty background effect for any video you wish to play. You can trivially adjust the size of the “pixels” on the canvas by adjusting the lines that set the cw and ch variables.

Directly manipulating video pixels

The last demo was cool, but it just let the browser do all the heavy lifting. The browser downscaled the video, drew it onto the canvas, and then upscaled the canvas pixels, all automatically. Let’s try our hand at doing some of this ourselves! Check out the demo to see this in action, where I convert the video to grayscale on the fly.

The HTML for the page is basically identical:

<video id=v controls loop>
    <source src=video.webm type=video/webm>
    <source src=video.ogg type=video/ogg>
    <source src=video.mp4 type=video/mp4>
</video>
<canvas id=c></canvas>

Nothing new here, so let’s move onto the script.

document.addEventListener('DOMContentLoaded', function(){
    var v = document.getElementById('v');
    var canvas = document.getElementById('c');
    var context = canvas.getContext('2d');
    var back = document.createElement('canvas');
    var backcontext = back.getContext('2d');

    var cw,ch;

    v.addEventListener('play', function(){
        cw = v.clientWidth;
        ch = v.clientHeight;
        canvas.width = cw;
        canvas.height = ch;
        back.width = cw;
        back.height = ch;
        draw(v,context,backcontext,cw,ch);
    },false);

},false);

function draw(v,c,bc,w,h) {
    if(v.paused || v.ended) return false;
    // First, draw it into the backing canvas
    bc.drawImage(v,0,0,w,h);
    // Grab the pixel data from the backing canvas
    var idata = bc.getImageData(0,0,w,h);
    var data = idata.data;
    // Loop through the pixels, turning them grayscale
    for(var i = 0; i < data.length; i+=4) {
        var r = data[i];
        var g = data[i+1];
        var b = data[i+2];
        var brightness = (3*r+4*g+b)>>>3;
        data[i] = brightness;
        data[i+1] = brightness;
        data[i+2] = brightness;
    }
    idata.data = data;
    // Draw the pixels onto the visible canvas
    c.putImageData(idata,0,0);
    // Start over!
    setTimeout(function(){ draw(v,c,bc,w,h); }, 0);
}

The script is a bit longer this time, because we’re actually doing some work. But it’s still really simple!

document.addEventListener('DOMContentLoaded', function(){
    var v = document.getElementById('v');
    var canvas = document.getElementById('c');
    var context = canvas.getContext('2d');
    var back = document.createElement('canvas');
    var backcontext = back.getContext('2d');

    var cw,ch;

    v.addEventListener('play', function(){
        cw = v.clientWidth;
        ch = v.clientHeight;
        canvas.width = cw;
        canvas.height = ch;
        back.width = cw;
        back.height = ch;
        draw(v,context,backcontext,cw,ch);
    },false);

This is almost the same as I had before, with two real differences.

First, I’m creating a second canvas and pulling the context out of it as well. This is a “backing canvas”, which I’ll use to perform intermediate operations before painting the final result into the visible canvas in the markup. The backing canvas doesn’t even need to be added to the document. It can just hang out here in my script. This strategy will be used a lot in later examples, and it’s quite useful in general, so take note of it.

Second, I’m waiting to resize the canvases until the video is played, rather than just sizing them immediately. This is because the <video> element probably hasn’t loaded its video up when the DOMContentLoaded event fires, so it’s still using the default size for the element. By the time it’s ready to play, though, it knows the size of the video and has sized itself appropriately. At that point, we can set up the canvases to be the same size as the video.

function draw(v,c,bc,w,h) {
    if(v.paused || v.ended) return false;
    bc.drawImage(v,0,0,w,h);

Same as the first demo, the draw() function begins by checking if it should stop, then just draws the video onto a canvas. Note that I’m drawing it onto the backing canvas, which, again, is just sitting in my script and isn’t displayed in the document. The visible canvas is reserved for the displaying the grayscale version, so I use the backing canvas to load up the initial video data.

    var idata = bc.getImageData(0,0,w,h);
    var data = idata.data;

Here’s the first new bit. You can draw something onto a canvas with either the normal canvas drawing functions or drawImage(), or you can just manipulate the pixels directly through the ImageData object. getImageData() returns the pixels from a rectangle of the canvas. In this case, I’m just getting the whole thing.

Warning! If you’re following along and trying to run these demos on your desktop, this is where you’ll probably run into trouble. The <canvas> element keeps track of where the data inside of it comes from, and if it knows that you got something from another website (for example, if the <video> element you painted into the canvas is pointing to a cross-origin file), it’ll “taint” the canvas. You’re not allowed to grab the pixel data from a tainted canvas. Unfortunately, file: urls count as “cross-origin” for this purpose, so you can’t run this on your desktop. Either fire up a web server on your computer and view the page from localhost, or upload it to some other server you control.

    for(var i = 0; i < data.length; i+=4) {
        var r = data[i];
        var g = data[i+1];
        var b = data[i+2];

Now, a quick note about the ImageData object. It returns the pixels in a special way in order to make them easy to manipulate. If you have, say, a 100×100 pixel canvas, it contains a total of 10,000 pixels. The ImageData array for it will then have 40,000 elements, because the pixels are broken up by component and listed sequentially. Each group of four elements in the ImageData array represent the red, green, blue, and alpha channels for that pixel. To loop through the pixels, just increment your counter by 4 every time, like I do here. Each channel, then, is an integer between 0 and 255.

        var brightness = (3*r+4*g+b)>>>3;
        data[i] = brightness;
        data[i+1] = brightness;
        data[i+2] = brightness;

Here, a quick bit of math converts the RGB value of the pixel into a single “brightness” value. As it turns out, our eyes respond most strongly to green light, slightly less so to red, and much less so to blue. So, I weight the channels appropriately before taking the average. Then, we just feed that single value back to all three channels. As we probably all know, when the red, green, and blue values of a color are equal, you get gray. (During this whole process, I’m completely ignoring the fourth member of each group, the alpha channel, because it’s always going to be 255.)

    idata.data = data;

Shove the modified pixel array back into the ImageData object…

    c.putImageData(idata,0,0);

…and then shove the whole thing into the visible canvas! We didn’t need to do any complicated drawing at all! Just grab the pixels, manipulate them, and shove them back in. So easy!

A final note: real-time full-video pixel manipulation is one of those rare places where micro-optimizations actually matter. You can see their effects in my code here. Originally, I didn’t pull the pixel data out of the ImageData object, and just wrote “var r = idata.data[i];” and so on each time, which meant several extra property lookups in every iteration of the loop. I also originally just divided the brightness by 8 and floored the value, which is slightly slower than bit-shifting by 3 places. In normal code, these sorts of things are completely insignificant, but when you’re doing them several million times per second (the video is 480×360, and thus contains nearly 200,000 pixels, each of which is individually handled roughly 100 times a second), those tiny delays add up into a noticeable lag.

More advanced pixel manipulation

You can operate on more than just a single pixel at a time, too, composing some fairly complex visual effects. As I noted at the end of the previous section, performance matters a lot here, but you’d be surprised what you can squeeze out with a little creativity. As you can see in the demo, I’ll be creating an emboss effect in this example, which requires you to use several input pixels together to compute the value of each output pixel.

Here’s the code. The HTML and most of the beginning code is identical to the previous example, so I’ve omitted everything but the draw() function:

function draw(v,c,bc,cw,ch) {
    if(v.paused || v.ended) return false;
    // First, draw it into the backing canvas
    bc.drawImage(v,0,0,cw,ch);
    // Grab the pixel data from the backing canvas
    var idata = bc.getImageData(0,0,cw,ch);
    var data = idata.data;
    var w = idata.width;
    var limit = data.length
    // Loop through the subpixels, convoluting each using an edge-detection matrix.
    for(var i = 0; i < limit; i++) {
        if( i%4 == 3 ) continue;
        data[i] = 127 + 2*data[i] - data[i + 4] - data[i + w*4];
    }
    // Draw the pixels onto the visible canvas
    c.putImageData(idata,0,0);
    // Start over!
    setTimeout(draw,20,v,c,bc,cw,ch);
}

Now let’s step through that.

function draw(v,c,bc,cw,ch) {
    if(v.paused || v.ended) return false;
    // First, draw it into the backing canvas
    bc.drawImage(v,0,0,cw,ch);
    // Grab the pixel data from the backing canvas
    var idata = bc.getImageData(0,0,cw,ch);
    var data = idata.data;

Same as the last example. Check to see if we should stop, then draw the video onto the backing canvas and grab the pixel data from it.

var w = idata.width;

The significance of this line needs some explanation. I’m already passing the canvas’s width into the function (as the cw variable), so why am I re-measuring its width here? Well, I was actually lying to you earlier when I explained how large the pixel array will be. The browser might have one pixel of canvas map to one pixel of ImageData, but browsers are allowed to use higher resolutions in the image data, representing each pixel of canvas as a 2×2 block of ImageData pixels, or maybe 3×3, or maybe even greater!

If they use a “high-resolution backing store”, as this is called, it means better display, as aliasing artifacts (jagged edges on diagonal lines) become much smaller and less noticeable. It also means that rather than a 100×100 pixel canvas giving you an ImageData.data object with 40,000 numbers, it might have 160,000 numbers instead. By asking the ImageData for its width and height, we ensure that we loop through the pixel data properly no matter whether the browser uses a low-res or high-res backing store for it.

It’s very important that you use this properly whenever you need the width or height of the data you pulled out as an ImageData object. If too many people screw it up and just use the canvas’s width and height instead, then browsers will be forced to always use a low-res backing store to be compatible with those broken scripts!

    var limit = data.length;
    for(var i = 0; i < limit; i++) {
        if( i%4 == 3 ) continue;
        data[i] = 127 + 2*data[i] - data[i + 4] - data[i + w*4];
    }

I’m grabbing the data’s length and stuffing it into a variable, so I don’t have to pay for a property access on every single iteration of the loop. (Remember, micro-optimizations matter when you’re doing real-time video manipulation!) Then I just loop through the pixels, like I did before. If the pixel happens to be for the alpha channel (every fourth number in the array), I can just skip it — I don’t want to change the transparency. Otherwise, I’ll do a little math to find the difference between the current pixel’s color channel and the similar channels of the pixels below and to the right, then just combine that difference with the “average” gray value of 127. This has the effect of making areas where the pixels are the same color a flat medium gray, but edges where the color suddenly changes will turn either bright or dark.

There’s another optimization here. Because I’m only comparing the current pixel with pixels “further ahead” in the data which I haven’t looked at yet, I can just store the changed value right back in the original data, because nothing will ever look at the current pixel’s data again after this point. This means I don’t have to allocate a big array to hold the results before turning it back into an ImageData object.

    c.putImageData(idata,0,0);
    setTimeout(draw,20,v,c,bc,cw,ch);

Finally, draw the modified ImageData object into the visible canvas, and set up another call to the function in 20 milliseconds. This is the same as the previous example.

Wrapping up

So, we’ve explored the basics of combining HTML5′s <canvas> and <video> elements today. The demos were very basic, but they illustrated all the essential techniques you’ll need to do something even cooler on your own:

1. You can draw a video directly onto a canvas.
2. When you draw onto a canvas, the browser will automatically scale the image for you if necessary.
3. When you display a canvas, the browser will again scale it automatically if the visible size is different from the size of the backing-store.
4. You can do direct pixel-level manipulation of a canvas by just grabbing the ImageData, changing it, and drawing it back in.

In Part 2 of this article [Ed: coming soon!], I’ll explore some more interesting applications of video/canvas integration, including a real-time video-to-ASCII converter!

video + canvas = magic originally appeared on HTML5 Doctor on October 20, 2010.

This SimpleQuiz is a bit different to previous SimpleQuizzes, since we’d like to use the result of this quiz to inform a design decision in the HTML5 spec. The question concerns how to mute a video in markup, and how this should be reflected in the DOM.

The use case is wanting to have multiple videos playing at once, but with all but one being muted at a time. This can be done today by setting .muted = true with script, but it would be more convenient to be able to mute with markup.

The .muted IDL attribute reflects the user setting of muted for the video element — if the user clicks “mute” in the native video controls, then the value of .muted changes, and vice versa. It would make sense for the browser to remember the mute setting for individual video elements on page reload (or later visit), so that the user doesn’t have to re-mute them.

This is where it starts to get hairy if we introduce a content attribute for muting. (Note that code fragments below aren’t real code, just suggestions.)

We could introduce muted="" which is reflected by .muted (i.e. if one changes, so does the other), but this would cause the DOM to mutate during parsing if the remembered setting doesn’t match the markup, i.e. you would get muted="" to appear in the DOM if the video is muted by the user even though there was no such attribute in the markup, which could confuse your scripts or style sheets if it’s not what you expected to happen.

We could introduce defaultmuted="" which is reflected by .defaultMuted, and just let the user setting change .muted, but defaultmuted="" is a bit long and ugly in markup.

We could have muted="" which is reflected by .defaultMuted and let the user setting change .muted, but it might be confusing for some people. On the other hand it is consistent with <input value> and .defaultValue.

What do you think? Which is the best option? Is there another option that is better than any of the above?

HTML5 Simplequiz #3: how to mute a video originally appeared on HTML5 Doctor on October 15, 2010.

<track> is a child of a <video> or <audio> element that links to a timed track, or time-based data. The kind of data is set via a kind attribute, which can take values of subtitles, captions, descriptions, chapters or metadata, depending on the type of information you’re adding to your media. These point to a source file containing timed text that the browser will expose via some kind of user interface, if the visitor requires additional data.

This will allow for “write once, use everywhere” accessibility; anyone linking to that file with a <video> or <audio> element that includes the element can access your information.

<track kind=captions src=blah.srt>

The file format is a new format called WebSRT, which competes with about 50 other timed formats, including some W3C formats (hence the omission of <track> from the W3C spec).

Added 9 July 2010: Here’s a readable overview of the SRT format.

Given that the format itself isn’t fully specified yet, it will be a while until we see implementation in browsers. But it’s good to know that there will be an official way to add accessibility information to media. Until then, I have a JavaScript hack to take timed <span>s out of an in-page transcript to superimpose over a video.

WebM video format

The big news of the last month is that Google open-sourced the VP8 video codec that it acquired when it took over On2 Technologies. When combined with the Vorbis audio codec (that Spotify uses) and wrapped in a subset of the Matroska container format, it’s collectively known as WebM.

All YouTube videos are being transcoded to WebM, and Adobe have also announced they will include it in their Flash player. It’s available in an Opera 10.60 beta, a Firefox testing build, and a Chromium dev channel. Even Microsoft have said that IE9 will support it if the codec is installed on the computer.

The VP8 video codec itself is high-quality (Google had said that Ogg Theora wasn’t good enough compression-to-quality for YouTube, but Theora was based on the VP3 precursor to VP8). It’s available for streaming too.

If you want to encode to WebM, you can try the Miro Video Converter utility. Although it doesn’t allow you to optimise settings, it’s very easy to use. As the codec becomes more widespread, expect to see many more tools for content creation, editing, and transcoding.

Once production versions of the browsers are available, you should encode your videos with WebM as the first option, Ogg for older versions of Opera, Firefox and Chrome, falling back to royalty-encumbered H.264 for Safari and links to downloads or a Flash player for legacy browsers:

<video controls>
<source src=video.webm type='video/webm; codecs="vorbis,vp8"'>
<source src=video.mp4 type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"'>
<source src=video.ogv type='video/ogg; codecs="theora, vorbis"'>
<!-- embed Flash here -->
<p>Your browser does not support video; download the <a href="video.webm">WebM</a>, <a href="video.mp4">mp4</a> or <a href="video.ogg">Ogg</a> video for off-line viewing.</p>
</video>

If, however, you’re having problems with the iPad, put the MP4 version first in the <source> element; apparently there’s a bug that causes the iPad only to see the first <source> element.

It’s a long haul, and it’s not over yet, but <track> and .webM show significant progress towards our goal of accessible, open, and royalty-free video playing natively in the browser.

Video: the track element and webM codec originally appeared on HTML5 Doctor on June 24, 2010.

Headers and sidebars

Ad asked:

Hello,

I have 2 questions:

1. If I have my main navigation above the masterhead would best practise be having the nav tag inside of a header tag with the h1 and h2 in a hgroup? Or should the nav and header tag be separate?

2. I am building a blog in HTML5. It has a blog-like sidebar with articles, contact info and about info in it. Would the best tag to wrap this in be section or aside?

Thank you so much for your time,
Ad Taylor

Placing your <nav> within your <header> is fine and valid. However, if it makes more sense to leave it outside, then you can do that too. You don’t need to put the <hgroup> inside the <nav> though.

See our articles on <header> and <nav> for more on this.

Regarding your second question, I would use <aside> (as we’ve done on the HTML5 doctor site) and then use multiple sections within that. Also see Bruce’s article on Designing a blog with HTML5.

Hope that helps.

Cheers, Rich

HTML5 <img> element

Martijn asked:

As you are, according to the slogan, “helping [me] implement HTML5 today,” I thought to bother you people with a spec. related question.

What should an UA do with an image without specified width and height attributes?

The dimension attributes part of the specification keeps stating “if specified” for every rule but doesn’t give any “if not specified”.

The part of the specification defining the img element itself does not state anything of importance about the dimension attributes apart from how the attributes in the DOM should be created by the UA.

Interesting is to note that they have omitted these attributes in all their img element examples.

In the dimension attribute section they go state the following.

– The dimension attributes are not intended to be used to stretch the image.

So we can only use them to make images smaller? This is odd as well so let’s say by stretch they mean to say both expending and shrinking in size. In this case the attributes can only be used for two cases:

1. To state the exact width and height of the image. Something that seems redundant unless not using those attributes means the UA can display the image at any size (which it might, as nothing about this is defined in the spec.).

2. To give a 0 in both attributes. By this I am telling the UA that the image is not to be seen by the user.

Am I missing something or is the specification missing this?

Looking forward to getting your prescription ;)

Kind Regards, Martijn

If no dimension attributes are specified, the browser will leave no space for the image, and once the image has been loaded (after the rest of the page), it will then need to reflow the entire page, as that’s the first time it knows the size of the image. This can cause the content you’re reading scrolling off the page.

If you give the size of the image as attributes in the HTML, the browser will leave space and render the image there once it’s loaded without reflowing the page. On a mobile phone, reflowing the page unnecessarily drains the battery, and users can get vertigo from the page’s text jumping around to accommodate images.

Thanks, Bruce

Section and Sectioning

Yanoo asked:

Hi,

What do you mean when you are talking about “sectioning”? And why don’t header and footer require sectioning?

I think about section and sectioning like about part of something defined. News, comment, page content, sidebar, *header*, and *footer*. Is it bad representation?

Sectioning relates to the headings in some block of related content and defining what is related to what in a hierarchy of headings (<h1>–<h6>). The outlining algorithm can produce a table of contents from the nested <section>, <article>, <nav>, and <aside> elements.

Headers and footers themselves do not change the outline; a header or footer may contain no headings. If a header or footer does contain a heading, then that heading does come into the outline. See our article on the section element.

Thanks, Bruce

Scaling the Video

Ian asked:

Hi there,

Not sure if this question is an appropriate one, but any help would be really appreciated.

I’m going about updating my video website, chutney.ie, and would love to implement HTML5. I am interested in replicating the scaling effect/style used on the Mozilla welcome page.

Not being overly knowledgeable in this area, I would love to know how to begin — is this effect a Flash based animation, or something that can be achieved with HTML alone?

Again, any nudge in the right direction would be great.

Many thanks,

Ian

I’m not sure how Mozilla did it, but you can use some of the Webkit and Mozilla transforms on the <video> element. For example, you can cause the video to grow on hover — see this example in Chrome, Safari, or Opera. You can also use the :target pseudoclass to initiate animations. Alternatively, you can use a JavaScript onClick event to create the same effect.

You can also combine <video> and <canvas> to provide some interesting results (laying the <canvas> over the <video>). For more on the <video> element and what it can do, please read the Introduction to HTML5 Video on dev Opera written by Bruce Lawson and Patrick Lauke.

Cheers, Rich

We need a <field> tag

John wrote in and proposed a field element:

Hey there. First off thanks for the site. I was excited to find it. I spent a little time on the W3C site and honestly couldn’t figure out how to submit suggestions there. So going to submit mine to you guys and maybe you can pass it on (if it is good) or point me to someone that can help. Ok to the point:

We are getting nice new tags to with HTML5 (nav, footer, etc) to help us create more semantic code. I think what we really need is a field tag — after all what are fieldsets sets of?

Everyone wraps up their labels and inputs with some element. Some of us do this with UL, some people do it with DT/DD, some with DIVs and some people out there insist that a form is tabular data.

We are all just bastardizing these elements because there is no clear semantic wrapper for field elements of a form.

I think you get the point. I’m trying to keep this email short. If you think there is anything to this argument, I have a more detailed summary (with example code) at:

http://john.mirick.me/2009/10/what-is-really-set-of-shouldnt-we-have.html

I would love to hear your thoughts on this.

Thanks!
John

While a field element would be nice — it would stop the argument over how best to mark-up forms — you have to ask yourself whether or not it actually adds any semantic or functional value to an HTML document. Yes, wrapping inputs and their labels in a field would make things easier to read and drop the need for the for attribute on the label (since the relationship can be assumed), but doing so would not be backwards compatible. In theory, we could continue to add for until there is a suitable time to drop it, but again I question the value of such a tag.

The purpose of wrapping form input/label pairs is generally to ease styling. My personal stance is that unless there are case studies showing how such an element can add more value to HTML forms, this proposal won’t make it very far.

Regards, Mike

Got a question for us?

That wraps up this round of questions. If you’ve got a query about the HTML5 spec or how to implement it, you can get in touch with us and we’ll do our best to help.

Your Questions Answered #5 originally appeared on HTML5 Doctor on March 2, 2010.

Will it work cross browser?

The videos will work natively in Safari and Chrome (well, YouTube would certainly have to, wouldn’t it?). They’ll also work in IE if you have ChromeFrame installed (which we all know is cheating). If you’re using a browser that doesn’t support HTML5 video it will default back to the Flash method they currently employ.

Both YouTube and Vimeo use the H.264 codec to encode the videos — and that’s where things start to get complicated.

Since no official codec is defined in the spec because browser manufacturers have chosen to use different codecs to render HTML5 video. Opera and Mozilla use the Ogg Theora codec (more on that later), while Apple and Google use H.264. Who knows which side of the fence Microsoft will come down on, but I feel duty bound to mention that the licensors of the H.264 codec include both Microsoft and Apple.

The elephant in the room

There’s one major problem here: Ogg Theora, the open source video codec supported by both Firefox and Opera, is not supported on either site. It’s a shame these browsers aren’t supported, especially since Firefox 3.6 was released last week and announced support for full-screen video in its native video player.

Why use Ogg?

As previously stated, Ogg is an open source codec, meaning it’s free to use and implement. By contrast, support for native H.264 decoding in a browser costs approximately $5 million per year. (See summary of AVC/H.264 license terms [PDF].) It also seems that people creating H.264 content will be liable for royalties starting in 2011.

Far be it from me to get into the ins and outs of the debate over which codec to use, but it seems that open source would be the better way to go. Silivia Pfeiffer (a contractor for Mozilla) claims that serving as Ogg Theora will reach more people than serving as H264.

Google, on the other hand, which owns YouTube has said that Theora is not a good enough codec, claiming “If [youtube] were to switch to theora and maintain even a semblance of the current youtube quality it would take up most available bandwidth across the Internet”, a claim hotly (and convincingly) contested in Greg Maxwell’s YouTube / Ogg/Theora comparison.

YouTube and Vimeo seem to have discounted Ogg. I’d love to find out if there are other plans to add support for Firefox and Opera.

Where does that leave us?

Ignoring the whole video codec debate, it leaves us in some ways back where we started — dealing with proprietary software or complicated licensing to create online content rather than using technologies that will allow us to have a truly open web.

With Apple, Google, Mozilla, and Opera not likely to budge on their respective codec choices for HTML5 video, we can only hope that the developers at market-leading sites like YouTube and Vimeo can implement cross-browser interoperable video just like Kroc Camen’s Video for Everybody. Judging by the aforementioned blog posts, both sites have more plans in the pipeline, which is encouraging.

The codec battles are only beginning, and we’re waiting to see which direction one big gun will point. Last year, Google (which owns YouTube) announced that is paying $104million for a company called On2 Technologies, “a leading creator of high-quality video compression technology” and which made the original VP3 codec which is the basis for Ogg Theora.

What are their plans? To release a “better” Theora-like codec into the community? We can only speculate.

Back to today. Although not perfect, these sites’ support for the <video> element is still a massive leap forward for HTML5. Who knows — if more large sites continue to adopt HTML5, maybe it’ll be ready in 2010 rather than 2022!

YouTube and Vimeo support HTML5 Video originally appeared on HTML5 Doctor on January 23, 2010.

However those of us more accustomed to the world of Post-Its, sketches, and .psds – the designers – haven’t been so vocal. Perhaps we’ve been distracted by the bright lights of CSS3 and those surface thrills we’ve longed for. (Rounded corners! Gradients! Transparency!) Or, alternatively, we’ve been in the thrall of @font-face and looking forward to the coming age of passable web typography.

Understandable. But it’s time designers got excited about HTML5 too.

Partly, it’s just good practice. Whatever your flavour of design – visual, web, interaction, user experience – knowing the native technology makes you better at your job. Just as composers should understand the capabilities of the orchestra’s instruments, designers need to understand the language of the web.

But there’s more to HTML5 than simply keeping our skills sharp. It could make a big difference to the way we design for the web.

Semantic elements

Information architects (and, by extension, user experience designers) should be excited by the new HTML5 elements – <nav>, <header>, <aside> and so on. While they won’t immediately revolutionise today’s web, they’re an investment for the future. Doing useful stuff with information is the central theme of IA, and therefore its practitioners should be at the forefront of the new experiences that machine-readable semantics will offer. HTML5 allows us to mark text up in a more meaningful way than a sea of <div>s, meaning we’ll soon see applications appearing at a sub-page level. We’ve started to scratch the surface – think about the Operator toolbar or customisable UIs à la iGoogle – but we’ll need detailed design thinking to work out how to bring the benefits of semantic richness to the end user.

APIs and other extensions

While it’s clear that some of the HTML5 APIs are far from perfect right now, when they’re refined they will offer us intriguing new opportunities and challenges.

Designers of location-based services should of course find the geolocation API invaluable. The contentEditable attribute gives us further power to make the web truly read/write without resorting to JavaScript and custom interfaces. New input types (eg type="search") can provide extra visual cues about input function, although of course this depends on the solutions chosen by the browser manufacturers.

Until now, it’s been easy to consider our domain as bounded by the viewport and the web server. But HTML5 is another step toward seamlessness: the merging of desktop, offline and online. For instance, the drag and drop API could see the line between online and desktop experience blur further. Local storage could allow for a web-like experience in areas of poor connectivity. This convergence is clearly a good thing, but we must also design how to expose those hidden seams at the user’s request. Users should stay in control of how their locations are published and what data is synchronised to their machine.

<video>, <audio>, <canvas>

There is of course something of a reported schism between the standards world and the Flash world. Some see the advent of these new media elements (particularly <canvas>) as heralding the death of Adobe’s poster child.

I don’t think this is either likely or desirable. Neither technology is perfect. Flash is, of course, proprietary and thus subject to the whims of a third party that stands between browser and user. <canvas> has known accessibility problems. But the two can live in harmony, if we play to their respective strengths. Some current Flash applications might be better suited to <canvas>, particularly those based around dynamic visualisation: graphs, animations, infographics. Some applications will benefit from the powerful capabilities of Flash: games, heavily interactive widgets.

This aside, there’s clearly a user experience benefit in not having to rely on an external plugin to play rich media elements, and it will be interesting to see the uptake of the <video> and <audio> elements. Although it will initially be down to browser makers to define the interface elements involved, we will need to figure out how to integrate them into everyday web experiences. The good news is that they can be styled in the same way as any other HTML element. If your visual aesthetic relies on slanted images with box shadows, it’s trivial to apply this to video too.

That said, we can’t ignore the elephant in the room: the thorny codec issue. I’m sure we’d all agree that the sooner it’s resolved, the better.

What can designers do?

Only the most patient and detail-oriented designer will relish the idea of reading the spec in full and arguing the finer points on the WHATWG list. That’s just not the way designers roll.

But as a community it’s important that we start talking about HTML5. If you’re new to HTML, now’s a great time to learn. The articles on this site and A preview of HTML5 give useful guidance on the differences between HTML5 and its predecessors. Above all, designers should get chatting with their developer friends: there can’t be many left who no longer have an opinion on this technology. How do they see their practices changing? What can we do today to prepare our sites for the advent of HTML5? How can we build on its strong points to make the web a better place?

We don’t yet know what we’ll accomplish with HTML5, but then it’s not often that the vocabulary of the web changes this deeply. However, one thing is clear: if we prepare now, we have a great chance to bring innovation to our users’ online lives.

This article was jointly published on Ineffable, Cennydd’s personal website.

Why designers should care about HTML5 originally appeared on HTML5 Doctor on October 14, 2009.

In this post we’re going to cover a few of the questions we’ve received that don’t require a full post answer but still need to be addressed. We’ll post more answers everytime we’ve collated a bunch of questions so don’t be afraid to ask, however groundbreaking or insignificant it might be. Ok here we go for starters…

Automated Conversion?

Andy Mabbett wrote to ask:

Are there (or will there be) any automated or semi-automated tools which will take my (valid) HTML 4.01 Strict documents and convert them to HTML5?

While we don’t know about any specific conversion tools for converting to HTML 5 from HTML 4.01. There are several articles (see below) describing how you can name your classes more semantically in preparation for using HTML 5.

• More on developing naming conventions, Microformats and HTML5
• Preparing for HTML5 with Semantic Class Names
• HTML5 id/class name cheatsheet

A final point on this is that I’m not sure that it would be wise to have a converter in order to move from HTML 4.01 to HTML 5, a machine however clever will not really be able to understand the true meaning of <aside> or <figure> for example. I’d be interested to hear other peoples thoughts on this though?

When should I start using HTML 5?

James asked us (presumably a little tongue in cheek):

Dear Doctor,

When will I know that it is time to start using HTML5 “for real”? Will there be an announcement?

yours etc,
James

Now I’d love there to be an official annoucement, in fact I might just announce it now! You can all start using HTML 5 today. There I said it, I feel so much better now. Seriously though, there is no reason for you not to start using HTML 5 now in sites you’re developing. I’m not saying that you have to use it religiously, but at you should at least start thinking about it in your development roadmaps. We can see on our sister site that there are a number of sites using HTML 5 “in the wild”. The amount of implementation varies greatly (some including <div>‘s around their <header> elements for styling purposes for example) but they have begun to incorporate HTML 5 right now.

Lastly, Molly recently gave a talk at @media which touched upon this subject, she told us that

Implementation trumps specification

which I happen to firmly believe, after all CSS 2.1 isn’t a formal specification yet but almost everyone is using it. If you’re not sure where to start then I suggest taking a look at what browser implementations are currently like, which is where this Wiki from the WHATWG comes in. Hope that answers your question James.

Main Body

Darren asked:

How do I markup the main body of a page, are there <content> tags? Can I serve a SWF Flash file using <VIDEO> tags?

Regards Darren

There isn’t a <content> tag (element), the most appropriate element for that would be <section>, however if only a single article is contained in that area you should use <article>.

As for the second part to your question, you *can’t* use a SWF file natively as the source to the <video> tag, or FLV file for that matter. However, for the foreseeable future you’re still going to need to embed flash for video, via nesting the flash within the <video> tag.

First you’ll use the <video> with nested source elements to fall through the different supported codecs. Then when all else fails, particularly for IE, you’ll serve up the flash video via the <object> and <embed> combo. This is because IE8 and lower doesn’t support the <video> you’re going to need to fall back on something that works.

Make sure you read Tom’s article about the <video> element, but you should also find this article on Video for Everybody useful to see how to degrade through the codecs.

More articles!

Although not really a question I wanted to drop this in, August wrote to tell us:

The <aside> article is awesome. Great explanation of a very semantic HTML5 element. More of those, please!

Thanks for the great feedback August, we’re happy to help, rest assured we’ve got plenty of articles lined up along those lines, in fact I know we’ve got a few crackers lined up for the next couple of weeks, so watch this space.

And finally…

If your question hasn’t been answered in this post or we haven’t got back to you directly then it’s more than likely that we’ll be covering it soon in a more detailed post so be sure to check back or subscribe to the RSS feed for all future articles.

Your questions answered #1 originally appeared on HTML5 Doctor on July 6, 2009.

Of course, this will only work in a few browsers: Safari 3.1+, Firefox 3.5+, and latest builds of Opera (oh, and potentially the next release of Chrome).

The ‘old’ way

Ugh, look at this horrid code:

<object classid="clsid:d27cdb6e-ae6d-11cf-96b8-444553540000" width="425" height="344" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0">
<param name="allowFullScreen" value="true" />
<param name="allowscriptaccess" value="always" />
<param name="src" value="http://www.youtube.com/v/oHg5SJYRHA0&hl=en&fs=1&" />
<param name="allowfullscreen" value="true" />
<embed type="application/x-shockwave-flash" width="425" height="344" src="http://www.youtube.com/v/oHg5SJYRHA0&hl=en&fs=1&" allowscriptaccess="always" allowfullscreen="true">
</embed>
</object>

Yuck. And as you know, you need Flash for this. Or it is often delivered via javascript as well so it is not perfect.

The HTML 5 way

Nice, clean, minimal code:

<video width="640"  height="360" src="http://www.youtube.com/demo/google_main.mp4"  controls autobuffer>
<p> Try this page in Safari  4! Or you can <a  href="http://www.youtube.com/demo/google_main.mp4">download the  video</a> instead.</p>
  </video>

Autoplay

The video tag has an attribute that allows the video to play when the page loads.

<video src="abc.mov" autoplay>
</video>

But autoplay is bad ok? Youtube and the like autoplay their videos. But before you push the launch button for your HTML 5 site, strongly consider whether it should autoplay a video.

Download

If the browser doesn’t know what to do with video tag, or if there is a display error, you can offer a download to the video instead:

Autobuffer

Autobuffer attribute is used when autoplay is not used but the page/site owner thinks the video will be watched at some point. The video is downloaded in the background, so when the user starts the video, it will be able to play at least some of the content. If both autoplay and autobuffer are used, then autobuffer is ignored.

It is worth pointing out at this point that the browser automatically downloads the video anyway, with or without autobuffer and there is nothing you can do about it. This has bandwidth and loading time issues, particularly if you have a lot of video on your page.

Poster

Use the poster attribute to display a frame of the video (as a .jpg, .png, whatever), in case the video doesn’t load for some reason. It can be a local image or elsewhere on the web

<video width="640" height="360" src="http://www.youtube.com/demo/google_main.mp" autobuffer controls poster="whale.png">
<p>Try this page in Safari 4! Or you can <a href="http://www.youtube.com/demo/google_main.mp4">download the video</a> instead.</p>
</video>

You should use the poster attribute because you don’t want the user to see nothing.

Controls

Adding this attribute means you can use your own play/pause/etc buttons for your video. Safari has lovely default controls but you can create your own.

Subtitles

There is no attribute for subtitles (yet – hello HTML 5 group) but just thought I’d mention it. Personally I love the way the JW FLV player handles it, but this is a very interesting read: http://blog.gingertech.net/2008/12/12/attaching-subtitles-to-html5-video/

But ideally, we will be able to delivery subtitles/description audio without javascript.

Current issues

Codecs

I won’t claim to be a video/audio codec expert but there are a few outstanding issues in this area, which this article does a good job explaining.

Internet Explorer

IE hasn’t a clue and as the market leader, will need to be addressed. Thankfully here at HTML 5 Doctor, we have a few hints and tips for IE that will be coming shortly.

Physical file

So far, the video tag needs to link to a physical file (see example at the top). What we need here is a way to just import, say a YouTube video, using the video tag, like we did with the embed tag.

When I set out writing this article, what I wanted to do was kick it back to 2008 and Rickroll the lot of you but so far I haven’t found a way to do it with the video tag.

Here’s what I think you should be able to do with the video tag:

<video src="http://www.youtube.com/v/oHg5SJYRHA0&hl=en&fs=1&" autoplay>
</video>

How awesome would that be?

Conclusion

I think there is a long way to go yet but this tag has real potential. The issue over codecs looks certain to run and run, but you can put fall backs in the code, like the download link. You could use a combination of the video, object, embed elements, but that markup will be pretty horrendous:

<video width="640" height="360" src="http://www.youtube.com/demo/google_main.mp4" autobuffer controls poster="whale.png">
<object classid="clsid:02bf25d5-8c17-4b23-bc80-d3488abddc6b" width="640"height="360" codebase="http://www.apple.com/qtactivex/qtplugin.cab">
<param value="http://www.youtube.com/demo/google_main.mp4">
<param value="true">
<param value="false">
<embed src="http://www.youtube.com/demo/google_main.mp4" width="640"height="360" autoplay="true" controller="false" pluginspage="http://www.apple.com/quicktime/download/">
</embed>
</object>
</video>

In the meantime

If you don’t want to use the video tag yet, then here is valid XHTML code to display a video:

<object width="460" height="265" data="http://vimeo.com/moogaloop.swf?clip_id=5072163" type="application/x-shockwave-flash">
<param value="http://vimeo.com/moogaloop.swf?clip_id=5072163"></param>
<param name="allowFullScreen" value="true"></param>
<param value="always"></param>
</object>

Further reading

• Editor’s Draft 17 June 2009
• Accessibility of HTML 5 video and audio elements
• Setting up a HTML 5 on-line video site
• Watch Video…without Flash
• YouTube HTML 5 demo
• Using the HTML 5 video tag with a Flash fallback
• Lessons From Building a Basic Video Player in HTML 5

The video element originally appeared on HTML5 Doctor on June 18, 2009.