We currently have no means to provide information about what’s happening or being said in the video, which means the video isn’t very accessible and the user can’t easily navigate to a particular section of the video. Thankfully, there’s a new format specification in the works called WebVTT (Web Video Text Tracks). As of now, it’s only in the WHATWG spec, but the recently established W3C Web Media Text Tracks Community Group should introduce a WebVTT spec to the W3C soon.

You may recall a similar format called WebSRT (Web Subtitle Resource Tracks) that was recently under discussion. WebSRT was renamed to, and has been replaced by, WebVTT.

A WebVTT (.vtt) file is simply plain text containing several types of information about the video:

Subtitles

The transcription or translation of the dialogue.

Captions

Similar to subtitles, but may also include sound effects and other audio information.

Descriptions

Intended to be a separate text file that describes the video through a screenreader.

Chapters

Intended to help the user navigate through the video.

Metadata

Information and content about the video, which isn’t intended to be displayed to the viewer by default, though you may wish to do so using JavaScript.

This article will mostly be talking about subtitles and captions, but it will briefly touch on chapters too.

Beyond the scope of this article but worth mentioning is the text track API, which, amongst other things, denotes how many text tracks there are and which ones have loaded and are ready for use. If you have used this API, let us know.

WEBVTT

<video width="640" height="480" controls>
  <source src="video.mp4" type="video/mp4" />
  <source src="video.webm" type="video/webm" />
  <track src="subtitles.vtt" kind="subtitles" srclang="en" label="English" />
  <!-- fallback for rubbish browsers -->
</video>

WEBVTT

1
00:00:01.000 --> 00:00:10.000
This is the first line of text, displaying from 1-10 seconds

2
00:00:15.000 --> 00:00:20.000
And the second line of text
separated over two lines

1
00:00:01.000 --> 00:00:10.000
Never gonna give you up <00:00:01.000> Never gonna let you down <00:00:05.000> Never gonna run around and desert you

Styling Options #

The previous examples specify the minimum configuration you need for subtitling and captioning, but you can style your captions too. Let’s start with the cue settings, which are done on the same line as the time settings:

D:vertical / D:vertical-lr

Display the text vertically rather than horizontally. This also specifies whether the text grows to the left (vertical) or to the right (vertical-lr).

L:X / L:X%

Either a number or a percentage. If a percentage, then it is the position from the top of the frame. If a number, this represents what line number it will be.

T:X%

The position of the text horizontally on the video. T:100% would place the text on the right side of the video.

A:start / A:middle / A:end

The alignment of the text within its box – start is left-aligned, middle is centre-aligned, and end is right-aligned.

S:X%

The width of the text box as a percentage of the video width.

To make use of these settings, put them alongside the time settings, like this:

00:00:01.000 --> 00:00:10.000 A:middle T:50%
00:00:01.000 --> 00:00:10.000 A:end D:vertical
00:00:01.000 --> 00:00:10.000 A:start T:100% L:0%

which would result in something like the following:

Along with the above cue settings, you can use inline styles for text:

Bold text

<b>Lorem ipsum</b>

Italic text

<i>dolor sit amet</i>

Underlined text

<u>consectetuer adipiscing</u>

Ruby text

<ruby>見<rt>み</rt></ruby>

You can even apply a CSS class to a section of text using <c.myClass>Lorem ipsum</c>, giving us many more styling options.

Finally, you can add a declaration representing the name of the voice: <v Tom>Hello world</v>. This declaration accomplishes three things:

1. The caption will display the voice (Tom) in addition to the caption text.
2. The name of the voice can be read by a screenreader, possibly event using a different voice for male or female names.
3. It offers a hook for styling so that, for example, all captions for Tom could be in blue.

Chapters

You can provide a chapter list for the video the same way you would provide subtitles or captions. Start with the same WEBVTT declaration, and then for each cue, declare the chapter number, the start and stop times, and the chapter title:

<track src="chapters.vtt" kind="chapters" srclang="en" />

WEBVTT

Chapter 1
00:00:01.000 --> 00:00:10.000>
Introduction to HTML5

Video Subtitling and WebVTT originally appeared on HTML5 Doctor on November 29, 2011.

This SimpleQuiz is a bit different to previous SimpleQuizzes, since we’d like to use the result of this quiz to inform a design decision in the HTML5 spec. The question concerns how to mute a video in markup, and how this should be reflected in the DOM.

The use case is wanting to have multiple videos playing at once, but with all but one being muted at a time. This can be done today by setting .muted = true with script, but it would be more convenient to be able to mute with markup.

The .muted IDL attribute reflects the user setting of muted for the video element — if the user clicks “mute” in the native video controls, then the value of .muted changes, and vice versa. It would make sense for the browser to remember the mute setting for individual video elements on page reload (or later visit), so that the user doesn’t have to re-mute them.

This is where it starts to get hairy if we introduce a content attribute for muting. (Note that code fragments below aren’t real code, just suggestions.)

We could introduce muted="" which is reflected by .muted (i.e. if one changes, so does the other), but this would cause the DOM to mutate during parsing if the remembered setting doesn’t match the markup, i.e. you would get muted="" to appear in the DOM if the video is muted by the user even though there was no such attribute in the markup, which could confuse your scripts or style sheets if it’s not what you expected to happen.

We could introduce defaultmuted="" which is reflected by .defaultMuted, and just let the user setting change .muted, but defaultmuted="" is a bit long and ugly in markup.

We could have muted="" which is reflected by .defaultMuted and let the user setting change .muted, but it might be confusing for some people. On the other hand it is consistent with <input value> and .defaultValue.

What do you think? Which is the best option? Is there another option that is better than any of the above?

HTML5 Simplequiz #3: how to mute a video originally appeared on HTML5 Doctor on October 15, 2010.

<track> is a child of a <video> or <audio> element that links to a timed track, or time-based data. The kind of data is set via a kind attribute, which can take values of subtitles, captions, descriptions, chapters or metadata, depending on the type of information you’re adding to your media. These point to a source file containing timed text that the browser will expose via some kind of user interface, if the visitor requires additional data.

This will allow for “write once, use everywhere” accessibility; anyone linking to that file with a <video> or <audio> element that includes the element can access your information.

<track kind=captions src=blah.srt>

The file format is a new format called WebSRT, which competes with about 50 other timed formats, including some W3C formats (hence the omission of <track> from the W3C spec).

Added 9 July 2010: Here’s a readable overview of the SRT format.

Given that the format itself isn’t fully specified yet, it will be a while until we see implementation in browsers. But it’s good to know that there will be an official way to add accessibility information to media. Until then, I have a JavaScript hack to take timed <span>s out of an in-page transcript to superimpose over a video.

WebM video format

The big news of the last month is that Google open-sourced the VP8 video codec that it acquired when it took over On2 Technologies. When combined with the Vorbis audio codec (that Spotify uses) and wrapped in a subset of the Matroska container format, it’s collectively known as WebM.

All YouTube videos are being transcoded to WebM, and Adobe have also announced they will include it in their Flash player. It’s available in an Opera 10.60 beta, a Firefox testing build, and a Chromium dev channel. Even Microsoft have said that IE9 will support it if the codec is installed on the computer.

The VP8 video codec itself is high-quality (Google had said that Ogg Theora wasn’t good enough compression-to-quality for YouTube, but Theora was based on the VP3 precursor to VP8). It’s available for streaming too.

If you want to encode to WebM, you can try the Miro Video Converter utility. Although it doesn’t allow you to optimise settings, it’s very easy to use. As the codec becomes more widespread, expect to see many more tools for content creation, editing, and transcoding.

Once production versions of the browsers are available, you should encode your videos with WebM as the first option, Ogg for older versions of Opera, Firefox and Chrome, falling back to royalty-encumbered H.264 for Safari and links to downloads or a Flash player for legacy browsers:

<video controls>
<source src=video.webm type='video/webm; codecs="vorbis,vp8"'>
<source src=video.mp4 type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"'>
<source src=video.ogv type='video/ogg; codecs="theora, vorbis"'>
<!-- embed Flash here -->
<p>Your browser does not support video; download the <a href="video.webm">WebM</a>, <a href="video.mp4">mp4</a> or <a href="video.ogg">Ogg</a> video for off-line viewing.</p>
</video>

If, however, you’re having problems with the iPad, put the MP4 version first in the <source> element; apparently there’s a bug that causes the iPad only to see the first <source> element.

It’s a long haul, and it’s not over yet, but <track> and .webM show significant progress towards our goal of accessible, open, and royalty-free video playing natively in the browser.

Video: the track element and webM codec originally appeared on HTML5 Doctor on June 24, 2010.