getUserMedia #

getUserMedia is an API that gives a web page access to a user’s camera and microphone via JavaScript. It’s supported in Opera 12 and Opera Mobile 12 for Android, and WebKit in Chrome Canary builds (instructions).

Like many other APIs, it’s not part of the “real” HTML5 spec. It started life as the HTML5 <device> element, then got moved into the W3C as part of the webRTC specifications. But let’s not taxonomise when we could be having fun.

The first thing we need to do when using super-cool new UIs is detect whether it’s supported:

 if (navigator.getUserMedia) {
  // do something cool
} else {
  // fallback code
}

For this article, our “do something cool” will be to grab the dominant colour of the current input stream from the camera. As a fallback, we have several options.

One option, if we’re using the default Android browser, is to show a form that automatically starts the camera when focussed:

<form enctype="multipart/form-data" method="post">
  <input type="file" accept="video/*;capture=camera" />
</form>

There’s also the capture attribute, a proposed extension from the W3C Device API Media Capture API that tells a user agent where to get the input data. Android extends the accept attribute. See more in David Calhoun’s Device API article. Alternative capture arguments are camcorder (for video), microphone, and filesystem.

Browsers that don’t understand the still-draft Media Capture API will simply ignore the capture attrbute and prompt the user to browse to a file on their file system for uploading.

Users on iOS are out of luck, as their devices don’t allow access to the file system.

getUserMedia API #

Let’s assume (because we’re optimists) that your user has a getUserMedia-enabled device. The API is nice and simple, as all Web APIs should be.

navigator.getUserMedia accepts two required arguments and an optional third.

The first argument tells the device which media you require access to, and it’s passed as a JavaScript object. So, if you only require access to the microphone, the first argument would be {audio: true}; for video-chatting, you would use {audio: true, video: true}.

The device decides which camera to use: User agents are encouraged to default to using the user’s primary or system default camera and/or microphone (as appropriate) to generate the media stream.

A previous version of the specification allowed hints to user agents about which camera to use. The API could specify “user” (the front camera on a phone) or “environment” (the rear camera on a phone). Debate continues about whether to reinstate this (read the ongoing conversation). One argument against is that letting sites know what cameras a device has could help Dr Malice of Evil Corp. fingerprint users after luring them to his site (which I find to be overly-cautious). Harald Alvestrand poses a conundrum: conferencing units may have a room camera, a document camera and a lectern camera, neither of which is attached to the physical box; which one of these is front?

But let’s not worry at the moment. Devices with more than one camera generally have a mechanism that allow the user to choose which one is used, so this gives the user control. Note also that the spec says User agents may allow users to use any media source, including pre-recorded media files.

The second argument is the success callback, the function to be executed on success, assuming that the user allows access and the device supports your request.

There is an optional third argument. This is the failure callback, the function to be executed if something went wrong. It’s optional, but only optional in the sense that washing your hands before you eat is optional: if you don’t do it, you leave yourself open to all sorts of bugs and your mum will shout at you.

Of course, it’s important to do feature detection before making an API call to ensure that the browser and device are capable, but even if feature detection succeeds, there is still much that can cause failure. For example, the user could deny permission or turn off the camera via a hardware switch, or the device itself could disable the camera.

So here’s how we recommend you use the API:

navigator.getUserMedia({audio: true, video: true}, success, error);

function success(stream) { 
  // ... use 'stream' ... 
} 

function error(){ 
  // display fallback content 
}

To check whether it’s working correctly, we can attach the output of the camera’s stream to the input of an HTML5 video on the page, like this:

<video autoplay></video>
<script>
if (navigator.getUserMedia) {
  navigator.getUserMedia({audio: true, video: true}, success, error);
  function success(stream)
  { 
    // ... use 'stream' ...
    var video = document.getElementsByTagName('video')[];
    video.src = stream; 
  }
  // ...
</script>

video = document.getElementById('video');