TTG + HTML5 Video

TTG Stage + Video

HTML5 has liberated web video from the confines of Flash players, has brought native video playback to modern web-browsers without the need of plugins, and allows video to be played on smartphones and other mobile devices. But it hasn’t made things easier; if anything, it’s made them harder. And it’s certainly not intuitive.

Mark Pilgrim’s article Video on the Web is one of the most thorough and informative pieces available on the subject of HTML5 video, but its thoroughness comes at a price. While providing an abridged view of HTML5 video, the article is nonetheless longwinded and difficult to read. In this article I will further abridge Mark’s writing, distilling the information to what is relevant and necessary to use HTML5 video with TTG plug-ins, and will hopefully provide a more digestible view of HTML5 video in the process.

Much of the information below is quoted directly from Mark’s article, and will be presented in three major sections:

1. Video containers, video and audio codecs 2. Tools for encoding video 3. Handling video with TTG plug-ins

If the information presented in this article seems complicated, that’s because it is. HTML5 is for web ninjas, those who want to deliver their content in the most optimized methods available and without the necessity for third-party plugins like Flash. If you feel your ninjutsu lacking in the area of the Web, that the information presented here is over your head or frightening, or if you’d just rather not be bothered, then don’t worry about it. TTG plug-ins provide other, simpler options — YouTube, Vimeo and JW Player support — that also allow for universal video playback on all devices, including the iPhone and you may find these methods preferable to HTML5 video.

TTG plug-ins supporting video include TTG CE2 Pages and TTG Stage.

Why HTML5?

Anyone who has visited YouTube.com in the past several years knows that you can embed video in a web page. But prior to HTML5, there was no standards-based way to do this. Virtually all the video you’ve ever watched “on the web” has been funneled through a third-party plug-in — maybe QuickTime, maybe RealPlayer, maybe Flash. (YouTube uses Flash.) These plugins integrate with your browser well enough that you may not even be aware that you’re using them. That is, until you try to watch a video on a platform that doesn’t support that plug-in.

HTML5 defines a standard way to embed video in a web page, using a <video> element. Support for the <video> element is still evolving, which is a polite way of saying it’s not quite as simple as it could be. But with modern browsers having finally implemented the <video> element to a greater or lesser extent, we’re finally able to play video without the aid of third-party plugins on devices ranging from desktop computers to mobile phones. And for those older browsers or devices not supporting the <video> element, we can still fallback on old solutions.

Video Containers, Video and Audio Codecs

The most important thing to understand about delivering video in HTML5 is that there is no one video format that is supported by all browsers. Instead, we must provide video in a variety of formats so that browsers may play whichever video file they do support.

For all appearances video files are just that: video files. In reality, however, they are video containers similar in nature to ZIP files. Just like a ZIP file can contain any sort of file within it, video container formats only define how to store things within them, not what kinds of data are stored. A video file usually contains multiple tracks, video and audio stored separately.

There are many container formats, but only a few we need concern ourselves with for the purposes of HTML5 video embedding.

MPEG 4, usually with an .mp4 or .m4v extension. The MPEG 4 container is based on Apple’s older QuickTime container (.mov). Movie trailers on Apple’s website still use the older QuickTime container, but movies that you rent from iTunes are delivered in an MPEG 4 container.

Ogg, usually with an .ogv extension. Ogg is an open standard, open source–friendly, and unencumbered by any known patents. Firefox 3.5, Chrome 4, and Opera 10.5 support the Ogg container format — including video (called “Theora”) and audio (called “Vorbis”) — natively, without platform-specific plugins. On the desktop, Ogg is supported out-of-the-box by all major Linux distributions, and you can use it on Mac and Windows by installing the QuickTime components or DirectShow filters, respectively. It is also playable with the excellent VLC on all platforms.

WebM is a new container format. It is technically similar to another format, called Matroska. WebM was announced in May, 2010. It is designed to be used exclusively with the VP8 video codec and Vorbis audio codec. It is supported natively, without platform-specific plugins, in the latest versions of Chromium, Google Chrome, Mozilla Firefox, and Opera. Adobe has also announced that a future version of Flash will support WebM video.

Flash Video, usually with an .flv extension. Flash Video is, unsurprisingly, used by Adobe Flash. Prior to Flash 9.0.60.184 (a.k.a. Flash Player 9 Update 3), this was the only container format that Flash supported. More recent versions of Flash also support the MPEG 4 container.

When you “watch a video,” your video player is doing at least three things at once:

1. Interpreting the container format to find out which video and audio tracks are available, and how they are stored within the file so that it can find the data it needs to decode next. 2. Decoding the video stream and displaying a series of images on the screen. 3. Decoding the audio stream and sending the sound to your speakers.

A video codec is an algorithm by which a video stream is encoded, i.e. it specifies how to do #2 above. Your video player decodes the video stream according to the video codec, then displays a series of images, or “frames,” on the screen. There are lossy and lossless video codecs, with lossy codecs being the most appropriate sort for web video. A lossy video codec means that information is being irretrievably lost during encoding. Like copying an audio cassette tape, you’re losing information about the source video, and degrading the quality, every time you encode. On the bright side, lossy video codecs can offer amazing compression rates by smoothing over blockiness during playback, to make the loss unnoticeable to the human eye.

There are tons of video codecs. Within the context of the Web, the three most relevant codecs are H.264, Theora, and VP8.

H.264 aims to provide a single codec for low-bandwidth, low-CPU devices (cell phones); high-bandwidth, high-CPU devices (modern desktop computers); and everything in between. To accomplish this, the H.264 standard is split into “profiles,” which each define a set of optional features that trade complexity for file size. Higher profiles use more optional features, offer better visual quality at smaller file sizes, take longer to encode, and require more CPU power to decode in real-time.

To give you a rough idea of the range of profiles, Apple’s iPhone supports Baseline profile, the AppleTV set-top box supports Baseline and Main profiles, and Adobe Flash on a desktop PC supports Baseline, Main, and High profiles. YouTube now uses H.264 to encode high-definition videos, playable through Adobe Flash; YouTube also provides H.264-encoded video to mobile devices, including Apple’s iPhone and phones running Google’s Android mobile operating system. Also, H.264 is one of the video codecs mandated by the Blu-Ray specification; Blu-Ray discs that use it generally use the High profile.

The H.264 standard is patent-encumbered; licensing is brokered through the MPEG LA group. H.264 video can be embedded in most popular container formats, including MP4 (used primarily by Apple’s iTunes Store) and MKV (used primarily by non-commercial video enthusiasts).

Theora is a royalty-free codec and is not encumbered by any known patents other than the original VP3 patents, which have been licensed royalty-free. Although the standard has been “frozen” since 2004, the Theora project (which includes an open source reference encoder and decoder) only released version 1.0 in November 2008 and version 1.1 in September 2009.

Theora video can be embedded in any container format, although it is most often seen in an Ogg container. All major Linux distributions support Theora out-of-the-box, and Mozilla Firefox 3.5 includes native support for Theora video in an Ogg container. And by “native”, I mean “available on all platforms without platform-specific plugins.”

VP8 is another video codec from On2, the same company that originally developed VP3 (later Theora). Technically, it is similar in quality to H.264 Baseline, with lots of potential for future improvements.

In 2010, Google acquired On2 and published the video codec specification and a sample encoder and decoder as open source. As part of this, Google also “opened” all the patents that On2 had filed on VP8, by licensing them royalty-free. (This is the best you can hope for with patents. You can’t actually “release” them or nullify them once they’ve been issued. To make them open source–friendly, you license them royalty-free, and then anyone can use the technologies the patents cover without paying anything or negotiating patent licenses.) As of May 19, 2010, VP8 is a royalty-free, modern codec and is not encumbered by any known patents, other than the patents that On2 (now Google) has already licensed royalty-free.

Like video codecs, audio codecs are algorithms by which an audio stream is encoded. Like video codecs, there are lossy and lossless audio codecs. And like lossless video, lossless audio is really too big to put on the web. The audio codec specifies how to do #3 — decoding the audio stream and turning it into digital waveforms that your speakers then turn into sound. And since we’re talking about lossy audio codecs, information is being lost during the recording → encoding → decoding → listening lifecycle. Different audio codecs throw away different things, but they all have the same purpose: to trick your ears into not noticing the parts that are missing.

There are gobs and gobs of audio codecs, but on the web, there are really only three you need to know about: MP3, AAC, and Vorbis.

MPEG-1 Audio Layer 3 is colloquially known as “MP3.” If you haven’t heard of MP3s, I don’t know what to do with you. Walmart sells portable music players and calls them “MP3 players.” Walmart. Anyway …

MP3s can contain up to 2 channels of sound. They can be encoded at different bitrates: 64 kbps, 128 kbps, 192 kbps, and a variety of others from 32 to 320. Higher bitrates mean larger file sizes and better quality audio, although the ratio of audio quality to bitrate is not linear. (128 kbps sounds more than twice as good as 64 kbps, but 256 kbps doesn’t sound twice as good as 128 kbps.) Furthermore, the MP3 format allows for variable bitrate encoding, which means that some parts of the encoded stream are compressed more than others. For example, silence between notes can be encoded at a low bitrate, then the bitrate can spike up a moment later when multiple instruments start playing a complex chord. MP3s can also be encoded with a constant bitrate, which, unsurprisingly, is called constant bitrate encoding.

The MP3 standard doesn’t define exactly how to encode MP3s (although it does define exactly how to decode them); different encoders use different psychoacoustic models that produce wildly different results, but are all decodable by the same players. The open source LAME project is the best free encoder, and arguably the best encoder period for all but the lowest bitrates.

The MP3 format (standardized in 1991) is patent-encumbered, which explains why Linux can’t play MP3 files out of the box. Pretty much every portable music player supports standalone MP3 files, and MP3 audio streams can be embedded in any video container. Adobe Flash can play both standalone MP3 files and MP3 audio streams within an MP4 video container.

Advanced Audio Coding is affectionately known as “AAC.” Standardized in 1997, it lurched into prominence when Apple chose it as their default format for the iTunes Store. The AAC format is patent-encumbered; licensing rates are available online.

AAC was designed to provide better sound quality than MP3 at the same bitrate, and it can encode audio at any bitrate. (MP3 is limited to a fixed number of bitrates, with an upper bound of 320 kbps.) AAC can encode up to 48 channels of sound, although in practice no one does that. The AAC format also differs from MP3 in defining multiple profiles, in much the same way as H.264, and for the same reasons. The “low-complexity” profile is designed to be playable in real-time on devices with limited CPU power, while higher profiles offer better sound quality at the same bitrate at the expense of slower encoding and decoding.

All current Apple products, including iPods, AppleTV, and QuickTime support certain profiles of AAC in standalone audio files and in audio streams in an MP4 video container. Adobe Flash supports all profiles of AAC in MP4, as do the open source MPlayer and VLC video players. For encoding, the FAAC library is the open source option; support for it is a compile-time option in mencoder and ffmpeg.

Vorbis is often called “Ogg Vorbis,” although this is technically incorrect. (“Ogg” is just a container format, and Vorbis audio streams can be embedded in other containers.) Vorbis is not encumbered by any known patents and is therefore supported out-of-the-box by all major Linux distributions and by portable devices running the open source Rockbox firmware. Mozilla Firefox 3.5 supports Vorbis audio files in an Ogg container, or Ogg videos with a Vorbis audio track. Android mobile phones can also play standalone Vorbis audio files. Vorbis audio streams are usually embedded in an Ogg or WebM container, but they can also be embedded in an MP4 or MKV container (or, with some hacking, in AVI). Vorbis supports an arbitrary number of sound channels.

There are open source Vorbis encoders and decoders, including OggConvert (encoder), ffmpeg (decoder), aoTuV (encoder), and libvorbis (decoder). There are also QuickTime components for Mac OS X and DirectShow filters for Windows.

Bring it all back to HTML5

As you can tell, video (and audio) is a complicated subject — and this was an abridged version of Mark Pilgrim’s abridged version! I’m sure you’re wondering how all of this relates to HTML5. Well, HTML5 includes a <video> element for embedding video into a web page. There are no restrictions on the video codec, audio codec, or container format you can use for your video. One <video> element can link to multiple video files, and the browser will choose the first video file it can actually play. It is up to you to know which browsers support which containers and codecs.

And it is by this method that TTG Stage CE embeds video using HTML5.

As of this writing, this is the landscape of HTML5 video:

  • Mozilla Firefox (3.5 and later) supports Theora video and Vorbis audio in an Ogg container. Firefox 4 also supports WebM.
  • Opera (10.5 and later) supports Theora video and Vorbis audio in an Ogg container. Opera 10.60 also supports WebM.
  • Google Chrome (3.0 and later) supports Theora video and Vorbis audio in an Ogg container. It also supports H.264 video (all profiles) and AAC audio (all profiles) in an MP4 container. Google Chrome 6.0 also supports WebM.
  • Safari on Macs and Windows PCs (3.0 and later) will support anything that QuickTime supports. In theory, you could require your users to install third-party QuickTime plugins. In practice, few users are going to do that. So you’re left with the formats that QuickTime supports “out of the box.” This is a long list, but it does not include WebM, Theora, Vorbis, or the Ogg container. However, QuickTime does ship with support for H.264 video (main profile) and AAC audio in an MP4 container.
  • Mobile phones like Apple’s iPhone and Google Android phones support H.264 video (baseline profile) and AAC audio (“low complexity” profile) in an MP4 container.
  • Adobe Flash (9.0.60.184 and later) supports H.264 video (all profiles) and AAC audio (all profiles) in an MP4 container.
  • Internet Explorer 9 will support some as-yet-unspecified profiles of H.264 video and AAC audio in an MP4 container.
  • Internet Explorer 8 has no HTML5 video support at all, but virtually all Internet Explorer users will have the Adobe Flash plugin. Later in this chapter, I’ll show you how you can use HTML5 video but gracefully fall back to Flash.

We can summarize this by codec/container as follows:

Theora + Vorbis + Ogg: Firefox 3.5+, Chrome 5.0+, Opera 10.5+

H.264 + AAC + MP4: Safari 3.0+, Chrome 5.0+, iPhone 3.0+, Android 2.0+

WebM: Chrome 6.0+, Opera 10.6+

Safari can play anything that Quicktime can play, and with the help of user-installed third-party plugins can also play Theora + Vorbis + Ogg and WebM. At the moment, Internet Explorer supports nothing; Internet Explorer 9+ will support H.264+AAC+MP4 and WebM.

A year from now, the landscape will look significantly different as WebM is implemented in multiple browsers, those browsers ship non-experimental WebM-enabled versions, and users upgrade to those new versions.

I’ve already said this, but it now bears repeating following the above information. The reason all of this needed to be explained is:

  • There is no single combination of containers and codecs that works in all HTML5 browsers.
  • This is not likely to change in the near future.
  • To make your video watchable across all of these devices and platforms, you’re going to need to encode your video more than once.

Licensing Issues with H.264 Video

Before we continue, I need to point out that there is a cost to encoding your videos twice. Well, there’s the obvious cost, that you have to encode your videos twice, and that takes more computers and more time than just doing it once. But there’s another real cost associated with H.264 video: licensing costs.

Remember when I first explained H.264 video, and I mentioned offhand that the video codec was patent-encumbered and licensing was brokered by the MPEG LA consortium. That turns out to be kind of important. If you read only one part of Mark Pilgrim’s article — from which this article is largely sourced — then make it this part:

http://diveintohtml5.org/video.html#licensing

Go on, read it. I’ll still be here when you get back.

Tools for Encoding Video

There are a plethora of video encoding applications and tools available for the various operating systems, most of them costly, suspicious (the sorts of applications you find via Google Ads, but never see mentioned in legitimate publications) and unnecessary. Most of what you will need is freely available and cross-platform, though not necessarily user friendly.

Miro Video Converter

Miro Video Converter is a free, open source video converter able to convert almost any video to MP4, WebM (vp8), Ogg Theora, or for Android, iPhone, and iPad. Batch conversion, custom sizing, and more!

Check it out: http://www.mirovideoconverter.com/

EasyHTML5Video

EasyHTML5Video is a tool I have discovered only recently, as I revise this article for the wiki, and has quickly become a favorite addition to my web video toolbox. In short, you feed your source video to the application and it spits the video back out in WebM, OGG and MP4 for both HTML5 and Flash fallback. It's a one-stop shop for HTML5 video, and I've even put it to use squashing video file sizes for use on my iPad. Versions are available for both Mac and PC, making it a wonderful solution for Lightroom users.

The catch? EasyHTML5Video costs $49 for a single website license, and $69 for an unlimited websites license. A free version is available, but it tags all of your videos with an obnoxious “EasyHTML5Video” marker; still, the free version lets you try it before throwing down the cash, which is nice.

Check it out: http://easyhtml5video.com/

FFmpeg

FFmpeg is the big daddy of video encoding tools — a complete, cross-platform solution to record, convert and stream audio and video. The catch is that it’s a command-line tool and needs to be compiled before you can use it. You would be forgiven if you turned tail and hid at the mention of “command-line tool” or “compile”, but FFmpeg is one of the best tools you can put into your video toolbox. There is a learning curve, but you should be able to get past it using documentation and some of the many tutorials available on the web.

YouTube user optikalefxx has an excellent tutorial on installing FFmpeg and LAME on Mac OS X. While I haven’t attempted to install FFmpeg on Windows, I expect the process is similar; Windows users should reference FFmpeg documentation.

Mac OS X users will also need to install Xcode, a free download from Apple, before they can use ffmpeg.

Mac OS X users might also consider FFmpegX, which leverages FFmpeg within a familiar Mac-style interface. The problem is that FFmpegX was last updated January 2008, which puts it pretty well behind the times. Personally, I’ve had mixed results from FFmpegX and prefer to use FFmpeg from the command-line.

FFmpeg2Theora

FFmpeg2Theora is another great command-line tool strictly for the creation of OGV video files for playback in Firefox and Chrome. If you’re looking at batch encoding a lot of Ogg video files and you want to automate the process, you should definitely check out ffmpeg2theora.

ffmpeg2theora is an open source, GPL-licensed application for encoding Ogg video. Pre-built binaries are available for Mac OS X, Windows, and modern Linux distributions. It can take virtually any video file as input, including DV video produced by consumer-level camcorders.

For a quick explanation on how to use ffmpeg2theora, jump to the relevant section of Mark Pilgrim’s article.

Handbrake

Licensing issues aside, the easiest way to encode H.264 video is Handbrake. HandBrake is an open source, GPL-licensed application for encoding H.264 video. Pre-built binaries are available for Windows, Mac OS X, and modern Linux distributions.

Again, Mark Pilgrim provides a nice walkthrough on how to use Handbrake to encode your video.

Firefogg

While I have no personal experience using the Firefogg extension for the Firefox web-browser, Mark Pilgrim recommends it as one option for creating OGV video and demonstrates its use.

Handling HTML5 video with TTG Plug-ins

TTG Stage + Video

Before we get started tackling video files, we must first setup our server for video. To do this, we must create a .htaccess file in the root of our domain, or edit the existing .htaccess file if one already exists. These files will often be hidden, so go into your FTP client’s settings and enable it to show hidden files. The code we want to add is this:

AddType video/ogg .ogv
AddType video/mp4 .mp4
AddType video/webm .webm

SetEnvIfNoCase Request_URI \.(og[gav]|mp4|m4a|webm)$ no-gzip dont-vary

The three AddType lines instruct the web server to serve video files with the appropriate MIME Type, which is very important. The last line will prevent servers from gzipping your video files, as some servers are setup to do by default.

With the .htaccess file saved, your server should be setup to deliver video. This is a one time setup and does not need to be repeated to add new videos to your site.

At it’s most basic, HTML5 embedding requires two video files for universal playback, an MP4 and an OGV. Safari, Chrome, iPhones and Android phones will play the MP4; Firefox and Opera will play the OGV file. Internet Explorer 9 will also play the MP4 file. Older browsers will fall back on JW Player to play the MP4 video using Flash.

This setup is reflected in the default HTML5 video settings for both TTG CE2 Pages and TTG Stage:

Default TTG HTML5 video settings

On export, TTG Stage CE creates a folder named videos into which you video files should be copied. These files should have identical file names, as shown below:

The /videos/ folder containing identically named MP4 and OGV files.

Optionally, additional video file types may be added to the array for improved device and browser performance and compatibility.

To add a WebM format video file, enable the “Include WebM Video Source” and drop a WebM video file into the videos folder.

The /videos/ folder containing identically named MP4, OGV and WebM files.

In addition to or instead of adding a WebM file, Quicktime’s “Save for Web” (“Export for Web” in some versions of Quicktime) feature may be used to create optimized video files. To enable support for these files in TTG Stage CE, enable “Include Quicktime Reference Video” and, optionally, “Use Quicktime Poster”.

Personally, I prefer not to bother with Quicktime and do not use these features. It's extra work with little, often negligible returns. But you do what you want.

TTG Stage CE setup to use web video files created by Quicktime, as well as a separately created WebM file.

The poster provides an image file that the browser can show while no video data is available. The poster image is intended to be a representative frame of the video (typically one of the first non-blank frames) that gives the user an idea of what the video is like. By default, the plug-in looks for JPG file within the /videos/ folder named the same as your videos; by enabling both “Include Quicktime Reference Video” and “Use Quicktime Poster”, the plug-in will instead instruct the video embed to use the poster created by Quicktime.

To create the Quicktime reference and video files, load your video into the Quicktime player. Go to the File menu and select “Save for Web…” to bring up this window:

 Quicktime's Save for Web options.

Into the Save As field, enter the name of the video file; this should be identical to the name of your MP4 and OGV files, so for the sake of our example this will be called “video”. When you click the Save button, Quicktime will create a new folder containing several files that you will want to move to your videos folder for use by your page:

The /videos/ folder containing identically named MP4, OGV and WebM files, and web video files created by Quicktime.

video.mov – the reference file for the HTML5 embed page. The browser looks at this file and, based upon the user agent, decides whether to load the larger Computer version of the video, or one of the smaller optimized versions.

video – Computer.m4v – A web resolution video for playback on computer screens.

video – iPhone (Cellular).3gp – A video optimized for playback over a 3G network connection on mobile devices such as the iPhone.

video – iPhone.m4v – A video optimized for playback on mobile devices over high bandwidth connections, such as when connected to the Internet via WiFi.

video.jpg – The poster image.

Do not alter the file names!

Finally, JW Player is used as a fallback so that video may be played on older browsers, such as Internet Explorer 8, not supporting the HTML5 <video> element. For licensing reasons, JW Player is not included with TTG Stage CE and must be installed separately. See the TTG Stage CE documentation for details.

By default, JW Player is instructed to load the MP4 file for playback by the “Fallback video format” setting. If a WebM video file is being used, JW Player can be instructed to use this file instead. And for those wishing it, ffmpeg may be used to create an FLV video file — a Flash video file — which can also be used by JW Player by setting “Fallback video format” to “flv”.

Coda

And that’s all there is to embedding HTML5 video into TTG Stage CE, though there’s still plenty to learn about the HTML5 <video> element for those willing to learn. For those not interested in learning, however, TTG Stage CE handles all of the underlying code, leaving only the actual video files as the user’s responsibility.

Nonetheless, if HTML5 video seems like too much a chore, TTG Stage CE’s other video options are simpler. When fed a single MP4 file, the JW Player embed method can be used to play that file in all browsers and on mobile devices. That same MP4 file, or any other video file supported by YouTube or Vimeo may be uploaded to those services and embedded into TTG Stage CE using the YouTube or Vimeo embed methods, also enabling playback in all browsers and on mobile devices. Regardless of your chosen embed method, TTG Stage CE has your web video needs covered.

That leaves the question, is HTML5 video necessary?

The answer is yes. And no.

If you are embedding video using YouTube, Vimeo or JW Player, then that video is being delivered to desktop computers using Flash. When a mobile user agent is detected however, YouTube, Vimeo and JW Player all deploy the video using an alternate HTML5 player. In these cases, YouTube and Vimeo will prepare your video file for mobile output. In the case of JW Player, you need to have provided an MP4 file; an FLV video file will not playback on mobile devices, for example. So yes, in that every one of these methods is delivering video to mobile devices using HTML5, HTML5 is absolutely necessary.

But no, you don’t necessarily need to know everything I’ve discussed in this article to use those embed methods. The reasons to use the HTML5 method instead of YouTube, Vimeo or JW Player are:

1) You’re a ninja.
2) You don’t want to be dependent upon third-party plugins — Flash, Quicktime, RealPlayer, etc. — at all.
3) You want to deliver video in the most optimal way for the end user’s user agent.

Whether you choose to use the HTML5 embed method or one of the simpler options is entirely up to you, and largely dependent upon the amount of effort and patience you want to put into the process. The tools are obtuse, but not impenetrable. The process is time-consuming, but not impossible. The payoff is there, but may not be justifiable for you. It’s your choice. And whatever you choose, TTG Stage CE works with you to deliver the best results possible.

Print/export