Hour 19. Encoding Your Own Video for Mobile

Last hour we learned about how to get an embedded video to play back in our application. Although we learned how to embed our own video, we did not see how to create the video that we used. This is what we cover in this hour.

Learning the Basics of Video Encoding

Every time you open or watch a video file you are setting into motion a complex process of file parsing and processing. To get started, let me give you a quick high-level view of how a video works.

Each video file is made up of at least one video stream and can be accompanied by several audio streams. When you watch a video file, the player must open the file and determine what data is available and how to display the video content on the screen and the audio content through your sound system. To make matters worse, video files can contain only audio, and they can contain only video. Let’s take a closer look at why this is possible by learning about the video container.

Video Containers

When most people talk about video files they seem to leave off what the file actually is. While techies obsess over the actual extension and container used, people in general have no idea what goes into a video, and what it does. To put it plainly, a container is exactly that—a file that can contain data. You can think of a container as a box that has room inside to hold or contain audio and video data. Figure 19.1 gives a diagram of a container.

Figure 19.1 A diagram of a basic container

Now before you start trying to stuff all sorts of video and audio files into a text file under the idea that is a suitable container, each container has limitations on what and how much it is allowed to hold.

If you are familiar with Windows, you may have heard about or even used .avi files. This file type was extremely common, and although it has now seen better days, can still be found today. A file with an extension of .avi is an Audio Video Interleaved, which is a generic description of what the file contains—a video stream and an audio stream. The default standard in Windows has changed to use the .wmv container, which stands for Windows Media Video.

Macintosh users are no doubt familiar with the QuickTime .mov container. This container is still in common use today and can be played back on other systems when QuickTime or another suitable player is installed.

WebM is a video container created specifically for web videos and can be used with HTML5 video on Opera, Firefox, and Chrome. Internet Explorer supports it when a plug-in has been installed. Other popular containers are Ogg and Matroska (.mkv), which are similar to the MPEG-4 container but usually contain different codecs. Speaking of codecs, let’s take a few minutes and talk about what they are, and why you need them.

Codecs

When videos are encoded, they use a coder/decoder or codec to store the data. Both video and audio streams have different codecs that they can use to achieve a mixture of compression ratio and quality. Some codecs use a better compression method and can achieve a higher quality picture and sound than other codecs, but then often require more processing power to decode the stream. This can create stuttering or frame skipping on the video when it is being watched. On the other hand, if you encode a video with very little compression you may end up with a file that is likely too big to stream or too pixelated if you cut back on your quality.

When dealing with mobile devices you have a limited number of codecs that are supported, and even fewer that are compatible among all devices. Let’s start by looking at the available codecs for audio.

Audio Codecs

Most videos have an audio track that is played back in sync with the video. Quite a few codecs are available that can be used to encode the audio to be played back from a container. Keep in mind that many audio codecs are available, and this list is just of the most common that are compatible with mobile devices.

WAV

WAV files are generally uncompressed audio files and are popular both because of how long they have been around and because they can be used with Windows, Macintosh, and Linux systems. Due to the uncompressed nature of this codec, files are extremely large. While you can use a WAV file for playback on mobile devices, you really should avoid using them in any video you plan on streaming due to bandwidth constraints.

WAV codec summary:

• Stored as .wav files.

• One to two channels at 8000-44100Hz with 4-16 bits per sample.

• Files must be less than 4GB each.

MP3

A format that became extremely popular in the 1990s is the MPEG-1/2 Audio Layer III or MP3 codec. This codec is the audio compression used with MPEG-2 files. MP3 files are highly favored as their compression is great for storing files, and allows easy portability. Not all MP3 files are created equal; some use a different bitrate than others. The bitrate controls the amount of compression used and the amount of room in the spectrum to contain the audio frequencies. Lower bitrates often sound distorted on some frequencies (most notably during cymbal crashes). Most MP3 files are encoded at a Constant Bit Rate that gives a dedicated amount of space to the audio that limits how much space is available. Variable Bit Rate encoding allows space to be shuffled from frequencies that are not currently in use to ones that are in an attempt to make the file stay small while allowing a more dynamic range of music to be heard. Regardless of some of the possible quality concerns, many online music stores including Google Music and Amazon MP3 use this format to distribute audio files. Due to the amount of mobile devices that support this codec, you should consider using it to encode your audio.

MP3 codec summary:

• Stored as .mp3 files.

• 1 to 5.1 multichannel at 16000-48000Hz at 32-320kbps.

• Can be encoded in either a Constant Bit Rate or a Variable Bit Rate.

• MP3 files do not contain Digital Rights Management.

Ogg Vorbis

Another audio format similar to MP3 is known as Ogg Vorbis. This codec is copyright free and endorsed by both Opera and Firefox as being the preferred audio codec in HTML5 videos. Ogg Vorbis is occasionally referenced as just Vorbis. This is because Ogg is a container and Vorbis is the actual codec used inside the container. The Ogg Vorbis codec is versatile and can encode files at a bitrate anywhere from 45kbps to 500kbps. It can also handle samples from 8kHz to 192kHz and can even handle up to 255 discrete channels. When encoding, the quality levels range from -2 to 10. This can make encoding somewhat of a chore as you may need to encode a file multiple times to make sure you are getting the quality you want. However, a quality level of 5 is generally a good value for use when streaming a file. As with all audio, though, your mileage may vary, and you may be able to get by with a lower quality encoding.

Ogg Vorbis codec summary:

• Files are encoded by quality level.

• The codec to use for video files played back in HTML5 on Opera and Firefox.

• Capable of being encoded at higher bitrates than MP3.

• Free from licensing and patent issues.

AAC

Anyone familiar with iTunes or devices running iOS have used the AAC codec. The Advanced Audio Coding or AAC codec is a popular audio codec that can compress audio with more quality than that of an MP3 file of the same size. AAC files are versatile and can have Digital Rights Management solutions embedded into them. AAC files can be encoded in Constant Bit Rate or by Average Bit Rate, which is similar to the Variable Bit Rate that MP3 files use. AAC files can also be encoded in different profiles such as AAC-LC (Low Complexity), HE-AAC (High Efficiency), and HE-AAC v2. Each of these profiles has a different amount of support and features in relation to compression, spectrum, and channel support. Of the three of these mentioned profiles, HE-AAC v2 is the newest and allows a very high rate of compression; unfortunately this and the HE-AAC codec are not compatible with most mobile devices. AAC-LC, however, is not only compatible with most mobile devices, but is highly recommended for use as the audio codec used in videos you want to stream.

AAC codec summary:

• Newer than the MP3 codec.

• Ability to compress with greater quality than other lossy codecs.

• Contains multiple encoding profiles.

• The AAC-LC profiles should be used for maximum compatibility.

• Can contain Digital Rights Management.

Now that we’ve covered a few of the codecs that can be used, let’s take a look at some of the video codecs available.

Video Codecs

Just like audio, video also has codecs that are used for compression. The following is a list of video codecs that can be used with HTML5 and mobile device playback.

VP8

VP8 is a relatively new codec on the field that is used with the WebM container with a Vorbis encoded audio stream. VP8 was acquired by Google and pushed as a replacement codec to the somewhat-legally challenged H.264 codec. VP8 is free to use as it is released from Google under a free patent license. The quality of VP8 is comparable to the existing H.264 codec. There is no official maximum bitrate, but keep in mind that the higher your bitrate, the more bandwidth it needs. Encoding with VP8 can be done in Variable Bitrate (VBR), Constant Bitrate (CBR), and Constrained Quality (CQ). VBR works like it does with some of the audio codecs. Sections of video with fast movement or color changes will be allocated more data bits, while sections that remain mostly still will receive less data bits. Encoding your video with a CBR setting does not actually keep the bitrate constant all the time, but instead offers slight fluctuations in the data bit distribution. Encoding with the CQ setting is similar to VBR, but places a limit on how much data can be spent in one section. When using the CQ setting, encoding the video with a two-pass system is highly recommended.

VP8 codec summary:

• Similar to H.264

• Supports VBR, CBR, and CQ encoding modes

• Capable of multipass encoding

• Compatible with Opera 10.60+, Firefox 4+, Chrome 6+, and IE9 (with WebM MF plug-in installed)

Theora

Theora is the video codec support by the Ogg container. This means that it is license free and can be used with any project or video distribution. Oddly enough, Theora is a spinoff of the VP3 codec (which continued on to become the VP8 codec). In relation to quality and compression, it falls in the middle of the current offering of video codecs. It is currently a well-supported video codec when it comes to use with HTML5 video as it is supported by many modern browsers. While other codecs focus on the bitrate to use, Theora uses a quality setting to adjust the bitrate as needed. When starting out with Theora, it is a good idea to make a few different files at various quality settings and choose the one that has the best balance of quality and file size for Internet delivery.

Theora codec summary:

• Similar to both H.264 and VP8

• Encoding based on quality instead of bitrate

• Compatible with Firefox 3.5+, Opera 10.50+, and Chrome 3.0+

H.264

H.264 actually has a few names. It can also be referred to as MPEG-4 Part 10 and AVC (Advanced Video Coding). I have been guilty on occasion of just referring to the codec as the container it is usually stored in, which is mp4. The H.264 codec is extremely popular, and you can find support for it on most mobile devices including the Sony PSP and iPod Touch. Because the codec can also be used on Blu-ray discs, there are profiles that devices can certify at to guarantee playback. Most mobile devices cannot play anything higher than the Baseline profile. While there are roughly 18 different profile settings, the three commonly found in encoders are Baseline, Main, and High. Each profile determines what features of the codec can be used. In addition to the profiles, you can also use levels to adjust the data available for encoding. The levels currently range from 1 to 5.2; however, most mobile devices do not support any level higher than 3.0. H.264 is a licensed codec, but it has been given a status of being free for encoding and decoding video over the Internet. This license may, however, be changed in the next few years as the license is up for renewal every five years. Depending on the program you use to encode, you may do multipass, CBR, and VBR encoded videos. Note that even though some programs offer this, it may not work and also may affect the playback functionality of your video.

H.264 codec summary:

• Very common codec, used in many commercial products

• Currently license free for videos distributed over the Internet

• Profile and level based encoding

• Supported by Internet Explorer 9+ and Safari 4+

Comparing Video Codecs and Mobile Devices

Video playback is important on today’s mobile devices. Users expect to be able to watch movies, news, and entertainment. Because of this, most devices have built-in support for many video types. The following sections provide a handy reference list of devices and relevant specs for video playback. This list is important to know as it can be used to help you determine what you should be aiming for when creating videos for end users.

Android

Android has good support for common video formats as well as a wide range of audio file support. Something that should be noted with devices running Android is that some codec support can be added by custom builds and by third-party manufacturers. For example, the Samsung Galaxy Player 5.0 has support for FLAC audio files even though it runs Android version 2.3.5.

• Native resolution: Varies on make and model of device

• Video codecs: H.263, H.264 AVC, MPEG-4 Baseline Profile, VP8 (Android 2.3.3+)

• Audio codecs: AAC LC 0-160kbps 8-48 kHz, HE-AACv1 0-160kbps 8-48kHz, HE-AACv2 0-160kbps 8-48kHz, AMR-NB 4.75-12.2kbps sampled @ 8kHz, AMR-WB 9 rates from 6.60-23.85kbit/s @ 16kHz, MP3 8-320kbps CBR/VBR, FLAC (Android 3.1+) recommended sample at 44.1 kHz but supports up to 48kHz, 16-bit recommended but support for 24-bit

• Video containers: .3gp, .mp4, .webm, .mkv

• Visit http://developer.android.com/guide/appendix/media-formats.html for changes in support

A challenge that you may encounter with Android devices is the wide resolution differeces between devices. This can be somewhat overcome by choosing a resolution for video that fits most devices. There will still be a problem on high resolution devices that may get either a small video playing back on their device or a video that is pixelated.

BlackBerry

BlackBerry has stepped up its media support game since the v6 of the BlackBerry OS. Devices running a previous OS have varied support for both audio and video codecs. For media content on your site, you can start by using the media that works on OS 6+ devices and then use device analytics and user feedback to adjust content to your consumer.

• Native resolution: Varies per device and OS

• Video codecs (v6 OS): Varies, but .3gp, .avi, .asf, .mp4, .mov, .wmv

• Audio codecs (v6 OS): .aac, .amr, .flac, .mp4, .m4a, .ogg, .wma, .wav

Similar to Android, the resolution of each device varies. This means you need to adopt a similar strategy for video playback. BlackBerry recently changed the way it lists supported audio and video codec for each device. Visit http://docs.blackberry.com/en/smartphone_users/?userType=1 and search for “media types”. This should return a few links to a current .pdf file for smartphones and another for tablets that contain the supported codecs for current devices. You may also use the navigtion menu on the page to find a link to Supported Media under the Tablets section, and also under the Smartphones an entry for BlackBerry Smartphones.

iPhone 4S

The iOS system that runs on the iPhone 4S allows some advanced playback features. While the original iPhone was limited on the encoding of the H.264 video it would play back, the current OS offers greater support and playback options.

• Native resolution: 640×960

• Video: H.264 Main Profile Level 3.1 - MPEG-4 2.5Mbps 640×480, 15/23.9/30 fps, 2 Simple Profile - also supports MJPEG

• Audio: AAC (8 to 320 Kbps), HE-AAC, MP3 (8 to 320 Kbps), MP3 VBR, Audible (formats 2, 3, 4, Audible Enhanced Audio, AAX, and AAX+), Apple Lossless, AIFF, and WAV

• Video containers: .m4v, .mp4, .mov

• Visit http://www.apple.com/iphone/specs.html for changes in support

While the iPhone 4S runs at a set resolution of 640×960, remember that the previous iPhone models prior to the iPhone 4 run with a resolution of 320×480. To complicate this a touch more, the iPad and iPad2 both have a resolution of 1024×768 while the new iPad contains a resolution of 2048×1536.

Windows Phone 7

The Windows Phone system currently runs with a set resolution for all devices. This makes designing for them nice as all screens display things in the same proportion. This may change shortly, but for now you can skip worrying too much about how the video is going to play back on all devices.

• Native resolution: 480×800

• Video codecs: WMV Simple, Main, and Advanced profile, MPEG-4 Part 2 Simple and Advanced profile, MPEG-4 Part 10 (H.264, AVC) Baseline, Main, and High Profile Level 3.0, and H.263

• Audio codecs: AAC-LC, HE-ACC v1 (AAC+), HE-ACC v2 (eAAC+)

• Video containers: .wmv, .3gp, .3g2, .mp4, .m4v, .amr

• Visit http://msdn.microsoft.com/en-us/library/ and search for media formats to learn about the latest media support

With the release of tablets and mobile devices pending, the support level for media playback may change soon. Microsoft has shown strong support for H.264 in HTML5 video playback, so with that in mind video support on mobile devices should stay near whatever can be played back in Internet Explorer.

Now that we have a list of what common mobile devices are capable of, let’s learn about the tools and settings used to encode videos for playback.

Encoding Video for Mobile Playback

If you’ve never encoded video before, it’s common to think that you need to use the absolute highest quality possible. While getting the highest quality possible may sound like the best idea, it will result in either a file size that is unacceptable for mobile delivery or a file that is unable to be played back on mobile devices. Luckily there are some tools you can use to help you learn how to create the perfect video. Sometimes the best way to learn is through a little investigation, and some slight reverse engineering. Let’s take a look at some tools to find out how videos are encoded.

Discovering Inspection Tools

Inspection tools work as analyzers that open a container file and tell you about the video and audio stream information. This is similar to the Get Info option in OSX and like viewing properties of a file in Windows. The biggest difference is that when using an inspector program you get a lot more detail. Figure 19.2 shows what is returned when using Get Info on OSX and viewing file properties in Windows.

Figure 19.2 Details about the video container are shown.

While the details given are relevant, such as file size and resolution, in Windows they do not tell you the bitrate used or even what codecs are used for each stream in the file. On OSX you see some basic information, but you are not shown the bitrate for the individual audio and video streams but instead are shown a combined total bitrate. You also are not shown extra codec features such as the level the video was encoded at. For that we need to use some other tools.

My absolute favorite video inspection tool is called MediaInfo and is available for Windows, Mac, and Linux users. It is available as a free, GPL licensed download on sourceforge at http://mediainfo.sourceforge.net/en/Download. Once you have downloaded and installed the application, you are ready to start inspecting files. When MediaInfo is opened, it presents an empty window divided into sections. To inspect a file, either use the program menu to select a file, or drag-and-drop the file into the program window. Once a file has been dropped, the window populates the sections with data from the file. Figure 19.3 shows the program inspecting a video.

Figure 19.3 MediaInfo inspecting a file

As you can see from Figure 19.3, the file in use has one video stream, and one audio stream. The video codec used was AVC, and the audio codec was AAC. In the sections below you can get more detailed information. The video section shows the bitrate the video was encoded at, the resolution of the video along with ratio, how many frames-per-second, and the actual codec along with applicable profile and level. Other encoding features also are shown when detected. The audio section displays the bitrate, sample rate, channels, and codec used.

While this is the best standalone solution for Windows users, other inspection tools available are as follows:

OSX

• Media Inspector—Available in the AppStore

• Invisor and Invisor Lite—Available in the AppStore

• VideoSpec—Available at http://videospec.free.fr/english/

• iMediaHUD—Available at http://www.iffmpeg.com/styled-6/index.html

Linux

• Themonospot—Available at http://www.integrazioneweb.com/themonospot/

Now that you know how to determine what is in a video file, you use this information to encode a video. Let’s look at some programs that you can use to encode your video.

Exploring Transcoding Programs

Many programs are available to help you encode your mobile video. Most of these programs work through a process known as transcoding. This process takes an already prepared video and changes the settings including codecs and bitrate used to create a new video file. The higher the quality of your source file, the higher quality your transcoded video will be.

MiroVideoConverter

The MiroVideoConverter (http://www.mirovideoconverter.com/) is a program available for free to Windows and Macintosh users. It is incredibly simple to use. Once the program has been opened, you drag and drop the file you want to transcode on the program window, pick an output format, and then click the Convert! button. The problem with the converter is that it does not give you any control over the settings used for the transcoding process. This makes it difficult to fine-tune your video files to the exact settings you want.

HandBrake

HandBrake (http://handbrake.fr/) is another open-source video transcoding program available for Windows, Macintosh, and Linux users. While the primary function of the program is more along the lines of creating personal DVD backup files, it can be used to create both .mp4 and .mkv files. Once the program has been installed, you open the file you want to transcode and then make adjustments to the output by using the Video, Audio, Subtitle, Advanced, and Chapter sections at the bottom of the program window.

MediaCoder

MediaCoder (http://www.mediacoderhq.com/) is a program that has gone through a bit of change in the last few years. When it first appeared it was an all-in-one solution, but it has since branched out into specialized versions. It is a Windows program, but the author mentions that by using an emulator it can be used on Linux and on Macintosh systems. MediaCoder is a transcoding program compatible with most format containers and codecs. It may occasionally throw an error when you attempt to create a file with incompatible codecs together. I have found that for Windows transcoders, this particular program gives you a considerable amount of flexibility and options. The program is built with a wizard to walk you through file creation, or you may adjust everything manually. If you are using the manual settings you can drag and drop the file or files you want transcoded into the program and then use the Settings tabs at the bottom to adjust the output type and format.

MPEG Streamclip

The last encoding/transcoding program we are going to cover is MPEG Streamclip (http://www.squared5.com/). It is available for both Macintosh and Windows systems. It contains a player and editor, and of course the ability to transcode files from one format into another. The program works similarly to the others listed. Once you have opened the program, you can drag and drop a video onto it. You can then use the File menu to view information about the streams contained in the video, and to export the video to another format.

Encoding a Video

Regardless of the program you end up using, if you want to support the widest number of devices possible there are some rules you need to follow:

• Encode with an .mp4 container.

While it is true that some browsers do not support .mp4 for HTML5 video playback, almost every mobile device supports the .mp4 container for native playback.

• Keep your bitrate low between 400-800 kbps.

The bitrate does affect how clear or pixelated your video is; however, it is also a key factor in file size and CPU load when decoding the video. If you can keep it in that range, you should be able to get acceptable quality and file size.

• Encode your audio with AAC-LC.

Of the AAC codecs this is the lowest in quality, but it is the most supported. Don’t worry too much about it; even at 48kbps it still sounds fairly good. Most encoders allow you to use a quality setting that creates a variable bitrate file that gives a good balance between file size and sound.

• Encode your video with H.264, Baseline, level 3.0, no B-Frames.

This should be pretty straightforward. Use AVC or H.264 and then if your encoding program allows you to choose a profile and level, choose Baseline and 3.0. If your encoding program does not allow these options, then turn off any extra options such as B-Frames.

• Unless you absolutely need HD video, use a resolution near 480×360 (4:3) or 480×270 (16:9).

The maximum resolution supported with level 3.0 is 720×576 when playing back at 24 frames per second. This really means that you will not be able to provide a true HD file anyway unless you use a file with a different level or container. While this is changing, legacy devices will continue to be an issue for several years.

If you follow those rules, the videos you create will play back on most smartphones, personal media players, and tablets.

To get an idea of how the settings should look let’s take a look at some figures. Figure 19.4 shows the export to MPEG-4 setting screen from MPEG Streamclip with the settings for mobile video. Figure 19.5 and Figure 19.6 show the Video and Advanced settings tab configurations needed when using HandBrake for transcoding mobile video.

Figure 19.4 The export screen of MPEG Streamclip using mobile-safe settings

Figure 19.5 The settings used on the Video tab for mobile-safe encoding

Figure 19.6 The settings used on the Advanced tab for mobile-safe encoding

Now that you can encode or transcode your video content, let’s look at how to get it to play back on mobile devices.

Delivering Video Content

When displaying video content on a mobile device you have a few options. You can use HTML5 video tags to display and control your video, you can use various JavaScript plug-ins, or you can link directly to the file allowing the device to play the file back in the native device player. Since we have already covered using the HTML5 video element, let’s take a look at some common plug-ins to help handle our mobile videos.

Using Plug-Ins and Players

Before you get too excited about the prospects of using a plug-in or player to handle all your video needs, there is one thing you need to do first. It doesn’t matter what web server you are running, but you must configure your MIME types or some of these plug-ins may not work. Either modify your server configuration file or if using an Apache server, add the following to an .htaccess file:

AddType video/ogg .ogv
AddType video/mp4 .mp4
AddType video/webm .webm

Many of the video plug-ins available for use are a clever collection of JavaScript and CSS allowing skinning support and device detection. Some players even allow fallback support to Flash for browsers that do not support HTML5. Several players on the Web today do not support mobile devices at all, so we’re going to take a look at the ones that do.

Projekktor

Projekktor (http://www.projekktor.com/) was one of the first HTML5 video players that I actually heard about. It is a fully featured player and can be tailored to suit almost anything you need. Projekktor is released under the GPL, so it is free to use in your projects. If you get stuck or need some support you have several options from free (in the form of an FAQ and document listing) to paid support. For nonmobile devices that browse this plug-in that do not support HTML5 video, a Flash fallback option is available.

MediaElement.js

MediaElement.js (http://mediaelementjs.com/) is quickly becoming the popular choice for guaranteed video playback regardless of browser and device type. It even allows you do decide how much support you want to give. This means that if you only want to provide one H.264 encoded video, then that is all you need to do. By doing so you limit older browsers, but it does save you a few lines of code and makes prototyping or testing a snap as you can always go back and add extra support later. It also has some built-in plug-ins that add a little extra functionality and/or eye candy to the player. The license is either GPL v2 or MIT, so you can feel free to use this in your web projects.

JPlayer

JPlayer (http://jplayer.org/) is another player that sports some excellent ideas. It can be used as both a video player and as an audio player. Just like the other players, it contains fallback support to Flash. Similar to MediaElement.js, you can get by with just an .mp4 file; however, for full compatibility you need to generate an .ogg and .webm file as well. It is licensed as GPL or MIT so you are free to include it in your web projects.

Video.js

Video.js (http://videojs.com/) is another option that provides a player with a common skin that works cross-browser. It also features integration with either Flash or Silverlight for fallback support. Unlike some of the previous players, it does not rely on including the jQuery library and can be run as a standalone or with others. Video.js is one of the easier players to use as you can include the required files from their CDN, and then create the HTML5 video element as you normally would with a few extra classes and a custom data attribute named data-setup. The license is a variant on the GPL known as the Lesser GPL, which allows you to use it on your site as long any improvements you make are contributed back to the Video.js team.

While there are always times when you would like use a specific interface that a player brings, occasionally it is better to just let the device handle the video. This is where using the native video player comes in.

Using the Native Player

The native video player varies on each mobile device; however, on most iOS and Android devices the player is a simple one that allows the playback of .mp4 files. To trigger one from your site, you can simply create a link directly to your video file as follows:

<a href="videos/demo.mp4">Watch a demo</a>

When the link is clicked, the browser should detect that it is a video file and ask what to do with the file, or it launches the native player for the device. When viewing is finished, the user is usually returned to the page that called the file.

Sometimes, however, this method can be a little unpredictable. While most iOS and Android devices play the .mp4 file when linked from a website, some devices and browsers download the file instead, which in turn forces the user to find the file and load it into the native player themselves. With my own testing I’ve found that by declaring the MIME types mentioned earlier, most devices play the video or at least ask whether you want to play the video before just downloading and storing the file. Another problem with using this method is that a user is shifted to the video screen and then forced to wait until the file has been downloaded enough to begin playback. This can be a little strange for users who are not expecting the transition, or the possibly long wait time.

Summary

In this hour you learned what video containers are and what they are capable of holding. You learned about the various codecs available for both video and audio streams and what the current specs are for several mobile devices in regards to video and audio playback capability.

You learned about some tools available for the inspection and creation of videos. This included learning about the basic rules of encoding a video with an almost guaranteed setting for playback on all video-capable mobile devices.

You also learned about some plug-ins available to integrate HTML5 video into your site that work with mobile devices. When plug-ins are not desired, you can link directly to a video file and have the device play the file inside the native player.

Q&A

Q. Are there any other video inspectors or encoders?

A. Yes, there are quite a few. I chose to cover the ones that either I have used personally or would fit the budget of a user just starting out. Many commercial programs such as Adobe Premiere and Final Cut have the capability to produce videos with the same constraints on them. There are also specialized encoders such as Firefogg that creates .ogg files for video playback.

Q. How do these videos work with jQuery Mobile?

A. Using jQuery Mobile doesn’t change the way any of the plug-ins are used, although you may have to change the load to the pageinit event.

Q. What happens if I try to view a video on a device that does not support HTML5, Flash, or any other fallback?

A. Generally when this happens you either get an error message about a failure to play back the video or get a direct link to download the video. As a developer what the end user sees is completely up to you.

Workshop

The workshop contains a quiz and some exercises to help you check your comprehension and understanding.

Quiz

1. What is the profile you should use for H.264 to maximize device compatibility?

2. What is the video codec used with an .ogg container?

3. What is a benefit to using a plug-in for video playback?

4. What version of the AAC codec should be used with mobile devices?

5. How can you use the native player from your site?

Answers

1. A profile of 3.0 allows maximum compatibility with mobile devices.

2. The Theora codec is used inside the .ogg container.

3. Plug-ins supply customization, ease of use, support for legacy browsers, and fallback support for other video delivery systems like Flash and Silverlight.

4. AAC-LC works on almost all mobile devices and as such should be used whenever possible.

5. If supported, the native player is triggered when direct link is used to the video file.

Exercises

1. Download MediaInfo and use it to analyze some video files on your computer. Compare the codec settings with what we learned this hour.

2. Create a video, or use the one from the previous exercise, and transcode the file into one that works on your device. Copy the file to your device and see whether it will play back. Take this a step further and create a simple HTML page with a direct link to the file. Use your mobile device to see whether the video will play in the native player.

3. Experiment with one of the player plug-ins and compare how it functions on various browsers and systems. If you have multiple devices available, examine the playback experience between them.