Video in ePub: Captioning, Storage and Other Thoughts

Note: While I talk primarily about ePub e-books, the same process, markup and scripts apply to a standard web page.

After finishing a draft of my fixed layout ePub I went back and researched the accessibility requirements for video on the web and how well supported they are in ePub e-books. I will present both the rationale and coding based on my ePub-based research and the article I wrote for the Web Platform Documentation project https://docs.webplatform.org/wiki/concepts/VTT_Captioning

Working with video in your ePub book presumes that you’re familiar, if not comfortable, with the process of manually creating an e-book. If you’re not then it’s better if you begin with a basic tutorial.

Reviewing video on the web

Ever since Mark Pilgrim wrote the video chapter of Dive into HTML5 the landscape has changed drastically. After a long fight Mozilla capitulated and now supports MP4 video, along with Safari, IE and Chrome. Opera is still the holdout, supporting only WebM and OGG video.

Most e-book rendering engines are WebKit based so, in theory, we should only need one version of the video but in the interest of working for multiple platforms we’ll keep at least two out of the three formats and work with them throughout the rest of the post.

What does the video look like

We define the size of the video with CSS (Optional, can also be defined in the element itself)

video {
  width: 320;
  height: 240;
}

We then define the video with standard HTML element, defining the formats for video in the order we do to make sure that the video will play in older versions of iOS and take into other idiosyncrasies as outline in Pilgrim’s page.

<video controls="controls" poster="video/Sintel.png">
  <source src="video/Sintel.mp4" type="video/mp4">
  <source src="video/Sintel.webm" type="video/webm">
</video>

In HTML we can write the control attribute as just control but ePub requires you to use the, somewhat sillier, controls="controls" instead.

We also add a type attribute to each source video as a hint for user agents (browsers and e-book readers) to use when deciding if they can play a given format.

Moving into ePub

Adding the video

The basic video in ePub is the same than the one we’d use in the open web. We’ll use the same video tag as our starting point with only MPEG-4 and WebM formats.

<video 
      controls="controls" 
      width="320" height="240" 
      poster="video/Sintel.png">
  <source src="video/Sintel.mp4" type="video/mp4">
  <source src="video/Sintel.webm" type="video/webm">
</video>

This will work as written in most reading systems. Still questioning if we need WebM and if so how pervasive it is in the e-book reader world.

Declaring it in the package

Because we are making the video part of the ePub package we need to make sure that we add the components of the video to the package.opf. We do not add them to the spine content because, at the basic level, they are part of a page and not independent content. We are not covering uses of video in the spine of a document.

The items that we added to the package file are listed below:

<!-- VIDEO AND VIDEO IMAGES -->
<item id="video1-mp4"         href="video/Sintel.mp4"    media-type="video/mp4"/>
<item id="video1-webm"        href="video/Sintel.webm"   media-type="video/webm"/>
<item id="video1-cover"       href="video/Sintel.png"    media-type="image/png"/>

As we will discuss later in the essay, one of the first questions that you need to consider is whether to package the video with the book or host it externally. For the purpose of this essay we’ll package the video with the book and discuss some alternatives in the section Video considerations for e-books.

Scripting user interaction

The first part of the JavaScript code is a generic set of functions that do three things:

  • checkReadingSystemSupport test whether a reader supports the features we’ll need for the video to work. The script does this by looping through the values in the neededFeatures variable and if the reader supports the feature then it continues, otherwise it returns false.
  • togglePlay checks if the video ended or if it has paused. If we meet either of these conditions we play or resume video playback; otherwise we pause the video
  • toggleControls checks if the default controls are visible. If they are then the script hides them, otherwise the scripts shows them
/**
* Shared functions used in all pages that use video
*/
function checkReadingSystemSupport() {
  var neededFeatures =["mouse-events", "dom-manipulation"];
  var support = typeof navigator.ePubReadingSystem !== 'undefined';
  if (support) {
    for (var i = 0; i < neededFeatures.length; i++) {
      if (!navigator.ePubReadingSystem.hasFeature(neededFeatures[i])) {
          return false;
      }
    }
  }
  return support;
}

function togglePlay() {
  var video = document.getElementsByTagName('video')[0];
  if (video.ended || video.paused) {
      video.play();
  } else {
      video.pause();
  }
}

function toggleControls() {
  var video = document.getElementsByTagName('video')[0];
  if (video.controls) {
    video.removeAttribute('controls', 0);
  } else {
    video.controls = 'controls';
  }
}

By themselves the functions are good but don’t do much for playing the video. This is where the second script comes in. Using the generic functions in the first script, the functions in the second script will take user input, click/tap or double click/double tap, and perform an action based on the input.

/**
* touch and keyboard based functions
*/

window.onload = function() { // equivalent to jQuery's $(document).ready
  var video = document.getElementsByTagName('video')[0];

  if(checkReadingSystemSupport()) {
     video.removeAttribute('controls', 0);;
  }

  video.addEventListener('click', function(e){
    e.preventDefault();
    togglePlay();
  }, false);

  video.addEventListener('dblclick', function(e){
    e.preventDefault();
    toggleControls();
  }, false);    

  video.addEventListener('keyup', function (e) {
    var k = e ? e.which : window.event.keyCode;
    if (k == 32) {
      e.preventDefault();
      togglePlay();
    }
  },false);
}

Testing

Before moving forward make sure that the video(s) and the poster image are in the directory specified in the page and that it matches the directory you used in the package (this issue tricked me the first time I added video to an e-book)

Package the files as you normally would, test that the video works and validate the book with epubcheck. This is the fist stage.

Video Considerations for e-books

As mentioned earlier, one of the first things to consider is whether to package the video with the e-book or host it remotely. They both have advantages and disadvantages.

Hosting videos remotely means that your users have to be online to play the video. As far as I know there is no way to cache the video and then play it back when the reader is offline.

Adding the video to the book increases the size of the book file and, with the more videos in the book, can come close to or get over the size limit of an ePub e-book. Some vendors (like Amazon) charge for the download based on the size of the file being downloaded.

Example book

The book cc-shared-culture presents multiple ways to add video to your e-book. I’ve chosen to allow both the default controls as well as a click/tap interface. My concern is always that the user knows how to play the video.

I’ve also created a shorter book with a video from theDurian Blender open movie project. It’s hosted in Github along with the book that uses the code and techniques discussed here

Creating captions

Defining captions.

  1. a title or explanation for a picture or illustration, especially in a magazine.
  2. a heading or title, as of a chapter, article, or page.
  3. Movies, Television. the title of a scene, the text of a speech, etc., superimposed on the film and projected on the screen.

From dictionary.com

What’s the difference between captions and subtitles

Although captions and subtitles are similar in the way we create them and add them to videos, they are different in purpose.

Captions serve primarily as an accessibility device that allows people with deaf or who are hard of hearing to fully access the video. Captions also help in situations where the video has no audio, the owner of the video muted it or provided no audio or the environment is too loud for people to listen to the audio.

Subtitles provide translation of the audio and, sometimes, other audio clues to other languages. A kind of subtitles, SDH (Subtitles for the Deaf and Hard of hearing), provides context for the subtitled audio.

Types of captions

As far as Web video captions are concerned there are two types of captions.

TTML (Timed Text Markup Language) is an XML-based captioning system. It is a World Wide Web Consortium recommendation. Internet Explorer is the only browser that supports the technology.

WebVTT (Web Video Text Tracks) is a community-led caption format (it is not a W3C draft or recommendation). It’s similar in structure to SRT captions and there was an earlier proposal called WebSRT. All modern browsers support this type of captions.

VTT captions in detail

We will concentrate in the captioning aspect of the VTT “spec” and will not address other aspects of VTT such as metadata, karaoke styling and other. If you want to read the specification or this HTML5 Doctor article on video subtitling.

At its simplest a VTT file is a text file formatted as shown below (and used with the Sintel video in the book)

WEBVTT

1
00:00:12.000 --> 00:00:15.000 A:middle T:10%
<v.gatekeeper>What brings you to the land
of the gatekeepers?

2
00:00:18.500 --> 00:00:20.500 A:middle T:80%
<v.sintel>I'm searching for someone.

3
00:00:36.500 --> 00:00:39.000 A:middle T:10%
<v.gatekeeper>A dangerous quest for a lone hunter.

4
00:00:41.500 --> 00:00:44.000 A:middle T:80%
<v.sintel>I've been alone for as long as I can remember.  

The hardest part of creating the captions is the timing. It requires hundredth of a second timing and we need to write all digits (even if they are 0) for the VTT cue to display and validate.

controlling positioning of the cue

In addition to adding the timed text we can control the placement of the caption inside the video element

According to HTML5 Doctor, we can use the following positioning attributes

D:vertical / D:vertical-lr

Display the text vertically rather than horizontally. This also specifies whether the text grows to the left (vertical) or to the right (vertical-lr).

L:X / L:X%

Either a number or a percentage. If a percentage, then it is the position from the top of the frame. If a number, this represents what line number it will be.

T:X%
The position of the text horizontally on the video. T:100% would place the text on the right side of the video.

A:start / A:middle / A:end

The alignment of the text within its box – start is left-aligned, middle is centre-aligned, and end is right-aligned. This syntax is similar to how SVG handles alignment

S:X%

The width of the text box as a percentage of the video width.

Some examples of the styles above:

00:00:01.000 --> 00:00:10.000 A:middle T:50%
00:00:01.000 --> 00:00:10.000 A:end D:vertical
00:00:01.000 --> 00:00:10.000 A:start T:100% L:0%

Built-in styles

Bold text: <b>Lorem ipsum</b>

Italic text: <i>dolor sit amet</i>

Underlined text: <u>consectetuer adipiscing</u>

Ruby text: <ruby>見<rt>み</rt></ruby>

Additional styles

You can apply a CSS class to a section of text using <c.myClass>Lorem ipsum</c>, giving us many more styling options.

You can also add a voice indicator to your cue using something like <code><v Tom>Hello world</v>. This declaration accomplishes three things:

  • The caption will display the voice (Tom) in addition to the caption text.
  • A screen reader can read the name of the voice, possibly event using a different voice for male or female names.
  • It offers a hook for styling so that all captions for Tom could be in blue.

Putting it all together

Now that we’ve built the video tag and we’ve taken a look at how to build the VTT caption track we’re ready to put them together. If we’re working with a single language caption file the result will look like this:

<video 
      controls="controls" 
      width="320" height="240" 
      poster="video/Sintel.png">
  <source src="video/Sintel.mp4" type="video/mp4">
  <source src="video/Sintel.webm" type="video/webm">
  <track src="sampleCaptions.vtt" kind="captions" srclang="en">
</video>

The code above is enough to add English captions to the video and have them play using the user agent (browser or reader) native ability.

Furthermore, we can specify multiple caption and subtitles tracks that will allow the user to select which language to view the captions in. The code allowing the user to choose between English captions, German and French subtitles looks like this:

<video 
      controls="controls" 
      width="320" height="240" 
      poster="video/Sintel.png">
  <source src="video/Sintel.mp4" type="video/mp4">
  <source src="video/Sintel.webm" type="video/webm">
  <track src="Sintel-en.vtt" kind="captions" srclang="en">
  <track src="Sintel-de.vtt" kind="subtitles" srclang="de">
  <track src="Sintel-fr.vtt" kind="subtitles" srclang="fr">
</video>

Testing

Prepare your book as you normally would. The testing now requires to test the captions; whether you can show them and whether you can switch th

Additional links and resources

The Trap of CDNs

The problem

There is a tricky issue when working with CDN during development. CDN requires an active Internet connection to actually load the script referenced as the source. If you are not online then jQuery will not load the first time you access the page or application and all other scripts will fail as they depend on jQuery (which couldn’t load from the CDN and had not local backup)

I first came across this issue when building a site that used carousels and jQuery based animations. I started working on the project while on the train and using the standard Google CDN load mechanism. None of the scripts in the page worked. It wasn’t until I saw the following snippet in the HTML5 Boilerplate that made things easier to work with.

A solution

The trick below uses jQuery but it also applies to any JavaScript library loaded through CDN.

<script src="//ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
<script>window.jQuery || 
document.write('<script src="js/vendor/jquery-1.11.1.min.js"></script>')</script>

We first load jQuery 1.11.1 from the Google CDN as we normally would.

Right after we load it from CDN we test if the jQuery object exists and, using a logical or statement (||). If it exists we use that and if it doesn’t then we load a local version of jQuery using document.write to inject the script tag into the document.

Pros and Cons

If you’re not careful this system defeats the idea of having CDNs . You end up with multiple copies of jQuery or other libraries spread throughout your projects that your browser, most likely will not cache. This will adversely affect performance.

As I don’t expect this situation to happen very often. If Google’s CDN goes down there are more serious issues to worry about than my app not working; still this is a good workaround to prevent my content not displaying properly just because of a CDN.

ePub package.opf generator

This is the first pass at a script to generate a basic package.opf file for epub3 ebooks using Python 2.X.

What it does

When you run it from the root of your ebook, the script will create a package.opf file, populate it with basic metadata as required by the epub3 specification, it will also create metadata and spine sections based on the content of the OEBPS directory.

How does the script do it

The script only uses modules from the default library. I want the a portable script and I don’t want to worry whether a module is compatible with 2.X and 3.X, compatible with either version or if uses a different syntax on each version.

At the top of the script we use the environment ‘shebang’ to declare the location of the Python executable without hard coding it.

We import the following modules, each for a specific purpose:

  • mimetypes to find the mime type of our files automatically
  • glob to create the list of files under OEBPS
  • os and os.path to create the items we populate our package file with

After loading the modules the first thing we do is initialize our mime-type database. This will make sure, as much as possible, that we match the file with the correct mime-type.

We then open our package.opf file in write mode.

The last step in this stage is to create the glob expression that will tell the rest of the scripts what files to work with.

We are now ready to create the content we’ll write to the file.

#!/usr/bin/env python 

import mimetypes
import glob
import os
import os.path

# Initialize the mimetypes database
mimetypes.init()
# Create the package.opf file
package = open('package.opf', 'w')

# WARNING: This glob will add all files and directories 
# to the variable. You will have to edit the file and remove
# empty directories and the package.opf file reference from
# both the manifest and the spine
package_content = glob.glob('OEBPS/**/*')

The second stage is to create the templates for the XML portions of the package. There are two things to notice with this part.

  • The XML elements are empty. I only create attributes as necessary
  • I create static templates and don’t use dynamic content because all the modules I found had issues when working with namespaces.

The three templates will be used when building the file.

template_top = '''<package xmlns="http://www.idpf.org/2007/opf"
  unique-identifier="book-id"
  version="3.0" xml:lang="en">
  <metadata >
    <!-- TITLE -->
    <dc:title></dc:title>
    <!-- AUTHOR, PUBLISHER AND PUBLICATION DATES-->
    <dc:creator></dc:creator>
    <dc:publisher></dc:publisher>
    <dc:date></dc:date>
    <meta property="dcterms:modified"></meta>
    <!-- MISC INFORMATION -->
    <dc:language>en</dc:language>
    <dc:identifier id="book-id"></dc:identifier>
    <meta name="cover" content="img-cov" />
  </metadata>
  <manifest>
  '''

template_transition = '''</manifest>
  <spine toc="ncx">'''

template_bottom = '''</spine>
</package>

The enumeration builds the dynamic section of the file. We first create two variables to hold the content and spine of the manifest.

For each element of our package_content (the content of the OEBPS directory) we do the following:

  • Set the basename variable to the part of the current item
  • Get the mime type for the item
  • Add the item XML tag to the manifest assigning it an ID, the base path and the mime type
  • Add the item to the spine by creating the idref element with an ID matching the one we used for the item tag above

When we complete this section, we have a list of all the files under OEBPS and are now ready to, finally, build the package file.

manifest = ""
spine = ""

for i, item in enumerate(package_content):
  basename = os.path.basename(item)
  mime = mimetypes.guess_type(item, strict=True)
  manifest += 't<item id="file_%s" href="%s" media-type="%s"/>n' % (i+1, basename, mime[0])
  spine += 'nt<itemref idref="file_%s" />' % (i+1)

After all the work, actually creating the file is almost anti climatic. We print each section in the following order:

  • template_top
  • manifest
  • template_transition
  • spine
  • template_bottom
# I don't remember my python all that well to remember 
# how to print the interpolated content. 
# This should do for now.
package.write(template_top)
package.write(manifest)
package.write(template_transition)
package.write(spine)
package.write(template_bottom)

An example of the complete file looks like this:

<package xmlns="http://www.idpf.org/2007/opf"
  unique-identifier="book-id"
  version="3.0" 
  xml:lang="en">
  <metadata >
    <!-- TITLE -->
    <dc:title></dc:title>
    <!-- AUTHOR, PUBLISHER AND PUBLICATION DATES-->
    <dc:creator></dc:creator>
    <dc:publisher></dc:publisher>
    <dc:date></dc:date>
    <meta property="dcterms:modified"></meta>
    <!-- MISC INFORMATION -->
    <dc:language>en</dc:language>
    <dc:identifier id="book-id"></dc:identifier>
    <meta name="cover" content="img-cov" />
  </metadata
  <manifest>
    <item id="file_1" href="styles.css" media-type="text/css"/>
    <item id="file_2" href="type" media-type="None"/>
    <item id="file_3" href="book_cover.jpg" media-type="image/jpeg"/>
  </manifest>

  <spine toc="ncx">
    <itemref idref="file_1" />
    <itemref idref="file_2" />
    <itemref idref="file_3" />
  </spine>
</package>

Thing to remember

This is not a complete solution. It is a starting point and it will require manual edits before it passes validation. It is still better than starting from scratch, at least in my opinion.

Things to work on

The first thing I need to figure out is how to skip or remove empty folders. In the example above the media folder needs to be removed manually before the package file will pass epubcheck validation.

Another thing I’ll have to research is whether the glob expression takes all the files we need. For geeks, how many levels deep does the glob expression go?

Exclusions

I’m sad to see the potential of exclusions not being used. No browser vendor supports the complete exclusion specification. IE 10+ is the one that comes the closes but doesn’t support the full set of exclusion features.

This is the best we have (and yes, it still feels weird to say that IE is the best we have in terms of a web feature)

The idea of exclusions is complementary to that of shapes. As a matter of fact, there was only one specification addressing both shapes and exclusions but they were split in 2012, I guess to ease development of at least one of the sections of the specification.

The spec has two primary CSS attributes: wrap-flow and wrap-through.

wrap-flow

Wrap-flow tells the browser how to wrap the content. One thing to notice is that instead of using left and right as attribute values it uses start and end to avoid confusions with right-to-left and top to bottom languages where the meaning of start and end is different.

I based each attribute definition on the how the specification defines it.

  • wrap-flow: auto;

This will not create an exclusion for floated elements. It has no effect on other, not floated, elements. This is the default value for wrap-flow

exclusion_wrap_side_auto

  • wrap-flow: both;

Flows content on both sides of the element

exclusion_wrap_side_both

  • wrap-flow: start;

Inline content can wrap on the start edge of the exclusion area (this would be the left edge for LTR languages.) It must leave the end edge clear

exclusion_wrap_side_left

  • wrap-flow: end;

Inline flow content can wrap on the end side of the exclusion area but must leave the area to the start edge of the exclusion area empty. This is the reverse of the start value.

exclusion_wrap_side_right

  • wrap-flow: maximum;

Inline flow content wraps on the side of the exclusion with the largest available space for the given line, and must leave the other side of the exclusion empty. The space can happen on either side of the content, as shown in the examples below:

Example of wrap-flow: maximum wrapped from the left side
Example of wrap-flow: maximum wrapped from the right side
Example of wrap-flow: maximum wrapped from the left side
Example of wrap-flow: maximum wrapped from the left side
  • wrap-flow: clear;

Inline content flows top and bottom of the exclusion, leaving the start and end sides clear.

exclusion_wrap_side_clear

wrap-through

This property controls whether content wraps around this particular element or not. According to the specification, if the value of the wrap-through property is to wrap:

The element inherits its parent node’s wrapping context. Its descendant inline content wraps around exclusions defined outside the element.

If the value is to none content will not wrap around the element

Combination of exclusions and shapes

Examples taken from the CSS WG use case wiki

One of the best things about exclusions is that they work almost intuitively with shapes as in the examples below. Note that because exclusions are a working draft, the syntax, is not finalized and, most likely, not be supported by your browser (even IE 10+)

I still chose to include the examples as an illustration of what, I hope, is to come

Basic shaped exclusion example

csswg_exclusions_v1

In a two column text frame we create a circle shape at the center and use the shape as an exclusion where we flow the content around both sides using wrap-flow: both;

Padding and margins in exclusions

csswg_exclusions_v7

Adding background to a shaped exclusion

csswg_exclusions_v8

Tutorials and Examples

Using media queries to handle HDPI screens

One of the things that I find infuriating about multi device development is that people can’t seem to agree on a resolution or a device aspect ratio. This has become even more complicated with the advent of retina devices (both hand-held, laptops and desktop devices).

Trying to keep an identical layout in different screen sizes is impossible. Even if we could create identical experiences, the amount of image replacement we have to do to accommodate the different resolutions so our images will not look like crap just is more than I’d like to deal with.

A compromise solution is to use Media Queries to target the work to the devices that we are planning to work with. This post will discuss a few ways in which we can use media queries to target devices based on resolution and orientation.

Before we start

Before we start it’s a good idea to gather references and start looking for a solution generic enough to work across our target devices. I found the following resources to start with:

NOTE: I used SASS to write all the queries below and included them in SASS stylesheets. I don’t like reinventing the wheel so these queries go into a SASS partial to use when needed.

The basic query: Device Pixel Ratio

I based the first query on bourbon hidpi-media-queries file and designed it as a generic query for hdpi browsers. Because it is generic we can’t really use it for specific devices or orientations but it provides a good starting point when tied to information from bjango (see link above).

// HiDPI mixin. 
@mixin hdpi($ratio: 1.3) {
  @media only screen and (-webkit-min-device-pixel-ratio: $ratio),
  only screen and (min--moz-device-pixel-ratio: $ratio),
  only screen and (-o-min-device-pixel-ratio: #{$ratio}/1),
  only screen and (min-resolution: #{round($ratio*96)}dpi),
  only screen and (min-resolution: #{$ratio}dppx)
  {
    @content;
  }
}

One of the things this will allow you to do is substitute images for the proper 2x or 3x resolution. The query will only match one value at a time so there shouldn’t be a problem in using multiple versions of the query with the DPI we want (1, 1.3, 1.5, 2 and maybe 3 if we’re targeting the really high-resolution devices.)

Apple and other hardware vendors have introduced retina / high DPI screens on desktop and laptop systems. We take this into account by introducing prefixed values for our media query corresponding to the vendors that need the prefix (-webkit for Safari, Chrome and Opera (after the blink adoption); -moz for Firefox and -o for older versions of Opera using the Presto rendering engine.)

Where this query falls short is when we try to target specific sizes or devices. That’s where our queries get a little more complicated and need more research.

Device specific queries: iOS devices

Where our first query worried only about DPI, our next set of queries address the following elements of the mobile experience:

  • DPI
  • Device size
  • Device orientation

For each query we create the SASS mixin using a default orientation value of all to use indepeendent of the device orientation. We can also create device specific portrait and landscape rules for each device.

Here are some queries specific to iPhones from 3 to 6+. Peculiarities for each device will be noted below the query.

iPhone 3

@mixin iphone3($orientation: all) {
  $deviceMinWidth: 320px;
  $deviceMaxWidth: 480px;
  $devicePixelRatio: 1;

  @if $orientation == all {
    @media only screen and (min-device-width: $deviceMinWidth) 
      and (max-device-width: $deviceMaxWidth)
      and (-webkit-device-pixel-ratio: $devicePixelRatio) {

      @content;
    }
  }
  @else {
    @media only screen and (min-device-width: $deviceMinWidth) 
    and (max-device-width: $deviceMaxWidth)
    and (-webkit-device-pixel-ratio: $devicePixelRatio) 
    and (orientation:#{$orientation}) {
      @content;
    }
  }
}

Since the iPhone 3 is standard DPI I debated whether to include it on the list. Decided to add it because it uses both landscape and portrait orientations issue and because it’s as far back as I wanted to go with this

iPhone 4 and iPhone 5

// iphone4
@mixin iphone4($orientation: all)
{
  $deviceMinWidth: 320px;
  $deviceMaxWidth: 480px;
  $devicePixelRatio: 2;
  $deviceAspectRatio: '2/3';

  @if $orientation == all
  {
    @media only screen 
    and (min-device-width: $deviceMinWidth) 
    and (max-device-width: $deviceMaxWidth)
    and (-webkit-device-pixel-ratio: $devicePixelRatio) 
    and (device-aspect-ratio: $deviceAspectRatio) {
      @content;
    }
  }
  @else {
    @media only screen and (min-device-width: $deviceMinWidth) 
    and (max-device-width: $deviceMaxWidth)
    and (-webkit-device-pixel-ratio: $devicePixelRatio) 
    and (device-aspect-ratio: $deviceAspectRatio) 
    and (orientation:#{$orientation})
    {
      @content;
    }
  }
}
/* iphone-5 */
@mixin iphone5($orientation: all)
{
  $deviceMinWidth: 320px;
  $deviceMaxWidth: 568px;
  $devicePixelRatio: 2;
  $deviceAspectRatio: '2/3';

  @if $orientation == all {
    @media only screen 
    and (min-device-width: $deviceMinWidth) 
    and (max-device-width: $deviceMaxWidth)
    and (-webkit-device-pixel-ratio: $devicePixelRatio) 
    and (device-aspect-ratio: $deviceAspectRatio) {
      @content;
    }
  }
  @else {
    @media only screen 
    and (min-device-width: $deviceMinWidth) 
    and (max-device-width: $deviceMaxWidth)
    and (-webkit-device-pixel-ratio: $devicePixelRatio) 
    and (device-aspect-ratio: $deviceAspectRatio) 
    and (orientation:#{$orientation}) {
      @content;
    }
  }
}

The main difference between the iPhone 4 and iPhone 5 is the device’s speed. Screen size and device pixel ratio are the same for both devices.

iPhone 6 and 6+

// iphone 6
@mixin iphone6($orientation: all)
{
//1334 ×750
  $deviceMinWidth: 750px;
  $deviceMaxWidth: 1334px;
  $devicePixelRatio: 2;
  $deviceAspectRatio: '40/71';

  @if $orientation == all
  {
    @media only screen 
      and (min-device-width: $deviceMinWidth) 
      and (max-device-width: $deviceMaxWidth)
      and (-webkit-device-pixel-ratio: $devicePixelRatio) 
      and (device-aspect-ratio: $deviceAspectRatio) {

      @content;
    }
  }
  @else {
    @media only screen 
      and (min-device-width: $deviceMinWidth) 
      and (max-device-width: $deviceMaxWidth)
      and (-webkit-device-pixel-ratio: $devicePixelRatio) 
      and (device-aspect-ratio: $deviceAspectRatio) 
      and (orientation:#{$orientation})
    {
      @content;
    }
  }
}
// iPhone 6+
// 1242 × 2208 px

@mixin iphone6plus($orientation: all) {
//1334 ×750
  $deviceMinWidth: 1242px;
  $deviceMaxWidth: 2208px;
  $devicePixelRatio: 3;
  $deviceAspectRatio: '40/71';

  @if $orientation == all
  {
    @media only screen 
      and (min-device-width: $deviceMinWidth) 
      and (max-device-width: $deviceMaxWidth)
      and (-webkit-device-pixel-ratio: $devicePixelRatio) 
      and (device-aspect-ratio: $deviceAspectRatio) {

      @content;
    }
  }
  @else {
    @media only screen 
      and (min-device-width: $deviceMinWidth) 
      and (max-device-width: $deviceMaxWidth)
      and (-webkit-device-pixel-ratio: $devicePixelRatio) 
      and (device-aspect-ratio: $deviceAspectRatio) 
      and (orientation:#{$orientation})
    {
      @content;
    }
  }
}

Unlike the iPhone 4 and 5 there is a real difference between the iPhone 6 and 6+. They have different device pixel ratio and different sizes. If we will support both devices then we need to use both queries.

Non Retina iPad

/* non-retina ipads (1 and 2) */
@mixin ipad($orientation: all) {
  $deviceMinWidth: 768px;
  $deviceMaxWidth: 1024px;

  @if ($orientation == all) {
    @media only screen and (min-device-width: $deviceMinWidth)
      and (max-device-width: $deviceMaxWidth) {

      @content;
    }
  }
  @else {
    @media only screen and (min-device-width: $deviceMinWidth)
    and (max-device-width: $deviceMaxWidth)
    and (orientation:#{$orientation}) {
      @content;
    }
  }
}

iPad 1 and 2 use the same size display, the same device pixel ratio and device aspect ratio. We don’t need specialized queries for each.

Retina iPad (iPad 3 and 4)

/* ipad-retina */
@mixin ipad-retina($orientation: all) {
$deviceMinWidth: 768px;
$deviceMaxWidth: 1024px;
$devicePixelRatio: 2;
$deviceAspectRatio: '4/3';

@if ($orientation == all)   {
  @media only screen and (min-device-width: $deviceMinWidth)
    and (max-device-width: $deviceMaxWidth)
    and (-webkit-device-pixel-ratio: $devicePixelRatio) {
      @content;
  }
}
@else   {
  @media only screen and (min-device-width: $deviceMinWidth)
    and (max-device-width: $deviceMaxWidth)
    and (-webkit-device-pixel-ratio: $devicePixelRatio)
    and (orientation:#{$orientation}) {

    @content;
  }
}

iPad 3 and 4 are retina devices

Device specific queries: Kindle Fire devices

I’m also working on queries specific to the Kindle Fire line of devices. These queries have not been tested as extensively as the iOS queries. If you use them and they work, please let me know via email or by posting an issue to report the results.

Non HDPI Kindle Fire

As with iPhones and iPad we use the non hdpi Kindle Fire as our baseline device.

/* Kindle fire*/
@mixin kindle-fire($orientation: all){
//Model           resolution      PPCM (PPI)  Pixel Ratio
//Kindle Fire     1024x600        67 (170)    1.0 (notHDPI)
$deviceMinWidth: 600px;
$deviceMaxWidth: 1024px;


@if ($orientation == all) {
@media only screen 
  and (min-device-width: $deviceMinWidth)
  and (max-device-width: $deviceMaxWidth) {

  @content;
  }
}

@else {
@media only screen 
  and (min-device-width: $deviceMinWidth)
  and (max-device-width: $deviceMaxWidth)
  and (orientation:#{$orientation}) {
  @content;
  }
}

Kindle Fire 7 Inch

@mixin kindlef-fire7in($orientation:all){
//                     resolution      PPCM (PPI)  Pixel Ratio
//Kindle Fire HD 7"    1280x800        85 (216)    1.5 hdpi
//Kindle Fire HDX 7"   1920x1200       127 (323)   1.5 xhdpi
$deviceMinWidth: 800px;
$deviceMaxWidth: 1200px;
$devicePixelRatio: 1.5;

@if ($orientation == all)   {
  @media only screen and (min-device-width: $deviceMinWidth)
    and (max-device-width: $deviceMaxWidth)
    and (-webkit-device-pixel-ratio: $devicePixelRatio) {

    @content;
  }
}
@else   {
  @media only screen and (min-device-width: $deviceMinWidth)
    and (max-device-width: $deviceMaxWidth)
    and (-webkit-device-pixel-ratio: $devicePixelRatio)
    and (orientation:#{$orientation}) {

    @content;
  }
}

There are two Kindle devices with 7″ screens. One uses HDPI and one uses as an XHDPI device. Need to do further testing to see if they both work well with the same resolution image or if we need to split the query into specific PPI devices

Kindle Fire 8.9 Inch

@mixin kindle-fire89in($orientation:all) {
//Model                   resolution  PPCM (PPI)  Pixel Ratio
//Kindle Fire HD 8.9"     1920x1200   100 (254)   1.5 hdpi
//Kindle Fire HDX 8.9"    2560x1600   133 (339)   1.5 xhdpi
$deviceMinWidth: 1200px;
$deviceMaxWidth: 1600px;
$deviceAspectRatio: '40/71';
$devicePixelRatio: 1.5;

@if ($orientation == all)   {
  @media only screen and (min-device-width: $deviceMinWidth)
    and (max-device-width: $deviceMaxWidth)
    and (-webkit-device-pixel-ratio: $devicePixelRatio) {

    @content;
  }
}
@else   {
  @media only screen and (min-device-width: $deviceMinWidth)
    and (max-device-width: $deviceMaxWidth)
    and (-webkit-device-pixel-ratio: $devicePixelRatio)
    and (orientation:#{$orientation}) {

    @content;
}
}

So how do we use these queries?

As of SASS 3.2 we can use the @content attribute to pass values to the function, not just parameters.

Using the mixins as defines above. We can do something like this to take into account for different pixel densities:

body {
/* Add other attributes as needed */

@include iphone5() { 
  /* styles independent of orientation*/
}
@include iphone5(portrait) {
  background-image: url(http://example.com/bck-iphone-portrait-x2.png);
}
@include iphone5(landscape) {
  background-image: url(http://www.example.com/bck-iphone-portrait-x2.png);
}

@include iphone6plus() {
  background-image: url(http://www.example.com/bck-iphonex3.png);
}


@include ipad(){ 
  background-image: url(http://www.example.com/bck-ipadx1.png);
}

If we’re only worried about pixel ratio, we can use our generic query to just replace some aspects of our content without worrying about device size or other device specific aspects. The sass rule looks like this:

body {

  /* generic content goes here */

  @include hdpi(1.3) {
    /* content for 1.3 HDPI devices */
  }

  @include hdpi(1.5) {
    /* content for 1.5 HDPI devices */
  }

  @include hdpi(2.0) {
    /* content for 2.0 HDPI devices */
  }

  @include hdpi(3.0) {
    /* content for 3.0 HDPI devices */
  }
}

Outstanding questions

Trying to future proof the queries I found out that the current editor draft of Media Queries Level 4 deprecates device-aspect-ratio as a valid media query.

I’m exploring whether to replace it with resolution as the CSS working group wants to use moving forward (as indicated in the email thread starting here) or wait until Media Queries Level 4 moves to Candidate Recommendation, at which point there will be several interoperable implementations (meaning Apple and Microsoft will get on with the program and adopt the same attributes as everyone else.

It is important to note that for iPhone and iPad queries we are ok with using device-pixel-ratio as it’s an Apple device and Safari supports it, where it will become problematic moving forward is non-Apple devices.

Theory and Practice of Digital Publishing

%d bloggers like this: