Web Performance Improvement: The Client Side Of The Equation

We’ve looked at a lot of server-side tricks to reduce the page load and improve users’ experience when accessing our content. Now we’ll look at client-side client hints and tools to optimize our content.

Client side resource hints

The basic way to preload a resource is to use the <link /> element with three attributes:

  • rel tells the browser the relationship between the current page and the resourced linked to
  • href gives the location of the resource to preload
  • as specifies the type of content being loaded. This is necessary for content prioritization, request matching, application of correct content security policy, and setting of the correct Accept request header.
<link rel="preload" href="late_discovered_thing.js" as="script">

Early loading fonts and the crossorigin attribute

Loading fonts is just the same as preloading other types of resources with some additional constraints

<link rel="preload"
      href="font.woff2"
      as="font"
      type="font/woff2"
      crossorigin>

You must add a crossorigin attribute when fetching fonts since they are fetched using anonymous mode CORS. Yes, even if your fonts are of the same origin as the page.

The type attribute is there to make sure that this resource will only get preloaded on browsers that support that file type. Only Chrome supports preload and it also supports WOFF2, but not all browsers that will support preload in the future may support the specific font type. The same is true for any resource type you’re preloading and which browser support isn’t ubiquitous.

Responsive Loading Links

Preload links have a media attribute that we can use to conditionally load resources based on a media query condition.

What’s the use case? Let’s say your site’s large viewport uses an interactive map, but you only show a static map for the smaller viewports.

You want to load only one of those resources. The only way to do that would be to load them dynamically using Javascript. If you use a script to do this you hide those resources from the preloader, and they may be loaded later than necessary, which can impact your users’ visual experience, and negatively impact your SpeedIndex score.

Fortunately, you can use preload to load them ahead of time, and use its media attribute so that only the required script will be preloaded:

<link rel="preload"
      as="image"
      href="map.png"
      media="(max-width: 600px)">

<link rel="preload"
      as="script"
      href="map.js"
      media="(min-width: 601px)">

Resource Hints

In addition to preload and server push, we can also ask the browser to help by providing hints and instructions on how to interact with resources.

For this section, we’ll discuss

  • DNS Prefetching
  • Preconnect
  • Prefetch
  • Prerender

DNS prefetch

This hint tells the browser that we’ll need assets from a domain so it should resolve the DNS for that domain as quickly as possible. If we know we’ll need assets from example.com we can write the following in the head of the document:

<link rel="dns-prefetch" href="//example.com">

Then, when we request a file from it, we’ll no longer have to wait for the DNS lookup. This is particularly useful if we’re using code from third parties or resources from social networks where we might be loading a widget from a <script>.

Preconnect

Preconnect is a more complete version of DNS prefetch. In addition to resolving the DNS it will also do the TCP handshake and, if necessary, the TLS negotiation. It looks like this:

<link rel="preconnect" href="//example.net">

Prefetching

This is an older version of preload and it works the same way. If you know you’ll be using a given resource you can request it ahead of time using the prefetch hint. For example an image or a script, or anything that’s cacheable by the browser:

<link rel="prefetch" href="image.png">

Unlike DNS prefetching, we’re actually requesting and downloading that asset and storing it in the cache. However, this is dependent on a number of conditions, as prefetching can be ignored by the browser. For example, a client might abandon the request of a large font file on a slow network. Firefox will only prefetch resources when “the browser is idle”.

Since we know have the preload API I would recommend using that API (discussed earlier) instead.

Prerender

Prerender is the nuclear option, since it will load all of the assets for a given document like so:

<link rel="prerender" href="http://css-tricks.com">

Steve Souders wrote a great explanation of this technique:

This is like opening the URL in a hidden tab – all the resources are downloaded, the DOM is created, the page is laid out, the CSS is applied, the JavaScript is executed, etc. If the user navigates to the specified href, then the hidden page is swapped into view making it appear to load instantly. Google Search has had this feature for years under the name Instant Pages. Microsoft recently announced they’re going to similarly use prerender in Bing on IE11.

But beware! You should probably be certain that the user will click that link, otherwise the client will download all of the assets necessary to render the page for no reason at all. It is hard to guess what will be loaded but we can make some fairly educated guesses as to what comes next:

  • If the user has done a search with an obvious result, that result page is likely to be loaded next.
  • If the user navigated to a login page, the logged-in page is probably coming next.
  • If the user is reading a multi-page article or paginated set of results, the page after the current page is likely to be next.

Tools to make your content smaller

These are some of the tools that I use to make my content slimmer, send fewer bytes through the wire and make the bytes I send through the wire load faster.

These are not all the tools that you can use to improve your site’s performance. They are the ones I use most frequently. As with many tools, your mileage may vary.

All the tool examples use Gulp. For any other systems, you’re on your own.

UNCSS

UnCSS takes a CSS stylesheet and one or more HTML files and removes all the unused CSS from the stylesheet and writes it back as CSS.

This is especially useful when working with third-party stylesheets and CSS frameworks where you, as the author may not have control over or may have downloaded the full framework for development and now need to slim it down for production

gulp.task('uncss', () => {
  return gulp
    .src('src/css/**/*.css')
    .pipe($.concat('main.css'))
    .pipe(
      $.uncss({
        html: ['index.html']
      })
    )
    .pipe(gulp.dest('css/main.css'))
    .pipe(
      $.size({
        pretty: true,
        title: 'Uncss'
      })
    );
});

Imagemin

Imagemin compresses images in the most popular formats (gif, jpg, png, and svg) to produce images better suited for use in the web.

What I like about Imagemin is that you can configure settings for each image type you’re working with.

Note that this uses the older implicit syntax for gulp-imagemin. Newer versions require you to be explicit on what format you’re configuring. For the newer syntax check the gulp-imagemin README.

gulp.task('imagemin', () => {
  return gulp
    .src('src/images/originals/**')
    .pipe(
      imagemin({
        progressive: true,
        svgoPlugins: [{ removeViewBox: false }, { cleanupIDs: false }],
        use: [mozjpeg()]
      })
    )
    .pipe(gulp.dest('src/images'))
    .pipe(
      $.size({
        pretty: true,
        title: 'imagemin'
      })
    );
});

Critical

Critical above the fold CSS and inline it on your page. This will reduce the load speed of your above the fold content. The example task below will take the above the fold content for all the specified screen sizes and will remove duplicate CSS before inlining it on the head of the page.

gulp.task('critical', () => {
  return gulp
    .src('src/*.html')
    .pipe(
      critical({
        base: 'src/',
        inline: true,
        css: ['src/css/main.css'],
        minify: true,
        extract: false,
        ignore: ['font-face'],
        dimensions: [
          {
            width: 320,
            height: 480
          },
          {
            width: 768,
            height: 1024
          },
          {
            width: 1280,
            height: 960
          }
        ]
      })
    )
    .pipe(
      $.size({
        pretty: true,
        title: 'Critical'
      })
    )
    .pipe(gulp.dest('dist'));
});

Generate Responsive Images

One of the biggest pain points of generating responsive images is that we have to create several versions of each image and then add them to the srcset attribute for each image or figure tag.

gulp-responsive will help with generating the images, not with writing the HTML. This plugin will let you create as many versions of an image as you need. You can target specific images or, like what I did with this example, target all images in a directory.

gulp.task('processImages', () => {
  return gulp
    .src(['src/images/**/*.{jpg,png}', '!src/images/touch/*.png'])
    .pipe(
      $.responsive({
        '*': [
          {
            // image-small.jpg is 200 pixels wide
            width: 200,
            rename: {
              suffix: '-small',
              extname: '.jpg'
            }
          },
          {
            // [email protected] is 400 pixels wide
            width: 200 * 2,
            rename: {
              suffix: '[email protected]',
              extname: '.jpg'
            }
          },
          {
            // image-large.jpg is 480 pixels wide
            width: 480,
            rename: {
              suffix: '-large',
              extname: '.jpg'
            }
          },
          {
            // [email protected] is 960 pixels wide
            width: 480 * 2,
            rename: {
              suffix: '[email protected]',
              extname: '.jpg'
            }
          },
          {
            // image-extralarge.jpg is 1280 pixels wide
            width: 1280,
            rename: {
              suffix: '-extralarge',
              extname: '.jpg'
            }
          },
          {
            // [email protected] is 2560 pixels wide
            width: 1280 * 2,
            rename: {
              suffix: '[email protected]',
              extname: '.jpg'
            }
          },
          {
            // image-small.webp is 200 pixels wide
            width: 200,
            rename: {
              suffix: '-small',
              extname: '.webp'
            }
          },
          {
            // [email protected] is 400 pixels wide
            width: 200 * 2,
            rename: {
              suffix: '[email protected]',
              extname: '.webp'
            }
          },
          {
            // image-large.webp is 480 pixels wide
            width: 480,
            rename: {
              suffix: '-large',
              extname: '.webp'
            }
          },
          {
            // [email protected] is 960 pixels wide
            width: 480 * 2,
            rename: {
              suffix: '[email protected]',
              extname: '.webp'
            }
          },
          {
            // image-extralarge.webp is 1280 pixels wide
            width: 1280,
            rename: {
              suffix: '-extralarge',
              extname: '.webp'
            }
          },
          {
            // [email protected] is 2560 pixels wide
            width: 1280 * 2,
            rename: {
              suffix: '[email protected]',
              extname: '.webp'
            }
          },
          {
            // Global configuration for all images
            // The output quality for JPEG, WebP and TIFF output formats
            quality: 80,
            // Use progressive (interlace) scan for JPEG and PNG output
            progressive: true,
            // Skip enalrgement warnings
            skipOnEnlargement: false,
            // Strip all metadata
            withMetadata: true
          }
        ]
      }).pipe(gulp.dest('dist/images'))
    );
});

Webpack

Webpack (and Rollup and Parcel) are resource bundlers. They will pack together resources to make downloads faster and payloads smaller.

As I documented in Revisiting Webpack I don’t take advantage of the full Webpack toolkit since most of the processing happens in Gulp before the resources get to Webpack for bundling and I don’t feel the need to reinvent the wheel every time I want to work in a project. Before you destroy me on comments or Twitter… this doesn’t mean that I won’t use Webpack to its fullest if the team or the project warrants it, none of my projects have so far.

Instead, I use Webpack inside Gulp to bundle the resources for my application. I use the same configuration file that I would use for a standalone Webpack-based application but most of the heavy lifting has already been done by the time we get here so we can slim the configuration file to only do Javascript bundling.

gulp.task('webpack-bundle', function() {
  return gulp
    .src('src/entry.js')
    .pipe(webpack(require('./webpack.config.js')))
    .pipe(gulp.dest('dist/'));
});

Summary and final considerations

I realize that and the amount of work we need to put in improving performance is big and that we won’t always see the results or that the results will only be small and people won’t notice.

But people do notice and they will leave your site if it doesn’t load fast enough. According to SOASTA’s post Google: 53% of mobile users abandon sites that take longer than 3 seconds to load

53% of visits to mobile sites are abandoned after 3 seconds. (This corresponds to research we did at SOASTA last year, where we found that the sweet spot for mobile load times was 2 seconds

Whether you want to or not, performance does affect a site’s number of visitors and, potentially, the company’s bottom line. So please evaluate your site’s performance and improve it where you can.

Your users and your boss will be thankful. 🙂

Links and Resources

Web Performance Improvement

One thing we as developers tend to forget is that not all web experiences are equal. One of the positive things I’ve found about AMP is that it reduces the fat of your web content by reducing the number of requests, reducing the amount of CSS that you can use and, mostly, by eliminating Javascript.

That’s good but I don’t think we need a whole new platform or library to do that. We need to be smarter about how we pack and serve our content. This is not new, some of the tools and tricks we’ll discuss in this section are old (in web terms).

Experiences outside the big city

The first thing we need to do is decide who our audience is. I don’t mean just figure out where they are coming from (although that’s important) but also figure out what their bandwidth and bandwidth cost is as this may have an effect on how they access the web and whether they will keep your application on their devices or not.

I love the two parallel views of the next billion users of the web, where they come from and what they can and cannot do with their data.

Bruce Lawson presents a more user-centered view of these next billion users and offers insights of emerging markets and things we can do to make their experiences easier.

Tal Oppenheimer presents a more technical view (and Chrome-centered) view of what we can do to improve the experience of your users who are outside the US and other bandwidth rich markets.

Some of the biggest takeaways for me:

  • How much does bandwidth cost for your users?
  • What devices are they using to access your app?
  • What kind of network are your users on?
  • How fast does your application load?

Answering these questions will help not only users in emerging markets but all your users, improve the overall site’s performance, and give users the perception that this is a fast page.

(Real) Performance Test

If you’ve read previous posts in this blog you’ve probably seen Alex’s video on why you should test performance on actual devices and why this is important so I’ll spare you the gory details and leave some important points to consider.

Even if you simulate a network connection, adjust network latency and modify other aspects of the browser’s network connection or use third party tool like Apple’s Network Link Conditioner the connection will not be a real mobile connection.

Alex goes into a lot of details and in much more depth than I could but the main take away is: Test your site in an actual device plugged into your laptop and using chrome://inspect

Server-side tools to improve performance

There are a few ways that we can help the user save data and improve their web experiences from the server without making any changes to the content and letting the server make the changes or fetch resources for you.

Proxy Browsers

Opera Mini is a proxy browser that is very popular in emerging markets where bandwidth is expensive and storage may not be as available as we are used to. It takes your request and forwards it to one of a number of Data Centers where they will make the request, download and process assets before sending the processed request back to your device for rendering.

Chrome in Android and UCBrowser do the same thing using different strategies.

Chrome uses the PageSpeed Module to process the pages it proxies.

UCBrowser uses a similar technology but I’m not fully familiar with the details of how it works.

The fact that these browsers proxy the page you want to access means that highly interactive applications will not work well in a proxy browser and some technologies will not work at all.

But if you pay a large percentage of your monthly salary for internet access, you have to swap memory cards to make sure that all your content is available to you or you expect pages to load faster than they do, then it makes more sense.

Configuring the server to preload resources

One of the cool things about resource hints is that we can implement them either on the server or the client. In this section, we’ll look at implementing them on the server with an example of preloading assets for a single page using Nginx or Apache.

The idea is as follows:

  1. When we hit demo.html we add link preload headers to load 3 resources: a stylesheet and two images
  2. The server will also set a cookie
  3. In subsequent visits, the server will check if the cookie exists and it will only load the link preload headers if it does not exist. If it exists it means the resources were already loaded and don’t need to be downloaded again

This is what the configuration looks like on Nginx. Note that this is the full configuration.

server {
    listen 443 ssl http2 default_server;

    ssl_certificate ssl/certificate.pem;
    ssl_certificate_key ssl/key.pem;

    root /var/www/html;
    http2_push_preload on;

    location = /demo.html {
        add_header Set-Cookie "resloaded=1";
        add_header Link $resources;
    }
}

map $http_cookie $resources {
    "~*resloaded=1" "";
    default "; as=style; rel=preload,
    ; as=image; rel=preload,
    ; as=style; rel=preload";
}

And this is how preloading the resources look like in an Apache HTTPD configuration. Unlike Nginx this is a partial configuration that does the following:

  1. Checks if the HTTP2 module is installed
  2. Checks if a resloaded cookie already exists
  3. Tests if the file requested is demo.html
  4. If it does then add the link preload headers
  5. Adds the resloaded cookie to check for in subsequent visits
<IfModule http2_module>
  SetEnvIf Cookie "resloaded=1" resloaded
  <Files "demo.html">
    Header add Link "</style.css>; as=style; rel=preload,
  </image1.jpg>; as=image; rel=preload,
  </image2.jpg>; as=style; rel=preload" env=!resloaded
    Header add Set-Cookie "resloaded=1; Path=/; Secure; HttpOnly" env=!resloaded
  </Files>
</IfModule>

We can load different resources based on the page we are working with and we can load resources that are needed by all pages and then additional resources that are specific to a page or set of pages.

The one problem when working with preload (and any other resource hint) on the server is that we don’t have an easy way to check if the client has already downloaded the file. That’s why we use a session cookie to handle the check, if it exists we’ll skip preloading the resources.

The Save-Data header

In addition to the tricks we’ve discussed above, we can take advantage of Chrome’s data saver feature and have Chrome send the request to a data compression proxy server that will reduce the size of the requested files by 60% according to Google.

As of Chrome 49 when the user has enabled the Data Saver feature in Chrome, Chrome will add a save-data HTTP Header with the value ‘on’ to each HTTP Request. The HTTP Header will not be present when the feature is turned off. Use this as a signal of intent from the user that they are conscious of the amount of data that they are using and not that their connection is going through the Data Compression Proxy. We can use this header to write conditional code as we’ll see in the following sections.

The first step is to add a header on the server side if the user agent (browser) sent the Save-Data header. This is what it looks like for Apache.

We also tell downstream proxies that data may change based on whether the Save-Data header is present or not.

# If the browser sent the Save-Data header
SetEnvIfNoCase ^Save-Data$ "on" data_saver="on"
# Unset link
Header unset Link env=data_saver
# Tell downstream servers that the response may change
# Based on the Save-Data header
Vary: Save-Data

Now that we’ve done the work on the server side we can look at client side code. These examples use PHP because that’s what I’m most familiar with because of my work on WordPress.

The first code block does the following:

  1. Create a saveData variable and set it to false by default
  2. Check the existence of a save_data header and that its value is on
  3. If both of the conditions in the prior step are true then set the saveData variable to true
// false by default.
$saveData = false;

// Check if the Save-Data header exists
// Check if Save-Data is set to a value of "on".
if  (isset($_SERVER["HTTP_SAVE_DATA"]) &&
    strtolower($_SERVER["HTTP_SAVE_DATA"]) === "on") {
      // Set saveData to true.
      $saveData = true;
}

Now that we know whether the Save_Data was enabled in the client we can use it to conditionally do things for people who are in data saver mode. For example, we may want to skip preloading resources for people in data saver mode.

We create a string with the elements we need to preload a resource, in this case, a stylesheet.

We check if the saveData variable is set to true and, if it is, we append the nopush option to the preload string. The nopush directive will skip preloading the resource.

We then add a Link header with the value of our preload variable as configured (with or without the nopush directive)

// `preload` like usual...
$preload = "; rel=preload; as=style";

if($saveData === true) {
  // ...but don't push anything if `Save-Data` is detected!
  $preload .= "; nopush";
}

header("Link: " . $preload);

Another thing we can do is conditionally use responsive images on our pages. In the example below, we will load a single image if the saveData variable is true and load a set of responsive images otherwise.

if ($saveData === true) {
  // Send a low-resolution version of the image for clients specifying `Save-Data`.
  ?><img src="butterfly-1x.jpg" alt="A butterfly perched on a flower."/>< ?php
}
else {
  // Send the usual assets for everyone else.
  ?><img src="butterfly-1x.jpg" srcset="butterfly-2x.jpg 2x, butterfly-1x.jpg 1x"
  alt="A butterfly perched on a flower."/>< ?php
}

We can also choose whether to load images at all or not. Just like we did with responsive image sets, we can choose to load images when not saving data (the saveData variable is set to false).

<p>This paragraph is essential content.
<p>The image below may be humorous, but it
is not critical to the content.</p>
< ?php
if ($saveData === false) {
  // Only send this image if `Save-Data` has NOT been detected.
  ?><img src="meme.jpg" alt="One does not simply consume data."/>< ?php
}?>

The last thing I wanted to highlight is a way to conditionally work with fonts and CSS in general.

The first part is to add a class to the HTML element based on whether saveData is true or false; this example will only add the class if we are saving data.

<html class="<?php if ($saveData === true): ?>save-data<?php endif; ?>">

The second part is the reverse of what I do when working with Fontface observer. I use web fonts by default and have a second set of selectors where the user is saving data and we don’t want to force them to load web fonts.

p,
li {
  font-family: 'Fira Sans', 'Arial', sans-serif;
}

.save-data p,
.save-data li {
  font-family: 'Arial', sans-serif;
}

If we’ve decided we want to do the work ourselves to make our content more efficient to download we can opt out of the Data compression proxy. The proxy respects the standard Cache-Control: no-transform directive and will not process resources that use that header.

Codec Comparison: 2-pass encoding

We’ve set up 1-pass encoding for all our target codecs and we have the resulting videos to compare. Do a 2-pass encoding changes file size and video quality?

Hardware Specs

  • Model Name: MacBook Pro
  • OS Version: MacOS 10.13 (High Sierra)
  • Processor Name: Intel Core i7
  • Processor Speed: 3.1 GHz
  • Number of Processors: 1
  • Total Number of Cores: 4
  • L2 Cache (per Core): 256 KB
  • L3 Cache: 8 MB
  • Memory: 16 GB

Codecs to test

The idea is to test the following codecs, both old and new, to get a better idea of they answer the questions

  • x264
    • Run through ffmpeg
  • x265
    • Run through ffmpeg
  • vp9
    • Run through ffmpeg
  • aom for av1 support
    • Compiled and installed from source

AVC/H264 is the current generation codec and what most people use to create a video on the web.

HEVC/H265 is the successor to H264 and produces smaller files of equivalent quality.

VP9 is the successor to VP8 and its primary use is in Youtube and some specific settings for HTML5 video (HDR video).

AV1 is an open source, royalty-free video codec developed by the Alliance for Open Media, a large consortium of software and hardware companies. The most attractive characteristics are:

  • It’s royalty free
  • It claims at least 20% better quality than HEVC at the cost of slower encode speeds

Once we’ve decided on the codecs we can start setting objectives to measure. Most of these are subjective and some need time before they can be fully tested since AV1 still doesn’t play in VLC and the versions that will play embedded in Firefox are, as of this writing, tied to specific commit hashes for the reference encoder/decoder implementation (explained in this Mozilla Hacks Article).

The basic questions that I seek to answer with this experiment:

  1. Is there’s any perceptible difference between codecs?
    • Do they look different? Is there a subjective difference?
    • Does a 2-pass encoding improve video quality?
  2. How the storage requirements change between formats
    • Does 2-pass encoding further reduce file size?
  3. Does 2-pass encoding improve encoding speed?

To answer these questions I’ll prepare the second round of videos using 2-pass encodings. In the 2-pass process, each codec will run twice with slightly different settings to create the two pass video.

The encoding scripts

The 2-pass encodings will, as much as possible, use default encoding strategies. The first pass will generate a log that the second pass will use, so don’t delete the log files.

I’ve also created a dummy null file to replace /dev/null. For some reason Bash on MacOS wants to overwrite /dev/null and I don’t think that’s healthy for Unix-based operating systems.

H264

# Encode x264
# Pass 1
ffmpeg -i ${filename} \
-preset slow -crf 22 \
-c:v libx264 -b:v 512k \
-pass 1 \
-c:a aac -b:a 9600 \
-f mp4 null

# Pass 2
ffmpeg -i ${filename} \
-c:v libx264 -b:v 512k \
-preset slow -crf 22 \
-pass 2 \
-c:a aac -b:a 9600 \
${filename}_2pass_h264.mp4

x265

# h265 Pass 1
ffmpeg -y -i ${filename} \
-c:v libx265 -b:v 512k \
-x265-params pass=1 \
-c:a aac -b:a 9600 \
-f mp4 null

# h265 Pass 2
ffmpeg -i ${filename} \
-c:v libx265 -b:v 512k \
-x265-params pass=2 \
-c:a aac -b:a 9600x \
${filename}-2pass-x265.mp4

VP9

# Pass 1
ffmpeg -i ${filename} \
-c:v libvpx-vp9 -b:v 512K \
-pass 1 -c:a libopus \
-f webm null

# Pass 2
ffmpeg -i ${filename} \
-c:v libvpx-vp9 -b:v 512K \
-pass 2 -c:a libopus \
${filename}_2pass_vp9.webm

Analysis and Review

1 and 2 pass encoding comparison
Format 1 pass encode 2 pass encode
Footloose original file 28.5 MB
Footloose x264 in MP4 container 27.7 MB 23.1 MB
Footloose x265 in MP4 container 13 MB 23.2 MB
Footloose VP9 in WebM container 24.1 MB 22 MB
Tears of Steel Original File 738.9 MB
Tears of Steel x264 in MP4 container 343.6 MB 56.1 MB
Tears of Steel x265 in MP4 container 110 MB 56.5 MB
Tears of Steel VP9 in WebM container 58.9 MB 55.6 MB

The differences between 1 and 2 pass encodings are puzzling in some aspects. It looks like 2 pass encoding works better for larger files but not necessarily for smaller files.

Footloose presented an interesting encoding exercise. The two-pass encoding didn’t reduce the file size as much as I expected and, in the case of x265, it increased the file size and not reduce it.

Tears of Steel provided a more consistent result in the 2 pass encoding even for VP9, that presented a very small 1 pass encoding, to begin with. However, all of the 2 pass encodings offer massive side reductions without a perceptible, to me, loss of quality.

The larger file also shows significantly larger file size savings; from 739 down to 55.6 MB for VP9, 56.1 for x264 and 56.5 for x265.

Even though results for other files may vary.

For Footloose, I’d recommend using x265 with a 1-pass encode.

Larger files (dimensions and file size), however, benefit from a 2 pass encoding. In all formats, the savings of a 2 pass encoding are significant enough to be worth the extra time it takes.

The examples I’ve used may or may not match the results. As with many things working with video I’d suggest you test all the formats and see what works best for your needs.

I have not included DASH video in these comparisons. A next step would be to take the x264 file and process it through the Shaka Packager as documented in Revisiting video encoding: DASH.

Or maybe the video will be small enough to play locally or through Youtube or Vimeo. Your mileage may vary 🙂

Codec Testing and Comparison

Warning

Message Date: 04/12/2018

The alliance for Open Media froze bitstream format on March 28, 2018 and announced version 1.0 of the codec on the same day. It is not clear if the bitstream work has actually been finalized or if this was done to present a “finished” product at NAB.

VLC has support for AV1 playback but it doesn’t work with the latest build of the encoder reference implementation. I have yet to play an AV1 file (that I encoded or downloadded from third parties) successfully in a release (3.0.1 on MacOS) or nightly (041218 on MacOS).

As of the writing of this article FFMPEG does not support AV1. I expect this to change now that the bitstream is frozen, however with all the active work going on with the codec, I don’t know when/if that will happen.

For AV1, I run the code from the https://aomedia.googlesource.com/aom/ and update it before every experiment using the instructions on the repository. Because I’m running the code from Git I’m playing with fire as the code may have bugs or other unexpected issues. The flip-side is that I get the most complete implementation of the codec and its tools available.

Note:

The Repository at https://github.com/caraya/video-encoding-tests contains the scripts used on this test along with 2 example videos for you to duplicate the tests on your system.

Hardware Specs

  • Model Name: MacBook Pro
  • OS Version: MacOS 10.13 (High Sierra)
  • Processor Name: Intel Core i7
  • Processor Speed: 3.1 GHz
  • Number of Processors: 1
  • Total Number of Cores: 4
  • L2 Cache (per Core): 256 KB
  • L3 Cache: 8 MB
  • Memory: 16 GB

Codecs to test

The idea is to test the following codecs, both old and new, to get a better idea of they answer the questions

  • x264
    • Run through ffmpeg
  • x265
    • Run through ffmpeg
  • vp9
    • Run through ffmpeg
  • aom for av1 support
    • Compiled and installed from source

AVC/H264 is the current generation codec and what most people use to create a video on the web.

HEVC/H265 is the successor to H264 and produces smaller files of equivalent quality.

VP9 is the successor to VP8 and its primary use is in Youtube and some specific settings for HTML5 video (HDR video).

AV1 is an open source, royalty-free video codec developed by the Alliance for Open Media, a large consortium of software and hardware companies. The most attractive characteristics are:

  • It’s royalty free
  • It claims at least 20% better quality than HEVC at the cost of slower encode speeds

Once we’ve decided on the codecs we can start setting objectives to measure. Most of these are subjective and some need time before they can be fully tested since AV1 still doesn’t play in VLC and the versions that will play embedded in Firefox are, as of this writing, tied to specific commit hashes for the reference encoder/decoder implementation (explained in this Mozilla Hacks Article).

The basic questions that I seek to answer with this experiment:

  1. Is there’s any perceptible difference between codecs?
    • Do they look different? Is there a subjective difference?
  2. How the storage requirements change between formats
  3. How long do they take to encode?
    • VP9 encoder is slow, is the speed acceptable?
    • AV1 is the slowest of all encoders, is the speed acceptable?

Encoders

Rather than use the native encoders which would make the test system specific, I’ve chosen to run as much as possible through FFMPEG; a cross-platform tool to work with video encoding and transcoding.

The only exception is AV1. For this encoder, I’ve downloaded the source code and compiled the reference implementation since the codec is not officially supported by FFMPEG yet.

To run the encodes through FFMPEG in MacOS (High Sierra) we need to install it. I chose to install it via Homebrew with the following flags:

  • –head (install from the HEAD of the Git Repository)
  • –with-tools (enable additional FFmpeg tools)
  • –with-x265 (adds x265 support)
  • –with-libvpx (addslibvpx support)
  • –with-opus (adds support for the Opus audio codec)

The source

To generate an uncompressed YUV420 baseline of the video we’ll use in the other encodings I ran the following command:

filename=$1

ffmpeg -i ${filename} -r 24 -vf format=yuv420p \
${filename}-source.y4m

filename=$1 assigns the first parameter to the variable filename. This is what allows the other parts of the script to use ${filename} instead of hardcoding the name everywhere. With this little trick, we can run multiple tests with different videos.

The first video we’ll encode is X264. This will create an AVC version of the video. I’ve chosen to use the slow preset for both AVC and HEVC encodings. You may want to play with the -preset value to see if faster or slower presets still make a difference.

ffmpeg -i ${filename} \
-c:v libx264 -preset slow -crf 22 \
-c:a copy -b:a 9k \
${filename}-h264.mp4

The next encoding is for X265. This should produce a higher quality video at the same bitrate. I don’t believe the difference is noticeable to the naked eye.

ffmpeg -i ${filename} \
-preset slow \
-c:v libx265 -crf 28 \
-c:a aac -b:a 44.1k \
${filename}-h265.mp4

VP9 is the current version of Google’s VPx line of codecs (acquired when they purchased On2 Corporation). It takes longer to encode but it provides smaller files at the same bitrate.

ffmpeg -i ${filename} \
 -c:v libvpx-vp9 \
 -b:v 512K \
 -c:a libopus -b:a 44.1k \
 ${filename}-vp9.webm

The final test is for AV1. This takes a really long time so it may work better if you leave the encode running over a long period of time (I’ve left mine running overnight).

aomenc  \
${filename} \
--passes=1 --pass=1 \
--fps=24/1 \
--end-usage=cq \
--target-bitrate=512  \
--width=640 --height=360 \
-o ${filename}-av1.webm

Results And Final Notes

I picked a short 5-minute clip to validate the concept and the scripts I’m using to run the comparisons and to create clips short enough that won’t run afoul of Github’s 100MB size limit. I know that using Git LFS would solve the problem but the free tier is limited in space and I don’t want to pay Github for the next tier of storage.

The results for encoding the Footloose clip are somewhat surprising in the Source to h264 conversion. If the files are encoded using the same codec and settings, how do you explain the 800 KB difference?

I also wasn’t expecting the VP video to be larger than h265. I think that’s because the audio bitrate is 48Khz rather than the 44.1Khz used in the other formats. Need to adjust the scripts to run the same audio bitrate throughout.

Encoding results for Footloose clip
Format File Size Notes
h264 (high Profile)/AAC in MP4 Container 28.5 MB Source for conversions
h264 (high Profile)/AAC in MP4 Container 27.7 MB Cannot explain the size difference
HEVC/AAC in MP4 Container 13 MB  
VP9 (profile 0)/Opus in WebM container 24.1 MB Default audio bitrate is higher than the ones chosen for x264 and x265. I’m still puzzled on the difference

In my local environment, I’m testing encodes with Tears of Steel. I’m expecting the results of these tests to better represent what full movie encoding in each format would look like.

Encoding results for Tears of Steel movie
Format File Size Notes
H264 High Profile/AAC in MOV container 738.9 MB Source File for all conversions
H264 High Profile/AAC in MP4 container 343.6 MB  
HEVC Main Profile/AAC in MP4 container 110 MB  
VP9 (profile 0)/Opus in WebM container 58.9 MB  

All our testing has been subjective. What we think looks best and uses the least amount of resources (bandwidth and file size) but it may not be an accurate assessment. There are objective quality measurement tools that may be good to use in addition to the subjective evaluation I’ve done here. Posible subject for a future post. 🙂

Playback is another issue I have yet to tackle for all formats. Unlike Bitmovin, who uses a specific version of the Encoder and Decoder tied to specific Git commits, I have been unable to use VLC or the HTML5 video element to play back AV1 content. It shouldn’t be long before major browsers (all members of the Alliance for Open Media) start implementing AV1 support (and in the case of Firefox to remove the restriction that goes along with Bitmovin’s implementation).

The last item that will drive the decision for me is encoding speed. VP9 and AV1 take significantly longer than x264 and x265 to create content so we’ll have to decide if these slower speeds are worth the file size and bandwidth savings.

As it stands right now AV1 is not usable. The encoding takes too long for it to work effectively in any sort of compression pipeline. VP9 is slower than either x264 or x265 but the file size reduction is worth the slower encoding speeds for video on demand.

I’m evaluating the command and parameters I’m using with AV1 to see if there are ways to optimize it for faster encodings. I’m also looking at 2-pass encoding to see if it improves encoding speed.

Links and Resources

Quick Note: Using details to create no-script accordions

Hat Tip to : CSS Tricks.

An accordion is a UX element to group related items like Frequently asked questions. Most libraries give you a way to create accordions. But there is an easier way to accomplish the same goal, use the details and summary elements.

The details element creates a disclosure widget where information is visible only in its “open” state. A summary or label can be provided using the <summary> element.

The companion summary element specifies a summary, caption, or legend for a <details> element’s disclosure box. Clicking the <summary> element toggles the state of the parent <details> element open and closed.

You also get keyboard navigation for free. Thanks to HTML!

The following HTML…

<h2>Frequently Asked Questions</h2>

<details>
  <summary>What is the population of New Orleans?</summary>
<p>Every year, I took a holiday. I went to Florence, this cafe
on the banks of the Arno. Every fine evening, I would sit there
and order a Fernet Branca. I had this fantasy, that I would look
across the tables and I would see you there with a wife maybe a
couple of kids. You wouldn't say anything to me, nor me to you.
But we would both know that you've made it, that you were happy.
I never wanted you to come back to Gotham. I always knew there
was nothing here for you except pain and tragedy and I wanted
something more for you than that. I still do.</p>
</details>

And this CSS code:

@import url('https://fonts.googleapis.com/css?family=Raleway:400,400i,700|Roboto:700');

details {
  margin: 1rem;
}
summary {
  font-family: 'Roboto', sans-serif;
}

p {
  text-indent: 1em;
  font-family: 'Raleway', sans-serif;
}

h2 {
  font-family: 'Roboto', sans-serif;
  font-size: 2.5em;
  font-weight: bold;
}

Produce the following page (shown as a Codepen Pen):

See the Pen Deatils Accordion by Carlos Araya (@caraya) on CodePen.