Understanding and Using Web Workers

Javascript does a lot of things decently enough but one of its biggest drawbacks is that it’s single-threaded. All scripts that run on a page or application run in the same execution context.

Web Workers provide one way to work around Javascript’s single-threaded execution model. The idea is that we have two scripts.

According to Surma:

Web Workers, also called “Dedicated Workers”, are JavaScript’s take on threads. JavaScript engines have been built with the assumption that there is a single thread, and consequently there is no concurrent access JavaScript object memory, which absolves the need for any synchronization mechanism. If regular threads with their shared memory model got added to JavaScript it would be disastrous, to say the least. […] Instead, we have been given Web Workers, which are basically an entire JavaScript scope running on a separate thread, without any shared memory or shared values.

Executing this script, for example, will prevent all other Javascript from executing until the script finished running and logging all 60000 numbers to the console.

i = 0;
while (i < 60000) {
  console.log("The number is " + i);
  i++;
}

When working with workers we have two scripts, the main script and the worker.

In the main script we load the worker like this:

if (typeof(Worker) !== "undefined") {
    worker = new Worker("worker.js");
}

In the worker script, we can do the heavy loading script without interrupting the main thread execution.

i = 0;
while (i < 200000) {
    postMessage("Web Worker Counter: " + i);
    i++;
}

It’s not all rosy though. Web Workers have quite a few limitations:

  • no access to the DOM: the Window object and the Document object are not available
  • they can communicate back with the main JavaScript program using messaging
  • they need to be loaded from the same origin (domain, port and protocol)
  • they don’t work if you serve the page using the file protocol (file://)

An important note is that the global scope of a Web Worker, instead of Window on the main thread, is a WorkerGlobalScope object.

Communication Between Workers and Main Thread

There are two main ways to communicate with a Web Worker:

In the main.js script, we use the worker’s postMessage method to pass the message we want to send.

There is a second optional parameter that contains an array of Transferable objects to transfer ownership of. If the ownership of an object is transferred, it becomes unusable (neutered) in the context it was sent from and becomes available only to the worker it was sent to.

Transferable objects are instances of classes like ArrayBuffer, MessagePort or ImageBitmap objects that can be transferred. null is not an acceptable value for the transfer.

main.js

const worker = new Worker('worker.js')
worker.postMessage('hello')

The worker script registers success and error messages that will be sent to the main script when the respective event triggers.

worker.js

onmessage = (event) => {
  console.log(event.data)
}

onerror = (event) => {
  console.error(event.message)
}

We can push messages from the worker back to the main script. You can have multiple event listeners answering with different messages to what the main script sends.

worker.js

onmessage = (event) => {
  console.log(event.data)
  postMessage('hey')
}

onerror = (event) => {
  console.error(event.message)
}

The main script posts messages to the worker with the data that we want the worker to use.

main.js

const worker = new Worker('worker.js')
worker.postMessage('hello')

worker.onmessage = (event) => {
  console.log(event.data)
}

Channel API: Another way to communicate with workers

There is another way to have workers and host pages, the Channel Messaging 7API.

The Channel Messaging API is a message bus that allows multiple channels to communicate with each other.

In the example below the main script listens on port1 and posts messages to the worker in port2.

main.js

const worker = new Worker('worker.js')
const messageChannel = new MessageChannel()

messageChannel.port1.addEventListener('message', (event) => {
  console.log(event.data)
})

worker.postMessage(data, [messageChannel.port2])

The worker script listens to messages and logs the results to console.

worker.js

addEventListener('message', (event) => {
  console.log(event.data)
})

A Web Worker can send messages back by posting a message to messageChannel.port2, like this:

addEventListener('message', (event) => {
  event.ports[0].postMessage(data)
})

Loading libraries in a Web Worker

Web Workers load scripts using the importScripts() global function defined in their WorkerGlobalScope object. The first line of your worker script should import the scripts (one or more) you need in your worker

importScripts('../utils/file.js',
  './something.js')

APIs available in Web Workers

Web Workers cannot access the DOM so you cannot interact with the window and document objects.

All is not gloom, you can use many other APIs like:

  • XHR or Fetch API
  • BroadcastChannel API
  • FileReader API
  • IndexedDB
  • Notifications API
  • Promises
  • Service Workers
  • Channel Messaging API
  • Cache API
  • Console API (console.log() and friends)
  • JavaScript Timers (setTimeout, setInterval…)
  • CustomEvents API: addEventListener() and removeEventListener()
  • current URL, which you can access through the location property in read-only mode
  • WebSockets
  • WebGL
  • SVG Animations

So: When Do We Use Web Workers

With all the limitations there are still plenty of things you can do with workers.

The first thing that comes to mind is to take those computationally heavy tasks off the main thread and make your application more responsive.

We are not limited to strings. While not an easy API to use it allows more transfer more complex data structures like arrayBuffers and imageBuffers.

I’m just starting to experiment with Workers and it’s an interesting area of research.

Links And Resources

How To Use Responsive Images

Based on Andreas Bovens’ article Responsive Images: Use Cases and Documented Code Snippets to Get You Started we will look at four use cases for responsive images:

  • Change image sizes based on responsive design rules (the sizes use case)
  • Optimize for high-dpi screens (the dpi use case)
  • Serve images in different image formats to browsers that support them (the mime use case)
  • Serve different art depending on certain contextual factors (the art direction use case)

We’ll look at some example native responsive images and how they apply to these use cases.

Viewport Size Based

The most basic case is to provide different images based on the width of the viewport.

The idea is to not load huge images for devices that don’t need or want them. Since we’re only worried about sizes we can use srcset to indicate the sizes that we want to use.

<img
    srcset="home-300.jpg 300w,
            home-400.jpg 400w,
            home-800.jpg 800w,
            home-1200.jpg 1200w">
    src="opera-400.jpg" alt="My House"

DPR based

We can also use responsive images to handle images that will look good regardless of the DPR factor of the device.

Some modern displays have up to 5x resolution so the standard image can range from 1×1 to o5x5 pixels per logical pixel

<img
  src="museum-outside-1x.jpg"
    alt="The Oslo Opera House"
  srcset= "museum-outside-2x.jpg 2x,
           museum-outside-3x.jpg 3x,
           museum-outside-4x.jpg 4x,
           museum-outside-5x.jpg 5x">

Media Query-like conditions

<img
  sizes="(max-width: 30em) 100vw,
         (max-width: 50em) 50vw,
         calc(33vw - 100px)"
  srcset="swing-200.jpg 200w,
          swing-400.jpg 400w,
          swing-800.jpg 800w,
          swing-1600.jpg 1600w"/>

When you use the srcset and sizes attributes on an <img /> element, you are providing information that the browser can use to make an informed decision about what image is appropriate for the user based on factors you as a developer won’t see or won’t care about.

If we provide browsers with information via srcset and sizes then browsers can make smarter decisions about the appropriate image source.

But none of that is possible when you use the &lt;picture&gt; element and its media attributes:

<picture>
  <source
    srcset="large.jpg"
    media="(min-width: 45em)"></source>
  <source
    srcset="med.jpg"
    media="(min-width: 32em)"></source>
  <img
    src="small.jpg"
    alt="The president giving an award." />
</picture>

When you use the picture element you’re telling the browser that it must use the first element where the media matches or the default element if none do.

The order of sources matters and you have to be sure that you have enough media conditions to cover all your cases and a good default.

Different Images for Portrait or Landscape

There are times when we may want to use different images for portrait and landscape modes.

This picture element will check for the device orientation and use the appropriate source material.

We could further refine this with viewport or DRP elements in the srcset attribute.

<picture>
  <source
    media="(orientation: portrait)"
    srcset="portrait.jpg"></source>
  <source
    media="(orientation: landscape)"
    srcset="med.jpg"
    ></source>
  <img
    src="portrait.jpg"
    alt="The president giving an award." />
</picture>

Different Formats for Different Browsers

Before WebP became widely supported (IE and Safari are the only browsers that don’t support it) we had to make sure that we served the right images for the right browsers.

In this picture element, both sources match the media so the browser will take the first source in document order and use that one if the browser supports the format and the second one if it doesn’t.

Finally, if it doesn’t understand the picture element, the browser will ignore it altogether and use the image element inside the picture.

<picture>
    <source
    media="(min-width: 1280px)"
    sizes="50vw"
    srcset="museum-fullshot-200.webp 200w,
      museum-fullshot-400.webp 400w,
      museum-fullshot-800.webp 800w,
      museum-fullshot-1200.webp 1200w"
        type="image/webp">
    <source
    media="(min-width: 1280px)"
    sizes="50vw"
    srcset="museum-fullshot-200.jpg 200w,
      museum-fullshot-400.jpg 400w,
      museum-fullshot-800.jpg 800w,
      museum-fullshot-1200.jpg 1200w">
    <img
    src="museum-closeup-400.jpg"
    alt="Museum Closeup"
    sizes="(min-width: 640px) 60vw, 100vw"
    srcset="museum-closeup-200.jpg 200w,
      museum-closeup-400.jpg 400w,
      museum-closeup-800.jpg 800w,
      museum-closeup-1200.jpg 1200w">
</picture>

Combining different techniques

This is a more extreme example of combining different techniques. We use multiple sources and multiple formats to make sure we reach as wide a userbase as possible.

<picture>
    <source
    media="(min-width: 1280px)"
    sizes="50vw"
    srcset="museum-fullshot-200.webp 200w,
      museum-fullshot-400.webp 400w,
      museum-fullshot-800.webp 800w,
      museum-fullshot-1200.webp 1200w"
        type="image/webp" />
    <source
    media="(min-width: 1280px)"
    sizes="50vw"
    srcset="museum-fullshot-200.jpg 200w,
      museum-fullshot-400.jpg 400w,
      museum-fullshot-800.jpg 800w,
      museum-fullshot-1200.jpg 1200w" />
    <source
    sizes="(min-width: 640px) 60vw, 100vw"
    srcset="museum-closeup-200.webp 200w,
      museum-closeup-400.webp 400w,
      museum-closeup-800.webp 800w,
      museum-closeup-1200.webp 1200w"
    type="image/webp" />
    <source
    sizes="(min-width: 640px) 60vw, 100vw"
    srcset="museum-closeup-200.jpg 200w,
      museum-closeup-400.jpg 400w,
      museum-closeup-800.jpg 800w,
      museum-closeup-1200.jpg 1200w" />
    <img
    src="museum-closeup-400.jpg"
    alt="Museum Closeup"
    sizes="(min-width: 640px) 60vw, 100vw"
    srcset="museum-closeup-200.jpg 200w,
      museum-closeup-400.jpg 400w,
      museum-closeup-800.jpg 800w,
      museum-closeup-1200.jpg 1200w">
</picture>

We have a lot of options regarding responsive images. This barely scratches the surface of leveraging responsive images but it’s a good start and a good tool to add to the arsenal.

Links and resources

Concepts and examples of responsive images

In working through automating image compression and responsive image generation I wanted to add more areas to the conversation about responsive images without cluttering the post about automation.

How should we get our source images?

The first thing to consider is what should be the ideal format for our source images. The Gulp task with manipulates the size of the images so it pays to have the largest possible image available to process. It’s always better to shrink an image than it is to enlarge it, besides you will compress the images later anyways.

There are ways to use Machine Learning to improve the quality of existing images even if we don’t have access to the originals.

Device Pixels versus Logical Pixels

We need to look at the difference between logical pixels and device pixels since this will affect the way we generate our responsive images and what types of images we want to use for our originals.

Let’s assume that we have an iPhone X with a resolution of 2436 x 1125 device pixels and Device Pixel Ratio (DPR) of 3.

The DPR is defined by the device manufacturer. Simply put, it refers to the number of physical pixels contained in one logical pixel. For example, a device with a DPR of 2 means that one logical pixel contains 4 (2 x 2) physical pixels. Similarly, a DPR of 3 implies that a single logical pixel is equivalent to 9 physical pixels.
from A Guide to Responsive Images on the Web

If we divide the height and width by the DPR We get the logical size of the screen in this case: 812 x 375 and that’s why the following CSS would work:

.example {
  width: 400px;
}

We’re telling the CSS parser to use 400 logical pixels instead of 400 device pixels.

Native Responsive Images

There are two ways to create native responsive images: the picture element and the srcset and sizes attributes. We’ll look at how each of these methods works and then we’ll look at the use cases.

Picture

Using picture and source elements in your page is similar to the video element. You use one or more source elements to indicate the different images that we want to use.

We can leverage the picture element to work with WebP in browsers that can support those formats.

The following code example gives us the following:

  1. If the width of the screen is 40em and the browser supports WebP use this source and the corresponding image based on device’s DPR
  2. If the width of the screen is 40ems and the browser does not support WebP use this source and the corresponding image based on device’s DPR
  3. If neither 1 and 2 are met uses this source and the corresponding image based on device’s DPR
  4. if the browser doesn’t understand the source attribute then use the img element. This must be the last children of a picture element
<picture>
  <!--1-->
  <source media="(min-width: 40em)"
    srcset="big.webp 1x, big-hd.webp 2x">
  <!--2-->
  <source media="(min-width: 40em)"
    srcset="big.jpg 1x, big-hd.jpg 2x">
  <!--3-->
  <source
    srcset="small.jpg 1x, small-hd.jpg 2x">
  <!--4-->
  <img src="fallback.jpg" alt="">
</picture>

srcset and sizes

srcset defines the set of images we want the browser to choose between and the DPR for each image. The browser will select the best image to use based on the device’s characteristics.

The 2x, 3x and 4x values indicate different DPR values. Where there is no value it is assumed to be 1x.

<img
  srcset= 'two.png 2x,
           three.png 3x,
           four.png 4x'
  src="small.jpg" alt="A rad wolf">

We can also add a width attribute instead of the DPR indicator. To assign the width of the image, we add a w to the pixel width number for the image.

The values for w can be almost any length value, e.g. em, rem, pixels, and viewport width, except percentages. The vw value is recommended as an alternative if a relative value is needed.

Only when we use width values we can add the sizes attribute.

The sizes attribute gives a set of media query-like conditions and width of images to use if the condition is met.

A media condition is not exactly a media query. It is part of a media query. It doesn’t allow us to specify media types, e.g. screen or print, but accepts the condition we usually add to a media type.

A valid media condition can be either –

  • A plain media condition, e.g. (min-width: 900px)
  • A “not” media condition, e.g. (not (orientation: landscape))
    An “and” media condition, e.g. (orientation: landscape) and (min-width: 900px)
  • An “or” media condition, e.g. ((orientation: portrait) or (max-width: 500px))

From: Responsive Images – The srcset and sizes Attributes

As I understand it the idea is that the browser will check the media conditions in the sizes attribute and, based on what condition is used, the browser will select the image to used from those available in the srcset.

The full example using srcset and sizes

<img
  srcset= "large.jpg 1024w,
          medium.jpg 640w,
          small.jpg 320w"
  sizes=  "(min-width: 48em) 33.3vw,
          (max-width: 25em) 75vw,
          100vw"
  src="small.jpg" alt="A rad wolf">

Automating image work

Since we are unlikely to compress images and less likely to create the individual images necessary to do a good job with responsive images, we have to leverage technologies to do it for us.

I use Gulp to build sites so I’ve incorporated image processing into my development and production pipeline.

There are two tasks for working with images in Gulp. I will cover them separately and explain the decisions I’ve made about them.

Install and configure the plugins

All these plugins are installed with Node. Like we do in all Node projects we first initialize an empty package.json. The --yes flag will accept all default parameters.

npm init --yes

We then install the packages that we’ll use in the tasks.

npm i -D gulp \
gulp-imagemin \
imagemin-mozjpeg \
imagemin-webp \
gulp-responsive

Once we have initialized our package.json and installed all the packages that we want to use, it’s time to start work in the Gulp configuration

The rest of the code in this post lives in a gulpfile.js file.

At the very top, we import Gulp and the plugins we want to use using the CommonJS require syntax.

// Imagemin and Plugins
const gulp = require('gulp');
const imagemin = require('gulp-imagemin');
const mozjpeg = require('imagemin-mozjpeg');
const webp = require('imagemin-webp');
const responsive = require('gulp-responsive');

Process Images

The first function is to compress images using Imagemin and the gulp-imagemin plugin.

This will handle SVG, GIF, PNG, JPG and WebP images.

function processImages() {
  return gulp.src('src/images/originals/**')
  .pipe(imagemin([
    imagemin.gifsicle({interlaced: true}),
    imagemin.optipng({optimizationLevel: 5}),
      imagemin.svgo({
          plugins: [
              {removeViewBox: true},
              {cleanupIDs: false}
      ]}),
    use: [
      mozjpeg(),
      webp({quality: 85})],
  ]))
  .pipe(gulp.dest('src/images'))
}

Although not strictly necessary, I chose to be explicit as to what plugins I’m using and what parameters each of these plugins uses.

Gifsicle, Optipng, and SVGO are bundled with Imagemin so we can just use them as-is. Mozjpeg and WebP are additional libraries that add functionality in addition to or instead of the default bundled libraries.

Generate Responsive Images

The next step is to generate a set of responsive images.

The idea is that for each JPG and PNG image we will generate a set of responsive images. I’ve reduced the code example below to a single size with 2 different resolutions.

Device Pixel versus Logical Pixel

Before we jump into the code we need to look at the difference between logical pixels and device pixels since this will affect the way we generate our responsive images and what types of images we want to use for our originals.

Let’s assume that we have an iPhone X with a resolution of 2436 x 1125 device pixels and Device Pixel Ratio (DPR) of 3.

The DPR is defined by the device manufacturer. Simply put, it refers to the number of physical pixels contained in one logical pixel. For example, a device with a DPR of 2 means that one logical pixel contains 4 (2 x 2) physical pixels. Similarly, a DPR of 3 implies that a single logical pixel is equivalent to 9 physical pixels.
from A Guide to Responsive Images on the Web

If we divide the height and width by the DPR We get the logical size of the screen in this case: 812 x 375 and that’s why the following CSS would work:

.example {
  width: 400px;
}

We’re telling the CSS parser to use 400 logical pixels instead of 400 device pixels.

The code

The code follows standard Gulp practices.

We first select the src for the task. In this case, we’ve chosen all the JPG and PNG files under src/images/hires.

We pipe the input through gulp-responsive to generate multiple versions of each image.

I’ve chosen to demonstrate one image into DPR resolutions and three formats, JPEG, PNG, and WebP.

The @2x images are using double the number of pixels to accommodate higher DPR values. When/if we wanted to add a 3x image we can add it like we did the 2x image in the examples below.

function generateResponsive() {
return gulp.src([
    'src/images/hires/**/*.{jpg,png}',
  ])
  .pipe(gulp.responsive({
    '*': [{
    // image-small.jpg is 200 pixels wide
      width: 200,
      rename: {
        suffix: '-small',
        extname: '.jpg',
      },
    }, {
    // [email protected] is 400 pixels wide
      width: 200 * 2,
      rename: {
        suffix: '[email protected]',
        extname: '.jpg',
      },
    }, {
    // image-small.png is 200 pixels wide
      width: 200,
      rename: {
        suffix: '-small',
        extname: '.png',
      },
    }, {
    // [email protected] is 400 pixels wide
      width: 200 * 2,
      rename: {
        suffix: '[email protected]',
        extname: '.png',
      },
    }, {
    // image-small.webp is 200 pixels wide
      width: 200,
      rename: {
        suffix: '-small',
        extname: '.webp',
      },
    }, {
    // [email protected] is 400 pixels wide
      width: 200 * 2,
      rename: {
        suffix: '[email protected]',
        extname: '.webp',
      },
    }, {
    // Global configuration for all images
    // The output quality for JPEG, WebP
    // and TIFF output formats
      quality: 80,
      // Use progressive (interlace) scan
      // for JPEG and PNG output
      progressive: true,
      // Skip enalrgement warnings
      skipOnEnlargement: true,
      // Strip all metadata
      withMetadata: true,
    }],
  })
      .pipe(gulp.dest('dist/images')));
}

We might not need all the images but it’s better to have them and no need them than need them and not have them.

Export the functions

The final step is to export the functions soo that Gulp can see them. It’s no different than exporting functions in ES2015 modules.

exports.imagemin = processImages;
exports.responsive = generateResponsive;

This will enable us to run them as gulp imagemin and gulp-responsive.

Loading scripts the right way for everyone

Differential loading is the technique where you load different content for different browsers that support different sets of Javascript features and APIs.

<script type="module" src="/js/modern.mjs"></script>
<script nomodule defer src="/js/legacy.js"></script>

This works awesome with modern browsers that understand type="module" and that will happily ignore nomodule.

The problem is that we can’t make that assumption safely. Some browsers will download the nomodule script twice and others that will download both scripts, even when they will only execute one of them.

Jeremy Wagner’s article A Less Risky Differential Serving Pattern proposes the following hack to make sure that all browsers will load a single version of the code for the page depending on whether they use modules or not.

<script>
  // Create a new script element 
  //to slot into the DOM.
  var scriptEl = document.createElement("script");

  // Check whether the script element
  // supports the `nomodule` attribute.
  if ("noModule" in scriptEl) {
    scriptEl.src = "/js/modern.mjs";
    scriptEl.type = "module";
  } else {
    scriptEl.src = "/js/legacy.js";
    scriptEl.defer = true;
  }

  document.body.appendChild(scriptEl);
</script>

In a separate article in the 2018 Performance Calendar entry Doing Differential Serving in 2019 he goes more in-depth on how to prepare the bundles that will differentially serve.