Progressive and Subcompact Books: Technical notes
# Progressive and Subcompact Books: Technical notes > **_This is meant as living document. Feedback is appreciated and will be incorporated when appropriate. The idea is to use this and its sister philosophical document as the basis for a proof of concept application_** ## Why a progressive web app? I’ll take the original list of attributes Frances and Alex described for a progressive web application and explain why I consider them important for web publications - Responsive: Let’s face it, reflowable ebooks could look much better. I’m not talking about typography but of essential layout and the inability to use modern web technologies because of the nature of the applications these expriences live inside of - Connectivity independent: It is a really powerful idea to be able to access our content offline. After we access the publication for the first time we no longer need to be online to read and when we’re online the reading experience is still faster because we are reading cached content rather than having to get from the network every timewe want to access it. - App-like-interactions: We can create shells where diverse pieces of content can live. More than one book or more than one issue of a magazine can exist inside our shell and all of it online on the web technologies that we, as developers, are already familiar with - Fresh: The process of updating our content no longer requires active participation. By changing our service worker we can trigger an automatic update process - Safe: We’re still talking about technologies so we need encryption to keep the content and the user interactions safe from eavesdropping - Discoverable: The underlying technologies of PWAs allow search engines to find them - Re-engageable: Once the user has accessed our content we have no way to bring them back to the content. Progressive web applications can leverage the engagement features such as Push Notifications to re-engage the user when there is new content or existing content has been updated - Installable: Each “book” is installable to the home screen through browser-provided prompts. Based on how much and often we interact with our books we can make them first class citizens of our online experience - Linkable: Progressive Web Apps are still web content. We can link to them, we can share them and we can leverage the full force of the web platform I’m leaving DRM out of the conversation deliberately. I don’t believe in restricting access to content people pay for and the licensing model for most ebook vendors leaves a lot to be desired. ## What browsers to target? The easiest answer is [Evergreen Browsers](http://eisenbergeffect.bluespire.com/evergreen-browsers/). But we need to think what that means beyond putting the list out. In my first, knee jerk reaction, the list is this: - Chrome - Opera - Firefox - Edge As Scott Hanselman [points out]([http://www.hanselman.com/blog/TheEvergreenWeb.aspx]) we have a responsibility as developers to keep our sites fresh as much as browser makers have a reponsibility to update their browsers in ways that won’t break “all of the web”. > In a world where we all write our websites with feature detection and (generally) gracefully degrade when features aren't around, things just work. But at the same time, it does make the Web itself a moving target. Can we afford to run a minimum common denominator? According to Brian Kardell’s [support gaps table](https://rawgit.com/bkardell/gaps/master/gaps-caniuse-2-2013.html), out of the 134 features tracked in caniuse.com are supported by at least 3 evergreen browsers without having support in Internet Explorer 9 or earlier. [this table which uses data from webbrowsercompatability.com](https://rawgit.com/bkardell/gaps/master/gaps-caniuse-2-2013.html) shows 460 of the properties/features that test for are available in most evergreen browsers but, again, pretty much none of them before IE9. From a developer’s point of view it’s an easy choice but from an end user’s perspective it’s not so simple. In [The problem with evergreen browsers](https://fragdev.com/blog/the-problem-with-evergreen-browsers) we are reminded again about some of the dawbacks of browser compatibility. The biggest one is still the fact that some corporate IT groups block manual and automatic browser updates so their users cannot take advantage of the new evergreen features until IT decides that the risk is warranted and that their legacy applications will work with the latest version of all browsers. So how do we deal with this support nightmare? Most of the web today works in a [progressive enhancement](http://alistapart.com/article/understandingprogressiveenhancement) paradigm. We start with the basic experience and then enhance the experience for browsers that can handle it. The opposite is [graceful degradation](http://zurb.com/word/graceful-degradation) where we provide the full experience as a starting point and degrade gracefully for browsers that don’t support the full package or which size is an issue. I’m choosing to work through graceful degradation. Starting with a full size experience we’ll explore how to make it work in smaller form factors without loosing sight that the evergreen browser is the primary experience for this particular use case. It is also important to realize that older browsers will still be able to see the content, just not in as rich a way as the target browsers can. ## Building an HTML reading platform There are are couple things I still struggle with when considering how to build this reading application: **Single page app or not** One of the biggest questions I always have when developing an application is wehter to make it a single page app or not. In the case of a book or portions where we know the length of the content we’d be ok using a Single Page Application, but if we’re trying to work with serial content or, in case of a book, we’re trying to provide a serial-like experience we need to load each document in addition to the shell. How to do that without duplicating navigation code in all pages and/or sections. We could build the app with content injection and only partial files but I’ve never been fully sold on that idea. What happens when you have javascript disabled or when you’re on LIE-FI (your device thinks it has connection but it’s so poor that it doesn’t work at all) and your device doesn’t connect but it continues to act asif the connection was working. I will start with a site built on vanilla HTML, CSS and Javascript with additional libraries that will be described where appropriate. This will allow me to concentrate on the application side rather than worry about the content creation. Others may have different opinions but this is a good place to start. ## Offline reading From my perspective the biggest drawback of any web-based application currently available is that the content would only be available online: the only solution for providing and offline experience left a lot to be desired as chronicled by Jake Archibald in [Application Cache is a Douchebag](http://alistapart.com/article/application-cache-is-a-douchebag) and it was written after a successful App Cache deployment. Then Service Workers came into the picture. It’s taken me a while to fully warmup to the idea of Service Workers. Yes, they do overcome the drawbacks from App Cache by making everything explicit… If you want the Service Worker to do something you have to explictly code it The downside to being forced into explicit behavior is that Service Workers requires coding with new technologies… we can’t rely on implicitb behavior anymore and the way these new APIs are written and coded is new and requires a different mindset than current ES5 code. If you’re coming from current ES5 environments the learning curve is a little steep as you have to learn Promises and the service worker API. It’s worth it, I assure you. Ok, so now we have a way to get the content to display whether the user is online or not. What’s next? Next we build the shell. ## Building the shell for the reader. The first thing I want to do is build the shell for the application. This also brings up the first set of questions: - How much of your content do you want to cache on install of the Service Worker so it’ll be ready to go when the user access your app the second time? - What parts of your reader’s structure will you cache? Why? First we’ll build the code for the Service Worker. This is a simple Service Worker that will do the following: - Load and cache the shell for the application - Cache any requests for content of our application before displaying it to the user ```javascript 'use strict'; // Chrome's currently missing some useful cache methods, this polyfill adds them. importScripts('serviceworker-cache-polyfill.js'); // Define constants for cache names and versions const SHELL_CACHE = 'shell-cache-v1'; const CONTENT_CACHE = 'content-cache-v1'; // Content to cache when the ServiceWorker is installed const SHELL_CONTENT = [ '/path/to/javascript.js', '/path/to/stylesheet.css', '/path/to/someimage.png', '/path/to/someotherimage.png', '/offline.html' ]; self.addEventListener('install', function(event) { event.waitUntil( caches.open(SHELL_CACHE).then(function(cache) { return cache.addAll(SHELL_CONTENT); }) ); }); self.addEventListener('activate', function(event) { event.waitUntil( caches.keys().then(function(cacheNames) { return Promise.all( cacheNames.filter(function(cacheName) { return cacheName.startsWith('shell-cache') && cacheName !== SHELL_CACHE; }).map(function(cacheName) { return caches.delete(cacheName); }) ); }) ); }); self.addEventListener('fetch', function(event) { event.respondWith( caches.match(event.request).then(function(response) { if (response) { return response; } return fetch(event.request).then(function(response) { return caches.open(CONTENT_CACHE).then(function(cache) { cache.put( event.request.url, response.clone() ); return response; }); }) }) ); }); ``` With this Service Worker we provide a consistent response regardless of the network (or lack thereof.) Think about it when you add a native application: the load time is very slow the first time but performance remains constant in subsequent visits; we no longer need to rely exclusively on the network for our applications. We will further enhance the service worker as we move along. We will also explore other ways to create Service Workers programmatically as we move through the design and development process. Since we’ll break down the content of our cache between the shell and the rest of our content it makes sense to cache as little as possible on our shell. This means we will cache the following items: - HTML index page (including inline CSS to render the [critical path](https://developers.google.com/web/showcase/case-study/service-workers-iowa)) - Javascript (if any) necessary to load the page (other than the Service Worker itself) - Any images needed for branding the site In the Service Worker demo above, all the assets need for the shell should be included in the `SHELL_CONTENT` constant. This will be picked up by the script and will add the files to the shell cache. For future visits the worker will check the cache first and use the content from there, if available and then, only if necessary, it will go to the network to get the resource we want and store them in the cache to speed things up for the next load. Now that we have the shell we can start playing with the content itself, how we’ll structure it and some additional tricks and extensions. ## Adding to homesscreen While it’s been a recent development for Chrome to change the heuristics regarding how to add web applications to the homescreen it only works in mobile and it only works for certain devices. The following code goes in the head of an HTML document and it provides basic support across platforms: ```markup ``` To activate newer features in Chrome we have to create a `manifest.json` file and explore what it does. We’ll shorten the file by removing some of the icon links ``` { "name": "Book Reader", "short_name": "Book Reader", "icons": [{ "src": "images/touch/icon-72x72.png", "sizes": "72x72", "type": "image/png" }, { "src": "images/touch/icon-96x96.png", "sizes": "96x96", "type": "image/png" }], "background_color": "#3E4EB8", "display": "standalone", "theme_color": "#2E3AA1" ``` We also have icons for multiple resolutions to accommodate for different resolutions. If you’re not interested in supporting all the resolutions (and I personally wouldn’t) you can skip the resolutions you are not working with. Chrome in supported platforms will use this information and create the application icon. The metadata we added to the head of the document will take care of iOS, Windows 8 and older versions of Chrome and Opera for Android. ## Build system I’ve given up on the idea of _just using_ the web rather than _building_ applications for the web. Partly because it’s an exercise in futility and partly because I can now see the advantages of such a system. I’ve been working on a [Gulp](http://gulpjs.com/) based system for a while now and documented the intial steps on [my blog](https://publishing-project.rivendellweb.net/?s=gulp%20workflow). I’ve created a [repository](https://caraya.github.io/gulp-starter/docs/gen/index.html) and [documentation](https://caraya.github.io/gulp-starter/docs/gen/index.html) as a starting point for my work on build systems. Because it is already thoroughly documented I will refer to one task that processes the CSS after the SCSS file is processed. Refer to the [gulp-starter Github repository](https://github.com/caraya/gulp-starter) repository for additional information and the gulpfile and package.json files to incorporate the build process into your own projects. In addition to these tools there are a couple libraries from Google that I want to use in the build process as they make life easier when building Service Worker scripts: [sw-precache](https://developers.google.com/web/showcase/2015/service-workers-iowa#precaching_with_wzxhzdk14sw-precachewzxhzdk15) and [sw-toolbox / shed](https://developers.google.com/web/showcase/2015/service-workers-iowa#wzxhzdk16sw-toolboxwzxhzdk17_for_all_our_dynamic_needs). You use sw-precache with your build system, Gulp in my case, to generate a list of the files to precache. This is much better than doing it manually… you only have one place to update and the build script will take care of the tedious process. One way to precache only some files for your project looks like this: ```javascript // This would most likely be defined elsewhere const rootDir = myApp; const filesToCache = [ rootDir + '/bower_components/**/*.{html,js,css}', rootDir + '/elements/**', rootDir + '/fonts/**', rootDir + '/images/**', rootDir + '/scripts/**', rootDir + '/styles/**/*.css', rootDir + '/manifest.json', rootDir + '/humans.txt', rootDir + '/favicon.ico', rootDir + '/data-worker-scripts.js' ]; // part of the sw-precache library swPrecache(filesToCache, callback); ``` sw-toolbox does the same for Service Workers. It provides 5 methods that prepackage some of the most often used caching strategies. ```javascript toolbox.networkFirst toolbox.cacheFirst toolbox.fastest toolbox.cacheOnly toolbox.networkOnly ``` If you want more information about these strategies, check [sw-toolbox API](https://googlechrome.github.io/sw-toolbox/docs/master/tutorial-api) and Jake Archibald’s [Offline Cookbook](https://jakearchibald.com/2014/offline-cookbook/) ## HTML and grid templates One of the advantages of working with evergreen browsers is that we can play with the latest CSS technologies and that most of the features that required plugins in the past (audio, video, SVG) are now baked into the platform so we need far fewer plugins to accomplish the same visual display. CSS also offers us many more native layout and display options than what we had even a year ago. The two I find the most intriguing are [flexbox](https://publishing-project.rivendellweb.net/new-in-the-css-horizon-css-flexbox/) and [grid layout](http://gridbyexample.com/). Flexbox allows you to create repeating content layouts like photo galleries or navigation menus. I’ve been playing with the spec for a while and have created demos like this [image gallery](http://codepen.io/caraya/full/NGoWKM/) to prove the concept. Galleries like this can be included directly on your HTML and don’t really need preprocessors, just a few additional classes on your markup and CSS definitions on your style sheet. Grids are newer and more intriguing. You can create a grid similar to [Skeleton](http://getskeleton.com/#grid), [Bootstrap](http://v4-alpha.getbootstrap.com/layout/grid/) or [Foundation](http://foundation.zurb.com/sites/docs/grid.html) and can be further refined with media queries. The advantage is that we don’t need an additional library and associated overhead for just the grid. These are two SASS mixins for a prototype grid system I’m currently working on. ```scss @mixin grid-wrapper ($columns: 12, $gutter: 8){ display: grid; margin: 0 auto; width: 100%; max-width: 960px; grid-template-columns: repeat($columns, 1fr); // $columns columns of equal width grid-template-rows: auto; // This should make new rows while respecting our column template grid-row-gap: ($gutter * 1px); grid-column-gap: ($gutter * 1px); } ``` The first mixin will create the grid itself. In its default configuration it will create a 960px wide grid with 12 columns and 8px gutter. ```scss .grid-container { @include grid-wrapper() } ``` The values for the columns and gutters are configurable, if we want a 16 column grid with 16pixel gutters we can do call the mixin like this: ````scss .grid-container { @include grid-wrapper(16, 16); } And SASS/CSS will create 16 equal columns with 16 pixel gutters between them and 16px gutters between rows. The second mixin will place content inside the grid. ```scss @mixin placement ($column-start, $column-end, $row-start, $row-end) { grid-row: $row-start / $row-end; grid-column: $column-start / $column-end; } ```` We do this by specifying row and column start and end for each element. ```scss .figure1 { @include placement(4, 5, 2, 4) } ``` Using the 16 column grid we created above, we’ll place the `figure ` with class `figure1` in the corresponding coordinates: - Starting Colum: 4 - Ending Column: 5 - Starting Row: 2 - Ending Row: 4 **Because the CSS Grid specification is not a recommendation yet it may change from what’s shown here. I’ll continue to test this and update the docs and mixins with appropriate code.** See Rachel Andrew’s [Grid by example](http://gridbyexample.com/) for ideas and exaples of what you can currently do with CSS Grids. As far as HTML is concerned there are several things we need to include in our index.html file before it’ll pass the test of a progressive web application. There are also fallbacks for iOS and older Android devices. Given all these requirements, the HTML for our index file may look like this: ```markup < !doctype html>