Reliable
index
layout: collection
override:tags: []
date: 2018-11-05
pathName: reliable
Adaptive loading with service workers
https://www.youtube.com/watch?v=fhqCwDP69PI
Users access websites through a wide variety of devices and network connections. Even in major cities, where mobile networks are fast and reliable, one can end up experiencing slower load times, for example, when commuting in the subway, in a car, or just when moving around.
In regions like emerging markets, this phenomenon is even more common, not only due to unreliable networks, but also because devices tend to have less memory and CPU processing power.
Adaptive loading is a web performance pattern that lets you adapt your site based on the user's network and device conditions.
The adaptive loading pattern is made possible by service workers, the Network Information API, the Hardware Concurrency API, and the Device Memory API. In this guide we explore how you can use service workers and the Network Information API to achieve an adaptive loading strategy.
Production case
Terra is one of the biggest media companies in Brazil. It has a large user base, coming from a wide variety of devices and networks.
To provide a more reliable experience to all their users, Terra combines service workers and the Network Information API to deliver lower quality images to users on 2G or 3G connections.
A screenshot of Terra's home page connected to different image qualities according to the connection type.
The company also found that the scripts and assets (like banners) loaded by ad networks were especially detrimental to users navigating in 3G or slower connections.
As is the case with many publishers, Terra serves AMP versions of their pages to users coming from search engines and other link sharing platforms. AMP pages are usually lightweight and help mitigate the impact of ads in performance by deprioritizing their load with respect to the main content of the page.
Taking that into consideration, Terra decided to start serving AMP versions of their pages not only to users coming from search engines, but also to those navigating their site in 3G connections or slower.
To achieve that, they use the Network Information API in the service worker to detect if the request comes from 3G or slower. If that's the case, they change the URL of the page to request the AMP version of the page instead.
A screenshot of Terra's article page connected to different image qualities according to the connection type.
Thanks to this technique, they send 70% less bytes to users on slower connections. The time spent in AMP pages is higher for 3G users and ads in AMP pages have a better CTR (click-through-rate) for that group.
Implement adaptive loading with Workbox
In this section we'll explore how Workbox can be used to implement adaptive loading strategies.
Workbox provides several runtime caching strategies out of the box. They are used to indicate how the service worker generates a response after receiving a
fetch event.
For example, in a Cache First strategy the
Request will be fulfilled using the cached response (if available). If there isn't a cached response, the
Request will be fulfilled by a network request and the response will be cached.
import {registerRoute} from 'workbox-routing';
import {CacheFirst} from 'workbox-strategies';
registerRoute(
new RegExp('/img/'),
new CacheFirst()
);
Caching strategies can be customized with Workbox plugins. These allow you to add additional behaviors by manipulating requests and responses during the lifecycle of a request. Workbox has several built-in plugins for common cases and APIs, but you can also define a custom plugin, and introduce some custom logic of your choice.
To achieve adapting loading, define a custom plugin, called, for example,
adaptiveLoadingPlugin:
const adaptiveLoadingPlugin = {
requestWillFetch: async ({request}) => {
const urlParts = request.url.split('/');
let imageQuality;
switch (
navigator && navigator.connection
? navigator.connection.effectiveType
: ''
) {
//...
case '3g':
imageQuality = 'q_30';
break;
//...
}
const newUrl = urlParts
.splice(urlParts.length - 1, 0, imageQuality)
.join('/')
.replace('.jpg', '.png');
const newRequest = new Request(newUrl.href, {headers: request.headers});
return newRequest;
},
};
The previous code does the following:
Implements a
requestWillFetch() callback: This is called whenever a network request is about to be made, so you can alter the
Request.
Checks the connection type, by using the Network Information API. Based on the status of the network, it creates a new URL part, indicating the quality of the image to fetch (e.g.
q_30 for 3G users).
Creates a new URL based on the dynamic
newPart value, and returns the new
Request to be made, based on that URL.
Next, pass the plugin to a
cacheFirst strategy containing a regular expression to match image URLs (e.g.
/img/):
workbox.routing.registerRoute(
new RegExp('/img/'),
workbox.strategies.cacheFirst({
cacheName: 'images',
plugins: [
adaptiveLoadingPlugin,
workbox.expiration.Plugin({
maxEntries: 50,
purgeOnQuotaError: true,
}),
],
}),
);
As a result, when requests for images are intercepted, the runtime caching strategy will try to fulfill the request from the cache. If it's not available, it will run the logic in the plugin, to decide which image quality to fetch from the network.
Finally the response will be persisted in the cache, and sent back to the page.
Cloudinary Workbox Plugin
Cloudinary, a video and image hosting service, has a Workbox Plugin that encapsulates the functionality explained in the previous section, making it even easier to implement.
Cloudinary and Workbox logos.
The plugin is designed to work with the Workbox webpack plugin. To implement it, use the
GenerateSW() class:
new workboxPlugin.GenerateSW({
swDest: 'sw.js',
importScripts: ['./cloudinaryPlugin.js'],
runtimeCaching: [
{
urlPattern: new RegExp('^https://res.cloudinary.com/.*/image/upload/'),
handler: 'CacheFirst',
options: {
cacheName: 'cloudinary-images',
plugins: [
{
requestWillFetch: async ({request}) =>
cloudinaryPlugin.requestWillFetch(request),
},
],
},
},
],
});
The previous code does the following:
Uses the
GenerateSW() class to configure webpack to generate a service worker in the destination indicated in
swDest.
Imports the cloudinary plugin script.
Defines a Cache First runtime caching strategy for requests for images to the Cloudinary CDN.
Passes the Cloudinary Workbox Plugin to adjust the image quality according to the network conditions.
Explore more adaptive loading strategies
You can go beyond this, by mapping device signals, like hardware concurrency and device memory to device categories and then serving different assets depending on the device type (low-, mid- or high-end).
App shell UX with service workers and streams
https://www.youtube.com/watch?v=fhqCwDP69PI
Single-page app (SPA) is an architectural pattern in which the browser runs JavaScript code to update the existing page when the user visits a different section of the site, as opposed to loading an entire new page.
This means that the web app doesn't perform an actual page reload. The History API is used instead to navigate back and forth through the user's history and manipulate the contents of the history stack.
Using this type of architecture can provide an app shell UX that's fast, reliable, and usually consumes less data when navigating.
In multi-page apps (MPAs) each time a user navigates to a new URL, the browser progressively renders HTML specific to that page. This means a full page refresh every time you visit a new page.
While both are equally valid models to use, you might want to bring some of the benefits of the app shell UX of SPAs to your existing MPA site.
In this article we'll analyze how you can achieve an SPA-like architecture in multi-page apps by combining partials, service workers, and streams.
Production case
DEV is a community where software developers write articles, take part in discussions, and build their professional profiles.
A screenshot of https://dev.to
Their architecture is a multi-page app based on traditional backend templating through Ruby on Rails. Their team was interested in some of the benefits of an app shell model, but didn't want to undertake a major architectural change or move away from their original tech stack.
Here's how their solution works:
First, they create partials of their home page for the header and the footer (
shell_top.html and
shell_bottom.html) and deliver them as standalone HTML snippets with an endpoint. These assets are added to the cache at the service worker
install event (what's commonly referred to as precaching).
When a navigation request is intercepted by the service worker, they create a streamed response by combining the cached header and footer with the main page content that just came from the server. The body is the only actual part of the page that requires fetching data from the network.
Dev's architecture consisting on static headers and footers that are cached and a body requested from the network.
The key element of this solution is the usage of streams, which enables incremental creations and updates of data sources. The Streams API also provides an interface for reading or writing asynchronous chunks of data, only a subset of which might be available in memory at any given time.
This way, the header of the page can be rendered as soon as it's picked from the cache, while the rest of the content is being fetched from the network. As a result, the navigation experience is so fast that users don't perceive an actual page refresh, only the new content (the body) being updated.
The resulting UX is similar to the app shell UX pattern of SPAs, implemented on a MPA site.
The previous section contains a quick summary of DEV's solution. For a more detailed explanation check out their blog post on this topic.
Implement an app shell UX architecture in MPAs with Workbox
In this section we'll cover a summary of the different parts involved in implementing an app shell UX architecture in MPAs.
For a more detailed post on how to implement this on a real site, check out Beyond SPAs: alternative architectures for your PWA.
The server
Partials
The first step is to adopt a site structure based on HTML partials. These are just modular pieces of your pages that can be reused across your site and also delivered as standalone HTML snippets.
The head partial can contain all the logic needed to style and render the header of the page. The navbar partial can contain the logic for the navigation bar, the footer partial the code that needs to execute there, and so forth.
The first time the user visits the site, your server generates a response by assembling the different parts of the page:
app.get(routes.get('index'), async (req, res) => {
res.write(headPartial + navbarPartial);
const tag = req.query.tag || DEFAULT_TAG;
const data = await requestData(…);
res.write(templates.index(tag, data.items));
res.write(footPartial);
res.end();
});
By using the
res (response) object's write() method, and referencing locally stored partial templates, the response can be streamed immediately, without getting blocked by any external data source. The browser takes this initial HTML and renders a meaningful interface and loading message right away.
The next portion of the page uses API data, which involves a network request. The web app can't render anything else until it gets a response back and processes it, but at least users aren't staring at a blank screen while they wait.
The service worker
The first time a user visits a site, the header of the page will be rendered faster, without having to wait for the body of the page. The browser still needs to go to the network to fetch the rest of the page.
After the first page load, the service worker is registered, allowing you to fetch the partials for the different static parts of the page (header, navbar, footer, etc.) from the cache.
Precaching static assets
The first step is to cache the partial HTML templates, so they are immediately available.
With Workbox precaching you can store these files at the
install event of the service worker and keep them up to date when changes are deployed to the web app.
Depending on the build process, Workbox has different solutions to generate a service worker and indicate the list of files to precache, including webpack and gulp plugins, a generic node module and a command line interface.
For a partials configuration like the one described earlier, the resulting service worker file should contain something similar to the following:
workbox.precaching.precacheAndRoute([
{
url: 'partials/about.html',
revision: '518747aad9d7e',
},
{
url: 'partials/foot.html',
revision: '69bf746a9ecc6',
},
// etc.
]);
Streaming
Next, add the service worker logic so that the precached partial HTML can be sent back to the web app immediately. This is a crucial part of being reliably fast. Using the Streams API within our service worker makes that possible.
Workbox Streams abstracts the details of how streaming works. The package lets you pass to the library a mix of streaming sources, both from caches and runtime data that might come from the network. Workbox takes care of coordinating the individual sources and stitching them together into a single, streaming response.
First, set up the strategies in Workbox to handle the different sources that will make up the streaming response.
const cacheStrategy = workbox.strategies.cacheFirst({
cacheName: workbox.core.cacheNames.precache,
});
const apiStrategy = workbox.strategies.staleWhileRevalidate({
cacheName: API_CACHE_NAME,
plugins: [
new workbox.expiration.Plugin({maxEntries: 50}),
],
});
The first strategy reads data that's been precached, like the partial HTML templates.
The second strategy implements the stale-while-revalidate caching logic, along with least-recently-used cache expiration logic once we reach 50 entries.
Next, tell Workbox how to use the strategies to construct a complete, streaming response, by passing in an array of sources as functions to execute immediately:
workbox.streams.strategy([
() => cacheStrategy.handle({
request: new Request(getCacheKeyForURL("/head.html")),
}),
() => cacheStrategy.handle({
request: new Request(getCacheKeyForURL("/navbar.html")),
}),
async ({event, url}) => {
const tag = url.searchParams.get('tag') || DEFAULT_TAG;
const listResponse = await apiStrategy.handle(…);
const data = await listResponse.json();
return templates.index(tag, data.items);
},
() => cacheStrategy.handle({
request: new Request(getCacheKeyForURL("/foot.html")),
}),
]);
The first two sources are precached partial templates read directly from the service worker's cache, so they'll always be available immediately.
The next source function fetches data from the network, and processes the response into the HTML that the web app expects.
Finally, a cached copy of the footer and closing HTML tags is streamed to complete the response.
Workbox takes the result from each source and streams it to the web app, in sequence, only delaying if the next function in the array hasn't completed yet.
As a result, the user immediately sees the page being painted. The experience is so fast that when navigating the header stays in its position without making the user perceive the full page refresh. This is very similar to the UX that the app shell SPA model provides.
Broadcast updates to pages with service workers
In some scenarios the service worker might need to proactively communicate with any of the active
tabs it controls to inform of a certain event. Examples include:
Informing the page when a new version of the service worker has been installed, so that the page
can show an "Update to refresh" button to the user to access the new functionality
immediately.
Letting the user know about a change on cached data that took place on the service worker side, by
showing an indication, like: "The app is now ready to work offline", or "New version of the
content available".
Diagram showing a service worker communicating with the page to send an update.
We'll call these types of use cases where the service worker doesn't need to receive a message from
the page to start a communication "broadcast updates". In this guide we'll review different
ways of implementing this type of communication between pages and service workers, by using standard
browser APIs and the Workbox library.
Check out Workers overview for a high-level explanation
of when to use web workers versus service workers and the rest of the
Communicate with workers series
for guides on other common use cases.
Production cases {: #production-cases }
Tinder {: #tinder }
Tinder PWA uses
workbox-window to listen to
important service worker lifecycle moments from the page ("installed", "controlled" and
"activated"). That way when a new service worker comes into play, it shows an "Update Available"
banner, so that they can refresh the PWA and access the latest features:
A screenshot of Tinder's webapp 'Update Available' functionality.
In the Tinder PWA, the service worker tells the page that a new version is ready, and the page shows users a "Update Available" banner.
Squoosh {: #squoosh }
In the Squoosh PWA, when the service worker has cached all of the necessary
assets to make it work offline, it sends a message to the page to show a "Ready to work offline"
toast, letting the user know about the feature:
A screenshot of Squoosh webapp 'Ready to work offline' functionality.
In the Squoosh PWA the service worker broadcasts an update to the page when cache is ready, and the page displays "Ready to work offline" toast.
Using Workbox {: #using-workbox }
Listen to service worker lifecycle events {: #listen-to-service-worker-lifecycle-events }
workbox-window provides a straightforward interface to listen to important service worker lifecycle
events.
Under the hood, the library uses client-side APIs like
updatefound
and statechange
and provides higher level event listeners in the
workbox-window object, making it easier for the
user to consume these events.
The following page code lets you detect every time a new version of the service worker is installed,
so you can communicate it to the user:
const wb = new Workbox('/sw.js');
wb.addEventListener('installed', (event) => {
if (event.isUpdate) {
// Show "Update App" banner
}
});
wb.register();
Inform the page of changes in cache data {: #inform-the-page-of-changes-in-cache-data }
The Workbox package
workbox-broadcast-update
provides a standard way of notifying window clients that a cached response has been updated. This is
most commonly used along with the StaleWhileRevalidate
strategy.
To broadcast updates add a
broadcastUpdate.BroadcastUpdatePlugin to your strategy options in the
service worker side:
import {registerRoute} from 'workbox-routing';
import {StaleWhileRevalidate} from 'workbox-strategies';
import {BroadcastUpdatePlugin} from 'workbox-broadcast-update';
registerRoute(
({url}) => url.pathname.startsWith('/api/'),
new StaleWhileRevalidate({
plugins: [
new BroadcastUpdatePlugin(),
],
})
);
In your web app, you can listen for these events like so:
navigator.serviceWorker.addEventListener('message', async (event) => {
// Optional: ensure the message came from workbox-broadcast-update
if (event.data.meta === 'workbox-broadcast-update') {
const {cacheName, updatedUrl} = event.data.payload;
// Do something with cacheName and updatedUrl.
// For example, get the cached content and update
// the content on the page.
const cache = await caches.open(cacheName);
const updatedResponse = await cache.match(updatedUrl);
const updatedText = await updatedResponse.text();
}
});
Using browser APIs {: #using-browser-apis }
If the functionality that Workbox provides is not enough for your needs, use the following browser
APIs to implement "broadcast updates":
Broadcast Channel API {: #broadcast-channel-api }
The service worker creates a BroadcastChannel
object and starts sending
messages to it. Any context (e.g. page) interested in receiving these messages can instantiate a
BroadcastChannel object and implement a message handler to receive messages.
To inform the page when a new service worker is installed, use the following code:
// Create Broadcast Channel to send messages to the page
const broadcast = new BroadcastChannel('sw-update-channel');
self.addEventListener('install', function (event) {
// Inform the page every time a new service worker is installed
broadcast.postMessage({type: 'CRITICAL_SW_UPDATE'});
});
The page listens to these events by subscribing to the
sw-update-channel:
// Create Broadcast Channel and listen to messages sent to it
const broadcast = new BroadcastChannel('sw-update-channel');
broadcast.onmessage = (event) => {
if (event.data && event.data.type === 'CRITICAL_SW_UPDATE') {
// Show "update to refresh" banner to the user.
}
};
This is a simple technique, but its limitation is browser support: at the moment of this writing,
Safari doesn't support this API.
Client API {: #client-api }
The Client API provides a straightforward
way of communicating with multiple clients from the service worker by iterating over an array of
Client objects.
Use the following service worker code to send a message to the last focused tab:
// Obtain an array of Window client objects
self.clients.matchAll(options).then(function (clients) {
if (clients && clients.length) {
// Respond to last focused tab
clients[0].postMessage({type: 'MSG_ID'});
}
});
The page implements a message handler to intercept these messages:
// Listen to messages
navigator.serviceWorker.onmessage = (event) => {
if (event.data && event.data.type === 'MSG_ID') {
// Process response
}
};
Client API is a great option for cases like broadcasting information to multiple active tabs. The
API is supported by all major browsers, but not all of its methods are. Check browser support before
using it.
Message Channel {: #message-channel }
Message Channel requires
an initial configuration step, by passing a port from the page to the service worker, to establish a
communication channel between them. The page instantiates a
MessageChannel object and passes a
port to the service worker, via the
postMessage() interface:
const messageChannel = new MessageChannel();
// Init port
navigator.serviceWorker.controller.postMessage({type: 'PORT_INITIALIZATION'}, [
messageChannel.port2,
]);
The page listens to messages by implementing an "onmessage" handler on that port:
// Listen to messages
messageChannel.port1.onmessage = (event) => {
// Process message
};
The service worker receives the port and saves a reference to it:
// Initialize
let communicationPort;
self.addEventListener('message', (event) => {
if (event.data && event.data.type === 'PORT_INITIALIZATION') {
communicationPort = event.ports[0];
}
});
From that point it can send messages to the page, by calling
postMessage() in the reference to the
port:
// Communicate
communicationPort.postMessage({type: 'MSG_ID' });
MessageChannel might be more complex to implement, due to the need of initializing ports, but it's
supported by all major browsers.
Next steps {: #next-steps }
In this guide we explored one particular case of Window to service worker communication:
"broadcast updates". The examples explored include listening to important service worker
lifecycle events, and communicating to the page about changes in content or cached data. You can think
of more interesting use cases where the service worker proactively communicates with the page,
without receiving any message previously.
For more patterns of Window and service worker communication check out:
Imperative caching guide: Calling a service worker from the page to
cache resources in advance (e.g. in prefetching scenarios).
Two-way communication: Delegating a task to a service worker (e.g.
a heavy download), and keeping the page informed on the progress.
Additional resources {: #additional-resources }
workbox-window
workbox-broadcast-update
Workbox 4: Implementing refresh-to-update-version flow using the workbox-window
module
Building resilient search experiences with Workbox
This codelab shows you how to implement a resilient search experience with Workbox. The demo app it uses contains a search box that calls a server endpoint, and redirects the user to a basic HTML page.
This codelab uses Chrome DevTools. Download Chrome if you don't already have it.
Measure
Before adding optimizations, it's always a good idea to first analyze the current state of the application.
In the new tab that just opened, check how the website behaves when going offline:
Open Chrome DevTools and select the Network panel.
In the Throttling drop-down list, select Offline.
In the demo app enter a search query, then click the Search button.
The standard browser error page is shown:
A screenshot of the default offline UX in the browser.
Provide a fallback response
The service worker contains the code to add the offline page to the precache list, so it can always be cached at the service worker
install event.
Usually you would need to instruct Workbox to add this file to the precache list at build time, by integrating the library with your build tool of choice (e.g. webpack or gulp).
For simplicity, we've already done it for you. The following code at
public/sw.js does that:
const FALLBACK_HTML_URL = '/index_offline.html';
…
workbox.precaching.precacheAndRoute([FALLBACK_HTML_URL]);
To learn more about how to integrate Workbox with build tools, check out the webpack Workbox plugin and the Gulp Workbox plugin.
Next, add code to use the offline page as a fallback response:
Add the following code to the bottom of
public/sw.js:
workbox.routing.setDefaultHandler(new workbox.strategies.NetworkOnly());
workbox.routing.setCatchHandler(({event}) => {
switch (event.request.destination) {
case 'document':
return caches.match(FALLBACK_HTML_URL);
break;
default:
return Response.error();
}
});
The Glitch UI says
workbox is not defined because it doesn't realize that the
importScripts() call on line 1 is importing the library.
The code does the following:
Defines a default Network Only strategy that will apply to all requests.
Declares a global error handler, by calling
workbox.routing.setCatchHandler() to manage failed requests. When requests are for documents, a fallback offline HTML page will be returned.
To test this functionality:
Go back to the other tab that is running your app.
Set the Throttling drop-down list back to Online.
Press Chrome's Back button to navigate back to the search page.
Make sure that the Disable cache checkbox in DevTools is disabled.
Long-press Chrome's Reload button and select
Empty cache and hard reload
to ensure that your service worker is updated.
Set the Throttling drop-down list back to Offline again.
Enter a search query, and click the Search button again.
The fallback HTML page is shown:
A screenshot of the custom offline UX in the browser.
Request notification permission
For simplicity, the offline page at
views/index_offline.html already contains the code to request notification permissions in a script block at the bottom:
function requestNotificationPermission(event) {
event.preventDefault();
Notification.requestPermission().then(function (result) {
showOfflineText(result);
});
}
The code does the following:
When the user clicks subscribe to notifications the
requestNotificationPermission() function is called, which calls
Notification.requestPermission(), to show the default browser permission prompt. The promise resolves with the permission picked by the user, which can be either
granted,
denied, or
default.
Passes the resolved permission to
showOfflineText() to show the appropriate text to the user.
Persist offline queries and retry when back online
Next, implement Workbox Background Sync to persist offline queries, so they can be retried when the browser detects that connectivity has returned.
Open
public/sw.js for edit.
Add the following code at the end of the file:
const bgSyncPlugin = new workbox.backgroundSync.Plugin('offlineQueryQueue', {
maxRetentionTime: 60,
onSync: async ({queue}) => {
let entry;
while ((entry = await queue.shiftRequest())) {
try {
const response = await fetch(entry.request);
const cache = await caches.open('offline-search-responses');
const offlineUrl = `${entry.request.url}¬ification=true`;
cache.put(offlineUrl, response);
showNotification(offlineUrl);
} catch (error) {
await this.unshiftRequest(entry);
throw error;
}
}
},
});
The code does the following:
workbox.backgroundSync.Plugin contains the logic to add failed requests to a queue so they can be retried later. These requests will be persisted in IndexedDB.
maxRetentionTime indicates the amount of time a request may be retried. In this case we have chosen 60 minutes (after which it will be discarded).
onSync is the most important part of this code. This callback will be called when connection is back so that queued requests are retrieved and then fetched from the network.
The network response is added to the
offline-search-responses cache, appending the
¬ification=true query param, so that this cache entry can be picked up when a user clicks on the notification.
To integrate background sync with your service, define a NetworkOnly strategy for requests to the search URL (
/search_action) and pass the previously defined
bgSyncPlugin. Add the following code to the bottom of
public/sw.js:
const matchSearchUrl = ({url}) => {
const notificationParam = url.searchParams.get('notification');
return url.pathname === '/search_action' && !(notificationParam === 'true');
};
workbox.routing.registerRoute(
matchSearchUrl,
new workbox.strategies.NetworkOnly({
plugins: [bgSyncPlugin],
}),
);
This tells Workbox to always go to the network, and, when requests fail, use the background sync logic.
Next, add the following code to the bottom of
public/sw.js to define a caching strategy for requests coming from notifications. Use a CacheFirst strategy, so they can be served from the cache.
const matchNotificationUrl = ({url}) => {
const notificationParam = url.searchParams.get('notification');
return (url.pathname === '/search_action' && (notificationParam === 'true'));
};
workbox.routing.registerRoute(matchNotificationUrl,
new workbox.strategies.CacheFirst({
cacheName: 'offline-search-responses',
})
);
Finally, add the code to show notifications:
function showNotification(notificationUrl) {
if (Notification.permission) {
self.registration.showNotification('Your search is ready!', {
body: 'Click to see you search result',
icon: '/img/workbox.jpg',
data: {
url: notificationUrl
}
});
}
}
self.addEventListener('notificationclick', function(event) {
event.notification.close();
event.waitUntil(
clients.openWindow(event.notification.data.url)
);
});
Test the feature
Go back to the other tab that is running your app.
Set the Throttling drop-down list back to Online.
Press Chrome's Back button to navigate back to the search page.
Long-press Chrome's Reload button and select
Empty cache and hard reload
to ensure that your service worker is updated.
Set the Throttling drop-down list back to Offline again.
Enter a search query, and click the Search button again.
Click subscribe to notifications.
When Chrome asks you if you want to grant the app permission to send notifications,
click Allow.
Enter another search query and click the Search button again.
Set the Throttling drop-down list back to Online again.
Once the connection is back a notification will be shown:
A screenshot of the full offline flow.
Conclusion
Workbox provides many built-in features to make your PWAs more resilient and engaging.
In this codelab you have explored how to implement the Background Sync API by way of the Workbox abstraction, to ensure that offline user queries are not lost, and can be retried once connection is back.
The demo is a simple search app, but you can use a similar implementation for more complex scenarios and use cases, including chat apps, posting messages on a social network, etc.
Explore DevTools Network panel
This codelab walks you through the process of interpreting all of the network
traffic for a somewhat complex sample application. At the end of the exercise,
you'll have the skills you need to figure out what your own web application is
loading and when it's making each request.
The screenshots and instructions in this codelab assume that you're using
Chrome. Each browser has its own DevTools experience, which might not match what
you see in this codelab.
Navigate to the Network Panel
Navigate to the Network panel to see the network traffic for the demo
application.
Reload the page to see the network traffic.
The Network panel shows all the assets loaded because of your initial
navigation:
Chrome DevTools' network panel.
The actual columns you see in the Network panel may be different; the
screenshot shows a simplified view with everything but the Name, Type, and
Waterfall columns hidden.
How to interpret the entries
Each row of recorded network traffic represents a single request and response
pair.
The first row, with type
document, is the initial navigation request for the
web app's HTML. This is the source for the waterfall; each of the subsequent
requests for additional assets (known as subresources of the main document) flow
from this original source.
The second and third rows, showing a CSS
stylesheet and a
script subresource
being loaded, are dependent requests that were initiated by the main document.
Looking at when those requests are made, the waterfall diagram shows that
they're not started until very late in the process of responding to the
navigation request.
Taken together, the requests for the HTML document, CSS, and
JavaScript are needed to display the full page during the
initial navigation.
Create some additional runtime requests
With the Network panel still open and recording, it's time to simulate something
common for a lot of web apps: additional API requests used to add more data to
the page after the initial navigation is complete.
Trigger these additional requests by clicking Find Me in the app and then
Allow in the popup that appears.
This will allow the site to access your current location:
The allow location permission prompt.
You could also deny geolocation permission, in which case the web app
will fall back to a default location.
Once the web app has a location to work with, clicking Find Nearby
Wikipedia Entries results in several additional network requests. You
should see something like this:
image
Interpret the new entries
As before, each row of recorded network traffic represents a single request
and response pair.
The first row of the new entries represents a request with a type of
fetch,
which corresponds to the
way the web app requests data
from the Wikipedia API.
The following rows all are images (
png or
jpeg) associated with the
Wikipedia entries. Although it's a little hard to see from the screenshot, their
entries in the Waterfall column directly flow from the API response.
For all of these additional requests, the when is going to vary based on how
long you've had the page open before you click on Find Nearby Wikipedia
Entries. Most important here is that the when is disconnected from the
initial navigation request. You can tell this from the large gap that exists in
the Waterfall display, representing a period of time that passed in between the
initial loading and when the Wikipedia API request is made.
Requests made after a gap of time following a navigation fall into the category
of "runtime requests," as opposed to the initial set of requests used to display
the page when you first navigated to it.
Summing things up
Having gone through the steps in this codelab, you're now familiar with the
tools you can use to analyze what any web application loads.
The Network panel helps you answer the question of what's being loaded, via
the URLs in the Name column and the data in the Type column, along with when
it's being loaded, via the waterfall display.
You've also seen that requests made by a web page can (usually) be grouped into
one of two categories:
Initial requests, made right after navigating to a new page, for the
HTML, JavaScript, CSS (and potentially other assets).
Runtime requests made as a result of user interaction with the page. This
can often start with a request to an API, and then flow into several
follow-up requests based on the API data retrieved.
Configuring HTTP caching behavior
This codelab shows you how to change the HTTP caching headers returned by a
Node.js-based web server, running the Express serving
framework. It will also show how to confirm that the caching behavior you expect
is actually being applied, using the Network panel in Chrome's DevTools.
While the specific instructions are tailored towards Express, the general
principles about choosing the correct caching headers apply to any web server
environment.
Get familiar with the sample project
These are the key files you will be working with in the sample project:
server.js contains the Node.js code that serves the web app's
content. It uses Express to handle HTTP requests
and responses. In particular,
express.static() is used to serve all of
the local files in the public directory, so the
serve-static
documentation
will come in handy.
public/index.html is the web app's HTML. Like most HTML files, it does not
contain any versioning information as part of its URL.
public/app.15261a07.js and
public/style.391484cf.css are the web app's JavaScript
and CSS assets. These files each contain a hash in their URLs,
corresponding to their contents. The
index.html is responsible for keeping
track of which specific versioned URL to load.
In the "real world", the process of assigning hashes and updating HTML
files to include references to the latest versioned URL would be handled by a
build tool, like
webpack.
For the purposes of this codelab, assume that the hashes were generated as part
of a build process that already took place.
Configure caching headers for our HTML
When responding to requests for URLs that don't contain versioning info, make
sure you add
Cache-Control: no-cache to your response messages. Along with
that, setting one of two additional response headers is recommended: either
Last-Modified
or
ETag. The
index.html falls into this category. You can break this down into two steps.
First, the
Last-Modified and
ETag headers are controlled by the
etag
and
lastModified
configuration options. Both of these options actually default to
true for all
HTTP responses, so in this current setup, you don't have to opt-in to get that
behavior. But you can be explicit in your configuration anyway.
Second, you need to be able to add in the
Cache-Control: no-cache header, but
only for your HTML documents (
index.html, in this case). The easiest way to
conditionally set this header is to write a custom
setHeaders function,
and within that, check to see if the incoming request is for an HTML document.
The static serving configuration in
server.js starts out as this:
app.use(express.static('public'));
Make the changes described above, and you should end up with something that
looks like:
app.use(express.static('public', {
etag: true, // Just being explicit about the default.
lastModified: true, // Just being explicit about the default.
setHeaders: (res, path) => {
if (path.endsWith('.html')) {
// All of the project's HTML files end in .html
res.setHeader('Cache-Control', 'no-cache');
}
},
}));
Configure caching headers for the versioned URLs
When responding to requests for URLs that contain
"fingerprint" or
versioning information, and whose contents are never meant to change, add
Cache-Control: max-age=31536000 to your responses. The
app.15261a07.js and
style.391484cf.css fall into this category.
Building off the
setHeaders function
used in the last step, you can add in additional logic to check whether a given
request is for a versioned URL, and if so, add the
Cache-Control: max-age=31536000 header.
The most robust way of doing this is to use a
regular expression
to see whether the asset being requested matches a specific pattern that you
know the hashes fall into. In the case of this sample project, it's always eight
characters from the set of digits 0–9 and the lowercase letters a–f (i.e.
hexadecimal characters). The hash
is always separated by a
. character on either side.
A regular expression that
matches those general rules
can be expressed as
new RegExp('\\.[0-9a-f]{8}\\.').
It helps to be as specific as possible when coming up with these rules,
to protect against future problems. A more general match, such as checking for
the
.js or
.css file extension, could end up being a problem down the road
if you end up adding in additional, unversioned JavaScript or CSS assets to your
project.
Modify the
setHeaders function so it looks like this:
app.use(express.static('public', {
etag: true, // Just being explicit about the default.
lastModified: true, // Just being explicit about the default.
setHeaders: (res, path) => {
const hashRegExp = new RegExp('\\.[0-9a-f]{8}\\.');
if (path.endsWith('.html')) {
// All of the project's HTML files end in .html
res.setHeader('Cache-Control', 'no-cache');
} else if (hashRegExp.test(path)) {
// If the RegExp matched, then we have a versioned URL.
res.setHeader('Cache-Control', 'max-age=31536000');
}
},
}));
Confirm the new behavior using DevTools
You can get familiar with the Network panel in Chrome's DevTools by
working through this codelab.
With the modifications to the static file server in place, you can check to make
sure that the right headers are being set by previewing the live app with the DevTools Network panel open.
Customize the columns that are
displayed in the Network panel to include the information that is most relevant, by right-clicking in
the column header:
Configuring DevTools' Network panel.
Here, the columns to pay attention to are
Name,
Status,
Cache-Control,
ETag, and
Last-Modified.
With the DevTools open to the Network panel, refresh the page.
After the page has loaded, you should see entries in the Network panel that look
like the following:
Network panel columns.
The first row is for the HTML document that you navigated to. This is properly
served with
Cache-Control: no-cache. The HTTP response status for that request
is a
304. This
means that the browser knew not to use the cached HTML immediately, but instead
made an HTTP request to the web server, using the
Last-Modified and
ETag
information to see if there was any update to the HTML that it already had in
its cache. The HTTP 304 response indicates that there is not updated HTML.
Cache-Control: no-cache doesn't mean "never used the cached copy". It
means "always check with the server first, and use the cached copy if there's a
HTTP 304 response."
The next two rows are for the versioned JavaScript and CSS assets. You should
see them served with
Cache-Control: max-age=31536000, and the HTTP status for
each is
200.
Because of the configuration used, there is no actual request being made to the
Node.js server, and clicking on the entry will show you additional detail,
including that the response came "(from disk cache)".
A network response status of 200.
The actual values for the ETag and Last-Modified columns don't matter much. The
important thing is to confirm that they're being set.
Summing things up
Having gone through the steps in this codelab, you're now familiar with how to
configure the HTTP response headers in a Node.js-based web server using Express,
for optimal use of the HTTP cache. You also have the steps you need to confirm
that the expected caching behavior is being used, via the Network panel in
Chrome's DevTools.
Working with service workers
This codelab shows you how to register a service worker from within your web
application, and use the Chrome DevTools to observe its behavior. It also
covers some debugging techniques that you might find useful when dealing with
service workers.
Get familiar with the sample project
The files in the sample project most relevant to this codelab are:
register-sw.js starts out empty, but it will contain the code used
to register the service worker. It's already being loaded via a
It registers a service worker, and adds image of a dog after 3 seconds.
Here's its service worker,
sw.js:
self.addEventListener('install', event => {
console.log('V1 installing…');
// cache a cat SVG
event.waitUntil(
caches.open('static-v1').then(cache => cache.add('/cat.svg'))
);
});
self.addEventListener('activate', event => {
console.log('V1 now ready to handle fetches!');
});
self.addEventListener('fetch', event => {
const url = new URL(event.request.url);
// serve the cat SVG from the cache if the request is
// same-origin and the path is '/dog.svg'
if (url.origin == location.origin && url.pathname == '/dog.svg') {
event.respondWith(caches.match('/cat.svg'));
}
});
It caches an image of a cat, and serves it whenever there's a request for
/dog.svg. However, if you run the above example, you'll see a dog the first time you load the page. Hit refresh, and you'll see the cat.
Note: Cats are better than dogs. They just are.
Scope and control
The default scope of a service worker registration is
./ relative to the script URL. This means if you register a service worker at
//example.com/foo/bar.js it has a default scope of
//example.com/foo/.
We call pages, workers, and shared workers
clients. Your service worker can only control clients that are in-scope. Once a client is "controlled", its fetches go through the in-scope service worker. You can detect if a client is controlled via
navigator.serviceWorker.controller which will be null or a service worker instance.
Download, parse, and execute
Your very first service worker downloads when you call
.register(). If your script fails to download, parse, or throws an error in its initial execution, the register promise rejects, and the service worker is discarded.
Chrome's DevTools shows the error in the console, and in the service worker section of the application tab:
Error displayed in service worker DevTools tab
Install
The first event a service worker gets is
install. It's triggered as soon as the worker executes, and it's only called once per service worker. If you alter your service worker script the browser considers it a different service worker, and it'll get its own
install event. I'll cover updates in detail later.
The
install event is your chance to cache everything you need before being able to control clients. The promise you pass to
event.waitUntil() lets the browser know when your install completes, and if it was successful.
If your promise rejects, this signals the install failed, and the browser throws the service worker away. It'll never control clients. This means we can't rely on
cat.svg being present in the cache in our
fetch events. It's a dependency.
Activate
Once your service worker is ready to control clients and handle functional events like
push and
sync, you'll get an
activate event. But that doesn't mean the page that called
.register() will be controlled.
The first time you load the demo, even though
dog.svg is requested long after the service worker activates, it doesn't handle the request, and you still see the image of the dog. The default is consistency, if your page loads without a service worker, neither will its subresources. If you load the demo a second time (in other words, refresh the page), it'll be controlled. Both the page and the image will go through
fetch events, and you'll see a cat instead.
clients.claim
You can take control of uncontrolled clients by calling
clients.claim() within your service worker once it's activated.
Here's a variation of the demo above which calls
clients.claim() in its
activate event. You should see a cat the first time. I say "should", because this is timing sensitive. You'll only see a cat if the service worker activates and
clients.claim() takes effect before the image tries to load.
If you use your service worker to load pages differently than they'd load via the network,
clients.claim() can be troublesome, as your service worker ends up controlling some clients that loaded without it.
Note: I see a lot of people including
clients.claim() as boilerplate, but I
rarely do so myself. It only really matters on the very first load, and due to
progressive enhancement the page is usually working happily without service
worker anyway.
Updating the service worker {: #updates}
In brief:
An update is triggered if any of the following happens:
A navigation to an in-scope page.
A functional events such as
push and
sync, unless there's been an update check within the previous 24 hours.
Calling
.register() only if the service worker URL has changed. However, you should avoid changing the worker URL.
Most browsers, including Chrome 68 and later, default to ignoring caching headers when checking for updates of the registered service worker script. They still respect caching headers when fetching resources loaded inside a service worker via
importScripts(). You can override this default behavior by setting the
updateViaCache option when registering your service worker.
Your service worker is considered updated if it's byte-different to the one the browser already has. (We're extending this to include imported scripts/modules too.)
The updated service worker is launched alongside the existing one, and gets its own
install event.
If your new worker has a non-ok status code (for example, 404), fails to parse, throws an error during execution, or rejects during install, the new worker is thrown away, but the current one remains active.
Once successfully installed, the updated worker will
wait until the existing worker is controlling zero clients. (Note that clients overlap during a refresh.)
self.skipWaiting() prevents the waiting, meaning the service worker activates as soon as it's finished installing.
Let's say we changed our service worker script to respond with a picture of a horse rather than a cat:
const expectedCaches = ['static-v2'];
self.addEventListener('install', event => {
console.log('V2 installing…');
// cache a horse SVG into a new cache, static-v2
event.waitUntil(
caches.open('static-v2').then(cache => cache.add('/horse.svg'))
);
});
self.addEventListener('activate', event => {
// delete any caches that aren't in expectedCaches
// which will get rid of static-v1
event.waitUntil(
caches.keys().then(keys => Promise.all(
keys.map(key => {
if (!expectedCaches.includes(key)) {
return caches.delete(key);
}
})
)).then(() => {
console.log('V2 now ready to handle fetches!');
})
);
});
self.addEventListener('fetch', event => {
const url = new URL(event.request.url);
// serve the horse SVG from the cache if the request is
// same-origin and the path is '/dog.svg'
if (url.origin == location.origin && url.pathname == '/dog.svg') {
event.respondWith(caches.match('/horse.svg'));
}
});
Note: I have no strong opinions on horses.
Check out a demo of the above. You should still see an image of a cat. Here's why…
Install
Note that I've changed the cache name from
static-v1 to
static-v2. This means I can set up the new cache without overwriting things in the current one, which the old service worker is still using.
This patterns creates version-specific caches, akin to assets a native app would bundle with its executable. You may also have caches that aren't version specific, such as
avatars.
Waiting
After it's successfully installed, the updated service worker delays activating until the existing service worker is no longer controlling clients. This state is called "waiting", and it's how the browser ensures that only one version of your service worker is running at a time.
If you ran the updated demo, you should still see a picture of a cat, because the V2 worker hasn't yet activated. You can see the new service worker waiting in the "Application" tab of DevTools:
DevTools showing new service worker waiting
Even if you only have one tab open to the demo, refreshing the page isn't enough to let the new version take over. This is due to how browser navigations work. When you navigate, the current page doesn't go away until the response headers have been received, and even then the current page may stay if the response has a
Content-Disposition header. Because of this overlap, the current service worker is always controlling a client during a refresh.
To get the update, close or navigate away from all tabs using the current service worker. Then, when you navigate to the demo again, you should see the horse.
This pattern is similar to how Chrome updates. Updates to Chrome download in the background, but don't apply until Chrome restarts. In the mean time, you can continue to use the current version without disruption. However, this is a pain during development, but DevTools has ways to make it easier, which I'll cover later in this article.
Activate
This fires once the old service worker is gone, and your new service worker is able to control clients. This is the ideal time to do stuff that you couldn't do while the old worker was still in use, such as migrating databases and clearing caches.
In the demo above, I maintain a list of caches that I expect to be there, and in the
activate event I get rid of any others, which removes the old
static-v1 cache.
You may not be updating from the previous version. It may be a service worker many versions old.
If you pass a promise to
event.waitUntil() it'll buffer functional events (
fetch,
push,
sync etc.) until the promise resolves. So when your
fetch event fires, the activation is fully complete.
The cache storage API is "origin storage" (like localStorage, and IndexedDB). If you run many sites on the same origin (for example,
yourname.github.io/myapp), be careful that you don't delete caches for your other sites. To avoid this, give your cache names a prefix unique to the current site, eg
myapp-static-v1, and don't touch caches unless they begin with
myapp-.
Skip the waiting phase
The waiting phase means you're only running one version of your site at once, but if you don't need that feature, you can make your new service worker activate sooner by calling
self.skipWaiting().
This causes your service worker to kick out the current active worker and activate itself as soon as it enters the waiting phase (or immediately if it's already in the waiting phase). It doesn't cause your worker to skip installing, just waiting.
It doesn't really matter when you call
skipWaiting(), as long as it's during or before waiting. It's pretty common to call it in the
install event:
self.addEventListener('install', event => {
self.skipWaiting();
event.waitUntil(
// caching etc
);
});
But you may want to call it as a results of a
postMessage() to the service worker. As in, you want to
skipWaiting() following a user interaction.
Here's a demo that uses
skipWaiting(). You should see a picture of a cow without having to navigate away. Like
clients.claim() it's a race, so you'll only see the cow if the new service worker fetches, installs and activates before the page tries to load the image.
skipWaiting() means that your new service worker is likely controlling pages that were loaded with an older version. This means some of your page's fetches will have been handled by your old service worker, but your new service worker will be handling subsequent fetches. If this might break things, don't use
skipWaiting().
Manual updates
As I mentioned earlier, the browser checks for updates automatically after navigations and functional events, but you can also trigger them manually:
navigator.serviceWorker.register('/sw.js').then(reg => {
// sometime later…
reg.update();
});
If you expect the user to be using your site for a long time without reloading, you may want to call
update() on an interval (such as hourly).
Avoid changing the URL of your service worker script {: #avoid-url-change}
If you've read my post on caching best practices, you may consider giving each version of your service worker a unique URL. Don't do this! This is usually bad practice for service workers, just update the script at its current location.
It can land you with a problem like this:
index.html registers
sw-v1.js as a service worker.
sw-v1.js caches and serves
index.html so it works offline-first.
You update
index.html so it registers your new and shiny
sw-v2.js.
If you do the above, the user never gets
sw-v2.js, because
sw-v1.js is serving the old version of
index.html from its cache. You've put yourself in a position where you need to update your service worker in order to update your service worker. Ew.
However, for the demo above, I have changed the URL of the service worker. This is so, for the sake of the demo, you can switch between the versions. It isn't something I'd do in production.
Making development easy {: #devtools}
The service worker lifecycle is built with the user in mind, but during development it's a bit of a pain. Thankfully there are a few tools to help out:
Update on reload
This one's my favourite.
DevTools showing 'update on reload'
This changes the lifecycle to be developer-friendly. Each navigation will:
Refetch the service worker.
Install it as a new version even if it's byte-identical, meaning your
install event runs and your caches update.
Skip the waiting phase so the new service worker activates.
Navigate the page.
This means you'll get your updates on each navigation (including refresh) without having to reload twice or close the tab.
Skip waiting
DevTools showing 'skip waiting'
If you have a worker waiting, you can hit "skip waiting" in DevTools to immediately promote it to "active".
Shift-reload
If you force-reload the page (shift-reload) it bypasses the service worker entirely. It'll be uncontrolled. This feature is in the spec, so it works in other service-worker-supporting browsers.
Handling updates
The service worker was designed as part of the extensible web. The idea is that we, as browser developers, acknowledge that we are not better at web development than web developers. And as such, we shouldn't provide narrow high-level APIs that solve a particular problem using patterns we like, and instead give you access to the guts of the browser and let you do it how you want, in a way that works best for your users.
So, to enable as many patterns as we can, the whole update cycle is observable:
navigator.serviceWorker.register('/sw.js').then(reg => {
reg.installing; // the installing worker, or undefined
reg.waiting; // the waiting worker, or undefined
reg.active; // the active worker, or undefined
reg.addEventListener('updatefound', () => {
// A wild service worker has appeared in reg.installing!
const newWorker = reg.installing;
newWorker.state;
// "installing" - the install event has fired, but not yet complete
// "installed" - install complete
// "activating" - the activate event has fired, but not yet complete
// "activated" - fully active
// "redundant" - discarded. Either failed install, or it's been
// replaced by a newer version
newWorker.addEventListener('statechange', () => {
// newWorker.state has changed
});
});
});
navigator.serviceWorker.addEventListener('controllerchange', () => {
// This fires when the service worker controlling this page
// changes, eg a new worker has skipped waiting and become
// the new active worker.
});
The lifecycle goes ever on
As you can see, it pays to understand the service worker lifecycle—and with that understanding, service worker behaviors should seem more logical, and less mysterious. That knowledge will give you more confidence as you deploy and update service workers.
Service workers and the Cache Storage API
You're locked in a struggle for network reliability. The browser's HTTP cache is
your first line of defense, but as you've learned, it's only really effective
when loading versioned URLs that you've previously visited. On its own, the HTTP
cache is not enough.
Fortunately, two newer tools are available to help turn the tide in your favor:
service workers
and the
Cache Storage API.
Since they're so often used together, it's worth learning about them both at the
same time.
Service workers
A service worker is built into the browser and controlled by a bit of extra
JavaScript code that you are responsible for creating. You deploy this alongside
the other files that make up your actual web application.
A service worker has some special powers. Among other duties, it patiently waits
for your web app to make an outgoing request, and then springs into action by
intercepting it. What the service worker does with this intercepted request is
up to you.
For some requests, the best course of action might be just to allow the request
to continue on to the network, just like what would happen if there were no
service worker at all.
For other requests, though, you can take advantage of something more flexible
than the browser's HTTP cache, and return a reliably fast response without
having to worry about the network. That entails using the other piece of the
puzzle: the Cache Storage API.
The Cache Storage API
The Cache Storage API opens up a whole new range of possibilities, by giving
developers complete control over the contents of the cache. Instead of relying
on a combination of HTTP headers and the browser's built-in heuristics,
the Cache
Storage API exposes a code-driven approach to caching. The Cache Storage API
is particularly useful when called from your service worker's JavaScript.
Wait… there's another cache to think about now?
You're probably asking yourself questions like "Do I still need to configure my
HTTP headers?" and "What can I do with this new cache that wasn't possible with
the HTTP cache?" Don't worry—those are natural reactions.
It's still recommended that you configure the
Cache-Control headers on your web
server, even when you know that you're using the Cache Storage API. You can
usually get away with setting
Cache-Control: no-cache for unversioned URLs,
and/or
Cache-Control: max-age=31536000 for URLs that contain versioning
information, like hashes.
When populating the Cache Storage API cache, the browser
defaults to checking for existing entries
in the HTTP cache, and uses those if found. If you're adding versioned URLs to
the Cache Storage API cache, the browser avoids an additional network request. The
flip side of this is that if you're using misconfigured
Cache-Control headers,
like specifying a long-lived cache lifetime for an unversioned URL, you can end
up
making things worse
by adding that stale content to the Cache Storage API. Getting your HTTP cache
behavior sorted is a prerequisite for effectively using the Cache Storage API.
As for what's now possible with this new API, the answer is: a lot. Some common
uses that would be difficult or impossible with just the HTTP cache include:
Use a "refresh in the background" approach to cached content, known as
stale-while-revalidate.
Impose a cap on the maximum number of assets to cache, and implement a
custom expiration policy to remove items once that limit is reached.
Compare previously cached and fresh network responses to see if
anything's changed, and enable the user to update content (with a button,
for example) when data has actually been updated.
Check out The Cache API: A quick guide to learn more.
API nuts and bolts
There are some things to keep in mind about the API's design:
Request
objects are used as the unique keys when reading and writing to these
caches. As a convenience, you can pass in a URL string like
'https://example.com/index.html' as the key instead of an actual
Request object, and the API will automatically handle that for you.
Response
objects are used as the values in these caches.
The
Cache-Control header on a given
Response is effectively ignored
when caching data. There are no automatic, built-in expiration or freshness
checks, and once you store an item in the cache, it will persist until your
code explicitly removes it. (There are libraries to simplify cache
maintenance on your behalf. They'll be covered later on in this series.)
Unlike with older, synchronous APIs such as
LocalStorage,
all Cache Storage API operations are asynchronous.
A quick detour: promises and async/await
Service workers and the Cache Storage API use
asynchronous programming concepts.
In particular, they rely heavily on promises to represent the future result of
async operations. You should familiarize yourself with
promises,
and the related
async/
await
syntax, before diving in.
Make an application reliable by registering a service worker.
Don't deploy that code… yet
While they provide an important foundation, and can be used as-is, both service
workers and the Cache Storage API are effectively lower-level building blocks,
with a number of edge cases and "gotchas". There are some higher-level tools
that can help smooth the difficult bits of those APIs, and provide all you need
to build a production-ready service worker. The next guide covers one such tool:
Workbox.
Learn while having fun. Check out the new Service Workies game!.
Two-way communication with service workers
In some cases, a web app might need to establish a two-way communication channel between the
page and the service worker.
For example: in a podcast PWA one could build a feature to let the user download episodes for
offline consumption and allow the
service worker to keep the page regularly informed about the progress, so the main
thread can update the UI.
In this guide we'll explore the different ways of implementing a two-way communication between
the Window and service
worker context, by exploring
different APIs, the Workbox library, as well as
some advanced cases.
Diagram showing a service worker and the page exchanging messages.
Check out Workers overview for a high-level explanation
of when to use web workers versus service workers and the rest of the
Communicate with workers series
for guides on other common use cases.
Using Workbox {: #using-workbox }
workbox-window is a set of
modules of the Workbox library that are intended
to run in the window context. The
Workbox
class provides a
messageSW() method to send a message to the instance's registered service worker and
await a response.
The following page code creates a new
Workbox instance and sends a message to the service worker
to obtain its version:
const wb = new Workbox('/sw.js');
wb.register();
const swVersion = await wb.messageSW({type: 'GET_VERSION'});
console.log('Service Worker version:', swVersion);
The service worker implements a message listener on the other end, and responds to the registered
service worker:
const SW_VERSION = '1.0.0';
self.addEventListener('message', (event) => {
if (event.data.type === 'GET_VERSION') {
event.ports[0].postMessage(SW_VERSION);
}
});
Under the hood the library uses a browser API that we'll review in the next section: Message
Channel, but abstracts many
implementation details, making it easier to use, while leveraging the wide browser
support this API has.
Diagram showing two-way communication between page and service worker, using Workbox Window.
Using Browser APIs {: #using-browser-apis }
If the Workbox library is not enough for your needs, there are several lower-level APIs available to
implement "two-way" communication between pages and service workers. They have some similarities
and differences:
Similarities:
In all cases the communication starts on one end via the
postMessage() interface and is received
on the other end by implementing a
message handler.
In practice, all the available APIs allow us to implement the same use cases, but some of them
might simplify development in some scenarios.
Differences:
They have different ways of identifying the other side of the communication: some of them use an
explicit reference to the other context, while others can communicate implicitly via a proxy
object instantiated on each side.
Browser support varies among them.
Diagram showing two-way communication between page and service worker, and the available browser APIs.
Broadcast Channel API {: #broadcast-channel-api }
The Broadcast Channel API
allows basic communication between browsing contexts via BroadcastChannel
objects.
To implement it, first, each context has to instantiate a
BroadcastChannel object with the same ID
and send and receive messages from it:
const broadcast = new BroadcastChannel('channel-123');
The BroadcastChannel object exposes a
postMessage() interface to send a message to any listening
context:
//send message
broadcast.postMessage({ type: 'MSG_ID', });
Any browser context can listen to messages via the
onmessage method of the
BroadcastChannel
object:
//listen to messages
broadcast.onmessage = (event) => {
if (event.data && event.data.type === 'MSG_ID') {
//process message...
}
};
As seen, there's no explicit reference to a particular context, so there's no need of obtaining a
reference first to the service worker or any particular client.
Diagram showing two-way communication between page and service worker, using a Broadcast Channel object.
The disadvantage is that, at the moment of this writing, the API has support from Chrome, Firefox
and Edge, but other browsers, like Safari, don't support it
yet.
Client API {: #channel-api }
The Client API allows you to obtain a
reference to all the
WindowClient objects representing the active tabs that the service worker is controlling.
Since the page is controlled by a single service worker, it listens to and sends messages to the
active service worker directly via the
serviceWorker interface:
//send message
navigator.serviceWorker.controller.postMessage({
type: 'MSG_ID',
});
//listen to messages
navigator.serviceWorker.onmessage = (event) => {
if (event.data && event.data.type === 'MSG_ID') {
//process response
}
};
Similarly, the service worker listens to messages by implementing an
onmessage listener:
//listen to messages
self.addEventListener('message', (event) => {
if (event.data && event.data.type === 'MSG_ID') {
//Process message
}
});
To communicate back with any of its clients, the service worker obtains an array of
WindowClient objects by executing
methods such as
Clients.matchAll() and
Clients.get(). Then it can
postMessage() any of them:
//Obtain an array of Window client objects
self.clients.matchAll(options).then(function (clients) {
if (clients && clients.length) {
//Respond to last focused tab
clients[0].postMessage({type: 'MSG_ID'});
}
});
Diagram showing a service worker communicating with an array of clients.
Client API is a good option to communicate easily with all the active tabs from a service worker
in a relatively straightforward way. The API is supported by all major
browsers,
but not all its methods might be available, so make sure to check browser support before
implementing it in your site.
In this guide we focus mostly on reliability use cases. Another interesting usage of
this API is synchronizing data across documents via a service worker. Check out this episode of
HTTP 203 to know more about it.
Message Channel {: #message-channel }
Message Channel requires
defining and passing a port from one context to another to establish a two-way communication
channel.
To initialize the channel, the page instantiates a
MessageChannel object and uses it
to send a port to the registered service worker. The page also implements an
onmessage listener on
it to receive messages from the other context:
const messageChannel = new MessageChannel();
//Init port
navigator.serviceWorker.controller.postMessage({type: 'PORT_INITIALIZATION'}, [
messageChannel.port2,
]);
//Listen to messages
messageChannel.port1.onmessage = (event) => {
// Process message
};
Diagram showing a page passing a port to a service worker, to establish two-way communication.
The service worker receives the port, saves a reference to it and uses it to send a message to the other
side:
let communicationPort;
//Save reference to port
self.addEventListener('message', (event) => {
if (event.data && event.data.type === 'PORT_INITIALIZATION') {
communicationPort = event.ports[0];
}
});
//Send messages
communicationPort.postMessage({type: 'MSG_ID'});
MessageChannel is currently supported by all major
browsers.
Advanced APIs: Background Sync and Background Fetch {: #advanced-apis:-background-sync-and-background-fetch }
In this guide we explored ways of implementing two-way communication techniques, for relatively
simple cases, like passing a string message describing the operation to perform, or a list of URLs
to cache from one context to the other. In this section we'll explore two APIs to handle specific
scenarios: lack of connectivity and long downloads.
Background Sync {: #background-sync }
A chat app might want to make sure that messages are never lost due to bad connectivity. The
Background Sync API lets you
defer actions to be retried when the user has stable connectivity. This is useful for ensuring that
whatever the user wants to send, is actually sent.
Instead of the
postMessage() interface, the page registers a
sync:
navigator.serviceWorker.ready.then(function (swRegistration) {
return swRegistration.sync.register('myFirstSync');
});
The service worker then listens for the
sync event to process the message:
self.addEventListener('sync', function (event) {
if (event.tag == 'myFirstSync') {
event.waitUntil(doSomeStuff());
}
});
The function
doSomeStuff() should return a promise indicating the success/failure of whatever it's
trying to do. If it fulfills, the sync is complete. If it fails, another sync will be scheduled to
retry. Retry syncs also wait for connectivity, and employ an exponential back-off.
Once the operation has been performed, the service worker can then communicate back with the page to
update the UI, by using any of the communication APIs explored earlier.
Google search uses Background Sync to persist failed queries due to bad connectivity, and retry
them later when the user is online. Once the operation is performed, they communicate the result to
the user via a web push notification:
Diagram showing a page passing a port to a service worker, to establish two-way communication.
Check out Resilient search experiences
to learn how to implement this feature using Workbox Background
Sync.
Background Fetch {: #background-fetch }
For relatively short bits of work like sending a message, or a list of URLs to cache, the options
explored so far are a good choice. If the task takes too long the browser will kill the service
worker, otherwise it's a risk to the user's privacy and battery.
The Background Fetch API
allows you to offload a long task to a service worker, like downloading movies, podcasts, or levels
of a game.
To communicate to the service worker from the page, use
backgroundFetch.fetch, instead of
postMessage():
navigator.serviceWorker.ready.then(async (swReg) => {
const bgFetch = await swReg.backgroundFetch.fetch(
'my-fetch',
['/ep-5.mp3', 'ep-5-artwork.jpg'],
{
title: 'Episode 5: Interesting things.',
icons: [
{
sizes: '300x300',
src: '/ep-5-icon.png',
type: 'image/png',
},
],
downloadTotal: 60 * 1024 * 1024,
},
);
});
The
BackgroundFetchRegistration object allows the page listen to the
progress event to follow
the progress of the download:
bgFetch.addEventListener('progress', () => {
// If we didn't provide a total, we can't provide a %.
if (!bgFetch.downloadTotal) return;
const percent = Math.round(
(bgFetch.downloaded / bgFetch.downloadTotal) * 100,
);
console.log(`Download progress: ${percent}%`);
});
Diagram showing a page passing a port to a service worker, to establish two-way communication.
The UI is updated to indicate the progress of a download (left). Thanks to service workers, the operation can continue running when all tabs have been closed (right).
Check out the Background Fetch
guide, which includes an
example podcast app along with its Glitch
code.
Next steps {: #next-steps }
In this guide we explored the most general case of communication between page and service workers
(bidirectional communication).
Many times, one might need only one context to communicate with the other, without receiving a
response. Check out the following guides for guidance how to implement unidirectional techniques in
your pages from and to the service worker, along with use cases and production examples:
Imperative caching guide: Calling a service worker from the page to
cache resources in advance (e.g. in prefetching scenarios).
Broadcast updates: Calling the page from the service worker to inform
about important updates (e.g. a new version of the webapp is available).
Workbox: your high-level service worker toolkit
Two APIs play a crucial role in building reliable web apps:
Service Worker
and Cache Storage. But
using them effectively—without introducing subtle bugs or bumping into edge
cases—can be a challenge. For example, errors in your service worker code can
cause caching problems; users might be shown out-of-date content or broken
links.
Workbox is a high-level
service worker toolkit built on top of the Service Worker and Cache Storage
APIs. It provides a production-ready set of libraries for adding offline support to
web apps. The toolkit is structured into two collections: tools that help manage
code that runs inside of your service worker, and tools that integrate with your
build process.
Runtime code
This is the code that runs inside of your service worker script and controls how
it intercepts outgoing requests and interacts with the Cache Storage API.
Workbox has a
dozen or so library modules in total,
that each handle a variety of specialized use cases. The most important modules
determine whether the service worker will respond (known as
routing),
and how it will respond (known as the
caching strategy).
Build integration
Workbox offers
command line,
Node.js module,
and
webpack plugin
tools that provide alternative ways to accomplish two things:
Create a service worker script based on a set of configuration
options. The generated service worker uses Workbox's runtime libraries
"under the hood" to put into action the caching strategies you configure.
Generate a list of URLs that should be
"precached",
based on configurable patterns to include and exclude files generated
during your build process.
Why should you use Workbox?
Using Workbox when building your service worker is optional—there are a number
of guides out there that walk through
common caching strategies
from a "vanilla" service worker perspective. If you do decide to use Workbox,
here are some of its benefits.
Cache management
Workbox handles cache updates for you, either tied in to your build process when
using precaching, or via configurable size/age policies when using runtime
caching. The underlying Cache Storage API is powerful, but it does not have any
built-in support for cache expiration. Tools like Workbox fill that gap.
Extensive logging and error reporting
When you're getting started with service workers, figuring out why something
is being cached (or, equally frustrating, why it isn't cached) is a challenge.
Workbox automatically detects when you're running a development version of your
website on
localhost, and turns on debug logging in your browser's JavaScript
console.
Workbox logging to the DevTools console
By following along with the log messages, you can get to the root of any
configuration or invalidation problems much more quickly than if you were going
it alone.
A tested, cross-browser codebase
Workbox is developed against a cross-browser test suite, and when possible,
automatically falls back to alternative implementations of features that are
missing from certain browsers.
The
workbox-broadcast-cache-update module
uses the
Broadcast Channel API
when available, and falls back to a
postMessage()-based
implementation on browsers that lack support.
The
workbox-background-sync module
uses the
Background Sync API
if possible, and if not, falls back to retrying queued events each time the
service worker starts up.
How should you use Workbox?
Framework integration
If you're starting a new project from scratch, you can take advantage of the
Workbox integration found in many popular starter kits and add-on plugins:
create-react-app
vue-cli
preact-cli
Gatsby
Next.js
Add Workbox to your existing build process
If you already have a build process for your site in place, dropping in the
appropriate
command line,
Node.js module,
or
webpack plugin
tool may be all you need to start using Workbox.
In particular, the Workbox command line interface makes it easy to get up and
running, featuring a
wizard mode that will check your local development
environment and suggest a reasonable default configuration that you could use
moving forward:
workbox wizard
? What is the root of your web app (i.e. which directory do you deploy)? src/
? Which file types would you like to precache? css, js, html
? Where would you like your service worker file to be saved? build/sw.js
? Where would you like to save these configuration options? workbox-config.js
To build your service worker, run
workbox generateSW workbox-config.js
as part of a build process. See the
generateSW documentation for details.
You can further customize your service worker by making changes to
workbox-config.js.
See the documentation of the options for details.
Use Workbox at runtime in an existing service worker
If you have an existing service worker and want to try out Workbox's runtime
libraries,
import Workbox from its official CDN
and start using it for runtime caching right away. This use
case means that you won't be able to take advantage of precaching (which
requires build-time integration), but it's great for prototyping and trying out
different caching strategies on the fly.
// Replace 3.6.3 with the current version number of Workbox.
importScripts('https://storage.googleapis.com/workbox-cdn/releases/3.6.3/workbox-sw.js');
workbox.routing.registerRoute(
new RegExp('\.png$'),
workbox.strategies.cacheFirst({
cacheName: 'images-cache',
})
);
Workers overview
This overview explains how web workers and service workers can improve the performance of your website, and when to use a web worker versus a service worker. Check out the rest of this series for
specific patterns of window and service worker communication.
How workers can improve your website {: #how-workers-can-improve-your-website }
The browser uses a single thread (the main
thread) to run all the JavaScript in
a web page, as well as to perform tasks like rendering the page and performing garbage collection.
Running excessive JavaScript code can block the main thread, delaying the browser from performing
these tasks and leading to a poor user experience.
In iOS/Android application development, a common pattern to ensure that the app's main thread
remains free to respond to user events is to offload operations to additional threads. In fact, in
the latest versions of Android, blocking the main thread for too long leads to an app
crash.
On the web, JavaScript was designed around the concept of a single thread, and lacks capabilities
needed to implement a multithreading model like the one apps have, like shared memory.
Despite these limitations, a similar pattern can be achieved in the web by using
workers to run scripts in background
threads, allowing them to perform tasks without interfering with the main thread. Workers are an
entire JavaScript scope running on a separate thread, without any shared memory.
In this post you'll learn about two different types of workers (web workers and service workers), their similarities and differences, and the most common patterns for using them in production websites.
Diagram showing two links between the Window object and a web worker and service worker.
Web workers and service workers {: #web-workers-and-service-workers }
Similarities {: #similarities }
Web workers
and service
workers
are two types of workers available to websites. They have some things in common:
Both run in a secondary thread, allowing JavaScript code to execute without blocking the main
thread and the user interface.
They don't have access to the
Window
and
Document objects, so they can't
interact with the DOM directly, and they have limited access to browser APIs.
Differences {: #differences }
One might think that most things that can be delegated to a web worker can be done in a service
worker and vice versa, but there are important differences between them:
Unlike web workers, service workers allow you to intercept network requests (via the
fetch event) and to listen for Push
API events in the background (via the
push event).
A page can spawn multiple web workers, but a single service worker controls all the active tabs
under the
scope it was
registered with.
The lifespan of the web worker is tightly coupled to the tab it belongs to, while the service
worker's
lifecycle is
independent of it. For that reason, closing the tab where a web worker is running will terminate
it, while a service worker can continue running in the background, even when the site doesn't have
any active tabs open.
For relatively short bits of work like sending a message, the browser won't likely
terminate a service worker when there are no active tabs, but if the task takes too long the browser
will terminate the service worker, otherwise it's a risk to the user's privacy and battery. APIs
like Background Fetch, that
can let you avoid the service worker's termination.
Use cases {: #use-cases }
The differences between both types of workers suggest in which situations one might want to use one
or the other:
Use cases for web workers are more commonly related to offloading work (like heavy
computations) to a secondary
thread, to avoid blocking the UI.
Diagram showing a link from the Window object to a web worker.
Example: the team that built the videogame PROXX wanted to leave the
main thread as free as possible to take care of user input and animations. To achieve that, they
used web workers to run the game logic and state
maintenance on a separate thread.
A screenshot of the videogame PROXX.
Service workers tasks are generally more related to acting as a network proxy, handling
background tasks, and things like caching and offline.
A screenshot of the videogame PROXX.
Example: In a podcast PWA, one might want to allow users
to download complete episodes to listen to them while offline. A service worker, and, in particular,
the Background Fetch API can
be used to that end. That way, if the user closes the tab while the episode is downloading, the task
doesn't have to be interrupted.
A screenshot of a Podcast PWA.
The UI is updated to indicate the progress of a download (left). Thanks to service workers, the operation can continue running when all tabs have been closed (right).
Tools and libraries {: #tools-and-libraries }
Window and worker communication can be implemented by using different lower level APIs. Fortunately,
there are libraries that abstract this process, taking care of the most common use cases. In this
section, we'll cover two of them that take care of window to web workers and service workers
respectively: Comlink and
Workbox.
A screenshot of the videogame PROXX.
Comlink {: #comlink }
Comlink is a small (1.6k)
RPC library that takes care of many
underlying details when building websites that use Web Workers. It has been used in websites
like PROXX and Squoosh. A summary of its motivations
and code samples can be found here.
Workbox {: #workbox }
Workbox is a popular library to build websites
that use service workers. It packages a set of best practices around things like caching, offline,
background synchronization, etc. The
workbox-window module provides a
convenient way to exchange messages between the service worker and the page.
Next steps {: #next-steps }
The rest of this series focuses on patterns for window and service worker communication:
Imperative caching guide: Calling a service worker from the page to
cache resources in advance (e.g. in prefetching scenarios).
Broadcast updates: Calling the page from the service worker to inform
about important updates (e.g. a new version of the website is available).
Two-way communication: Delegating a task to a service worker
(e.g. a heavy download), and keeping the page informed on the progress.
For patterns of window and web worker communication check out: Use web workers to run JavaScript
off the browser's main thread.