Compression Dictionary Transport

Compression Dictionary Transport is a way of using a shared compression dictionary to dramatically reduce the transport size of HTTP responses.
Overview
Compression algorithms are used in HTTP to reduce the size of resources downloaded over the network, reducing bandwidth cost and the time taken to load pages. Lossless HTTP compression algorithms work by finding redundancy in the source: for example, places where text like the string 
"function" is repeated. They then include just one copy of the redundant string, and replace occurrences of it in the resource with references to that copy. Since the references are shorter than the string, the compressed version is shorter.
NOTE:
A previous attempt at this technology was called SDCH (Shared Dictionary Compression for HTTP) but was never widely supported and was removed in 2017. Compression Dictionary Transport is a better-specified and more robust implementation with broader industry consensus.
For example, take this JavaScript:
function a() {
  console.log("Hello World!");
}
function b() {
  console.log("I am here");
}
This could be compressed by replacing repeated strings with references to a previous location and number of characters, like this:
function a() {
  console.log("Hello World!");
}
[0:9]b[10:20]I am here[42:46]
In this example, 
[0:9] refers to copying the 9 characters starting at character 0. Note that this is a simplified example to demonstrate the concept; the actual algorithms are more complex.
Clients can then reverse the compression after download to recreate the original, uncompressed resource.
Compression dictionaries
Algorithms like Brotli compression and Zstandard compression achieve even greater efficiency by allowing the use of dictionaries of commonly encountered strings, so you don't need any copies of them in the compressed resource. These algorithms ship with a predefined default dictionary that is used when compressing HTTP responses.
Compression Dictionary Transport builds on this by enabling you to provide your own dictionary, which is especially applicable to a particular set of resources. The compression algorithm can then reference it as a source of bytes when compressing and decompressing the resource.
Assuming the references from the previous example are included in that common dictionary, this could be further reduced to this:
[d0:9]a[d10:20]Hello World![d42:46]
[d0:9]b[d10:20]I am here[d42:46]
The dictionary can be a separate resource that is only required for Compression Dictionary Transport, or a resource that the website needs anyway.
For example, suppose your website uses a JavaScript library. You would typically load a specific version of the library, and might include the version name in the name of the library, like 
<script src="my-library.v1.js">. When the browser loads your page, it will fetch a copy of the library as a subresource.
If you then update to v2 of the library, most of the library's code will probably be the same. Sites can therefore greatly reduce the size of the download for 
my-library.v2.js by telling the browser to use 
my-library.v1.js as a compression dictionary for 
my-library.v2.js. All strings that are common between v1 and v2 don't need to be included in the download for v2, because the browser already has them. Most of the download size of 
my-library.v2.js is then just the delta between the two versions.
Compression Dictionary Transport can achieve an order of magnitude more compression than compression using a default built-in dictionary: see Compression dictionary transport examples for some real-life results.
Dictionary format
A compression dictionary doesn't follow any specific format, nor does it have a specific MIME type. They are regular files.
Previous versions of files typically have lots of similar content, which is why they make excellent dictionaries.
Using a previous version of a file as a dictionary allows the compression algorithm to efficiently reference all the unchanged content, and capture the relatively small differences in the new version. This approach is called delta compression.
Another approach is to list common strings (for example, your HTML templates) together in a new 
dictionary.txt file so it can be used to compress HTML pages on the website. You can optimize this further using specialized tooling, for example Brotli's dictionary generator, which reduces dictionaries to their minimum size with minimal overlap.
Dictionaries can also compress binary formats effectively. For example, WASM binary files are large resources that can also benefit from delta compression.
Existing resource as a dictionary
To use a resource as a dictionary, the server should include the Use-As-Dictionary header in the response that provides the resource:
Use-As-Dictionary: match="/js/app.*.js"
The value of this header indicates the resources that can use this resource as a dictionary: in this case, that includes any resources whose URLs match the given pattern.
When a resource is later requested that matches the given pattern (for example, 
app.v2.js), the request will include a SHA-256 hash of the available dictionary in the Available-Dictionary header, along with 
dcb and/or 
dcz values in the Accept-Encoding header (for delta compression using Brotli or ZStandard as appropriate):
Accept-Encoding: gzip, br, zstd, dcb, dcz
Available-Dictionary: :pZGm1Av0IEBKARczz7exkNYsZb8LzaMrV7J32a2fFG4=:
The server can then respond with an appropriately encoded response with the chosen content encoding given in the Content-Encoding header:
Content-Encoding: dcb
If the response is cacheable, it must include a Vary header to prevent caches from serving dictionary-compressed resources to clients that don't support them or serving the response compressed with the wrong dictionary:
Vary: accept-encoding, available-dictionary
An optional 
id can also be provided in the Use-As-Dictionary header to allow the server to more easily find the dictionary file if they do not store the dictionary by the hash:
Use-As-Dictionary: match="/js/app.*.js", id="dictionary-12345"
If this is provided, the value will be sent in future requests in the Dictionary-ID header:
Accept-Encoding: gzip, br, zstd, dcb, dcz
Available-Dictionary: :pZGm1Av0IEBKARczz7exkNYsZb8LzaMrV7J32a2fFG4=:
Dictionary-ID: "dictionary-12345"
The server must still check the hash from the 
Available-Dictionary header — the 
Dictionary-ID is additional information for the server to identify the dictionary but does not replace the need for the 
Available-Dictionary header.
Separate dictionary
An HTML document can also provide a compression dictionary to the browser. Such a dictionary shouldn't be a resource the browser is already downloading via an element such as a script tag. There are two methods to do this:
Include a link element whose 
rel attribute is set to 
compression-dictionary:
<link rel="compression-dictionary" href="/dictionary.dat" />
Reference the dictionary using the Link header:
Link: </dictionary.dat>; rel="compression-dictionary"
The browser downloads this dictionary during idle time, and the response must include the Use-As-Dictionary header:
Use-As-Dictionary: match="/js/app.*.js"
From here, the process is similar to the previous example when a matching resource is requested.
Dictionary freshness
The response that carries the Use-As-Dictionary header also decides, through its Cache-Control header, whether the browser stores the dictionary and for how long. The browser uses a stored dictionary only while that response is still fresh, or while the 
stale-while-revalidate directive still allows the response to be served stale. In practice:
Specifying 
no-store prevents the browser from storing the response at all, while 
no-cache allows it to be stored but requires revalidation before each reuse. Either way the browser never uses the response as a dictionary, regardless of the other directives it carries.
Specifying 
max-age=0 combined with 
stale-while-revalidate=<seconds> still lets the browser store the dictionary, which then lasts as long as the 
stale-while-revalidate window.
Specifying 
must-revalidate cancels any 
stale-while-revalidate window, so the dictionary lasts only as long as 
max-age. With 
max-age=0, the browser never stores it.
The 
s-maxage directive doesn't apply because the browser is a private cache.
HTML documents are affected more than other resources because servers often send them with 
no-cache or 
max-age=0, must-revalidate so that the browser always revalidates them. Such a document cannot offer itself as a dictionary for its next version, regardless of how the 
match pattern is written.
In such cases, the browser silently drops the dictionary. It accepts the 
Use-As-Dictionary header without an error, yet a later request that matches the pattern carries no Available-Dictionary header. Developer tools can surface this issue; for example, Chrome reports the reason in the Issues panel. The issue reads "The response can't be used as a dictionary because its freshness is expired", along with the name of the response that offered the dictionary.
Creating dictionary-compressed responses
Dictionary-compressed responses can use either the Brotli or ZStandard algorithms, with two extra requirements: they must also include a magic header and embedded dictionary hash.
Dictionary-compressed resources can be dynamically created, but for static resources it can be better to create them in advance at build time. When using prior versions as dictionaries, you need to decide how many delta-compressed versions to create — for the last version only, or for the last X versions for some value of X.
Given a dictionary file named 
dictionary.text and a file to compress named 
data.text, the following Bash command will compress the file using Brotli, producing a compressed file named 
data.txt.dcb:
echo -en '\xffDCB' > data.txt.dcb && \
openssl dgst -sha256 -binary dictionary.txt >> data.txt.dcb && \
brotli --stdout -D dictionary.txt data.txt >> data.txt.dcb
Given the same input files, the following Bash command will compress the file using ZStandard, producing a compressed file named 
data.txt.dcz:
echo -en '\x5e\x2a\x4d\x18\x20\x00\x00\x00' > data.txt.dcz && \
openssl dgst -sha256 -binary dictionary.txt >> data.txt.dcz && \
zstd -D dictionary.txt -f -o tmp.zstd data.txt && \
cat tmp.zstd >> data.txt.dcz
For this to work, you need OpenSSL installed locally as well as Brotli or ZStandard.
Restrictions
Compression algorithms are at risk of security attacks, so there are several restrictions for Compression Dictionary Transport, including:
Dictionaries must be same-origin with the resource using the dictionary.
Dictionary-compressed resources must be same-origin with the document origin, or follow the CORS rules, and so be requested with the 
crossorigin attribute and served with an appropriate Access-Control-Allow-Origin header.
Dictionaries are bound by the usual HTTP Cache partitioning and so cannot be shared between origins even if they download the same resources. The dictionary will need to be downloaded again for each origin.
Additionally, dictionaries could themselves become tracking vectors, so browsers may restrict this feature when cookies are disabled or when other extra privacy protections are enabled.
As with other resources, if a website uses the Content-Security-Policy header, the compression dictionary must be an allowed source for it to be loaded.
In particular, when loading a separate dictionary using 
<link rel="compression-dictionary">, the 
connect-src directive (or 
default-src, if 
connect-src is not set) must allow the dictionary location.
Specifications
Browser compatibility
See also
Glossary terms:
Brotli compression
Zstandard compression
<link rel="compression-dictionary">
Accept-encoding
Content-encoding
Available-Dictionary
Dictionary-ID
Use-As-Dictionary
RFC 9842: Compression Dictionary Transport
Resources for Compression Dictionary Transport