#Production Module Optimizations

When shipping ES modules in production, there are currently two major performance optimizations to apply - code splitting and preloading.

Code splitting optimizations are available for native ES modules in bundlers like esbuild or RollupJS. Code splitting ensures that for any two modules that always load together, they will always be inlined into the same module file as a network-optimized chunk module (or even inlined into the entry point module itself where possible).

Then preloading solves the delayed latency waterfall of module graph discovery - modules only execute after every module in the static import graph has been loaded, and modules only load after their parents have been loaded.

#modulepreload

Preloading for ES modules is provided by the <link rel="modulepreload" href="..."/> tag in browsers. There's a great article on it from the Google Developers 2017 Updates back when it was first shipped in Chrome.

It is advisable to inject modulepreload tags for all deep dependencies whenever possible so that this entirely eliminates the latency cost of module loading, and that is the primary benefit of static preloading in the first place.

Another major benefit of modulepreload is that it is currently the only mechanism to support full integrity for all loaded modules using the "integrity" attribute. For example if app.js loads dependency.js loads library.js, we can write:


<link rel="modulepreload" href="/src/app.js" integrity="sha384-Oe38ELlp8iio2hRyQiz2P4Drqc+ztA7jb7lONj7H3Cq+W88bloPxoZzuk6bHBHZv"/>
<link rel="modulepreload" href="/src/dependency.js" integrity="sha384-kjKb2aJJUT956WSU7Z0EF9DZyHy9gdvPOvIWbcEGATXKYxJfkEVOcuP1q20GT2LO"/>
<link rel="modulepreload" href="/src/library.js" integrity="sha384-Fwh0dF5ROSVrdd/vJOvq0pT4R6RLZOOvf6k+ijkAyUtwCP7B0E3qHy8wbot/ivfO"/>
<script type="module" src="/src/app.js"></script>

The waterfall is eliminated as the preloads cause app.js, dependency.js and library.js to now load immediately in parallel, and with integrity on all scripts we can fully secure the module execution environment.

#Polyfilling modulepreload

One issue with this feature is it is only implemented in Chromium browsers right now, but a polyfill can be constructed with the following code:


<script>
  function processPreload () {
    const fetchOpts = {};
    if (script.integrity)
      fetchOpts.integrity = script.integrity;
    if (script.referrerpolicy)
      fetchOpts.referrerPolicy = script.referrerpolicy;
    if (script.crossorigin === 'use-credentials')
      fetchOpts.credentials = 'include';
    else if (script.crossorigin === 'anonymous')
      fetchOpts.credentials = 'omit';
    else
      fetchOpts.credentials = 'same-origin';
    fetch(link.href, fetchOpts).then(res => res.ok && res.arrayBuffer());
  }
  for (const link of document.querySelectorAll('link[rel=modulepreload]'))
    processPreload(link);
</script>

It is important to ensure that the fetch call immediately reads the response to completion to avoid possible race conditions that can result in double network fetches, per the .then(res => res.ok && res.arrayBuffer()) above.

Furthermore, if we want to add dynamic preload support to this polyfill, then we can use mutation observers:


new MutationObserver(mutations => {
  for (const mutation of mutations) {
    if (mutation.type !== 'childList') continue;
    for (const node of mutation.addedNodes) {
      if (node.tagName === 'LINK' && node.rel === 'modulepreload')
        processPreload(node);
    }
  }
}).observe(document, { childList: true, subtree: true });

This won't polyfill deep dependency preloading, but covers the majority use case and enables "integrity" checks in all modules browsers by priming the internal integrity map.

In this way we can get full preload and integrity support in production modules environments in all browsers.

The above polyfill is included in the latest 0.12.1 release of ES Module Shims, which provides a combination of polyfills for various ES modules features, particularly for import maps.

#Integrity Limitations

A major issue with using modulepreload as the primary integrity approach is there is no easy way to provide integrity upfront for lazy loaded modules without preloading them immediately. For production workflows today, the best approach would be to construct a custom dynamic import wrapper that lazily injects preloads just before triggering dynamic import.

That is a lot of work though and the friction is likely so prohibitive I'd be surprised if anyone is even shipping an approach like this. Yet module integrity is an absolutely crucial feature for using ES module CDNs.

Possible future specifications that might interact with module preloading and integrity include:

  • Import assertions: An import 'mod' assert { integrity: '...' } syntax has been suggested but is not yet specified or implemented for import assertions.

    Unfortunately this feature suffers from the issue that it undoes the primary performance benefit of import maps in allowing all modules in a graph to be independently cached with far-future expires. So while it is useful for certain specific cases, as a general solution to this problem it would be a step backwards.

  • Import Map Integrity: I have suggested an integrity attribute specification for import maps, allowing them to act as the point of orchestration here. The difficulty here is getting buy-in from browsers, which has been unsuccessful so far.

  • Milestones: This is an experimental performance approach with a current Proposal Doc and Chrome CL designed to allow specifying something about the conditions under which a script should be loaded for more fine-grained loading optimization. Unfortunately there are currently no plans to support this feature for preloads so that means it cannot in its current form solve the problem of deep graph content integrity with lazy module loading.

  • Lazy Preloads: Another design might be to have an attribute on a preload tag to indicate that it should not be preloaded, but its integrity value should still apply. I suggested this on WebAppSec but it seems there is confusion in the combination of preloading and integrity here.

  • Web Bundles: From the FAQ, it seems like the current state of integrity for Web Bundles it is being treated as a follow-up proposal due to the high bandwidth cost of hash verification.

Perhaps the concept of generalized module integrity folds into a centralized integrity manifest for the web, possibly combined along with other permissions / security features in a security manifest.

Further, ideally such integrity schemes wouldn't even specify per-resource integrity at all, since is actually quite inefficient due to the high bandwidth involved. Instead verification could benefit from the breadth of approaches we have for optimized integrity including merkle trees optimized to chunking boundaries or even something more exotic.

#Call to Action

The basic principle that one should be able to visit a website on the internet with all executed code being verified against an integrity hash is an absolutely fundamental security property.

Getting this done well and optimally will require some novel work and we need to actively engage with browser vendors and standards bodies to ensure that this security property can be fully and easily enabled for the future web platform without friction.

#JSPM Generator Preloading

JSPM Generator is a low-level tool I've been working on for generating import maps against module CDNs or local package paths. The latest version now includes support for static tracing of module dependencies allowing for constructing these preload tags for module graphs. Work on refining these generator APIs and workflows is ongoing.

Support for these features is also included in the Online Generator for JSPM.io, by toggling the "Preload" and "Integrity" boxes at the top right of the application.

Here's a demonstration of that in action, toggling preloading for separate CDN dependencies:

Production modules workflows have come a far way!