Airbnb Moves from Webpack to Metro, Enjoys Shorter Build Times
Like many webscale companies, Airbnb has experienced growing pains with bundlers as its codebase grew. But even after its codebase quadrupled, the company was able to speed UX changes to the front end when it migrated its JavaScript code bundler from Webpack to Metro in 2018.
With build performance significantly improved, UI changes appeared 80% faster, as per the Time to Interactive (TTI) metric. Even the slowest production build, one compiling 49,000 modules (JavaScript files) is now 55% faster — down to 13.8 minutes from 30.5 minutes with Webpack.
Airbnb’s own Page Performance Scores also improved (~1%) with those pages built with Metro.
For reference, after the codebase quadrupled around 2018, the average page refresh time for a simple one-line code change was anywhere between 30 seconds to two minutes depending on the project size.
Airbnb software engineer Rae Liu covered some of the differences between Webpack and Metro and discussed some of the migration challenges in this recent blog post.
What Is Metro?
Developed by Meta, Metro is the open source JavaScript code bundler for React Native. This article will cover a custom version of Metro as Airbnb’s stack doesn’t include React Native. Airbnb engineers worked directly with Metro’s engineers at Meta as well as on teams themselves to further develop the technology.
Metro breaks bundling down into three steps in the following order: resolution, transformation, and serialization.
- Resolution: resolve the import/ require statements
- Transformation: transpile code (source to source compiler which converts modern Typescript/ JavaScript source code into JavaScript and backwards compatible with old browsers), an example tool is babel
- Serialization: combine the transformed files into bundles
In development, Airbnb engineers created a Metro server with custom endpoints to handle building dependency graphs and source maps, translation, and bundling JS and CSS files. For production builds, they ran Metro as a Node API to handle resolution, transformation, and serialization.
The migration took place in two phases. The first priority was the Metro development server as the slow Webpack development server was the source of significant development productivity costs. The second migration phase focused on bringing Metro to feature parity with Webpack and running an A/B test between Metro and Webpack in production.
Two Key Differences Between Metro and Webpack
Process JavaScript Bundles On Demand
Webpack pre-compiles the entire project on startup while Metro only compiles what is needed. What does this mean? A JavaScript bundle is technically just a serialized dependency graph, where an entry point is the root of the graph.
At Airbnb, every frontend project has a Node server that matches a route to a specific entry point. When a web page is requested, the DOM includes script tags with the development JavaScript URLs. Webpack needs to know all the entry points for all the pages before it can start bundling while Metro only needs one entry point and can process JavaScript bundles as requested.
Unseen in the following image, a developer makes a change to Page A:
In both 1a and 1b in the diagram above the browser loads Page A (1) the requests the entryPageA.js file from the bundler (2) and the bundler responds to the browser with the appropriate bundles (4). The difference between figures 1a and 1b lies in action (3) as the Webpack diagram compiles entry points for pages B and C while Metro does not as the developer-only modified Page A in the example.
One of the largest frontend projects at Airbnb has 26,000 unique modules with the median number of modules per page being ~7.2 modules. The number of modules Airbnb ultimately has to process doubles to roughly 48,000 due to their use of server-side rendering. After putting Metro’s compile-on-demand model into action, approximately 70% is now taking place.
Multilayered Cache
Airbnb leverages Metro’s Multilayered Caching feature with persistent and non-persistent caches. Metro does provide more caching flexibility by allowing engineers to define the cache implementation, including mixing different types of cache layers.
Airbnb ordered their caching layers by order of priority. If a result is not found in one cache layer, the next layer will be used until the result is found. Compared with the default Metro implementation without a cache, hitting a remote read-only cache resulted in a 56% faster server build in a project compiling 22,000 files.
The third caching layer is a remote read-only cache rather than a read-write cache as writing to a remote cache incurs costly network calls especially on a slow network. This decision saved an additional 17% build time in development.
Webpack does have a caching layer though it does differ from what Metro offers.
Bundle Splitting
One of the technical challenges detailed in Airbnb’s blog post is Bundle Splitting. This is the process of splitting the bundles by dynamic import boundaries also known as code splitting. The out-of-the-box Metro solution produced giant ~ 5MiB bundles per entry point which were taxing on browser resources, network latency, and unable to HTTP cache.
In the image above, import(‘./file’) represents the dynamic import boundaries. The bundle on the left-hand side (3a) is broken down to three smaller bundles on the right (3b). The additional bundles are requested when the import(‘./file’) statements are executed.
Suppose fileA.js changed, the entire bundle needs to be re-downloaded for the browser to pick up the change in fileA.js. With bundles split by dynamic import illustrated in Figure 3b, a change in fileA.js only results in the re-downloading of the fileA.js bundle. The rest of the bundles can reuse the browser cache.
In production, there is no development server and the bundles are prebuilt. Airbnb engineers took some inspiration from Webpack’s bundle splitting algorithm and implemented a similar mechanism to split the Metro dependency graphs. The resulting bundle size decreased by ~20% (1549 KB –> 1226 KB) on airbnb.com as compared to the development splitting by dynamic import boundaries.
Development bundles were optimized differently as it takes time to run the bundle splitting algorithm and the engineers didn’t want to waste time splitting bundle size in development. In the instance of development, page load performance was prioritized over minimizing bundle size.
The Metro and Webpack bundle size metrics are comparable.
In Conclusion
The biggest Airbnb frontend project compiling ~48,000 modules (including server and browser compilations) saw a drop in the average build time by ~55% from 30.5 minutes to 13.8 minutes. Airbnb Page Performance Scores improved around 1% for pages built with Metro which was a nice surprise as the goal was a neutral result. Overall, the implementation of Metro is widely successful.
Metro has solved the bundling issues Airbnb was facing but the engineers also recognize that new technologies have emerged since their decision to move forward with Metro and that Metro isn’t a general-purpose JavaScript bundler.
