Istio and Envoy WebAssembly Extensibility, One Year On

An update on Envoy and Istio's WebAssembly-based extensibility effort.

Mar 5, 2021 | By Pengyuan Bian - Google

One year ago today, in the 1.5 release, we introduced WebAssembly-based extensibility to Istio. Over the course of the year, the Istio, Envoy, and Proxy-Wasm communities have continued our joint efforts to make WebAssembly (Wasm) extensibility stable, reliable, and easy to adopt. Let’s walk through the updates to Wasm support through the Istio 1.9 release, and our plans for the future.

WebAssembly support merged in upstream Envoy

After adding experimental support for Wasm and the WebAssembly for Proxies (Proxy-Wasm) ABI to Istio’s fork of Envoy, we collected some great feedback from our community of early adopters. This, combined with the experience gained from developing core Istio Wasm extensions, helped us mature and stabilize the runtime. These improvements unblocked merging Wasm support directly into Envoy upstream in October 2020, allowing it to become part of all official Envoy releases. This was a significant milestone, since it indicates that:

wasm-extensions Ecosystem Repository

As an early adopter of the Envoy Wasm runtime, the Istio Extensions and Telemetry working group gained a lot of experience in developing extensions. We built several first-class extensions, including metadata exchange, Prometheus stats, and attribute generation. In order to share our learning more broadly, we created a wasm-extensions repository in the istio-ecosystem organization. This repository serves two purposes:

The guide currently covers WebAssembly extension development and unit testing with C++, as well as integration testing with a Go test framework, which simulates a real runtime by running a Wasm module with the Istio proxy binary. In the future, we will also add several more canonical extensions, such as an integration with Open Policy Agent, and header manipulation based on JWT tokens.

Wasm module distribution via the Istio Agent

Prior to Istio 1.9, Envoy remote data sources were needed to distribute remote Wasm modules to the proxy. In this example, you can see two EnvoyFilter resources are defined: one to add a remote fetch Envoy cluster, and the other one to inject a Wasm filter into the HTTP filter chain. This method has a drawback: if remote fetch fails, either due to bad configuration or transient error, Envoy will be stuck with the bad configuration. If a Wasm extension is configured as fail closed, a bad remote fetch will stop Envoy from serving. To fix this issue, a fundamental change is needed to the Envoy xDS protocol to make it allow asynchronous xDS responses.

Istio 1.9 provides a reliable distribution mechanism out of the box by leveraging the xDS proxy inside istio-agent and Envoy’s Extension Configuration Discovery Service (ECDS).

istio-agent intercepts the extension config resource update from istiod, reads the remote fetch hint from it, downloads the Wasm module, and rewrites the ECDS configuration with the path of the downloaded Wasm module. If the download fails, istio-agent will reject the ECDS update and prevent a bad configuration reaching Envoy. For more detail, please see our docs on Wasm module distribution.

Remote Wasm module fetch flow
Remote Wasm module fetch flow

Istio Wasm SIG and Future Work

Although we have made a lot of progress on Wasm extensibility, there are still many aspects of the project that remain to be completed. In order to consolidate the efforts from various parties and better tackle the challenges ahead, we have formed an Istio WebAssembly SIG, with aim of providing a standard and reliable way for Istio to consume Wasm extensions. Here are some of the things we are working on:

If you would like to join us, the group will meet every other week Tuesdays at 2PM PT. You can find the meeting on the Istio working group calendar.

We look forward to seeing how you will use Wasm to extend Istio!