Extend envoy through wasm and rust

Time:2021-6-17

If you already know what istio, envoy, wasm and rust have in common and just want to start building your filters, you can skip to part 2 – building filters.

Part 1: evolution of grid

As I have mentioned many times in the past few years, service grid is the next stage of cloud native infrastructure development.

History of istio scalability

However, like istio, each grid implementation continues to develop at its own speed. As Google and IBM support istio, istio is ahead of its competitors in all aspects related to its functions. Unfortunately – prior to version 1.5, istio was also known for the performance problems caused by many architectural decisions made at the beginning of project development. One of the main sources of performance bottlenecks is a component called mixer. Its main responsibilities in mesh network include implementing traffic policy and collecting telemetry data. With the help of adapters, mixer completes all these tasks. Adapters is an extension mechanism that can easily integrate istio with third-party policies and telemetry systems. On the downside, mixers reside in the data path of each network request and cause intolerable delays when overloaded. Check out thisBenchmark ReportTo see how istio before 1.5 works in high traffic.

The istio team has been working on performance issues. One of the tough decisions they had to make was to remove the mixer and move telemetry and policy settings to the mesh agent. In the case of istio, these agents are envoy instances. But what about the mixer Adapter? In order not to give up the scalability of supporting throughput, we must design a new solution. The envoy proxy now allows the webassembly module to be loaded for use in its network filter chain.

Wasm and rust

I won’t waste your time describing what webassembly is and the future of secure lightweight computing workloads. There are other blog posts about this. I’m just saying that envoy now supports wasm, which means that people can write filters using C + +, assemblescript or rust. This is the last choice that makes me most excited – because I like rust very much and have been looking for a practical project to use it.

About webassembly hub

If you want to quickly start creating your own wasm based envoy filter, the good people at solo. IO have provided some tools to help you get started. theirWebAssemblyHubAnd wasme command-line utilities allow you to create filters, package filters, and deploy them to GLOO, istio, or stand-alone envoy. But as it happens – they still don’t have built-in rust support.

In general, rust’s proxy SDK seems to lag behind C + + and assemblescript. When trying to make the rust based filter work, I even thought the SDK was not ready. But thanks to Victor charypar, who was very happy to respond to my GitHub question, I finally realized that there was a problem with my envoy version.

Anyway – finally, my filter starts to work, and Part 2 of this article goes on to describe the steps needed to build an envoy filter based on wasm:

Part 2: building filters

Get your toolbox ready

The extent to which you use rust may vary – so just in case you still don’t have rust and cargo installed, go ahead. On Linux and MacOS systems, the operation is as follows:

curl https://sh.rustup.rs -sSf | sh

If all goes well, you will see the following:

Rust is installed now. Great!

Remember – we’re dealing with some cutting-edge features here, so a basic rust installation is not enough. We also need to install the rust nightly toolchain and support for wasm compilation targets. Fortunately, it’s very easy. You just need to run:

rustup toolchain install nightly
rustup target add wasm32-unknown-unknown

Now we have all the tools ready – let’s initialize the project:

Create Library

The wasm filter is compiled from the rust library project, so first we need to create the project:

cargo new --lib my-wasm-filter

This will create a template library project in the “my wasm filter” directory. You’ll find a lib.rs file in the Src / directory, as well as a cargo.toml file that tells cargo how to build your project.

Set library type

The generated library is loaded from envoy’s C + + code, so there is no need to include any rust specific information in it. Therefore, we will set the library type to “cdylib” as defined here. This will result in smaller binaries. To do this, open your cargo.toml file, and then add under the [lib] section:

[lib] 
crate-type = ["cdylib"]

Proxy WASM SDK

The envoy filter must be based on the SDK provided by the proxy wasm project. In particular, the first version of the proxy wasm trust SDK has been released here at crates.io. Therefore, we need to add it to our cargo.toml as a dependency:

[dependencies] 
proxy-wasm =  "0.1.0"

Now we can write the code.

Write the code

At the time of writing – there is no detailed documentation for writing the envoy wasm filter. My code is based on the example in the proxy SDK project and the CPP based filter example in the envoy wasm project.

Basically, what we need to do is:

  • Implement basic context features for our filter.
  • Implement an httpcontext feature, which inherits the basic context feature.
  • Override the context API method to handle the corresponding initialization and HTTP headers / events from the host.
  • By calling proxy_ wasm::set_ http_ Context initializes the context.

Note: our example only deals with HTTP headers. If we are dealing with other flows and initialization events – we need to implement streamcontext or rootcontext accordingly.

Implement the filter function

I decided to implement a very simple scenario in which every request sent to our service needs to be authorized by sending a token, and then the validity of the token is checked by the filter. If the token is validated – the request is passed to the service. Otherwise – 403 response is returned to the caller.

The token validation check is very clumsy – we check if the token is prime. To do that – let’s add another dependency to cargo.toml:

[dependencies] 
proxy-wasm =  "0.1.0"  
primes  =  "0.3.0"

Specific implementation:

Let’s create a context:

struct  PrimeAuthorizer  {    
    context_id: u32,    
}

And implement the context class for it

impl  Context  for  PrimeAuthorizer  {}

Note – we don’t need to implement any methods in context, because we do all the work at the L7 level – only the HTTP header is processed. But we still need to include it in our code, because our context must implement basic context features.

Now let’s get down to work – let’s implement httpcontext. In fact, we only need to implement one method:HttpContext :: on_http_request_headers-Verify the header when it arrives:

Extend envoy through wasm and rust

As you can see – the method callsself.get_http_request_headerTo find the header named “token” and check its priority. If the token is prime, thenself.resume_http_request()Pass the request further to the target cluster. Otherwise, 403 response will be returned, prompting “access forbidden”. The method returns an action enumeration that tells envoy whether to continue the request processing or to pause and wait for the next request.

Test filter

Although it’s very easy to build a filter – the work of verifying it has proved a bit painful. The official envoy binary still does not have built-in wasm support. But I assume that if I build my own envoy binary with wasm enabled – the filter should load normally. The binaries I build are based on thisGitHub repositoryBut unfortunately – Envoy crashed due to “unable to load wasm module due to lack of import”: env.proxy_ get_ configuration”

Next – I decided to usewasme CLIThe tool uses the envoy image of to test the built filter. This is a wrapper:https://quay.io/repository/so…

But this time envoy collapsed, “wasm lacks malloc / free.”. When trying to solve this problem, I opened a GitHub problem on the proxy wasm trust project. Thanks to Victor charypar, I finally realized that I should use the official proxy image of istiohttps://hub.docker.com/r/isti…. Especially the 1.5.0 tag. Once I build a mirror based on that – everything’s OK! Hooray, my filter did what I expected!

Complete the test in the docker compose file. My final project can be found here:

https://github.com/otomato-gh/proxy-wasm-rust

All you need to do is:

  • Clone code repository
  • cargo +nightly build –target=wasm32-unknown-unknown –release

    This will create the file “myenvoyfilter. Wasm” in < your clone Directory > / target / wasm32 unknown unknown unknown. The name of the wasm file is defined by this configuration in cargo.toml:

    [lib]  
    name  =  "myenvoyfilter"
  • docker-compose up –build

    This will build a new envoy image based on istio / proxyv2, and pull the hasicorp / HTTP echo image, which is used as the target service.

    You should see something like this:

proxy_1        | [2020-04-17 13:15:36.931][14][debug][wasm] [external/envoy/source/extensions/common/wasm/wasm.cc:285] Thread-Local Wasm created 4 now active


proxy_1        | [2020-04-17 13:15:36.935][14][debug][upstream] [external/envoy/source/common/upstream/cluster_manager_impl.cc:1084] membership update for TLS cluster web_service added 1 removed 0

Note that envoy is set to port 18000 of the listening host. And our new filter is loaded from our build target:

  proxy:  
   ..  
    volumes:  
      -  ./envoy/envoy.yaml:/etc/envoy.yaml  
      -  ./target/wasm32-unknown-unknown/release/myenvoyfilter.wasm:/etc/myenvoyfilter.wasm  
    ..  
   ports:  
    -  "18000:80"

Now go to another prompt and verify that everything is OK by sending a curl request:

 curl -H "token":"323232" 0.0.0.0:18000 
 Access forbidden. 
 curl -H "token":"32323" 0.0.0.0:18000 
 "Welcome to WASM land"

Feel free to try other prime and non prime numbers or strings. Let me know if you are successful.

Conclusion:

  • Wasm filter allows to extend envoy function
  • We can use rust to build wasm filters for envoy
  • Currently, the envoy binary in istio1.5 + is required to successfully execute this filter.