Our benchmark reveals a WASM VM that runs with minimum overhead

Harshil Jani

— 2 min read

In our last blog, We ran the benchmarks on Wasmi and TinyWasm to find out the overhead of instantiating a WASM module. We found that the overhead was around 1.5 Mb for TinyWasm and 3.2 Mb for Wasmi. Which was high for a simple module.

We got to learn about optimization techniques from other rust developers. One of it was about running the binaries in release mode with --release to find the actual benchmarking.

Apart from it, we have also included benchmarks for running n number of wasm instantiation altogether on a same server on different threads. And to our benchmarks, we tried with another WASM VM which is WAMR (Web Assembly Micro-Runtime).

See full code here

Observations

For all three WASM VM, We ran it for 10 module instantiation in different threads and with the release mode. Here's what we found the overhead to be for each one of them.

Tinywasm

Running 10 Instances of TinyWASM
Memory Start 3012 kB
Memory After 3636 kB
Wasm Overhead 0 kB
Memory After 3636 kB
Wasm Overhead 496 kB
Memory After 3636 kB
Wasm Overhead 0 kB
Memory After 3636 kB
Wasm Overhead 624 kB
Memory After 3636 kB
Wasm Overhead 624 kB
Memory After 3636 kB
Wasm Overhead 184 kB
Memory After 3636 kB
Wasm Overhead 0 kB
Memory After 3636 kB
Wasm Overhead 0 kB
Memory After 3636 kB
Wasm Overhead 624 kB
Memory After 3636 kB
Wasm Overhead 0 kB

The Overhead was between 496kB-624kB per module.

Wasmi

Running 10 Instances of Wasmi
Memory Start 3052 kB
Memory After 4076 kB
Wasm Overhead 1024 kB
Memory After 4204 kB
Wasm Overhead 384 kB
Memory After 4204 kB
Wasm Overhead 128 kB
Memory After 4204 kB
Wasm Overhead 128 kB
Memory After 4204 kB
Wasm Overhead 0 kB
Memory After 4204 kB
Wasm Overhead 0 kB
Memory After 4204 kB
Wasm Overhead 0 kB
Memory After 4204 kB
Wasm Overhead 0 kB
Memory After 4204 kB
Wasm Overhead 0 kB
Memory After 4204 kB
Wasm Overhead 0 kB

The Overhead was relatively very low for wasmi between 128kB-1024kB. With that we found that the initial load up is bit high and then relatively its coming less with mode modules initialized.

Wamr

Running 10 Instances of Wamr
Memory Start 2864 kB
Memory After 3120 kB
Wasm Overhead 256 kB
Memory After 3248 kB
Wasm Overhead 128 kB
Memory After 3248 kB
Wasm Overhead 0 kB
Memory After 3248 kB
Wasm Overhead 0 kB
Memory After 3248 kB
Wasm Overhead 0 kB
Memory After 3248 kB
Wasm Overhead 0 kB
Memory After 3248 kB
Wasm Overhead 0 kB
Memory After 3248 kB
Wasm Overhead 0 kB
Memory After 3376 kB
Wasm Overhead 128 kB
Memory After 3376 kB
Wasm Overhead 128 kB

The overhead for WAMR is the least compared to other VMs and it is between 128Kb-256Kb and it takes only 3MB of RAM to run the entire program.

We had the questions in last post about

  • Is the RAM goal of 10 MB realistic?
  • Is there any VM that will allow embedded language using only 500 kb or less?

With our benchmarks, we can be sure that yes it is possible for running a CMS under 10 MB of RAM. And WAMR is one such VM that takes least RAM space and also the least overhead for initializing same number of module.

Up Next

We have decided to move ahead with WAMR as our WASM VM. Now we need to benchmark for different HTTP Servers and a DB pool. That will help us get an overall idea of RAM requirements.

Read more