Working Go code from Elixir Utilizing Net Meeting

I used to be engaged on an Elixir undertaking lately and one explicit open supply package deal that I needed to utilize was written in Go. I searched for various methods to interface with Go code from Elixir. I do know that it’s pretty trivial to interface Rust code from Elixir utilizing a NIF however there isn’t something comparable for Go that I got here throughout. Most of my analysis led me in the direction of utilizing Ports and I used to be about to offer in earlier than realizing that WASI (WebAssembly System Interface) can also be an possibility. I had been listening to about Net Meeting for some time now and this was nearly as good of a second as any to offer it a attempt. I additionally got here throughout Wasmex which appeared like a great choice to run WASI binaries in Elixir. Philipp Tessenow (creator of wasmex) was instrumental in getting this complete factor to work and guiding me to the ultimate working resolution so thanks tessi!

There have been a number of issues I wanted to determine:

1. Easy methods to compile Go code to WASI
2. Easy methods to run a WASI binary in an Elixir host
3. Easy methods to move information into the WASI binary
4. Easy methods to get information out of the WASI binary

Let’s undergo them one after the other.

1. Easy methods to compile Go code to WASI

This one was simple. Some fast Google-fu knowledgeable me that I want to make use of tinygo because it helps WASI as an output goal. The primary Go compiler additionally added WASM assist however so far as I do know, it doesn’t at the moment compile to the WASI goal.

That is the pattern Go code that I used for testing this complete setup:

package deal predominant

import "fmt"

func predominant() {}

//export greet
func greet() {
  fmt.Println("Hiya world")
}

And I used the next command to compile the code to WASI:

$ tinygo construct -o predominant.wasm -scheduler=none --no-debug -target wasi predominant.go

Tinygo makes use of feedback to information the compiler concerning which capabilities have to be exported to a WASI host. One such remark is // export greet within the code above. This may export the greet operate and we are able to name it from the WASI host (Elixir/Wasmex). There are additionally some annotations you’ll be able to add to inform tinygo to import a operate from the WASI host however I’m not going to be utilizing that right here. We additionally want so as to add an empty predominant operate for the code to work.

You possibly can mess around with the tinygo flags within the construct command I shared above. I discovered this mix of flags to generate the smallest WASI/WASM binary.

2. Easy methods to run a WASI binary in an Elixir host

This one was additionally pretty simple to determine. After you have wasmex put in as a mixture dependency, you’ll be able to run a WASI binary like this:

binary = File.learn!("./native/predominant.wasm")
{:okay, pid} = Wasmex.start_link(%{bytes: binary, wasi: true}) 
Wasmex.call_function(pid, "greet", [])

3. Easy methods to move information into the WASI binary

Now I had to determine move information into the WASI binary. This one was a bit tough as there are a few methods to go about this. The 2 most well-known ones are passing information in through stdin or copying information to some shared WASI reminiscence. You can even parametrize your capabilities and take direct arguments like this:

func add(a, b int32) int32 {
	return a + b
}

Nonetheless, this doesn’t work with strings. WASI specification at the moment doesn’t assist string inputs. As a result of I needed to move some strings as enter, I couldn’t merely parametrize my operate like this.

I attempted to utilize WASI reminiscence and move enter by that technique utilizing a bunch of completely different tutorials however wasn’t in a position to get wherever. I used to be getting all kinds of errors and since I used to be new to the world of Net Meeting I made a decision to go for the trail of least resistance and ended up utilizing pipes.

Right here is how one can move binary enter to Go utilizing pipes:

binary = File.learn!("./native/predominant.wasm")
{:okay, stdin_pipe} = Wasmex.Pipe.new()
wasi = %Wasmex.Wasi.WasiOptions{args: [], stdin: stdin_pipe}
{:okay, pid} = Wasmex.start_link(%{bytes: binary, wasi: wasi})

Wasmex.Pipe.write(stdin_pipe, "Hiya world!")
Wasmex.Pipe.search(stdin_pipe, 0)
{:okay, []} = Wasmex.call_function(pid, :greet, [])

This code will put "Hiya world" within the stdin pipe which you can learn from the Go facet in your greet operate like this:

func greet() {
    // Right here information will comprise "Hiya world"
    information, _ := io.ReadAll(os.Stdin)
}

4. Easy methods to get information out of the WASI binary

Much like how you utilize a stdin pipe, you can also make use of a stdout pipe. Let’s first modify the Go code to output the greeting to stdout:

import "fmt"

func greet() {
    information, _ := io.ReadAll(os.Stdin)
    fmt.Println("👋 Knowledge from Elixir:", string(information))
}

Now, let’s make use of the stdout pipe:

binary = File.learn!("./native/predominant.wasm")
{:okay, stdout_pipe} = Wasmex.Pipe.new()
{:okay, stdin_pipe} = Wasmex.Pipe.new()
wasi = %Wasmex.Wasi.WasiOptions{args: [], stdout: stdout_pipe, stdin: stdin_pipe}
{:okay, pid} = Wasmex.start_link(%{bytes: binary, wasi: wasi})

# Put information in stdin pipe
Wasmex.Pipe.write(stdin_pipe, "Yasoob right here!")
Wasmex.Pipe.search(stdin_pipe, 0)

# Name the greet operate
{:okay, []} = Wasmex.call_function(pid, :greet, [])

# Learn information from stdout
Wasmex.Pipe.search(stdout_pipe, 0)
IO.places(Wasmex.Pipe.learn(stdout_pipe))

This could print 👋 Knowledge from Elixir: Yasoob right here! on the display screen.

Conclusion

I had enjoyable studying about Net Meeting and determining that it isn’t too onerous to name Go code from Elixir utilizing WASI. I’m fairly positive there should be some caveats to this method because the complexity of the Go code will increase however for most elementary use instances this can be a excellent resolution. I haven’t benchmarked the reminiscence utilization of this resolution however that isn’t a priority for my undertaking at this stage. If push involves shove, I can at all times go ahead with a Ports-based resolution.

This small endeavor made me notice that even when Elixir doesn’t have a related library or package deal, I can look in the direction of Go as effectively and interface Go from Elixir. Rust was at all times an possibility through Rustler however now Go can also be a viable possibility if one thing doesn’t exist in Rust both (as was the case in my undertaking).

I hope this text helps those that try to do one thing comparable 😄