Would rewrite be worth it? just a distant thought

[sleipnir]

https://github.com/ergo-services/ergo

[marcellanz]

A rewrite of Massa in Go?

[sleipnir]

Yes, just a mental exercise on my part.
I believe in the power of erlang and the current implementation, the current weak point I think lies in the grpc that is not doing well in the benchmarks out there.
But this Ergo framework does a great job replicating the principles of Erlang in Go and I was wondering what it would be like to see it running in production.

[sleipnir]

But I don't intend to invest in Go in the short term, obviously from the point of view of our kubernetes controller it would be possible to run multiple proxies implementations, including more than one in the same cluster

[marcellanz]

I think this is a great question, really. I have many thoughts, let me collect them...

[sleipnir]

...I believe in the power of erlang and the current implementation, the current weak point I think lies in the grpc that is not doing well in the benchmarks out there...

Obviously I'm sure that if we invest a little time in grpc I think we could improve performance, basically from what I've seen from the code there is an excess of staggered processes to meet a single request.
If @wesleimp wants to spend the time on this, I've already forked the project on behalf of our organization and I know more or less the main points that would have to be changed, but unfortunately I don't have time to check that out right now. I'm focused on the release of our first release.

[marcellanz]

Would rewrite be worth it?

I'm not sure about the motivation. Let me know, it might be different from what I currently see why Erlang/Elixir and the BEAM fit.

I really wanted to write the proxy in Go, @sleipnir you might remember helogale. I love Go and for me from a tools and language perspective I'd really like to write our runtime in Go. On the other hand, the reason why we've chosen the BEAM was that it brings a lot "included" with it when it comes to building a distributed system able to scale massively. For many years the Erlang ecosystem did not get my attention at all. For certain aspects, Erlang as a language seems "too strange", its ecosystem perhaps a bit outdated, but all languages and stacks have their drawbacks. One could wish Erlang has a more "modern" base and one could wish perhaps is not implemented in C/C++, but so many system still are. Scared to get the VM crashed on the C/C++ level? => that is possible with many other systems. Go and Rust bring that fresh air or green-field feeling that really is able to abandon C/C++ codebases. For Erlang this might happen or not, later or possibly never. Like with the Linux Kernel, I'd like to see an incremental model where parts of it can be implemented in Rust over time.

For many coming to Erlang, is Erlang, OTP and the BEAM too strange? With Elixir our language of choice so far, that more modern feeling, like what probably Akka felt when they copied all the good parts of Erlang and implemented their "OTP in Scala", enables us to get attention of a broader community with its more pleasing style as a language. Its base is given and obviously very solid. Who knows where Erlang could be if Bonér would have been able to choose Erlang and not had to copy-port it to the JVM. No question, Akka is a high quality piece of technology, but it also copied many principles of Erlang to built their Actor based system.

I still struggle a bit with Elixir, it gets better, but I'm very used to have a statically typed language at hand – I'll get to used to it I think :) I even sometimes like Erlang better because of its simpler structure as a language.

So in term of our choice of Erlang, why not copy the good parts of Akka and bring them to the BEAM as it was done some years ago the other way around. Then use its capabilities to proof we can build it on top of OTP and the BEAM and be productive to implement all the features we envision for the runtime. I don't see us building this from the ground up yet other than perhaps "special proxies" for certain use-cases like some performance or size constrained environments, where a proxy in Go or Rust fits better; think of "just" an Action–Protocol proxy. Otherwise, I see the BEAM to have too many advantaged out of the box. Then I see Erlang and its architecture to be such an obvious match to implement a FaaS Serverless runtime with the state models we aim for. It's the modern "Telephone Switch" in the era of the Cloud- and Serverless-Computing used as a control-plane at first and its data-plane.

References of alternative Erlang VM efforts.
https://github.com/WhatsApp/erlt
https://github.com/rusterlium/rustler
https://github.com/archseer/enigma
https://news.ycombinator.com/item?id=23031379
https://www.michielrook.nl/2016/11/strangler-pattern-practice/
https://github.com/kvakvs/ErlangRT
https://www.youtube.com/watch?v=e7T6p8ynVuM

[sleipnir]

Hi Marcel, thanks for your reply. I see you've spent some time thinking about this and that's very good.
I agree with most of your arguments and I have no doubt that we made an excellent choice with Elixir and BEAM, elixir is without a doubt the most powerful language we currently have and BEAM is an exceptional piece of software when it comes to resilience, concurrency and distribution . There is no doubt that we are fine with our choices.
What motivated me to open the discussion is because I like to do the mental exercise of thinking about other ways to solve a certain problem, in this way I expand my way of thinking and seeing the world.
One thing I like to point out is that I wouldn't feel the urge to change the language or runtime if it weren't for keeping the same philosophy from which BEAM comes from, that is, the actor model. So the only viable alternative possibilities I see today are Ergo (Go) or Lunatic (Rust WebAssembly).
And yes, we could have specific proxy versions to satisfy certain tasks (edge ​​computing, high network performance, etc), and in that sense I think Ergo is very worth studying because its compatibility with the Distributed Erlang protocol brings the advantage of being able to use a mixed configuration of proxies participating in the same cluster.

Anyway I just wanted to open up to more thoughts.

Note: I'm not a type-addicted developer and I don't think this would be a reason to adopt a proxy language or not, as I've spent time with languages ​​that use both static and dynamic typing and have gotten used to both styles. But I tend to agree with Matz, creator of Ruby, that we should stop writing programs using machine languages ​​and use languages ​​with more natural characteristics, therefore, expressing like I consider an excess of zeal with the cpu, despite knowing that the possibility of using explicit types will help in debugging problems. I can finally sleep peacefully with types or without types.
But I understand your point with types and I find this type of questioning natural in learning a new programming paradigm

[sleipnir]

Opening a new parenthesis that might not fit in this discussion, I think we can try a mixed approach with Elixir and Gleam, so we can easily add types to the project. We can keep all the OTP part of the project like Applications, GenServers, and Supervisors in Elixir and use Gleam to define the service modules.
I think this is a productive approach but it would require a big refactor, we can try this after we release the Action protocol.
Wdyt?

[sleipnir]

@wesleimp I would love to hear your ideas too

[sleipnir]

A point I touched on briefly would be about the ability to schedule different proxies to run on the same cluster, in this sense we could use k8s tenants and labels to deploy a version of the proxy written in Go on Edge Nodes (suddenly the client uses something like kube-edge), and also schedule workloads with the proxy using Lunatic (rust) on this same cluster on Wasi Nodes (with krustlet), or even running the browser with something like Lumen and those Nodes running specific proxies could live with the default proxy written in Elixir on conventional Nodes.
I think that opens up to interesting possibilities.

[ralphlaude]

Interesting discussions and thoughts. I haven't spent time on this topic. But I think it would be better to stick with the Erlang Ecosystem. It is nice to try to mix it with Go and Rust and I am not sure we have the capacities to deal with that now. This is my humble opinion on this.

[sleipnir]

Interesting discussions and thoughts. I haven't spent time on this topic. But I think it would be better to stick with the Erlang Ecosystem. It is nice to try to mix it with Go and Rust and I am not sure we have the capacities to deal with that now. This is my humble opinion on this.

Hi @ralphlaude, it's always nice to see you interacting with us. Thanks for your comment.

The point I would like to make here is that the motivator of my post is for us to continue to work out in our minds new challenges and new ways of thinking as we build our project. In fact, we are not going down that path at the moment, as you mentioned, nor do we have the technical conditions to do so now. But this does not prevent us from imagining, and from elaborating new future or hypothetical paths, and this mental exercise is what I came to propose in this discussion here.

New ideas are always welcome ;)