feat(*)!: Updates wasi-messaging interface with feedback

[thomastaylor312]

This PR incorporates various points of feedback from discussion amongst the interface champions, users of Wasm projects, and discussion in the CNCF wasm WG. This smooths over some of the rough edges and adds support for request/reply paradigms. Some of the bigger changes are:

• Addition of request/reply as an optional interface included in a separate world
• Removal of the custom format type in favor of an optional content type string
• Addition of the topic field to the message type
• Concrete errors in form of a variant

[thomastaylor312]

I took a first stab at error cases here. Please feel free to suggest more that might need to be included

[lukewagner]

I'm only starting to understand the conceptual model of this proposal, so I just had some possibly-naive questions to start with:

[thomastaylor312]

Thanks for all the great feedback! Gimme until tomorrow and I should be able to make all the changes

[thomastaylor312]

Ok I've pushed up some additional changes as discussed

[danbugs]

These changes are great. Thanks for your work, @thomastaylor312 !

Mostly LGTM, just one last note - I'd just probably not delete the examples.md document. I think it can be a useful recource for people to see how an interface like this will be used "in the real world" at a quick glance. Do you think we could bring it back or achieve that goal in some other way?

[thomastaylor312]

My plan was to actually implement something with this interface and then copy over. I deleted purely because it was completely out of date. I am also fine to put something back together once people are ok with the changes. I'll do that before we merge

[brooksmtownsend]

Overall looks nice. The new request-reply interface looks really great! Left some comments for discussion

[devigned]

I like where this is going. I have a handful of comments and questions.

[devigned]

It would be helpful to see some examples of usage with the changes to the interface.

[thomastaylor312]

Yeah, once we're in agreement with things, I am going to update this

[yordis]

Suggested change

	send: func(c: client, ch: channel, m: message) -> result<_, error>;
	send: func(c: client, ch: channel, m: message, opts: option<send-options>) -> result<_, error>;

record send-options {
  timeout-ms: option<u32>
}

I expect the timeout to be configurable. How are you envisioning the situation around it?

Are you envisioning to add the value to the message itself?

[thomastaylor312]

See this thread above for more information around ttl/timeout. Right now this only matters if you're in a request/reply

[yordis]

I wasn't thinking of the ttl message itself, but the network timeout, if you will, something around the lines of "I give you 2 seconds to tell me you have received (not process) the message" per request-based

[yordis]

Suggested change

	reply: func(reply-to: borrow<message>, reply: message) -> result<_, error>;
	reply: func(reply-to: borrow<message>, reply: message, opts: option<reply-options>) -> result<_, error>;

record reply-options {
  timeout-ms: option<u32>
}

Same over here. How do we deal with the timeout config?

[thomastaylor312]

A timeout would only matter here if you're expecting the thing you sent the reply to respond with a confirmation. More complex patterns like what Kafka Streams and NATS JetStream provide are going to be layered on top of this. Those types of patterns can handle things like acks and double_acks. Everything in this package, with the exception of request reply (where the request should have an eventual timeout), is doing basic pub sub with no explicit guarantees around deliverability

[yordis]

Suggested change

	abandon: func() -> result<_, error>;
	abandon: func(abandon-msg) -> result<_, error>;

record abandon-msg {
  reason: ...
}

I don't know if it would be worth adding some reason for abandonment. I'm just generalizing my thoughts here to see what sticks.
These would be friendly messages in the Competing Consumer model.

Maybe it is not worth having it as a core functionality; people could add it as an add-on.

Action	Description
Unknown	The client does not know what action to take. Let the server decide.
Park	Park the message and do not resend. Put it on poison queue.
Retry	Explicitly retry the message.
Skip	Skip this message do not resend and do not put in poison queue.

[devigned]

AMQP structures related to this are here: http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-messaging-v1.0-os.html#type-rejected.

[thomastaylor312]

What do we think about having this just be a string rather than a specific state? I do like having additional context for sure, but am trying to keep this as simple as possible

[yordis]

@thomastaylor312 yeah, that is why I left the type out.

[thomastaylor312]

@danbugs @devigned As I was adding the new request options, I thought about how we want to handle this. Currently, request returns a list of messages. If we are returning a list of messages, then it makes sense we might want to specify (optionally) the number of replies we expect for things like scatter/gather operations. I am not totally sold on the idea, but put it in here so it was easy to grok. I think there are some options here:

• Keep support for receiving multiple responses and have this optional expected-replies field (more complex, but more flexible in what you can use it for)
• Make the request method only return a single message (this is the simplest, but does limit you to more of a client-server type pattern)
• Maybe something in between? I can't quite think of one, but I don't have a monopoly on thoughts around messaging 😅

[danbugs]

Make the request method only return a single message (this is the simplest, but does limit you to more of a client-server type pattern)

I think this would just cause people to make wrappers around the single reply mechanism for multiple-reply, so I'd avoid it.

Now, going over your other suggestion, I think the main question we should be asking is: What if the implementor doesn't really offer support for said feature? How should the implementor behave if, regardless, the field is set? And I'm not just saying this about expected-replies, but other opts too like timeout-ms. Rather than having options all together, I'm debating if we wouldn't be better off w/ entirely different functions, so that an implementor can make an explicit choice about implementing them or not. What are your thoughts, @thomastaylor312 ?

[thomastaylor312]

If people wanted to opt out of implementations, each one of those functions would have to be a separate interface, which could get messy. I also feel like that no matter where each of those functions live, it would start to get really annoying to have to find the right function for each thing you're doing, but that is more of a minor gripe to me than the former. So my initial thought is that I don't quite think that would work.

The comment about options is interesting. A timeout value is simple enough that even if a client doesn't natively support it, it is really easy to add that via code. So I feel like that is fairly easy. I could say the same about expected replies, but I can also start to see the slippery slope here. At what point do additional options cross over into "yeah that is going to require custom logic for every implementation." Anyway, if my first answer is correct in that many functions would be a bad idea (which it could not be), then the best path forward would be to use options but to add even more documentation that we must be very careful adding anything else to it

[yordis]

What are the expectations around the Message ID? Such metadata is so critical that it would be good to have one way to figure this out.

[thomastaylor312]

Is a message ID actually important to a guest here? Most of the time those are used in acks or other advanced operations. Each implementation can take the wit type message and convert it to/assign it an ID based on implementation details IMO.

[yordis]

From the guest's perspective, I think it is critical for the guest to control the identity of the message, the guest is most likely the component aware of the domain, while the host is at the platform level.

What the host would do with it shouldn't be part of the spec. That is a different story. Still, guest could find a component that provides the guarantees required around the dedupe and whatnot.

It's worth saying that the ID doesn't mean unique; it means an identifier.

[yordis]

For whatever is worth, "messaging.message_id" OpenTelemetry Semantic Conventions Trace exists. Just to point to the important of such information

[yordis]

Suggested change

	constructor(topic: topic, data: list<u8>, content-type: option<string>, metadata: option<list<tuple<string, string>>>);
	constructor(topic: topic, data: list<u8>, opts: option<message-options>);

record message-options {
  content-type: option<string>
  metadata: option<list<tuple<string, string>>>
}

I feel it will be prudent to avoid breaking changes in the function signature

[thomastaylor312]

@lukewagner I'm curious on your thoughts here on how constructors should be handled. In particular:

1. Is it idiomatic to have another record type you use in a constructor?
2. Are there any other considerations around breaking a constructor API that we should take into account here? At this point I don't think the @since and @unstable would necessarily help here

[lukewagner]

To carve out a way to add optional fields in a semver-compatible way (at least in a 0.2 timeframe before we relax subtyping), you can use a resource type with methods for each field, which is what wasi-http does with request-options.

[lukewagner]

Changes are looking good! Just a few more ideas/suggestions:

[lukewagner]

I think readability might be enhanced by renaming this function to something like set-subscriptions: func(topics: list<string>) -> result<_, error>; (and deleting config). Then, if we want to add other kinds of configuration in the future, we can just add other functions (which is a semver compatible change, unlike, atm, adding fields to records). Also, do we have to worry about the list of topics growing large, and then each time you add 1 topic, it's an O(n) operation? If so, there could be add-subscription, remove-subscription, etc. But it's also fine to wait and see if this is a problem.

[lukewagner]

Nit: to be symmetric with wasi:http, how about renaming the guest interface to incoming-handler, and renaming the handler function to handle?

[lukewagner]

Given that handler now takes a single message, could we remove the complete/abandon methods from the message resource and instead use the return value of handler to indicate completion/abandonment? This has a few advantages:

• it avoids the cross-product of spec-questions of what happens if the message was { completed, abandoned, neither } x and the result of handler was { ok, error, trapped }, all of which otherwise need precise answers
• it avoids questions of how complete and abandon interact with request-reply
• it's more like normal success/failure control flow

[thomastaylor312]

I'm glad you brought this up. I actually started doing it this way and then was unsure if it was a good idea. Now that it also came from someone else I feel more confident 😅

[lukewagner]

I'd suggest copying request-options in wasi-http and making this a resource, with methods to set/get each field. This has the advantage of allowing semver-compatible field additions over time.

[lukewagner]

I think you need c to be a borrow<client>. Also, it seems like expected-replies is a pretty essential part of how this function works (it explains the list in the return type) so maybe it should be a proper parameter instead of being "hidden" in request-options? It sounds like maybe it's not common to use a value other than 1, so what about improving the ergonomics of the common case with:

request: func(c: borrow<client>, m: message, opts: option<request-options>) -> result<mesage, error>;
request-list: func(c: borrow<client>, m: message, expected-replies: u32, opts: option<request-options>) -> result<list<message>, error>;

[lukewagner]

What if instead of reply being an imported interface, it was instead an exported interface:

interface incoming-request-reply-handler {
  handle: func(msg: message) -> result<message, error>;
}

Here, the reply is just the success case of the result. This can avoid some of the weird spec interaction questions like I was pointing out in my other comment on complete. Thus, there could be a request-reply world that looks like:

world request-reply-messaging {
  include messaging;
  import outgoing-request-reply-handler;
  export incoming-request-reply-handler;
}

[lukewagner]

I've generally found that type definitions like this that assign a name to string mostly just add indirection that the reader has to chase down without providing any real abstraction (e.g., it's a semver breaking change to attempt to change topic to equal any other type, and generated bindings will fully expose the fact that topic is actually a string). So I'd suggest deleting topic and relying on the parameter/field names to convey topic-ness.

[lukewagner]

To carve out a way to add optional fields in a semver-compatible way (at least in a 0.2 timeframe before we relax subtyping), you can use a resource type with methods for each field, which is what wasi-http does with request-options.