Idea: Friendly Streams

[Stebalien]

Libp2p has streams, but they're really annoying:

1. They take up resources when not in use, so we usually treat them as ephemeral. But this gets really annoying when we want to send ordered messages on a stream that's usually idle (gossip, etc.).
2. They don't migrate across connections (proposal in Add a Stream Migration spec #406).
3. They won't re-try and/or re-form on failure. We have quite a bit of code to try sending a message on a stream, only to open a new stream and re-try.

We should consider adding user-space stream layer that can:

1. Suspend the underlying transport stream to free all buffers.
2. Migrates and re-creates streams.

[Stebalien]

Hm. Well, this would require leaving goroutines open. Which isn't great.

Maybe we're really just looking for in-order messages of some form?

[Wondertan]

Hey @Stebalien. I've been working on a Go-specific side project https://github.com/celestiaorg/go-libp2p-messenger, which solves most of the annoying points you've mentioned:

• It does migrate across connections. Also gracefully handles duplicates.
• It re-creates streams on failure and even keeps msgs around to be resent automatically on re-create.
• It abstracts away streams with minimalistic message API and manages stream lifecycles. Hence controls resources allocated for streams

Other points about Messenger:

• It is user space and is supposed to be an instance per protocol
  • Keeps two routines around for state management
  • Keeps two routines per each peer/stream, but we can easily clean up idle streams: Cleanup idle streams celestiaorg/go-libp2p-messenger#24 and free up resources, so effectively, for protocols like identify, it will only take two state management routines most of the time.
• Inspired by stream management from go-libp2p-pubsub, but vastly improves it
  • Decouples peer/stream management from main processLoop
  • Decouples inbound and outbound msg and stream processing
• Works with plain streams and does not have any footprint on the wire

[Stebalien]

That's exactly what I'm looking for. But I would consider suspending idle streams because we currently have to reserve fairly large receive buffers per-stream (I think?).

[Stebalien]

Ah, wait, that's what your issue is referring to.

[Stebalien]

Are there any ordering guarantees?

[Wondertan]

Yes, as I use native channels there atm

(If I understood your question correctly)

[Wondertan]

(if there are any messenger issues or feature requests, I am happy to collaborate/implement them)