Some meows can be missing from the query tier

[arnaud-lb]

Hi!

Thanks for this article, and especially for this repository! Some things are talked about a lot, yet there is not much code to read.

I've found some issues in the published code: It seems that the query tier can get inconsistent quite easily: Some meows may not be available through the query tier.

For instance, here an event is published through NATS (I believe), but if something goes wrong, the query tier will never hear of this meow.

Later, in the query tier, an attempt is made to index a meow in Elasticsearch. If it fails, the neow will never be available in search.

This is quite critical, because it will happen during any of these events:

• NATS is unreachable from meower (temporary network hickup) while trying to send a message
• NATS is down temporarily while trying to send a message
• query is unreachable from NATS (temporary network hickup) while trying to forward a message to it
• query is down temporarily while trying to forward a message to it

[tinrab]

Yes, all this can happen. It could be resolved with a saga. I have a possible implementation on my To-Do list 😸

[ghost]

Would event store pattern help ?

Example:
https://medium.com/@ivanjaros/diy-startup-simple-event-store-with-boltdb-e8146d386753

[tinrab]

@gedw99 Yes, if you can guarantee that events will be delivered at least once. If the event gets properly handled on the query side, then the delivery can be considered successful.

[ghost]

Recently i started experimenting with the whole issue of co-ordination or data ( and single source of it) using CRDT and automerge. Its still not 100% solved architecturally though.