Inter-repository (team) dependencies

[heliopais]

Hi Dagster team,
First of all, I would like to thank you for all the great work done so far on Dagster! Our team has been having a lot of fun trying it out.

Our general question is about how Dagster is meant to be used by different teams in the same organisation, which are developing their pipelines independently. Reading the docs it feels like each team should have its own repository, which are then combined in a workspace (as shown in the Multi-location Workspace example).

In this setup, let's say that we have two teams, and that a solid in Team A's pipeline takes as input one of the outputs produced by a solid in Team B's pipeline. How would we have Dagster make Team A's pipeline "wait" for the necessary outputs from Team B's pipeline to be produced?

Or maybe we are thinking about the problem in the wrong way and there is a better setup for this use case?

[alangenfeld]

How would we have Dagster make Team A's pipeline "wait" for the necessary outputs from Team B's pipeline to be produced?

Great question, at this moment there is not really a good way to do this but its a feature we are currently working towards. Aspiring to have something for 0.10.0 which is should release around end of year.

[heliopais]

Thanks for the answer, and glad to hear we may have this supported soon! Awaiting (even more) eagerly for 0.10.0