-
Notifications
You must be signed in to change notification settings - Fork 928
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug] Grafting can skip copying data sources #5465
Comments
The fix might be simple: the whole of copying private data sources runs outside of a transaction. We should just wrap all of it in a txn. |
@lutter those transactions could run for days, couldn't they? |
Copying private data sources should be very quick; at the very worst, it's a table with 100k rows, but usually much less. The actual data copying, which can take days, is already broken into txns that should take about 3 minutes |
Ah, understood, thanks! |
We've seen subgraphs up to 1.9 million data sources. Not sure if that matters at all for how this should be handled. |
True; I just looked through the code again, and it does one But we need to first address correctness, which to me is more important than performance issues, though they can be devastating, too. |
Bug report
The following sequence of steps will lead to a grafted (or copied) subgraph with missing dynamic data sources:
If at step (2), the graft had already copied data sources, and the rewind doesn't remove all of them, upon resuming, we will skip copying data sources because of this code
The result will be a graft that seems to be ok, but is missing dynamic data sources.
The text was updated successfully, but these errors were encountered: