Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jube continue has quadratic complexity without specific purpose #4

Open
jprotze opened this issue Aug 23, 2023 · 1 comment
Open

jube continue has quadratic complexity without specific purpose #4

jprotze opened this issue Aug 23, 2023 · 1 comment

Comments

@jprotze
Copy link

jprotze commented Aug 23, 2023

jube continue seems to check done/error files in all work package directories while processing one work package (according to strace). This results in at least 2n^2 file accesses when only 2n should be necessary.
For a larger experiment space and slow meta data server, this can result in significant execution time.

@wellmann99
Copy link

I had a quick look at the problem using a few examples. It turns out that the done and error file check happens more often than expected and needed, and even more often than described here. However, due to the complexities involved, we need a little more time to analyse and fix the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants