Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make sure only one import is performed at the time #258

Open
niconoe opened this issue Sep 11, 2023 · 1 comment
Open

Make sure only one import is performed at the time #258

niconoe opened this issue Sep 11, 2023 · 1 comment
Assignees

Comments

@niconoe
Copy link
Member

niconoe commented Sep 11, 2023

We recently had performance issues that made the import take too long (more than 24hrs for RIPARIAS), therefore the import process is restarted the next day while the first one is still in process, this is something to be avoided.

@niconoe niconoe self-assigned this Sep 11, 2023
@niconoe niconoe added the enhancement New feature or request label Sep 12, 2023
@niconoe
Copy link
Member Author

niconoe commented Sep 12, 2023

After a quick investigation, I didn't found any integrated mechanism nor easy solution for that. Possible solutions:

  • create a file / an entry in the database / a Redis key to keep track of the running status (similar to Django maintenance mode)
  • actually use the status of django-maintenance-mode (skip import if already in maintenance mode) => doesn't seems great since we only put in maintenance mode in the middle of the process. Also, the site could be in maintenance mode for other reasons than a running import.
  • Check if using RQ's scheduler feature (in place of cron) could help avoiding the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant