Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Celery Progress Bar #41

Closed
WayneLambert opened this issue Jul 1, 2020 · 5 comments
Closed

Celery Progress Bar #41

WayneLambert opened this issue Jul 1, 2020 · 5 comments
Assignees
@WayneLambert
Labels
app: scraping Issues relating to the `scraping` app enhancement New feature or request feature Feature for the project Stale Issue

Comments

@WayneLambert
Copy link
Owner

The results could be scraped more quickly using Python's multiprocessing module.

The app is also a good candidate to implement a progress bar.

Tutorial / Package: Celery Progress Bar with Django
@WayneLambert WayneLambert self-assigned this Jul 1, 2020
@WayneLambert WayneLambert added app: scraping Issues relating to the `scraping` app enhancement New feature or request feature Feature for the project labels Jul 1, 2020
@github-actions
Copy link
Contributor

github-actions bot commented Aug 4, 2020

This issue is stale since it has been open for 10 days without any activity. If the 'Stale Issue' label hasn't been removed or a comment made on the issue, it will be closed after 5 days of inactivity.

@github-actions github-actions bot closed this as completed Aug 9, 2020
@WayneLambert WayneLambert reopened this Aug 13, 2020
@WayneLambert
Copy link
Owner Author

The celery bar feature cannot be implemented just yet because it depends upon installing celery. The package django-google-analytics-app is not yet updated to support celery >=4.0, therefore an update is first required to it so that PipEnv can resolve the dependencies.

A PR has been submitted to initiate the process of updating the package.

@WayneLambert WayneLambert reopened this Aug 19, 2020
@WayneLambert WayneLambert mentioned this issue Aug 24, 2020
Closed
@WayneLambert
Copy link
Owner Author

This issue can now be closed since an alternative caching solution was implemented.

Refer to issue #60

WayneLambert added a commit that referenced this issue Aug 25, 2020
@WayneLambert
Update timeout for Redis caching
ddefca5 
The previous commit implemented caching, however seemingly the caching
expired.

Updates the global `CACHES` setting to include a `TIMEOUT` setting.

Contributes towards: #60
Eliminates requirement for: #41
@WayneLambert WayneLambert reopened this Sep 15, 2020
@WayneLambert WayneLambert pinned this issue Sep 15, 2020
@WayneLambert WayneLambert changed the title EU Referendum Results Celery Progress Bar Sep 15, 2020
@WayneLambert
Copy link
Owner Author

3 environment variables will need to be added to the config vars in Heroku.

CELERY_BROKER_URL
CELERY_RESULT_BACKEND
CELERY_BACKEND_URL

WayneLambert added a commit that referenced this issue Sep 15, 2020
@WayneLambert
Update Scraping App
7d0105f 
Addresses the following:

- Removal of redundant templates
- Updates to development settings/config

Linked Issue: #41
@github-actions
Copy link
Contributor

This issue is stale since it has been open for 10 days without any activity. If the 'Stale Issue' label hasn't been removed or a comment made on the issue, it will be closed after 5 days of inactivity.

@github-actions github-actions bot closed this as completed Oct 1, 2020
@WayneLambert WayneLambert unpinned this issue Oct 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@WayneLambert WayneLambert

Labels
app: scraping Issues relating to the `scraping` app enhancement New feature or request feature Feature for the project Stale Issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@WayneLambert