Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[cosmos-admin] Script in charge of cleaning the staging HDFS folders #102

Open
frbattid opened this issue Oct 8, 2015 · 0 comments
Open

[cosmos-admin] Script in charge of cleaning the staging HDFS folders #102

frbattid opened this issue Oct 8, 2015 · 0 comments
Labels
backlog

Comments

@frbattid
Copy link
Member

frbattid commented Oct 8, 2015

Some times, the /user/any_user/.staging folder, a temporal location for MapReduce job data, is not cleaned once a job has finished (it seems to be a bug). This, in the mid/long term, consumes a lot of HDFS space.

It is necessary a script in charge of cleaning these staging folders. It is very important to delete the temporal data for those MR jobs not running at the current moment. Something similar to this or this.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
backlog
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@frbattid