Restarted from enhancement to aggregation script and small fixes

[scrasmussen]

TYPE: bugfix

KEYWORDS: precipitation, restarts, ideal tests, Python requirements

SOURCE: Soren Rasmussen, NCAR

DESCRIPTION OF CHANGES:

• fixes to aggregation process to prevent mixing output data from runs started at different times:
  • added restarted_from global attribute to NetCDF files
  • before aggregating output files, aggregation script will:
    1. delete aggregated files created on and after restarted_from date
    2. if restarted_from is equal to Not Restarted, delete all aggregated files
• small fixes to scripts that generate the ideal testing files icar_options.nml, init.nc and forcing.nc
• git will ignore files produced by ideal test
• added instructions and files for installing ICAR Python tool dependencies
• Tiedke to Tiedtke spelling fixes
• removed .travis.yml since Travis CI is no longer used
• this PR has same changes that are in Issue 182 Fix: Xarray Future Warning #183

ISSUE: Fixes #180 and negative precip

TESTS CONDUCTED: Used the ideal testing files to do runs testing the new features in the aggregate script. Tested behavior of script with different restarted_from NetCDF values. Ran new aggregate script on old outputted files that had negative precip and it fixed the issue.

NOTES: Optional, as appropriate. Delete if not used. Included only once for
new features requiring several merge cycles. Changes to default behavior are
also note worthy.

Checklist

• Closes issue code error in helpers/genNetCDF/Topography.py #180
• Tests added (unit tests and/or regression/integration tests)
• Backwards compatible
• Requires new files? If so, how to generate them.
• Fully documented

[gutmann]

I think we agreed that matrix-nio could be removed from the requirements/environment, can you do that and anything else you made a note of when we talked last week?

[scrasmussen]

Hi @gutmann, sorry for the slow changes, took three days of PTO so was a bit slow getting to this but the things we talked about should be encapsulated in these new commits. I changed the README to describe the additional Python package requirements.

I should note that the Bunch package is 12 years old and doesn't support Python 3. I can look at updating the make_domain.py script if you'd like.

[gutmann]

Thanks, if I recall correctly, "bunch" is a ~10 line python module... either it could be updated or we can just drop it. I suspect none of the scripts that are actively used anymore still use it. Let's just move all the scripts that do rely on it to some sort of deprecated archive folder for reference. I realize we could just delete them and they will still exist in git, but nobody would find them if we do that, and they could still be useful for someone to modify for future projects.

[gutmann]

Weird, looking at that bunch repo, it has a python3 compatibility, does that not work? Also, funny that the core 10-line functionality has now ballooned into 400+ lines of code with all the comments, yaml/json support, etc. Useful, but then with just 10-lines I knew what it was doing 🙃

[scrasmussen]

PR should be good to go, here is a summary of changes since there were a lot of things stuck into this one:

• Output Changes
  • Added restarted_from attribute
  • Bundled init statements for output/restart objects into single init
    subroutine call: main/driver.f90:75, output_obj:9-19
• Aggregate Scripts
  • Added restarted_from attribute to NetCDF files
  • aggregate_parallel_files.py:246 delete if after restarted_from data,
    else just print a warning
• Other Python Scripts
  • README with dependency and setup instructions for Python Scripts
  • Pandas dependency removed
  • Small fixes to run with updated packages
  • Will use local Bunch and other packages in helpers/lib directory
• removed Travis CI file
• Tiedtke name fix