Removing traces of Old Commondata

[Radonirinaunimi]

Once #1678 and #1708 are merged, we should remove all the traces of the old commondata. This includes:

• Remove buildmaster #2032
• Update the theory part of all ported commondata files so that they don't use any apfelcomb keys (e.g., instead of apfelcomb: shifts just shifts can be used. Remove apfelcomb flags #2013
• Complete the library of processes: https://github.com/NNPDF/nnpdf/blob/master/validphys2/src/validphys/process_options.py such that the kinematics_transform module is no longer used, i.e, that all processes can use the identity transformation (and so the key can be dropped).
• Update the kinematic part of all ported commondata files so that they are not just k1, k2, k3 but have instead variable names. #2025
• Update the outdated documentation
  • Update "Adding new data" tutorials in the docs #1988
  • Add the nnpdf data package and remove APFELcomb and buildmaster from nnpdfmodules.rst
• Update to new commondata names in docs, examples and tests #1969
• Use only new theories and data in the tests #1940
• Add a test to the CI so that all new commondata files get read and inspected in some way at every commit (e.g., check that the kinematic_coverage is indeed what's in the kinematics file or that the number of bins match, etc.) (assigned: Add a test for all (new) commondata #1990)
• Add new documentation for stuff that has not been included before (for instance, the naming convention for the process variables)
• Utils for the new commondata #1693
• Execute data generation in the CI #1457

Once the process options module is finished

• Remove all old commondata files for which a translation exists already.
• Drop the mandatory status of the kinematic override key.
• Stop using k1, k2, k3 inside validphys and allow for the dataframe of kinematics to be the complete information (don't drop the minmax)
• relax the constraint that kinematic_coverage has to contain 3 variables

Once 4.1 is out

• Remove all old commondata files (even if they have not been translated)
• Remove the old commondata parser and old loader from validphys

(edit by @scarlehoff: added a hierarchy to the items and some extra ones)

[scarlehoff]

@RoyStegeman @Radonirinaunimi

just to make sure that several of us don't end up working on the same stuff, if you want to tackle one of these issues either open an issue and assign yourself or put your name here in the list

in any case, next Wednesday I was planning to go through the list and assign tasks to everyone

[RoyStegeman]

I assigned myself to updating the dataset names in the examples, so if I have time to work on any of these things, that's the one I'll address first.

[Radonirinaunimi]

I will assign myself starting from the first one on this list and the two issues #1940 and ...