Merged cwl-v1.3 into cwltool

cwltool/process.py

@@ -125,6 +125,7 @@ def filter(self, record: logging.LogRecord) -> bool:
 ]
 
 cwl_files = (
+    "Base.yml",
     "Workflow.yml",
     "CommandLineTool.yml",
     "CommonWorkflowLanguage.yml",

cwltool/schemas/v1.3.0-dev1/CITATION

@@ -0,0 +1,28 @@
+To cite the Common Workflow Language standards in a publication, please use:
+
+Amstutz, Peter; Crusoe, Michael R; Tijanić, Nebojša; Chapman, Brad;
+Chilton, John; Heuer, Michael; Kartashov, Andrey; Kern, John; Leehr, Dan;
+Ménager, Hervé; Nedeljkovich, Maya; Scales, Matt; Soiland-Reyes, Stian;
+Stojanovic, Luka (2016): Common Workflow Language, v1.0. Standards,
+Common Workflow Language working group. https://w3id.org/cwl/v1.0/
+https://doi.org/10.6084/m9.figshare.3115156.v2
+
+@data{cwl,
+  doi = {10.6084/m9.figshare.3115156.v2},
+  url = {https://doi.org/10.6084/m9.figshare.3115156.v2},
+  author = {Peter Amstutz; Michael R. Crusoe; Nebojša Tijanić; Brad Chapman;
+John Chilton; Michael Heuer; Andrey Kartashov; John Kern; Dan Leehr;
+Hervé Ménager; Maya Nedeljkovich; Matt Scales; Stian Soiland-Reyes;
+Luka Stojanovic
+ },
+  publisher = {Figshare},
+  institution = {Common Workflow Language working group},
+  title = {Common Workflow Language, v1.0},
+  year = {2016}
+}
+
+# Are you editing this file?
+# Synchronize any changes made with
+# README.md
+# and
+# https://github.com/common-workflow-language/user_guide/blob/gh-pages/CITATION

cwltool/schemas/v1.3.0-dev1/CODE_OF_CONDUCT.md

@@ -102,7 +102,6 @@ As a stop gap measure until a more formal governance structure is adopted, the
 following individuals make up the leadership of the CWL Project: Peter Amstutz,
 John Chilton, Michael R. Crusoe, and Nebojša Tijanić.
 
-To report an issue with anyone on the team you can escalate to Ward Vandewege
- (Curoverse) [email protected], Anton Nekrutenko (Galaxy)
+To report an issue with anyone on the team you can escalate to Anton Nekrutenko (Galaxy)
 anton AT bx DOT psu DOT edu, C. Titus Brown (UC Davis) [email protected], or
 Brandi Davis-Dusenbery (Seven Bridges Genomics) [email protected].

cwltool/schemas/v1.3.0-dev1/CONFORMANCE_TESTS.md

@@ -144,7 +144,7 @@ We will use this single entry to explain the format
      Must include one or more of the following tags: `command_line_tool`, `expression_tool` or `workflow`.
      If the test does not test any optional features, the tag `required` is required.
 
-Because `conformance_tests.yaml` is a `schema-salad` processed document, [`$import`](https://www.commonwl.org/v1.2/SchemaSalad.html#Import)
+Because `conformance_tests.yaml` is a `schema-salad` processed document, [`$import`](https://www.commonwl.org/v1.3/SchemaSalad.html#Import)
 can be used to organize the tests into separate files.
 
 Currently, the main file is too big (over 3400 lines); we are slowly re-organizing it.
@@ -166,7 +166,7 @@ At any level, if there is an extra field, then that will be considered an error.
 An exception to this is `class: File` and `class: Directory` objects, the `cwl-runner` under test can add additional fields here without causing a test to fail.
 Likewise, if you don't want to test some aspect of a `class: File` or `class: Directory` object (like `nameext`) you can just omit it.
 
-[According to the CWL standards](https://www.commonwl.org/v1.2/CommandLineTool.html#File), the format of the `location` field in 
+[According to the CWL standards](https://www.commonwl.org/v1.3/CommandLineTool.html#File), the format of the `location` field in 
 `class: File` and `class: Directory` is implementation specific and we should not be testing them.
 Please remember to use `location: Any` for them.
 
@@ -181,17 +181,19 @@ Likewise, please do not test the `path` for `class: File` and `class: Directory`
 
 To add a new conformance test:
 1. Ensure the CWL document you have tests the desired feature or aspect.
-2. All `CommandLineTool`s need a software container (via `DockerRequirement`) for better reproducibility, preferably under `hints`.
+2. The `cwlVersion` should be the latest version (`cwlVersion: v1.2`), unless
+   testing the mixing of versions as in the `tests/mixed-versions` directory.
+3. All `CommandLineTool`s need a software container (via `DockerRequirement`) for better reproducibility, preferably under `hints`.
      Please limit your container usage to the following: 
      - `dockerPull: docker.io/alpine:latest`
      - `dockerPull: docker.io/bash:4.4`
      - `dockerPull: docker.io/debian:stable-slim`
      - `dockerPull: docker.io/python:3-slim`
 4. Run your test using the CWL reference runner (`cwltool`) or another CWL runner
      that shows the correct behavior to collect the output, or confirm that validation/execution fails as expected
-3. Add the CWL document and output object to the subdirectory `tests` in this repository.
-4. Fill out a new entry in [conformance_tests.yaml](conformance_tests.yaml) following the [format of the conformance test file](#format-of-the-conformance-test-file)
-5. Send a pull request to [current staging branch for the next revision of the CWL standards](https://github.com/common-workflow-language/cwl-v1.2/tree/1.2.1_proposed) 
+5. Add the CWL document and output object to the subdirectory `tests` in this repository.
+6. Fill out a new entry in [conformance_tests.yaml](conformance_tests.yaml) following the [format of the conformance test file](#format-of-the-conformance-test-file)
+7. Send a pull request to [current staging branch for the next revision of the CWL standards](https://github.com/common-workflow-language/cwl-v1.2/tree/1.2.1_proposed) 
      with your changes
 
 ## Tags for conformance tests

cwltool/schemas/v1.3.0-dev1/CommandLineTool.yml

@@ -1,4 +1,4 @@
-saladVersion: v1.1
+saladVersion: v1.3
 $base: "https://w3id.org/cwl/cwl#"
 
 $namespaces:
@@ -42,14 +42,20 @@ $graph:
       This specification represents the latest development version from the
       CWL group.
 
-      ## Changelog
+      Documents should use `cwlVersion: v1.3.0-dev1` to make use of new
+      syntax and features introduced in v1.3.0-dev1.  Existing v1.2 documents
+      should be trivially updatable by changing `cwlVersion`, however
+      CWL documents that relied on previously undefined or
+      underspecified behavior may have slightly different behavior in
+      v1.3.0-dev1.
 
+      ## Changelog for v1.3.0-dev1
 
       See also the [CWL Workflow Description, v1.3.0-dev1 changelog](Workflow.html#Changelog).
       For other changes since CWL v1.0, see the
       [CWL Command Line Tool Description, v1.1 changelog](https://www.commonwl.org/v1.1/CommandLineTool.html#Changelog)
       and
-      [CWL Command Line Tool Description, v1.2 changelog](https://www.commonwl.org/v1.2/CommandLineTool.html#Changelog).
+      [CWL Command Line Tool Description, v1.2.1 changelog](https://www.commonwl.org/v1.2/CommandLineTool.html#Changelog).
 
       ## Purpose
 
@@ -483,7 +489,7 @@ $graph:
         type: File
         streamable: true
 
-    stdin: ${inputs.an_input_name.path}
+    stdin: $(inputs.an_input_name.path)
     ```
 
 - name: stdout
@@ -1081,7 +1087,7 @@ $graph:
     Modify the behavior of CommandLineTool to generate a single string
     containing a shell command line.  Each item in the `arguments` list must
     be joined into a string separated by single spaces and quoted to prevent
-    intepretation by the shell, unless `CommandLineBinding` for that argument
+    interpretation by the shell, unless `CommandLineBinding` for that argument
     contains `shellQuote: false`.  If `shellQuote: false` is specified, the
     argument is joined into the command string without quoting, which allows
     the use of shell metacharacters such as `|` for pipes.
@@ -1159,7 +1165,7 @@ $graph:
         do not exceed the capacity of the node.
 
         Processes sharing a core must have the same level of isolation
-        (typically a container or VM) that they would normally.
+        (typically a container or VM) that they would normally have.
 
         The reported number of CPU cores reserved for the process,
         which is available to expressions on the CommandLineTool as

cwltool/schemas/v1.3.0-dev1/CommonWorkflowLanguage.yml

@@ -1,4 +1,4 @@
-saladVersion: v1.1
+saladVersion: v1.3
 $base: "https://w3id.org/cwl/cwl#"
 
 $namespaces:

cwltool/schemas/v1.3.0-dev1/Operation.yml

@@ -1,4 +1,4 @@
-saladVersion: v1.1
+saladVersion: v1.3
 $base: "https://w3id.org/cwl/cwl#"
 
 $namespaces: