[IBCDPE-589] Catch up `main` with `dev` #9

BWMac · 2023-07-11T02:44:11Z

This PR merges dev into main to get main caught up. all changes have been tested and are working well with the current version of py-dcqc.

This reverts commit c89c113.

…9/nf-dcqc [ORCA-119] Implement Nextflow workflow for DCQC

…ocker-hub Use copy of container image from GitHub Container Registry

…7/lazy-staging [ORCA-167] Stage files lazily

…add_csv_update_process [ORCA-175] Adds functionality to update input CSV with DCQC status

…improve_file_mounting [ORCA-234] Improve file path handling

…report_config [ORCA-254] Adds information for output files and Tower reports

github-actions · 2023-07-11T02:45:39Z

`nf-core lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit 26b9c53

+| ✅ 130 tests passed       |+
#| ❔  13 tests were ignored |#
!| ❗  16 tests had warnings |!

❗ Test warnings:

files_exist - File not found: assets/multiqc_config.yml
files_exist - File not found: lib/WorkflowNf-dcqc.groovy
nextflow_config - Config manifest.version should end in dev: '0.1'
pipeline_todos - TODO string in README.md: Write a 1-2 sentence summary of what data the pipeline is for and what it does
pipeline_todos - TODO string in README.md: Add full-sized test dataset and amend the paragraph below if applicable
pipeline_todos - TODO string in README.md: Fill in short bullet-pointed list of the default steps in the pipeline
pipeline_todos - TODO string in README.md: Update the example "typical command" below used to run the pipeline
pipeline_todos - TODO string in README.md: If applicable, make list of people who have also contributed
pipeline_todos - TODO string in README.md: Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file.
pipeline_todos - TODO string in README.md: Add bibliography of tools and data used in your pipeline
pipeline_todos - TODO string in output.md: Write this documentation describing your workflow's output
pipeline_todos - TODO string in usage.md: Add documentation about anything specific to running your pipeline. For general topics, please point to (and add to) the main nf-core website.
pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your prefered methods description, e.g. add publication citation for this pipeline
pipeline_todos - TODO string in WorkflowMain.groovy: Add Zenodo DOI for pipeline after first release
pipeline_name_conventions - Naming does not adhere to nf-core conventions: Contains non alphanumeric characters
schema_lint - Schema $id should be https://raw.githubusercontent.com/sage-bionetworks-workflows/nf-dcqc/master/nextflow_schema.json
Found https://raw.githubusercontent.com/sage/dcqc/master/nextflow_schema.json

❔ Tests ignored:

files_exist - File is ignored: CODE_OF_CONDUCT.md
files_exist - File is ignored: assets/nf-core-dcqc_logo_light.png
files_exist - File is ignored: docs/images/nf-core-dcqc_logo_light.png
files_exist - File is ignored: docs/images/nf-core-dcqc_logo_dark.png
files_exist - File is ignored: .github/ISSUE_TEMPLATE/config.yml
files_exist - File is ignored: .github/workflows/awstest.yml
files_exist - File is ignored: .github/workflows/awsfulltest.yml
nextflow_config - Config variable ignored: manifest.name
nextflow_config - Config variable ignored: manifest.homePage
files_unchanged - File does not exist: CODE_OF_CONDUCT.md
files_unchanged - File does not exist: .github/ISSUE_TEMPLATE/config.yml
actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/nf-dcqc/nf-dcqc/.github/workflows/awstest.yml
multiqc_config - 'assets/multiqc_config.yml' not found

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-nf-dcqc_logo_light.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-nf-dcqc_logo_light.png
files_exist - File found: docs/images/nf-core-nf-dcqc_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: lib/nfcore_external_java_deps.jar
files_exist - File found: lib/NfcoreSchema.groovy
files_exist - File found: lib/NfcoreTemplate.groovy
files_exist - File found: lib/Utils.groovy
files_exist - File found: lib/WorkflowMain.groovy
files_exist - File found: main.nf
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes.config
files_exist - File found: modules.json
files_exist - File found: pyproject.toml
files_exist - File not found check: Singularity
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: docs/images/nf-core-nf-dcqc_logo.png
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: params.show_hidden_params
nextflow_config - Config variable found: params.schema_ignore_params
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/CONTRIBUTING.md matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - lib/nfcore_external_java_deps.jar matches the template
files_unchanged - lib/NfcoreSchema.groovy matches the template
files_unchanged - lib/NfcoreTemplate.groovy matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
files_unchanged - pyproject.toml matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
readme - README Nextflow minimum version badge matched config. Badge: 22.10.4, Config: 22.10.4
readme - README Nextflow minimum version in Quick Start section matched config. README: 22.10.4, Config: 22.10.4
template_strings - Did not find any Jinja template strings (83 files)
schema_lint - Schema lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'

Run details

nf-core/tools version 2.7.2
Run at 2023-07-11 02:45:21

thomasyu888

Nice!

swarmia · 2023-07-11T05:40:57Z

✅ Linked to Story IBCDPE-589 · DCQC: merge dev branch into main

BrunoGrandePhD added 30 commits January 25, 2023 16:30

Create initial working version with test_txt

54910f0

Update dcqc container image

eb016e8

Fix file types in test data

fc1c283

Add new output JSON arg for dcqc stage-target

5fb2173

Fix copy-paste typo in variable name

e1dc69f

Ensure consistent paths between processes

87cfbc0

Ensure consistent container entrypoint

566af19

Use containerOptions for disabling entrypoint

0a2acab

Remove containerOptions, which didn't work

7379403

Address nf-core linting issues

6e6ad5e

Auto-fix files_unchanged linting issues

27c7847

Address other linting issues

3f50b88

Fix issues with test profiles

6f67aee

Add test files to repo to avoid Synapse URLs

7e07644

Update Synapse URLs with GitHub HTTP URLs

e097072

Remove requirement for SYNAPSE_AUTH_TOKEN

50e8c9a

Revert Synapse to HTTPS URLs

942e08a

Re-enable SYNAPSE_AUTH_TOKEN secret

8ed7ee6

Fix path for dummy file

6ea775f

Update test profiles

5a5786d

Update dcqc docker tag to latest

56526c8

Pass in Synapse auth token

5dd0843

Fix typo in Nextflow version

ee0b932

Use consistent minimum Nextflow version

c89c113

Revert "Use consistent minimum Nextflow version"

41cb054

This reverts commit c89c113.

Merge pull request #1 from Sage-Bionetworks-Workflows/bgrande/ORCA-11…

e67f5d6

…9/nf-dcqc [ORCA-119] Implement Nextflow workflow for DCQC

Avoid Docker Hub

57924f3

Merge pull request #2 from Sage-Bionetworks-Workflows/bgrande/avoid-d…

529d080

…ocker-hub Use copy of container image from GitHub Container Registry

Increase memory allocation

8fa6b80

Add params for required_tests & skipped_tests

fe66fbe

BrunoGrandePhD and others added 23 commits April 3, 2023 19:59

Add comment about default container name

7f433fb

Merge pull request #4 from Sage-Bionetworks-Workflows/bgrande/ORCA-16…

1e5f076

…7/lazy-staging [ORCA-167] Stage files lazily

adds update_input process. exports updated csv to outdir

b90b775

adds tower.yml with csv report

f53c0eb

missing $

619dc40

updates input_txt and test config for amber status testing

1142435

try asterisk path

a2e2633

Merge pull request #5 from Sage-Bionetworks-Workflows/bwmac/ORCA-175/…

afa9251

…add_csv_update_process [ORCA-175] Adds functionality to update input CSV with DCQC status

point to test container

6a77760

try type dir

7141603

revert to file mount

6952137

debug modes

3b2bbb0

print staged file

2cf274a

remove debugs

cffd1ad

change back to container

dd3da47

undo spacing

dda2ac0

Merge pull request #7 from Sage-Bionetworks-Workflows/bwmac/orca-234/…

efa839a

…improve_file_mounting [ORCA-234] Improve file path handling

adds reports and output info

9d4250c

update for nf-core lint failures

aceca71

prettier fix

8b199f0

reverts commits 8b199f0 and aceca71

ca3b8d2

prettier fix

8810870

Merge pull request #8 from Sage-Bionetworks-Workflows/bwmac/orca-254/…

26b9c53

…report_config [ORCA-254] Adds information for output files and Tower reports

BWMac marked this pull request as ready for review July 11, 2023 02:45

BWMac requested a review from thomasyu888 July 11, 2023 02:45

thomasyu888 approved these changes Jul 11, 2023

View reviewed changes

thomasyu888 changed the title ~~Catch up main with dev~~ [IBCDPE-589] Catch up main with dev Jul 11, 2023

BWMac merged commit 76694ae into main Jul 11, 2023
12 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[IBCDPE-589] Catch up `main` with `dev` #9

[IBCDPE-589] Catch up `main` with `dev` #9

BWMac commented Jul 11, 2023 •

edited

Loading

github-actions bot commented Jul 11, 2023

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

thomasyu888 left a comment

swarmia bot commented Jul 11, 2023

[IBCDPE-589] Catch up main with dev #9

[IBCDPE-589] Catch up main with dev #9

Conversation

BWMac commented Jul 11, 2023 • edited Loading

github-actions bot commented Jul 11, 2023

nf-core lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

thomasyu888 left a comment

Choose a reason for hiding this comment

swarmia bot commented Jul 11, 2023

[IBCDPE-589] Catch up `main` with `dev` #9

[IBCDPE-589] Catch up `main` with `dev` #9

BWMac commented Jul 11, 2023 •

edited

Loading

`nf-core lint` overall result: Passed ✅ ⚠️