Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ERROR: Snakemake failed #112

Open
baidefeng opened this issue Jun 11, 2024 · 4 comments
Open

ERROR: Snakemake failed #112

baidefeng opened this issue Jun 11, 2024 · 4 comments

Comments

@baidefeng
Copy link

When i run "hecatomb install", an snakemake error occurred. Do you have any ideas how to deal with it? Thanks a lot.

[2024:06:11 18:14:25] Copying system default config to hecatomb.out/hecatomb.config.yaml
[2024:06:11 18:14:25] Updating config file with new values
[2024:06:11 18:14:25] Writing config file to hecatomb.out/hecatomb.config.yaml
[2024:06:11 18:14:25] ------------------
[2024:06:11 18:14:25] | Runtime config |
[2024:06:11 18:14:25] ------------------

hecatomb:
addHost:
minViralAlnLen: 1000
args:
assembly: merged
conda_prefix: /data5/baidefeng/hecatomb/hecatomb/snakemake/conda
configfile: hecatomb.out/hecatomb.config.yaml
custom_aa: null
custom_nt: null
example_profile: false
fastqc: false
host: human
log: hecatomb.out/hecatomb.log
output: hecatomb.out
profile: null
search: sensitive
simulate: false
snake_args: []
system_config: /data5/baidefeng/hecatomb/hecatomb/snakemake/config/config.yaml
system_workflow_profile: /data5/baidefeng/hecatomb/hecatomb/snakemake/profile/default/config.yaml
threads: 32
trim: fastp
use_conda: true
workflow_profile: hecatomb.out/hecatomb.profile
assembly:
canu: correctedErrorRate=0.16 maxInputCoverage=10000 minInputCoverage=0 corOutCoverage=10000
corMhapSensitivity=high corMinCoverage=0 useGrid=False stopOnLowCoverage=False
genomeSize=10M -nanopore
flye: -g 1g
megahit: --presets meta-large
metaflye: --meta -g 1g --nano-raw
mmseqs:
fast: -s 4
filtAA: --min-length 30 -e 1e-5
filtNT: --min-length 90 -e 1e-20
linclustParams: --kmer-per-seq-scale 0.3 -c 0.8 --cov-mode 1 --min-seq-id 0.97
--alignment-mode 3
sensitive: --start-sens 2 --sens-steps 3 -s 7
taxIdIgnore: 0 1 2 10239 131567 12429 2759
resources:
big:
cpu: 32
mem: 64000
time: '24:00:00'
lrg:
cpu: 16
mem: 32000
time: 04:00:00
med:
cpu: 8
mem: 16000
time: 04:00:00
ram:
cpu: 2
mem: 16000
time: 04:00:00
sml:
cpu: 1
mem: 2000
time: 01:00:00
trimnami:
qc:
compression: 1
cutadapt: -m 40
fastp: --qualified_quality_phred 15 --length_required 90 --cut_tail --cut_tail_window_size
25 --cut_tail_mean_quality 15 --dedup --dup_calc_accuracy 4 --trim_poly_x --detect_adapter_for_pe
filtlong: --min_length 1000 --keep_percent 95
hostRemoveFlagstat: -f 4 -F 3584
minimapIndex: -I 8G
prinseq: -min_len 60 -min_qual_mean 25 -ns_max_n 1 -derep 1 -out_format 0 -trim_tail_left
5 -trim_tail_right 5 -ns_max_n 5 -trim_qual_type min -trim_qual_left 30 -trim_qual_right
30 -trim_qual_window 10

[2024:06:11 18:14:25] Copying system default config to hecatomb.out/hecatomb.profile/config.yaml
[2024:06:11 18:14:25] ---------------------
[2024:06:11 18:14:25] | Snakemake command |
[2024:06:11 18:14:25] ---------------------

snakemake -s /data5/baidefeng/hecatomb/hecatomb/snakemake/workflow/DownloadDB.smk --configfile hecatomb.out/hecatomb.config.yaml --cores 32 --use-conda --conda-prefix /data5/baidefeng/hecatomb/hecatomb/snakemake/conda --workflow-profile hecatomb.out/hecatomb.profile
Using profile hecatomb.out/hecatomb.profile and workflow specific profile hecatomb.out/hecatomb.profile for setting default command line arguments.
Config file /data5/baidefeng/hecatomb/hecatomb/snakemake/workflow/../config/config.yaml is extended by additional config specified via the command line.
Config file /data5/baidefeng/hecatomb/hecatomb/snakemake/workflow/../config/dbFiles.yaml is extended by additional config specified via the command line.
Config file /data5/baidefeng/hecatomb/hecatomb/snakemake/workflow/../config/immutable.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Your conda installation is not configured to use strict channel priorities. This is however crucial for having robust and correct environments (for details, see https://conda-forge.org/docs/user/tipsandtricks.html). Please consider to configure strict priorities by executing 'conda config --set channel_priority strict'.
Creating conda environment hecatomb/snakemake/workflow/envs/curl.yaml...
Downloading and installing remote packages.
CreateCondaEnvironmentException:
Could not create conda environment from /data5/baidefeng/hecatomb/hecatomb/snakemake/workflow/envs/curl.yaml:
Command:
conda env create --quiet --file "/data5/baidefeng/hecatomb/hecatomb/snakemake/conda/6b0e909bfbff6795553256ec89c7507f_.yaml" --prefix "/data5/baidefeng/hecatomb/hecatomb/snakemake/conda/6b0e909bfbff6795553256ec89c7507f_"
Output:
Collecting package metadata (repodata.json): ...working... failed

>>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

Traceback (most recent call last):
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda/exception_handler.py", line 17, in __call__
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda_env/cli/main.py", line 56, in do_call
    exit_code = getattr(module, func_name)(arguments, parser)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda/notices/core.py", line 124, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda_env/cli/main_create.py", line 162, in execute
    result[installer_type] = installer.install(
                             ^^^^^^^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda_env/installers/conda.py", line 51, in install
    unlink_link_transaction = solver.solve_for_transaction(
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda/core/solve.py", line 154, in solve_for_transaction
    unlink_precs, link_precs = self.solve_for_diff(
                               ^^^^^^^^^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda/core/solve.py", line 215, in solve_for_diff
    final_precs = self.solve_final_state(
                  ^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda/core/solve.py", line 358, in solve_final_state
    ssc = self._collect_all_metadata(ssc)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda/common/io.py", line 84, in decorated
    return f(*args, **kwds)
           ^^^^^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda/core/solve.py", line 572, in _collect_all_metadata
    index, r = self._prepare(prepared_specs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda/core/solve.py", line 1286, in _prepare
    reduced_index = get_reduced_index(
                    ^^^^^^^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda/core/index.py", line 287, in get_reduced_index
    new_records = SubdirData.query_all(
                  ^^^^^^^^^^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda/core/subdir_data.py", line 158, in query_all
    result = tuple(
             ^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/concurrent/futures/_base.py", line 619, in result_iterator
    yield _result_or_cancel(fs.pop())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/concurrent/futures/_base.py", line 317, in _result_or_cancel
    return fut.result(timeout)
           ^^^^^^^^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/data/meta/miniconda3/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda/core/subdir_data.py", line 143, in subdir_query
    return tuple(
           ^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda/core/subdir_data.py", line 165, in query
    self.load()
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda/core/subdir_data.py", line 265, in load
    _internal_state = self._load()
                      ^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda/core/subdir_data.py", line 322, in _load
    repodata, state = fetcher.fetch_latest_parsed()
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda/gateways/repodata/__init__.py", line 758, in fetch_latest_parsed
    parsed, state = self.fetch_latest()
                    ^^^^^^^^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda/gateways/repodata/__init__.py", line 820, in fetch_latest
    cache.load_state()
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda/gateways/repodata/__init__.py", line 626, in load_state
    self.load(state_only=True)
  File "/data/meta/miniconda3/lib/python3.11/site-packages/conda/gateways/repodata/__init__.py", line 573, in load
    with self.cache_path_state.open("r+") as state_file, lock(state_file):
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/meta/miniconda3/lib/python3.11/pathlib.py", line 1044, in open
    return io.open(self, mode, buffering, encoding, errors, newline)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/data/meta/miniconda3/pkgs/cache/47929eba.info.json'

$ /data/meta/miniconda3/bin/conda-env create --quiet --file /data5/baidefeng/hecatomb/hecatomb/snakemake/conda/6b0e909bfbff6795553256ec89c7507f_.yaml --prefix /data5/baidefeng/hecatomb/hecatomb/snakemake/conda/6b0e909bfbff6795553256ec89c7507f_

environment variables:
CIO_TEST=
CONDA_AUTO_UPDATE_CONDA=false
CONDA_BACKUP_ADDR2LINE=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-
addr2line
CONDA_BACKUP_AR=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-ar
CONDA_BACKUP_AS=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-as
CONDA_BACKUP_BUILD=x86_64-conda-linux-gnu
CONDA_BACKUP_CC=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-cc
CONDA_BACKUP_CC_FOR_BUILD=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-cc
CONDA_BACKUP_CFLAGS=-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-
strong -fno-plt -O2 -ffunction-sections -pipe -isystem
/data/meta/miniconda3/envs/kraken2/include -march=nocona
-mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-
plt -O2 -ffunction-sections -pipe -isystem
/data/meta/miniconda3/envs/kraken2/include
CONDA_BACKUP_CMAKE_PREFIX_PATH=/data/meta/miniconda3/envs/kraken2:/data/meta/miniconda3/envs/kraken2/
x86_64-conda-linux-gnu/sysroot/usr
CONDA_BACKUP_CONDA_BUILD_SYSROOT=/data/meta/miniconda3/envs/kraken2/x86_64-conda-linux-gnu/sysroot
CONDA_BACKUP_CONDA_TOOLCHAIN_BUILD=x86_64-conda-linux-gnu
CONDA_BACKUP_CONDA_TOOLCHAIN_HOST=x86_64-conda-linux-gnu
CONDA_BACKUP_CPP=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-cpp
CONDA_BACKUP_CPPFLAGS=-DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem
/data/meta/miniconda3/envs/kraken2/include -DNDEBUG
-D_FORTIFY_SOURCE=2 -O2 -isystem
/data/meta/miniconda3/envs/kraken2/include
CONDA_BACKUP_CXX=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-c++
CONDA_BACKUP_CXXFILT=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-c++filt
CONDA_BACKUP_CXXFLAGS=-fvisibility-inlines-hidden -std=c++17 -fmessage-length=0
-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-
strong -fno-plt -O2 -ffunction-sections -pipe -isystem
/data/meta/miniconda3/envs/kraken2/include -fvisibility-inlines-hidden
-std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-
vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-
sections -pipe -isystem /data/meta/miniconda3/envs/kraken2/include
CONDA_BACKUP_CXX_FOR_BUILD=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-c++
CONDA_BACKUP_DEBUG_CFLAGS=-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-
all -fno-plt -Og -g -Wall -Wextra -fvar-tracking-assignments
-ffunction-sections -pipe -isystem
/data/meta/miniconda3/envs/kraken2/include -march=nocona
-mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt
-Og -g -Wall -Wextra -fvar-tracking-assignments -ffunction-sections
-pipe -isystem /data/meta/miniconda3/envs/kraken2/include
CONDA_BACKUP_DEBUG_CPPFLAGS=-D_DEBUG -D_FORTIFY_SOURCE=2 -Og -isystem
/data/meta/miniconda3/envs/kraken2/include -D_DEBUG
-D_FORTIFY_SOURCE=2 -Og -isystem
/data/meta/miniconda3/envs/kraken2/include
CONDA_BACKUP_DEBUG_CXXFLAGS=-fvisibility-inlines-hidden -std=c++17 -fmessage-length=0
-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-
all -fno-plt -Og -g -Wall -Wextra -fvar-tracking-assignments
-ffunction-sections -pipe -isystem
/data/meta/miniconda3/envs/kraken2/include -fvisibility-inlines-hidden
-std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-
vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -Wall -Wextra
-fvar-tracking-assignments -ffunction-sections -pipe -isystem
/data/meta/miniconda3/envs/kraken2/include
CONDA_BACKUP_DEBUG_FFLAGS=-fopenmp -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-
protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem
/data/meta/miniconda3/envs/kraken2/include -fopenmp -march=nocona
-mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt
-Og -g -Wall -Wextra -fcheck=all -fbacktrace -fimplicit-none -fvar-
tracking-assignments -ffunction-sections -pipe -fopenmp -march=nocona
-mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-
plt -O2 -ffunction-sections -pipe -isystem
/data/meta/miniconda3/envs/kraken2/include
CONDA_BACKUP_DEBUG_FORTRANFLAGS=-fopenmp -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-
protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem
/data/meta/miniconda3/envs/kraken2/include -fopenmp -march=nocona
-mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt
-Og -g -Wall -Wextra -fcheck=all -fbacktrace -fimplicit-none -fvar-
tracking-assignments -ffunction-sections -pipe -fopenmp -march=nocona
-mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-
plt -O2 -ffunction-sections -pipe -isystem
/data/meta/miniconda3/envs/kraken2/include
CONDA_BACKUP_ELFEDIT=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-elfedit
CONDA_BACKUP_F77=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-gfortran
CONDA_BACKUP_F90=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-gfortran
CONDA_BACKUP_F95=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-f95
CONDA_BACKUP_FC=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-gfortran
CONDA_BACKUP_FC_FOR_BUILD=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-gfortran
CONDA_BACKUP_FFLAGS=-fopenmp -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-
protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem
/data/meta/miniconda3/envs/kraken2/include -fopenmp -march=nocona
-mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-
plt -O2 -ffunction-sections -pipe -isystem
/data/meta/miniconda3/envs/kraken2/include
CONDA_BACKUP_FORTRANFLAGS=-fopenmp -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-
protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem
/data/meta/miniconda3/envs/kraken2/include -fopenmp -march=nocona
-mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-
plt -O2 -ffunction-sections -pipe -isystem
/data/meta/miniconda3/envs/kraken2/include
CONDA_BACKUP_GCC=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-gcc
CONDA_BACKUP_GCC_AR=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-gcc-ar
CONDA_BACKUP_GCC_NM=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-gcc-nm
CONDA_BACKUP_GCC_RANLIB=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-gcc-
ranlib
CONDA_BACKUP_GFORTRAN=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-gfortran
CONDA_BACKUP_GPROF=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-gprof
CONDA_BACKUP_GXX=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-g++
CONDA_BACKUP_HOST=x86_64-conda-linux-gnu
CONDA_BACKUP_LD=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-ld
CONDA_BACKUP_LDFLAGS=-Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,
--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined
-Wl,-rpath,/data/meta/miniconda3/envs/kraken2/lib -Wl,-rpath-
link,/data/meta/miniconda3/envs/kraken2/lib
-L/data/meta/miniconda3/envs/kraken2/lib -Wl,-O2 -Wl,--sort-common
-Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--
gc-sections -Wl,--allow-shlib-undefined
-Wl,-rpath,/data/meta/miniconda3/envs/kraken2/lib -Wl,-rpath-
link,/data/meta/miniconda3/envs/kraken2/lib
-L/data/meta/miniconda3/envs/kraken2/lib
CONDA_BACKUP_LD_GOLD=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-ld.gold
CONDA_BACKUP_NM=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-nm
CONDA_BACKUP_OBJCOPY=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-objcopy
CONDA_BACKUP_OBJDUMP=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-objdump
CONDA_BACKUP_RANLIB=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-ranlib
CONDA_BACKUP_READELF=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-readelf
CONDA_BACKUP_SIZE=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-size
CONDA_BACKUP_STRINGS=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-strings
CONDA_BACKUP_STRIP=/data/meta/miniconda3/envs/kraken2/bin/x86_64-conda-linux-gnu-strip
CONDA_BACKUP_build_alias=x86_64-conda-linux-gnu
CONDA_BACKUP_host_alias=x86_64-conda-linux-gnu
CONDA_DEFAULT_ENV=hecatombDev
CONDA_EXE=/data5/baidefeng/miniconda3/bin/conda
CONDA_PREFIX=/data5/baidefeng/miniconda3/envs/hecatombDev
CONDA_PREFIX_1=/data/meta/miniconda3
CONDA_PREFIX_10=/data/meta/miniconda3/envs/metawrap
CONDA_PREFIX_11=/data/meta/miniconda3
CONDA_PREFIX_12=/data/meta/miniconda3/envs/metawrap
CONDA_PREFIX_13=/data/meta/miniconda3
CONDA_PREFIX_14=/data/meta/miniconda3/envs/metawrap
CONDA_PREFIX_15=/data/meta/miniconda3/envs/drep
CONDA_PREFIX_16=/data/meta/miniconda3/envs/coverm
CONDA_PREFIX_17=/data/meta/miniconda3/envs/humann3
CONDA_PREFIX_18=/data/meta/miniconda3/envs/gtdbtk2.3
CONDA_PREFIX_19=/data/meta/miniconda3
CONDA_PREFIX_2=/data/meta/miniconda3/envs/kneaddata
CONDA_PREFIX_20=/data/meta/miniconda3/envs/eggnog
CONDA_PREFIX_21=/data/meta/miniconda3
CONDA_PREFIX_22=/data/meta/miniconda3/envs/kneaddata
CONDA_PREFIX_23=/data/meta/miniconda3
CONDA_PREFIX_24=/data/meta/miniconda3/envs/kneaddata
CONDA_PREFIX_25=/data/meta/miniconda3
CONDA_PREFIX_26=/data/meta/miniconda3/envs/kneaddata
CONDA_PREFIX_27=/data/meta/miniconda3
CONDA_PREFIX_28=/data/meta/miniconda3/envs/kneaddata
CONDA_PREFIX_29=/data/meta/miniconda3
CONDA_PREFIX_3=/data/meta/miniconda3/envs/humann3
CONDA_PREFIX_30=/data/meta/miniconda3/envs/kneaddata
CONDA_PREFIX_31=/data/meta/miniconda3/envs/humann3
CONDA_PREFIX_32=/data/meta/miniconda3/envs/humann2
CONDA_PREFIX_33=/data/meta/miniconda3/envs/lefse
CONDA_PREFIX_34=/data/meta/miniconda3/envs/kraken2
CONDA_PREFIX_35=/data/meta/miniconda3
CONDA_PREFIX_36=/data/meta/miniconda3/envs/humann2
CONDA_PREFIX_37=/data/meta/miniconda3
CONDA_PREFIX_38=/data/meta/miniconda3/envs/kneaddata
CONDA_PREFIX_39=/data/meta/miniconda3
CONDA_PREFIX_4=/data/meta/miniconda3/envs/kraken2
CONDA_PREFIX_40=/data/meta/miniconda3/envs/kneaddata
CONDA_PREFIX_41=/data/meta/miniconda3/envs/humann3
CONDA_PREFIX_42=/data/meta/miniconda3
CONDA_PREFIX_43=/data/meta/miniconda3/envs/humann3
CONDA_PREFIX_44=/data/meta/miniconda3
CONDA_PREFIX_45=/data/meta/miniconda3/envs/humann3
CONDA_PREFIX_46=/data/meta/miniconda3
CONDA_PREFIX_47=/data/meta/miniconda3/envs/kneaddata
CONDA_PREFIX_48=/data/meta/miniconda3
CONDA_PREFIX_49=/data5/baidefeng/miniconda3
CONDA_PREFIX_5=/data/meta/miniconda3/envs/megahit
CONDA_PREFIX_50=/data/meta/miniconda3
CONDA_PREFIX_51=/data5/baidefeng/miniconda3
CONDA_PREFIX_52=/data/meta/miniconda3
CONDA_PREFIX_53=/data5/baidefeng/miniconda3
CONDA_PREFIX_54=/data/meta/miniconda3
CONDA_PREFIX_55=/data5/baidefeng/miniconda3
CONDA_PREFIX_56=/data5/baidefeng/miniconda3/envs/checkv
CONDA_PREFIX_57=/data5/baidefeng/miniconda3
CONDA_PREFIX_58=/data5/baidefeng/miniconda3/envs/checkv
CONDA_PREFIX_59=/data5/baidefeng/miniconda3
CONDA_PREFIX_6=/data/meta/miniconda3
CONDA_PREFIX_60=/data5/baidefeng/miniconda3/envs/hecatomb
CONDA_PREFIX_61=/data5/baidefeng/miniconda3
CONDA_PREFIX_62=/data5/baidefeng/miniconda3/envs/snakemake_env
CONDA_PREFIX_63=/data5/baidefeng/miniconda3/envs/hecatomb
CONDA_PREFIX_64=/data5/baidefeng/miniconda3/envs/snakemake_env
CONDA_PREFIX_65=/data5/baidefeng/miniconda3
CONDA_PREFIX_66=/data5/baidefeng/miniconda3/envs/snakemake_env
CONDA_PREFIX_67=/data5/baidefeng/miniconda3/envs/hecatomb
CONDA_PREFIX_68=/data5/baidefeng/miniconda3/envs/snakemake
CONDA_PREFIX_69=/data5/baidefeng/miniconda3/envs/hecatomb
CONDA_PREFIX_7=/data/meta/miniconda3/envs/megahit
CONDA_PREFIX_70=/data5/baidefeng/miniconda3
CONDA_PREFIX_71=/data5/baidefeng/miniconda3/envs/hecatomb
CONDA_PREFIX_72=/data5/baidefeng/miniconda3
CONDA_PREFIX_73=/data5/baidefeng/miniconda3/envs/hecatomb
CONDA_PREFIX_74=/data5/baidefeng/miniconda3
CONDA_PREFIX_75=/data5/baidefeng/miniconda3/envs/hecatomb
CONDA_PREFIX_76=/data5/baidefeng/miniconda3
CONDA_PREFIX_77=/data5/baidefeng/miniconda3/envs/hecatomb
CONDA_PREFIX_78=/data5/baidefeng/miniconda3
CONDA_PREFIX_79=/data5/baidefeng/miniconda3/envs/hecatomb
CONDA_PREFIX_8=/data/meta/miniconda3/envs/eggnog
CONDA_PREFIX_80=/data5/baidefeng/miniconda3/envs/snakemake
CONDA_PREFIX_81=/data5/baidefeng/miniconda3
CONDA_PREFIX_82=/data5/baidefeng/miniconda3/envs/snakemake_env
CONDA_PREFIX_83=/data5/baidefeng/miniconda3
CONDA_PREFIX_84=/data5/baidefeng/miniconda3/envs/hecatomb
CONDA_PREFIX_85=/data5/baidefeng/miniconda3
CONDA_PREFIX_86=/data5/baidefeng/miniconda3/envs/hecatomb
CONDA_PREFIX_87=/data5/baidefeng/miniconda3
CONDA_PREFIX_88=/data5/baidefeng/miniconda3/envs/snakemake_env
CONDA_PREFIX_89=/data5/baidefeng/miniconda3/envs/hecatomb
CONDA_PREFIX_9=/data/meta/miniconda3/envs/kraken2
CONDA_PREFIX_90=/data5/baidefeng/miniconda3/envs/snakemake
CONDA_PREFIX_91=/data5/baidefeng/miniconda3
CONDA_PREFIX_92=/data5/baidefeng/miniconda3/envs/hecatomb
CONDA_PREFIX_93=/data5/baidefeng/miniconda3
CONDA_PREFIX_94=/data5/baidefeng/miniconda3/envs/hecatomb
CONDA_PREFIX_95=/data5/baidefeng/miniconda3
CONDA_PREFIX_96=/data5/baidefeng/miniconda3/envs/hecatomb
CONDA_PREFIX_97=/data5/baidefeng/miniconda3
CONDA_PROMPT_MODIFIER=(hecatombDev)
CONDA_PYTHON_EXE=/data5/baidefeng/miniconda3/bin/python
CONDA_ROOT=/data/meta/miniconda3
CONDA_SHLVL=98
CURL_CA_BUNDLE=
GTDBTK_DATA_PATH=/data/meta/db/gtdb
JAVA_LD_LIBRARY_PATH=/data/meta/miniconda3/envs/kneaddata/lib/jvm/lib/server
LD_LIBRARY_PATH=/usr/lib/R/lib:/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/default-
java/lib/server
LD_PRELOAD=
PATH=/data5/baidefeng/miniconda3/envs/hecatombDev/bin:/data/meta/miniconda3
/bin:/data/meta/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/us
r/sbin:/usr/bin:/sbin:/bin:/data/meta/db/EasyMicrobiome/linux:/data/me
ta/db/EasyMicrobiome/script:/usr/bin:/snap/bin:/usr/local/go/bin:/data
/meta/db/EasyMicrobiome/linux:/data/meta/db/EasyMicrobiome/script:/dat
a/meta/db/EasyMicrobiome/linux:/data/meta/db/EasyMicrobiome/script:/da
ta/meta/db/EasyMicrobiome/linux:/data/meta/db/EasyMicrobiome/script:/d
ata/meta/db/EasyMicrobiome/linux:/data/meta/db/EasyMicrobiome/script:/
usr/bin:/usr/local/go/bin:/data/meta/db/EasyMicrobiome/linux:/data/met
a/db/EasyMicrobiome/script:/data/meta/db/EasyMicrobiome/linux:/data/me
ta/db/EasyMicrobiome/script:/data/meta/db/EasyMicrobiome/linux:/data/m
eta/db/EasyMicrobiome/script:/data/meta/db/EasyMicrobiome/linux:/data/
meta/db/EasyMicrobiome/script
REQUESTS_CA_BUNDLE=
RMARKDOWN_MATHJAX_PATH=/usr/lib/rstudio-server/resources/mathjax-27
RS_RPOSTBACK_PATH=/usr/lib/rstudio-server/bin/rpostback
RS_SERVER_RPC_SOCKET_PATH=/var/run/rstudio-server/rstudio-rserver/session-server-rpc.socket
SSL_CERT_FILE=

 active environment : /data5/baidefeng/miniconda3/envs/hecatombDev
active env location : /data5/baidefeng/miniconda3/envs/hecatombDev
        shell level : 98
   user config file : /data5/baidefeng/.condarc

populated config files :
conda version : 23.7.4
conda-build version : not installed
python version : 3.11.4.final.0
virtual packages : __archspec=1=x86_64
__glibc=2.35=0
__linux=6.5.0=0
__unix=0=0
base environment : /data/meta/miniconda3 (read only)
conda av data dir : /data/meta/miniconda3/etc/conda
conda av metadata url : None
channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /data/meta/miniconda3/pkgs
/data5/baidefeng/.conda/pkgs
envs directories : /data5/baidefeng/.conda/envs
/data/meta/miniconda3/envs
platform : linux-64
user-agent : conda/23.7.4 requests/2.29.0 CPython/3.11.4 Linux/6.5.0-28-generic ubuntu/22.04.2 glibc/2.35
UID:GID : 1009:1001
netrc file : None
offline mode : False

An unexpected error has occurred. Conda has prepared the above report.
If you suspect this error is being caused by a malfunctioning plugin,
consider using the --no-plugins option to turn off plugins.

Example: conda --no-plugins install

Alternatively, you can set the CONDA_NO_PLUGINS environment variable on
the command line to run the command without plugins enabled.

Example: CONDA_NO_PLUGINS=true conda install

@beardymcjohnface
Copy link
Collaborator

Hi, it looks like an error building the conda env. I think this is the key line here: PermissionError: [Errno 13] Permission denied: '/data/meta/miniconda3/pkgs/cache/47929eba.info.json'

do you have read permissions for that conda package cache directory? are you using a module for miniconda or did you install your own version?

@baidefeng
Copy link
Author

Thanks for your response. When i solve the permission problem and run "hecatomb install ", another ERROR occurred as below. Do you know how to fix it?

Error in rule download_taxdump:
jobid: 63
output: /home/baidefeng/miniconda3/envs/hecatomb/lib/python3.12/site-packages/hecatomb/snakemake/workflow/../databases/tax/taxonomy/citations.dmp, /home/baidefeng/miniconda3/envs/hecatomb/lib/python3.12/site-packages/hecatomb/snakemake/workflow/../databases/tax/taxonomy/delnodes.dmp, /home/baidefeng/miniconda3/envs/hecatomb/lib/python3.12/site-packages/hecatomb/snakemake/workflow/../databases/tax/taxonomy/division.dmp, /home/baidefeng/miniconda3/envs/hecatomb/lib/python3.12/site-packages/hecatomb/snakemake/workflow/../databases/tax/taxonomy/gc.prt, /home/baidefeng/miniconda3/envs/hecatomb/lib/python3.12/site-packages/hecatomb/snakemake/workflow/../databases/tax/taxonomy/gencode.dmp, /home/baidefeng/miniconda3/envs/hecatomb/lib/python3.12/site-packages/hecatomb/snakemake/workflow/../databases/tax/taxonomy/merged.dmp, /home/baidefeng/miniconda3/envs/hecatomb/lib/python3.12/site-packages/hecatomb/snakemake/workflow/../databases/tax/taxonomy/names.dmp, /home/baidefeng/miniconda3/envs/hecatomb/lib/python3.12/site-packages/hecatomb/snakemake/workflow/../databases/tax/taxonomy/nodes.dmp, taxdump.tar.gz
conda-env: /home/baidefeng/miniconda3/envs/hecatomb/lib/python3.12/site-packages/hecatomb/snakemake/conda/af2fc30e385f331538790795233e05db_
shell:

    curl https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump.tar.gz -o taxdump.tar.gz
    curl https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump.tar.gz.md5 | md5sum -c
    mkdir -p /home/baidefeng/miniconda3/envs/hecatomb/lib/python3.12/site-packages/hecatomb/snakemake/workflow/../databases/tax/taxonomy
    tar xvf taxdump.tar.gz -C /home/baidefeng/miniconda3/envs/hecatomb/lib/python3.12/site-packages/hecatomb/snakemake/workflow/../databases/tax/taxonomy

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

@beardymcjohnface
Copy link
Collaborator

I'm not sure why it's failed but you should try running those commands manually one by one. Was there any other error message?

@baidefeng
Copy link
Author

When i rerun "hecatomb install", no error occurred again. Then i run "hecatomb test --threads 2" and there is an error.

Error in rule index_host_genome:
jobid: 4
input: /home/baidefeng/miniconda3/envs/hecatomb/lib/python3.12/site-packages/hecatomb/snakemake/workflow/../databases/host/human/masked_ref.fa.gz
output: hecatomb.out/trimnami/temp/masked_ref.fa.idx
log: hecatomb.out/trimnami/logs/index_host_genome.log (check log file(s) for error details)
conda-env: /home/baidefeng/miniconda3/envs/hecatomb/lib/python3.12/site-packages/hecatomb/snakemake/conda/d0948b388800f98818a19a437160e856_/lib/python3.12/site-packages/trimnami/workflow/conda/94ed3c58c658bd219e7cdedfab82ff53_
shell:
minimap2 -t 2 -I 8G -d hecatomb.out/trimnami/temp/masked_ref.fa.idx /home/baidefeng/miniconda3/envs/hecatomb/lib/python3.12/site-packages/hecatomb/snakemake/workflow/../databases/host/human/masked_ref.fa.gz &> hecatomb.out/trimnami/logs/index_host_genome.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Logfile hecatomb.out/trimnami/logs/index_host_genome.log: empty file

Removing output files of failed job index_host_genome since they might be corrupted:
hecatomb.out/trimnami/temp/masked_ref.fa.idx
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
cat .snakemake/log/2024-06-18T101546.068754.snakemake.log >> hecatomb.out/trimnami/trimnami.log
Complete log: .snakemake/log/2024-06-18T101546.068754.snakemake.log
[2024:06:18 10:16:20] ERROR: Snakemake failed
[Tue Jun 18 10:16:20 2024]
Error in rule run_trimnami:
jobid: 1
input: hecatomb.out/results/hecatomb.samples.tsv, hecatomb.out/processing/temp/trimnami.config.yaml
output: hecatomb.out/trimnami/results/fastp/A13-04-182-06_TAGCTT_R1.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-04-182-06_TAGCTT_R2.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-04-182-06_TAGCTT_RS.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-12-250-06_GGCTAC_R1.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-12-250-06_GGCTAC_R2.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-12-250-06_GGCTAC_RS.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-135-177-06_AGTTCC_R1.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-135-177-06_AGTTCC_R2.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-135-177-06_AGTTCC_RS.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-151-169-06_ATGTCA_R1.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-151-169-06_ATGTCA_R2.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-151-169-06_ATGTCA_RS.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-252-114-06_CCGTCC_R1.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-252-114-06_CCGTCC_R2.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-252-114-06_CCGTCC_RS.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-253-140-06_GTCCGC_R1.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-253-140-06_GTCCGC_R2.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-253-140-06_GTCCGC_RS.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-255-183-06_GTGGCC_R1.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-255-183-06_GTGGCC_R2.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-255-183-06_GTGGCC_RS.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-256-115-06_GTTTCG_R1.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-256-115-06_GTTTCG_R2.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-256-115-06_GTTTCG_RS.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-256-117-06_ACTGAT_R1.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-256-117-06_ACTGAT_R2.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-256-117-06_ACTGAT_RS.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-258-124-06_CGTACG_R1.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-258-124-06_CGTACG_R2.host_rm.fastq.gz, hecatomb.out/trimnami/results/fastp/A13-258-124-06_CGTACG_RS.host_rm.fastq.gz
conda-env: /home/baidefeng/miniconda3/envs/hecatomb/lib/python3.12/site-packages/hecatomb/snakemake/conda/d0948b388800f98818a19a437160e856_
shell:
trimnami run fastp --reads hecatomb.out/results/hecatomb.samples.tsv --configfile hecatomb.out/processing/temp/trimnami.config.yaml --host /home/baidefeng/miniconda3/envs/hecatomb/lib/python3.12/site-packages/hecatomb/snakemake/workflow/../databases/host/human/masked_ref.fa.gz --output hecatomb.out/trimnami --threads 2 --minimap sr --workflow-profile hecatomb.out/hecatomb.profile ;
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
cat .snakemake/log/2024-06-18T101542.755942.snakemake.log >> hecatomb.out/hecatomb.log

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@beardymcjohnface @baidefeng