Nanodisco difference: no data for any chunks?

[ecpierce]

Hi @fanggang,

I ran nanodisco difference, and everything seemed to run okay I think (output below). However when I look at the stdout files in the differences log, all of them look like this ("no data for chunk #x") :

[1] "2022-12-06 19:25:42 UTC"
[1] " Prepare index for EligoWGA."
[1] " Extract read mapped on chunks for EligoWGA."
[1] " Link fast5 to fasta for EligoWGA."
[1] " Prepare index for Eligo."
[1] " Extract read mapped on chunks for Eligo."
[1] " Link fast5 to fasta for Eligo."
[1] "2022-12-06 19:26:39 UTC"
[1] "Processing chunk #22"
[1] " Preparing EligoWGA input data for chunk #22"
[1] " Preparing Eligo input data for chunk #22"
[1] " Correcting mapping."
[1] " No data for chunk #22"
[1] " Remove temporary files."
[1] "2022-12-06 19:26:48 UTC"

When I look at the .rds file of a chunk in R, this is what I see:

So it seems like it also has no data? Any ideas what is happening?

(base) em:/data$ singularity exec --bind .:/mnt nanodisco.sif nanodisco chunk_info -r /mnt/eligo4weekWGAconsensus.fasta
Number of chunks: 5841

(base) em:/data$ singularity exec --bind .:/mnt nanodisco.sif nanodisco difference -nj 2 -nc 2 -p 4 -i /mnt/eligopreprocess/ -o /mnt/eligopreprocess/differences/ -w EligoWGA -n Eligo -r /mnt/eligo4weekWGAconsensus.fasta

Computers / CPU cores / Max jobs to run
1:local / 16 / 2

Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete
local:2/261/100%/66.4s Regional downsampling: 1356 reads from EligoWGA chunk #525 (contig_854:6535-10588,+; 4052 bp).
Regional downsampling: 1324 reads from EligoWGA chunk #525 (contig_854:6563-10588,-; 4024 bp).
Regional downsampling: 832 reads from Eligo chunk #525 (contig_854:8013-10588,+; 2574 bp).
Regional downsampling: 226 reads from Eligo chunk #525 (contig_854:8745-10587,-; 1841 bp).
Localized downsampling: 265 reads from Eligo chunk #525.
local:2/263/100%/66.6s Regional downsampling: 2585 reads from EligoWGA chunk #523 (contig_854:1-7760,+; 7758 bp).
Regional downsampling: 2697 reads from EligoWGA chunk #523 (contig_854:1-7786,-; 7784 bp).
No regional downsampling for EligoWGA chunk #524: region too long (contig_854:9-10588,+; 10578 bp).
No regional downsampling for EligoWGA chunk #524: region too long (contig_854:3-10588,-; 10584 bp).
Localized downsampling: 3789 reads from EligoWGA chunk #524.
Regional downsampling: 2903 reads from Eligo chunk #523 (contig_854:1-6665,+; 6663 bp).
Regional downsampling: 2864 reads from Eligo chunk #523 (contig_854:1-6627,-; 6625 bp).
Regional downsampling: 3229 reads from Eligo chunk #524 (contig_854:3128-10588,+; 7459 bp).
Regional downsampling: 2318 reads from Eligo chunk #524 (contig_854:3454-10578,-; 7123 bp).
Localized downsampling: 59 reads from Eligo chunk #524.
local:2/284/100%/67.5s No regional downsampling for EligoWGA chunk #570: region too short (contig_229:9716-9733,+; 16 bp).
Localized downsampling: 3 reads from EligoWGA chunk #570.
local:2/303/100%/67.9s Random downsampling: 13973 reads from EligoWGA chunk #607 (contig_988:1-4760,+; 4758 bp).
Random downsampling: 13944 reads from EligoWGA chunk #607 (contig_988:1-4760,-; 4758 bp).
local:2/308/100%/67.8s Random downsampling: 5037 reads from EligoWGA chunk #617 (contig_500:1-503,+; 501 bp).
Random downsampling: 4911 reads from EligoWGA chunk #617 (contig_500:1-503,-; 501 bp).
Random downsampling: 813 reads from Eligo chunk #617 (contig_500:1-503,+; 501 bp).
Regional downsampling: 4 reads from Eligo chunk #617 (contig_500:13-393,-; 379 bp).
No regional downsampling for Eligo chunk #617: region too short (contig_500:409-482,-; 72 bp).
Localized downsampling: 13 reads from Eligo chunk #617.
local:2/332/100%/67.2s Random downsampling: 1143 reads from EligoWGA chunk #666 (contig_539:1-1150,+; 1148 bp).
Random downsampling: 1198 reads from EligoWGA chunk #666 (contig_539:1-1150,-; 1148 bp).
local:2/435/100%/65.6s Regional downsampling: 156 reads from EligoWGA chunk #872 (contig_579:8-3507,+; 3498 bp).
No regional downsampling for EligoWGA chunk #872: region too short (contig_579:4344-4350,+; 5 bp).
Regional downsampling: 1 reads from EligoWGA chunk #872 (contig_579:4355-5277,+; 921 bp).
Regional downsampling: 179 reads from EligoWGA chunk #872 (contig_579:8-4173,-; 4164 bp).
Regional downsampling: 3 reads from EligoWGA chunk #872 (contig_579:4197-5550,-; 1352 bp).
Localized downsampling: 21 reads from EligoWGA chunk #872.
local:2/436/100%/65.5s Regional downsampling: 27 reads from EligoWGA chunk #873 (contig_579:4755-6803,+; 2047 bp).
Regional downsampling: 45 reads from EligoWGA chunk #873 (contig_579:4569-7022,-; 2452 bp).
No regional downsampling for EligoWGA chunk #873: region too short (contig_579:7034-7038,-; 3 bp).
No regional downsampling for EligoWGA chunk #873: region too short (contig_579:7110-7117,-; 6 bp).
No regional downsampling for EligoWGA chunk #873: region too short (contig_579:7122-7125,-; 2 bp).
No regional downsampling for EligoWGA chunk #873: region too short (contig_579:7127-7238,-; 110 bp).
No regional downsampling for EligoWGA chunk #873: region too short (contig_579:7240-7282,-; 41 bp).
Localized downsampling: 125 reads from EligoWGA chunk #873.
local:2/469/100%/65.0s Error in { : task 2 failed - "task 1 failed - "invalid 'size' argument""
Calls: prepare.index -> %do% ->
Execution halted
local:2/511/100%/64.8s Random downsampling: 20081 reads from EligoWGA chunk #1024 (contig_817:1-4781,+; 4779 bp).
Random downsampling: 20374 reads from EligoWGA chunk #1024 (contig_817:1-4781,-; 4779 bp).
Random downsampling: 4145 reads from Eligo chunk #1024 (contig_817:1-4781,+; 4779 bp).
Random downsampling: 4099 reads from Eligo chunk #1024 (contig_817:1-4781,-; 4779 bp).
Localized downsampling: 324 reads from Eligo chunk #1024.
local:2/547/100%/65.0s Regional downsampling: 1382 reads from EligoWGA chunk #1098 (contig_816:409-5608,+; 5198 bp).
Regional downsampling: 1282 reads from EligoWGA chunk #1098 (contig_816:833-5608,-; 4774 bp).
Error in { : task 1 failed - "task 1 failed - "invalid 'size' argument""
Calls: prepare.index -> %do% ->
Execution halted
local:2/832/100%/66.6s No regional downsampling for Eligo chunk #1665: region too short (contig_922:718-720,+; 1 bp).
No regional downsampling for Eligo chunk #1665: region too short (contig_922:869-883,+; 13 bp).
No regional downsampling for Eligo chunk #1665: region too short (contig_922:886-944,+; 57 bp).
No regional downsampling for Eligo chunk #1665: region too short (contig_922:953-988,+; 34 bp).
Regional downsampling: 1 reads from Eligo chunk #1665 (contig_922:999-1229,+; 229 bp).
No regional downsampling for Eligo chunk #1665: region too short (contig_922:1325-1334,+; 8 bp).
No regional downsampling for Eligo chunk #1665: region too short (contig_922:1379-1384,+; 4 bp).
No regional downsampling for Eligo chunk #1665: region too short (contig_922:1422-1534,+; 111 bp).
No regional downsampling for Eligo chunk #1665: region too short (contig_922:1548-1576,+; 27 bp).
Regional downsampling: 239 reads from Eligo chunk #1665 (contig_922:1653-3663,+; 2009 bp).
Regional downsampling: 2 reads from Eligo chunk #1665 (contig_922:3895-4193,+; 297 bp).
Regional downsampling: 4 reads from Eligo chunk #1665 (contig_922:4813-5140,+; 326 bp).
No regional downsampling for Eligo chunk #1665: region too short (contig_922:546-657,-; 110 bp).
No regional downsampling for Eligo chunk #1665: region too short (contig_922:740-747,-; 6 bp).
No regional downsampling for Eligo chunk #1665: region too short (contig_922:816-988,-; 171 bp).
No regional downsampling for Eligo chunk #1665: region too short (contig_922:992-999,-; 6 bp).
Regional downsampling: 3 reads from Eligo chunk #1665 (contig_922:1004-1243,-; 238 bp).
No regional downsampling for Eligo chunk #1665: region too short (contig_922:1316-1351,-; 34 bp).
No regional downsampling for Eligo chunk #1665: region too short (contig_922:1376-1385,-; 8 bp).
No regional downsampling for Eligo chunk #1665: region too short (contig_922:1541-1593,-; 51 bp).
No regional downsampling for Eligo chunk #1665: region too short (contig_922:1599-1606,-; 6 bp).
No regional downsampling for Eligo chunk #1665: region too short (contig_922:1618-1619,-; 0 bp).
Regional downsampling: 355 reads from Eligo chunk #1665 (contig_922:1632-3859,-; 2226 bp).
Regional downsampling: 32 reads from Eligo chunk #1665 (contig_922:3867-4571,-; 703 bp).
No regional downsampling for Eligo chunk #1665: region too short (contig_922:4762-4781,-; 18 bp).
No regional downsampling for Eligo chunk #1665: region too short (contig_922:4817-4818,-; 0 bp).
Regional downsampling: 2 reads from Eligo chunk #1665 (contig_922:4823-5117,-; 293 bp).
Localized downsampling: 158 reads from Eligo chunk #1665.
Regional downsampling: 42 reads from Eligo chunk #1666 (contig_922:4886-5762,+; 875 bp).
No regional downsampling for Eligo chunk #1666: region too short (contig_922:6432-6492,+; 59 bp).
No regional downsampling for Eligo chunk #1666: region too short (contig_922:6495-6497,+; 1 bp).
No regional downsampling for Eligo chunk #1666: region too short (contig_922:6516-6592,+; 75 bp).
No regional downsampling for Eligo chunk #1666: region too short (contig_922:6732-6740,+; 7 bp).
No regional downsampling for Eligo chunk #1666: region too short (contig_922:6751-6782,+; 30 bp).
No regional downsampling for Eligo chunk #1666: region too short (contig_922:7694-7751,+; 56 bp).
Regional downsampling: 8 reads from Eligo chunk #1666 (contig_922:4933-5496,-; 562 bp).
Localized downsampling: 124 reads from Eligo chunk #1666.
local:2/833/100%/66.6s No regional downsampling for Eligo chunk #1667: region too short (contig_922:10533-10713,+; 179 bp).
No regional downsampling for Eligo chunk #1667: region too short (contig_922:10720-10789,+; 68 bp).
Regional downsampling: 148 reads from Eligo chunk #1667 (contig_922:10825-12344,+; 1518 bp).
No regional downsampling for Eligo chunk #1667: region too short (contig_922:11291-11292,-; 0 bp).
Regional downsampling: 0 reads from Eligo chunk #1667 (contig_922:11397-11622,-; 224 bp).
No regional downsampling for Eligo chunk #1667: region too short (contig_922:11637-11639,-; 1 bp).
No regional downsampling for Eligo chunk #1667: region too short (contig_922:11642-11648,-; 5 bp).
No regional downsampling for Eligo chunk #1667: region too short (contig_922:11655-11659,-; 3 bp).
No regional downsampling for Eligo chunk #1667: region too short (contig_922:11664-11691,-; 26 bp).
No regional downsampling for Eligo chunk #1667: region too short (contig_922:11707-11733,-; 25 bp).
No regional downsampling for Eligo chunk #1667: region too short (contig_922:11735-11736,-; 0 bp).
No regional downsampling for Eligo chunk #1667: region too short (contig_922:11738-11741,-; 2 bp).
No regional downsampling for Eligo chunk #1667: region too short (contig_922:11744-11856,-; 111 bp).
No regional downsampling for Eligo chunk #1667: region too short (contig_922:11877-11879,-; 1 bp).
No regional downsampling for Eligo chunk #1667: region too short (contig_922:11890-11917,-; 26 bp).
No regional downsampling for Eligo chunk #1667: region too short (contig_922:11930-11946,-; 15 bp).
No regional downsampling for Eligo chunk #1667: region too short (contig_922:14722-14726,-; 3 bp).
No regional downsampling for Eligo chunk #1667: region too short (contig_922:14736-14737,-; 0 bp).
No regional downsampling for Eligo chunk #1667: region too short (contig_922:14739-14742,-; 2 bp).
No regional downsampling for Eligo chunk #1667: region too short (contig_922:14746-14754,-; 7 bp).
Localized downsampling: 50 reads from Eligo chunk #1667.
Regional downsampling: 1119 reads from Eligo chunk #1668 (contig_922:16036-20649,+; 4612 bp).
No regional downsampling for Eligo chunk #1668: region too short (contig_922:16747-16748,-; 0 bp).
No regional downsampling for Eligo chunk #1668: region too short (contig_922:16755-16773,-; 17 bp).
No regional downsampling for Eligo chunk #1668: region too short (contig_922:16790-16793,-; 2 bp).
No regional downsampling for Eligo chunk #1668: region too short (contig_922:16848-16854,-; 5 bp).
No regional downsampling for Eligo chunk #1668: region too short (contig_922:16891-16917,-; 25 bp).
No regional downsampling for Eligo chunk #1668: region too short (contig_922:16930-16932,-; 1 bp).
No regional downsampling for Eligo chunk #1668: region too short (contig_922:16935-16941,-; 5 bp).
No regional downsampling for Eligo chunk #1668: region too short (contig_922:16943-16948,-; 4 bp).
No regional downsampling for Eligo chunk #1668: region too short (contig_922:16954-16955,-; 0 bp).
No regional downsampling for Eligo chunk #1668: region too short (contig_922:17128-17132,-; 3 bp).
No regional downsampling for Eligo chunk #1668: region too short (contig_922:17142-17153,-; 10 bp).
No regional downsampling for Eligo chunk #1668: region too short (contig_922:17205-17237,-; 31 bp).
No regional downsampling for Eligo chunk #1668: region too short (contig_922:17246-17248,-; 1 bp).
Regional downsampling: 674 reads from Eligo chunk #1668 (contig_922:17277-20588,-; 3310 bp).
Localized downsampling: 111 reads from Eligo chunk #1668.
local:2/835/100%/66.6s Regional downsampling: 1645 reads from Eligo chunk #1669 (contig_922:19217-25498,+; 6280 bp).
Regional downsampling: 1628 reads from Eligo chunk #1669 (contig_922:19390-25519,-; 6128 bp).
Localized downsampling: 49 reads from Eligo chunk #1669.
Regional downsampling: 864 reads from Eligo chunk #1670 (contig_922:24407-28316,+; 3908 bp).
No regional downsampling for Eligo chunk #1670: region too short (contig_922:28327-28333,+; 5 bp).
No regional downsampling for Eligo chunk #1670: region too short (contig_922:28336-28430,+; 93 bp).
No regional downsampling for Eligo chunk #1670: region too short (contig_922:28433-28438,+; 4 bp).
No regional downsampling for Eligo chunk #1670: region too short (contig_922:28451-28628,+; 176 bp).
No regional downsampling for Eligo chunk #1670: region too short (contig_922:28631-28636,+; 4 bp).
Regional downsampling: 1151 reads from Eligo chunk #1670 (contig_922:24385-29020,-; 4634 bp).
Localized downsampling: 62 reads from Eligo chunk #1670.
local:2/946/100%/66.4s Random contig downsampling: 5803 reads from EligoWGA chunk #1894 (contig_502:1-6667,+; 6665 bp).
Random contig downsampling: 5608 reads from EligoWGA chunk #1894 (contig_502:1-6667,-; 6665 bp).
Localized downsampling: 3138 reads from EligoWGA chunk #1894.
Regional downsampling: 2447 reads from Eligo chunk #1894 (contig_502:1-5716,+; 5714 bp).
Regional downsampling: 2452 reads from Eligo chunk #1894 (contig_502:1-5606,-; 5604 bp).
local:2/950/100%/66.6s Random contig downsampling: 2315 reads from EligoWGA chunk #1895 (contig_502:1-6667,+; 6665 bp).
Random contig downsampling: 2132 reads from EligoWGA chunk #1895 (contig_502:1-6667,-; 6665 bp).
Localized downsampling: 5751 reads from EligoWGA chunk #1895.
Regional downsampling: 901 reads from Eligo chunk #1895 (contig_502:4184-6667,+; 2482 bp).
Regional downsampling: 318 reads from Eligo chunk #1895 (contig_502:4330-6655,-; 2324 bp).
Localized downsampling: 127 reads from Eligo chunk #1895.
local:2/1094/100%/66.3s Random contig downsampling: 13845 reads from EligoWGA chunk #2190 (contig_944:1-6857,+; 6855 bp).
Random contig downsampling: 13971 reads from EligoWGA chunk #2190 (contig_944:1-6857,-; 6855 bp).
Localized downsampling: 2051 reads from EligoWGA chunk #2190.
Error in { : task 2 failed - "task 1 failed - "invalid 'size' argument""
Calls: prepare.index -> %do% ->
Execution halted
local:2/1098/100%/66.5s Random contig downsampling: 10880 reads from EligoWGA chunk #2191 (contig_944:1-6857,+; 6855 bp).
Random contig downsampling: 10760 reads from EligoWGA chunk #2191 (contig_944:1-6857,-; 6855 bp).
Localized downsampling: 3777 reads from EligoWGA chunk #2191.
Regional downsampling: 2528 reads from Eligo chunk #2191 (contig_944:2168-6857,+; 4688 bp).
Regional downsampling: 2429 reads from Eligo chunk #2191 (contig_944:2177-6857,-; 4679 bp).
local:2/1105/100%/66.5s No regional downsampling for EligoWGA chunk #2210: region too short (contig_205:8-116,-; 107 bp).
Localized downsampling: 234 reads from EligoWGA chunk #2210.
No regional downsampling for Eligo chunk #2210: region too short (contig_205:1-191,-; 189 bp).
Localized downsampling: 3028 reads from Eligo chunk #2210.
local:2/1135/100%/66.4s Regional downsampling: 736 reads from Eligo chunk #2272 (contig_893:6531-10064,+; 3532 bp).
No regional downsampling for Eligo chunk #2272: region too short (contig_893:6393-6406,-; 12 bp).
No regional downsampling for Eligo chunk #2272: region too short (contig_893:6410-6539,-; 128 bp).
Regional downsampling: 744 reads from Eligo chunk #2272 (contig_893:6552-10098,-; 3545 bp).
Localized downsampling: 5 reads from Eligo chunk #2272.
local:2/1136/100%/66.4s Regional downsampling: 0 reads from Eligo chunk #2273 (contig_893:9955-10166,+; 210 bp).
Regional downsampling: 13 reads from Eligo chunk #2273 (contig_893:9900-10291,-; 390 bp).
Localized downsampling: 112 reads from Eligo chunk #2273.
local:2/1381/100%/66.5s Regional downsampling: 3 reads from EligoWGA chunk #2764 (contig_973:9731-10407,+; 675 bp).
No regional downsampling for EligoWGA chunk #2764: region too short (contig_973:9230-9239,-; 8 bp).
No regional downsampling for EligoWGA chunk #2764: region too short (contig_973:9292-9299,-; 6 bp).
Regional downsampling: 17 reads from EligoWGA chunk #2764 (contig_973:9307-10535,-; 1227 bp).
Localized downsampling: 155 reads from EligoWGA chunk #2764.
local:2/1382/100%/67.0s Regional downsampling: 419 reads from EligoWGA chunk #2765 (contig_973:9788-15720,+; 5931 bp).
Regional downsampling: 396 reads from EligoWGA chunk #2765 (contig_973:9757-15687,-; 5929 bp).
Localized downsampling: 88 reads from EligoWGA chunk #2765.
Regional downsampling: 410 reads from EligoWGA chunk #2766 (contig_973:14101-20351,+; 6249 bp).
Regional downsampling: 448 reads from EligoWGA chunk #2766 (contig_973:14019-20570,-; 6550 bp).
Localized downsampling: 73 reads from EligoWGA chunk #2766.
local:2/1383/100%/66.9s Regional downsampling: 316 reads from EligoWGA chunk #2767 (contig_973:19526-24642,+; 5115 bp).
No regional downsampling for EligoWGA chunk #2767: region too short (contig_973:24645-24656,+; 10 bp).
No regional downsampling for EligoWGA chunk #2767: region too short (contig_973:24658-24665,+; 6 bp).
No regional downsampling for EligoWGA chunk #2767: region too short (contig_973:24667-24697,+; 29 bp).
Regional downsampling: 318 reads from EligoWGA chunk #2767 (contig_973:19015-24312,-; 5296 bp).
Localized downsampling: 46 reads from EligoWGA chunk #2767.
local:2/1413/100%/67.0s Regional downsampling: 2643 reads from EligoWGA chunk #2828 (contig_4:2-7160,-; 7157 bp).
Error in { : task 1 failed - "task 1 failed - "invalid 'size' argument""
Calls: prepare.index -> %do% ->
Execution halted
local:2/1421/100%/67.0s Random downsampling: 1019 reads from EligoWGA chunk #2841 (contig_1:1-920,+; 918 bp).
Random downsampling: 964 reads from EligoWGA chunk #2841 (contig_1:1-920,-; 918 bp).
local:2/1447/100%/66.9s No regional downsampling for EligoWGA chunk #2896: region too short (contig_777:1-59,+; 57 bp).
No regional downsampling for EligoWGA chunk #2896: region too short (contig_777:1-59,-; 57 bp).
No regional downsampling for EligoWGA chunk #2896: region too short (contig_777:61-107,-; 45 bp).
Localized downsampling: 2851 reads from EligoWGA chunk #2896.
No regional downsampling for Eligo chunk #2896: region too short (contig_777:1-60,+; 58 bp).
No regional downsampling for Eligo chunk #2896: region too short (contig_777:1-107,-; 105 bp).
Localized downsampling: 7794 reads from Eligo chunk #2896.
local:2/1531/100%/67.9s Regional downsampling: 468 reads from EligoWGA chunk #3063 (contig_998:2585-7507,+; 4921 bp).
Regional downsampling: 930 reads from EligoWGA chunk #3063 (contig_998:2968-11062,-; 8093 bp).
Localized downsampling: 586 reads from EligoWGA chunk #3063.
No regional downsampling for EligoWGA chunk #3064: region too long (contig_998:3622-15693,+; 12070 bp).
No regional downsampling for EligoWGA chunk #3064: region too long (contig_998:3137-15092,-; 11954 bp).
Localized downsampling: 5172 reads from EligoWGA chunk #3064.
local:2/1532/100%/68.4s No regional downsampling for EligoWGA chunk #3065: region too long (contig_998:6772-22369,+; 15596 bp).
No regional downsampling for EligoWGA chunk #3065: region too long (contig_998:4568-18893,-; 14324 bp).
Localized downsampling: 6429 reads from EligoWGA chunk #3065.
No regional downsampling for EligoWGA chunk #3066: region too long (contig_998:9580-25243,+; 15662 bp).
No regional downsampling for EligoWGA chunk #3066: region too long (contig_998:9710-26255,-; 16544 bp).
Localized downsampling: 5451 reads from EligoWGA chunk #3066.
local:2/1533/100%/69.7s No regional downsampling for EligoWGA chunk #3069: region too long (contig_998:25576-38937,+; 13360 bp).
No regional downsampling for EligoWGA chunk #3069: region too long (contig_998:22711-35972,-; 13260 bp).
Localized downsampling: 5693 reads from EligoWGA chunk #3069.
Regional downsampling: 1620 reads from EligoWGA chunk #3070 (contig_998:30961-40324,+; 9362 bp).
Regional downsampling: 256 reads from EligoWGA chunk #3070 (contig_998:29651-36324,-; 6672 bp).
Localized downsampling: 606 reads from EligoWGA chunk #3070.
local:2/1534/100%/70.0s Regional downsampling: 9 reads from EligoWGA chunk #3071 (contig_998:38988-40392,+; 1403 bp).
Localized downsampling: 125 reads from EligoWGA chunk #3071.
local:2/1536/100%/70.0s No regional downsampling for EligoWGA chunk #3067: region too long (contig_998:11464-29307,+; 17842 bp).
No regional downsampling for EligoWGA chunk #3067: region too long (contig_998:15851-32325,-; 16473 bp).
Localized downsampling: 7507 reads from EligoWGA chunk #3067.
No regional downsampling for EligoWGA chunk #3068: region too long (contig_998:19681-34064,+; 14382 bp).
No regional downsampling for EligoWGA chunk #3068: region too long (contig_998:18602-35194,-; 16591 bp).
Localized downsampling: 7165 reads from EligoWGA chunk #3068.
local:2/1563/100%/70.0s No regional downsampling for Eligo chunk #3127: region too short (contig_488:6212-6248,-; 35 bp).
Localized downsampling: 428 reads from Eligo chunk #3127.
local:2/1572/100%/71.5s No regional downsampling for EligoWGA chunk #3145: region too long (contig_974:567-15433,+; 14865 bp).
No regional downsampling for EligoWGA chunk #3145: region too long (contig_974:573-15310,-; 14736 bp).
Localized downsampling: 6466 reads from EligoWGA chunk #3145.
No regional downsampling for EligoWGA chunk #3146: region too long (contig_974:787-17360,+; 16572 bp).
No regional downsampling for EligoWGA chunk #3146: region too long (contig_974:902-17379,-; 16476 bp).
Localized downsampling: 10022 reads from EligoWGA chunk #3146.
local:2/1573/100%/71.4s No regional downsampling for EligoWGA chunk #3147: region too long (contig_974:1547-17793,+; 16245 bp).
No regional downsampling for EligoWGA chunk #3147: region too long (contig_974:2220-17718,-; 15497 bp).
Localized downsampling: 9443 reads from EligoWGA chunk #3147.
No regional downsampling for EligoWGA chunk #3148: region too long (contig_974:4778-18252,+; 13473 bp).
No regional downsampling for EligoWGA chunk #3148: region too long (contig_974:4891-18104,-; 13212 bp).
Localized downsampling: 4579 reads from EligoWGA chunk #3148.
local:2/1679/100%/71.7s No regional downsampling for EligoWGA chunk #3361: region too long (contig_981:3131-16875,+; 13743 bp).
No regional downsampling for EligoWGA chunk #3361: region too long (contig_981:3362-16994,-; 13631 bp).
Localized downsampling: 5298 reads from EligoWGA chunk #3361.
Regional downsampling: 933 reads from EligoWGA chunk #3362 (contig_981:8621-17461,+; 8839 bp).
Regional downsampling: 960 reads from EligoWGA chunk #3362 (contig_981:8493-17500,-; 9006 bp).
No regional downsampling for EligoWGA chunk #3362: region too short (contig_981:17506-17514,-; 7 bp).
Localized downsampling: 504 reads from EligoWGA chunk #3362.
local:2/1680/100%/71.8s No regional downsampling for EligoWGA chunk #3359: region too long (contig_981:1002-12867,+; 11864 bp).
No regional downsampling for EligoWGA chunk #3359: region too short (contig_981:1159-1162,-; 2 bp).
No regional downsampling for EligoWGA chunk #3359: region too long (contig_981:1164-11838,-; 10673 bp).
Localized downsampling: 3186 reads from EligoWGA chunk #3359.
No regional downsampling for EligoWGA chunk #3360: region too long (contig_981:1641-15968,+; 14326 bp).
No regional downsampling for EligoWGA chunk #3360: region too long (contig_981:2178-16177,-; 13998 bp).
Localized downsampling: 5874 reads from EligoWGA chunk #3360.
local:2/1681/100%/71.8s Regional downsampling: 222 reads from EligoWGA chunk #3363 (contig_986:2926-6098,+; 3171 bp).
Localized downsampling: 171 reads from EligoWGA chunk #3363.
Regional downsampling: 138 reads from EligoWGA chunk #3364 (contig_986:3351-6101,+; 2749 bp).
Localized downsampling: 157 reads from EligoWGA chunk #3364.
local:2/1809/100%/71.5s Random downsampling: 9061 reads from EligoWGA chunk #3619 (contig_38:1-2179,+; 2177 bp).
Random downsampling: 9188 reads from EligoWGA chunk #3619 (contig_38:1-2179,-; 2177 bp).
Random downsampling: 2050 reads from Eligo chunk #3619 (contig_38:1-2179,+; 2177 bp).
Random downsampling: 2046 reads from Eligo chunk #3619 (contig_38:1-2179,-; 2177 bp).
local:2/1864/100%/71.3s No regional downsampling for Eligo chunk #3728: region too short (contig_88:245-302,-; 56 bp).
Localized downsampling: 465 reads from Eligo chunk #3728.
local:2/1974/100%/71.1s Regional downsampling: 1518 reads from EligoWGA chunk #3948 (contig_41:1027544-1027766,-; 221 bp).
Localized downsampling: 220 reads from EligoWGA chunk #3948.
Regional downsampling: 1121 reads from Eligo chunk #3948 (contig_41:1027548-1027761,-; 212 bp).
Localized downsampling: 288 reads from Eligo chunk #3948.
local:2/2014/100%/71.4s No regional downsampling for EligoWGA chunk #4030: region too long (contig_928:6-16883,+; 16876 bp).
No regional downsampling for EligoWGA chunk #4030: region too long (contig_928:4-16864,-; 16859 bp).
Localized downsampling: 11437 reads from EligoWGA chunk #4030.
local:2/2015/100%/71.6s No regional downsampling for EligoWGA chunk #4033: region too long (contig_928:1945-16890,+; 14944 bp).
No regional downsampling for EligoWGA chunk #4033: region too long (contig_928:2370-16880,-; 14509 bp).
Localized downsampling: 3637 reads from EligoWGA chunk #4033.
local:2/2041/100%/72.1s No regional downsampling for EligoWGA chunk #4031: region too long (contig_928:6-16889,+; 16882 bp).
No regional downsampling for EligoWGA chunk #4031: region too long (contig_928:5-16878,-; 16872 bp).
Localized downsampling: 17163 reads from EligoWGA chunk #4031.
No regional downsampling for EligoWGA chunk #4032: region too long (contig_928:6-16889,+; 16882 bp).
No regional downsampling for EligoWGA chunk #4032: region too long (contig_928:5-16879,-; 16873 bp).
Localized downsampling: 14261 reads from EligoWGA chunk #4032.
Regional downsampling: 74 reads from Eligo chunk #4031 (contig_928:5364-6685,-; 1320 bp).
No regional downsampling for Eligo chunk #4031: region too short (contig_928:6721-6722,-; 0 bp).
No regional downsampling for Eligo chunk #4031: region too short (contig_928:6817-6851,-; 33 bp).
No regional downsampling for Eligo chunk #4031: region too short (contig_928:6857-6859,-; 1 bp).
No regional downsampling for Eligo chunk #4031: region too short (contig_928:6931-6932,-; 0 bp).
No regional downsampling for Eligo chunk #4031: region too short (contig_928:6942-6955,-; 12 bp).
No regional downsampling for Eligo chunk #4031: region too short (contig_928:6962-6963,-; 0 bp).
No regional downsampling for Eligo chunk #4031: region too short (contig_928:7009-7031,-; 21 bp).
Regional downsampling: 66 reads from Eligo chunk #4031 (contig_928:7189-8617,-; 1427 bp).
Regional downsampling: 279 reads from Eligo chunk #4031 (contig_928:5462-8470,+; 3007 bp).
No regional downsampling for Eligo chunk #4031: region too short (contig_928:8476-8480,+; 3 bp).
Localized downsampling: 111 reads from Eligo chunk #4031.
Regional downsampling: 143 reads from Eligo chunk #4032 (contig_928:10201-12013,-; 1811 bp).
No regional downsampling for Eligo chunk #4032: region too short (contig_928:10542-10547,+; 4 bp).
No regional downsampling for Eligo chunk #4032: region too short (contig_928:10688-10692,+; 3 bp).
No regional downsampling for Eligo chunk #4032: region too short (contig_928:10733-10734,+; 0 bp).
No regional downsampling for Eligo chunk #4032: region too short (contig_928:10738-10853,+; 114 bp).
No regional downsampling for Eligo chunk #4032: region too short (contig_928:10867-10877,+; 9 bp).
Regional downsampling: 15 reads from Eligo chunk #4032 (contig_928:10901-11656,+; 754 bp).
No regional downsampling for Eligo chunk #4032: region too short (contig_928:11682-11687,+; 4 bp).
No regional downsampling for Eligo chunk #4032: region too short (contig_928:11689-11691,+; 1 bp).
No regional downsampling for Eligo chunk #4032: region too short (contig_928:11848-11851,+; 2 bp).
Localized downsampling: 83 reads from Eligo chunk #4032.
local:2/2146/100%/71.7s Regional downsampling: 667 reads from EligoWGA chunk #4294 (contig_985:3675-5208,+; 1532 bp).
Regional downsampling: 581 reads from EligoWGA chunk #4294 (contig_985:3675-5140,-; 1464 bp).
Localized downsampling: 85 reads from EligoWGA chunk #4294.
local:2/2147/100%/71.7s Regional downsampling: 143 reads from EligoWGA chunk #4295 (contig_985:3678-5208,+; 1529 bp).
Regional downsampling: 101 reads from EligoWGA chunk #4295 (contig_985:3678-5140,-; 1461 bp).
Localized downsampling: 486 reads from EligoWGA chunk #4295.
local:2/2148/100%/71.7s Regional downsampling: 461 reads from EligoWGA chunk #4298 (contig_985:21468-22889,+; 1420 bp).
Regional downsampling: 547 reads from EligoWGA chunk #4298 (contig_985:21468-22894,-; 1425 bp).
Localized downsampling: 113 reads from EligoWGA chunk #4298.
local:2/2165/100%/71.9s Regional downsampling: 1443 reads from EligoWGA chunk #4332 (contig_174:39783-43210,+; 3426 bp).
Regional downsampling: 1327 reads from EligoWGA chunk #4332 (contig_174:39783-43210,-; 3426 bp).
Regional downsampling: 389 reads from Eligo chunk #4332 (contig_174:39685-43208,+; 3522 bp).
Regional downsampling: 1010 reads from Eligo chunk #4332 (contig_174:39642-43210,-; 3567 bp).
Localized downsampling: 521 reads from Eligo chunk #4332.
local:2/2166/100%/72.0s Regional downsampling: 2552 reads from EligoWGA chunk #4333 (contig_174:39783-43210,+; 3426 bp).
Regional downsampling: 2437 reads from EligoWGA chunk #4333 (contig_174:39783-43210,-; 3426 bp).
Regional downsampling: 9254 reads from EligoWGA chunk #4334 (contig_247:957-5404,+; 4446 bp).
Regional downsampling: 9157 reads from EligoWGA chunk #4334 (contig_247:957-5394,-; 4436 bp).
Regional downsampling: 2193 reads from Eligo chunk #4333 (contig_174:39685-43210,+; 3524 bp).
Regional downsampling: 2180 reads from Eligo chunk #4333 (contig_174:39685-43210,-; 3524 bp).
local:2/2167/100%/72.0s Regional downsampling: 6817 reads from EligoWGA chunk #4335 (contig_247:957-5416,+; 4458 bp).
Regional downsampling: 6515 reads from EligoWGA chunk #4335 (contig_247:957-5395,-; 4437 bp).
local:2/2169/100%/72.0s Regional downsampling: 10967 reads from EligoWGA chunk #4340 (contig_856:1-7763,+; 7761 bp).
Regional downsampling: 10812 reads from EligoWGA chunk #4340 (contig_856:1-7763,-; 7761 bp).
Regional downsampling: 1473 reads from Eligo chunk #4340 (contig_856:3-5712,+; 5708 bp).
Regional downsampling: 1492 reads from Eligo chunk #4340 (contig_856:3-5767,-; 5763 bp).
Localized downsampling: 55 reads from Eligo chunk #4340.
local:2/2170/100%/72.0s Regional downsampling: 9030 reads from EligoWGA chunk #4341 (contig_856:1-7763,+; 7761 bp).
Regional downsampling: 8879 reads from EligoWGA chunk #4341 (contig_856:1-7763,-; 7761 bp).
Regional downsampling: 776 reads from Eligo chunk #4341 (contig_856:3983-7381,+; 3397 bp).
Regional downsampling: 693 reads from Eligo chunk #4341 (contig_856:4218-7381,-; 3162 bp).
Localized downsampling: 105 reads from Eligo chunk #4341.
local:2/2222/100%/71.7s Regional downsampling: 359 reads from EligoWGA chunk #4443 (contig_540:4-549,+; 544 bp).
Random downsampling: 522 reads from EligoWGA chunk #4443 (contig_540:1-549,-; 547 bp).
local:2/2248/100%/71.7s No regional downsampling for EligoWGA chunk #4495: region too short (contig_201:9096-9133,-; 36 bp).
No regional downsampling for EligoWGA chunk #4495: region too short (contig_201:9168-9178,-; 9 bp).
Localized downsampling: 6 reads from EligoWGA chunk #4495.
local:2/2651/100%/72.3s Regional downsampling: 1041 reads from EligoWGA chunk #5301 (contig_52:2939941-2941468,-; 1526 bp).
Regional downsampling: 1008 reads from EligoWGA chunk #5301 (contig_52:2939941-2941468,+; 1526 bp).
Regional downsampling: 1118 reads from EligoWGA chunk #5302 (contig_52:2939941-2941468,+; 1526 bp).
Regional downsampling: 1151 reads from EligoWGA chunk #5302 (contig_52:2939941-2941468,-; 1526 bp).
local:2/2739/100%/72.7s Random downsampling: 5274 reads from EligoWGA chunk #5479 (contig_501:1-590,+; 588 bp).
Random downsampling: 5188 reads from EligoWGA chunk #5479 (contig_501:1-590,-; 588 bp).
Random downsampling: 446 reads from EligoWGA chunk #5480 (contig_935:1-3054,+; 3052 bp).
Random downsampling: 445 reads from EligoWGA chunk #5480 (contig_935:1-3054,-; 3052 bp).
Localized downsampling: 1722 reads from EligoWGA chunk #5480.
Regional downsampling: 100 reads from Eligo chunk #5479 (contig_501:4-588,+; 583 bp).
Regional downsampling: 27 reads from Eligo chunk #5479 (contig_501:21-588,-; 566 bp).
Localized downsampling: 168 reads from Eligo chunk #5479.
local:2/2748/100%/72.7s Regional downsampling: 1355 reads from EligoWGA chunk #5498 (contig_37:302-735,+; 432 bp).
Regional downsampling: 1310 reads from EligoWGA chunk #5498 (contig_37:287-735,-; 447 bp).
local:2/2752/100%/72.8s No regional downsampling for EligoWGA chunk #5506: region too long (contig_852:1-11322,+; 11320 bp).
No regional downsampling for EligoWGA chunk #5506: region too long (contig_852:1-11322,-; 11320 bp).
Localized downsampling: 38798 reads from EligoWGA chunk #5506.
Regional downsampling: 1183 reads from Eligo chunk #5506 (contig_852:3682-5211,-; 1528 bp).
Regional downsampling: 1208 reads from Eligo chunk #5506 (contig_852:3682-5211,+; 1528 bp).
local:2/2777/100%/73.1s Regional downsampling: 1334 reads from EligoWGA chunk #5558 (contig_895:3452-4766,+; 1313 bp).
Regional downsampling: 1258 reads from EligoWGA chunk #5558 (contig_895:3452-4766,-; 1313 bp).
Regional downsampling: 288 reads from Eligo chunk #5558 (contig_895:3452-4752,+; 1299 bp).
Regional downsampling: 99 reads from Eligo chunk #5558 (contig_895:3460-4749,-; 1288 bp).
Localized downsampling: 152 reads from Eligo chunk #5558.
local:2/2782/100%/73.2s No regional downsampling for EligoWGA chunk #5507: region too long (contig_852:1-11322,+; 11320 bp).
No regional downsampling for EligoWGA chunk #5507: region too long (contig_852:1-11322,-; 11320 bp).
Localized downsampling: 44280 reads from EligoWGA chunk #5507.
No regional downsampling for EligoWGA chunk #5508: region too long (contig_852:1-11322,+; 11320 bp).
No regional downsampling for EligoWGA chunk #5508: region too long (contig_852:1-11322,-; 11320 bp).
Localized downsampling: 12908 reads from EligoWGA chunk #5508.
Regional downsampling: 384 reads from Eligo chunk #5507 (contig_852:3704-5211,-; 1506 bp).
Regional downsampling: 1242 reads from Eligo chunk #5507 (contig_852:7541-9070,-; 1528 bp).
Regional downsampling: 338 reads from Eligo chunk #5507 (contig_852:3704-5211,+; 1506 bp).
Regional downsampling: 1226 reads from Eligo chunk #5507 (contig_852:7541-9070,+; 1528 bp).
Localized downsampling: 627 reads from Eligo chunk #5507.
local:2/2892/100%/72.7s Random downsampling: 1901 reads from EligoWGA chunk #5785 (contig_249:1-2533,+; 2531 bp).
Random downsampling: 1785 reads from EligoWGA chunk #5785 (contig_249:1-2533,-; 2531 bp).
Localized downsampling: 2927 reads from EligoWGA chunk #5785.
local:2/2901/100%/72.6s Random downsampling: 1719 reads from EligoWGA chunk #5803 (contig_248:1-2533,+; 2531 bp).
Random downsampling: 1692 reads from EligoWGA chunk #5803 (contig_248:1-2533,-; 2531 bp).
Localized downsampling: 2961 reads from EligoWGA chunk #5803.
local:2/2916/100%/72.6s Random downsampling: 1890 reads from EligoWGA chunk #5833 (contig_250:1-2533,+; 2531 bp).
Random downsampling: 1928 reads from EligoWGA chunk #5833 (contig_250:1-2533,-; 2531 bp).
Localized downsampling: 2909 reads from EligoWGA chunk #5833.
local:0/2921/100%/72.5s

[ecpierce]

@touala My issue seems similar to this issue #53
My merged file only has positions reported are only every 5000 bp and only reverse sequences

[touala]

Hello @ecpierce,

Thank you for using nanodisco. The numerous downsampling message indicate that there is mapped data on contigs. My next best guess would be an issue with fast5 format, likely vbz compression. I've had issues like that: #23, #26, #28, #31, #34, #46, #58.

The fast5 compression solution (i.e. vbz) broke the compatibility with the older nanopolish's version I'm still currently using. You can use ont-fast5-api to convert the fast5 encoding to gzip with:

compress_fast5 -t 20 --recursive -i <input_vbz.fast5> -s <input_gzip.fast5> -c gzip

Please let me know if this fixed you issue.

Best,

Alan

[ecpierce]

Hi @touala ,

Thanks for your response. I reran with gzip compressed fast5 files, but I am having the same issue. Everything seems to run fine, I get downsampling messages, but the .rds files look the same (reported every 5000 bp and only reverse sequences and otherwise empty). Do you have other suggestions?

Best,
Emily

[touala]

Hi @ecpierce,

Would it be possible to share some of your data (fast5 + reference) so I can take a look?

Alan

[pms-bio]

Hi, I do have the same problem (output from merge step with zero differences).
I've tried different guppy versions for --fast5_out and with compressed and non-compressed. I also tested with different models .cfg wondering if that would affect (even if I didn't think so) Then I checked the generated .bams in the preprocess with modbam2bed and neither native or wga datasets seem to have modbase annotations. Thus, it seems normal that the nanodisco difference step will output "no differences".

Furthermore, during guppy_basecaller to get --fast5_out, I also flagged --bam_out and did a rough evaluation of the generated *.bam for modbases using modbam2bed. Those bams actually include a wide number of putative mod bases and there is a sharp difference (as expected) between native and wga datasets. Therefore, the rawdata seems ok.

Thus, it seems that there is an issue related with the nanodisco preprocess step.
I was wondering if it would be feasible to feed nanodisco with --bam_out from guppy on the preprocess step? (this would be also helpful taking into account that --fast5_out was deprecated in the latest guppy versions)

I'll be looking forward for some hints to solve this
Pedro

[touala]

Hi @Pedroo,

Thank you for the feedback. In nanodisco, we do not use the modified base information outputted by Guppy but we directly use the raw signal leveraging nanopolish features. The .bam is used for mapping the read sequences to a reference and nanopolish linked the raw signal to specific read positions.

As you mentioned, it's possible that we are facing a new incompatibility from latest Guppy version. @Haidermanzer in #64 seems to have similar issue and offer to send some of his data. I'll try to diagnose from it but feel free to also share yours.

Alan

[pms-bio]

Hi @touala,
I just tested the same datasets with GUPPY version: 6.4.2+97a7f06 and NANOPLISH (v0.14.0) and everything went fine (modbases were detected). I also run mCaller (which requires nanopolish output) since I'm mostly interested in 6mA and it worked as expected when comparing wga vs native datasets. How much data do you require for it? I glad to share and help with this. I 'm looking forward to process all my data with nanodisco.

[ecpierce]

Hi @touala,
Sure, I'm happy to share the data with you. I will send it to the email you suggested for Haidermanzer in #64. Hopefully we are all having the same issue and can get it sorted out.

[touala]

Hello @ecpierce and @Pedroo,

I've just responded to #64 with a couple of commands sorting out an issue similar to yours (here). Please could you take a look and let me know if this is helping?

Thank you very much @Pedroo for helping troubleshooting.

And thank you @ecpierce for sharing your dataset. I've not downloaded it yet as I was focusing on the one from #64's user. If the issue remains, I'll look into yours.

Best,

Alan

[ecpierce]

@touala I tried with your suggestions from #64 and I'm still not getting data for any chunks (at least on a subset of the data). Do you mind trying with my dataset and see if you can see anything?

[touala]

Sure, no problem. I'm downloading the dataset now. Will get back to you ASAP.

[touala]

I think I've found the issue. It appears that the dataset is generated using R10 chemistry. Unfortunately, we currently only support R9.4 pores. The fundamental differences in pore structure between R9 and R10 do not allow using the method as is. This is mentioned in FAQ Q9. To be complete, we did not validate nanodisco on PromethION datasets either, but this should work.

[ecpierce]

Hi @touala, thanks for figuring that out. I appreciate your help working through this. Such a bummer though. Are you actively developing the method for R10, and if so, do you know when that might be ready? Since nanopore is actively discontinuing R9, I'm sure there will be a lot of interest in nanodisco that works for newer chemistries.

[ecpierce]

@touala follow-up question- do you know of any currently existing tools similar to nanodisco that do work with R10?

[touala]

Hi @ecpierce. Yes we are currently working on R10.4.1 support. The fast turnaround time of ONT kits' release made us reluctant to invest a lot of time early on, but with the discontinuation of R9 we are ready to move on. We already have a clear roadmap for uplifting the method but, unfortunately, I don't have clear ETA as it will mostly depend on unknown unknowns...

[pms-bio]

@touala
I've tested :
compress_fast5 -t 20 --recursive -c gzip -i Nativefast5_basecalled -s Nativefast5_basecalled_gzip
compress_fast5 -t 20 --recursive -c gzip -i WGAfast5_basecalled -s WGAfast5_basecalled_gzip

and it solved the issue of no chunks. Meanwhile, when I run "nanodisco characterize" it refers "Models for Guppy version 6.3.8+d9e0f64 is not yet available" should I change Guppy version or is it fine?

[fanggang]

Emily, for your question below: to the best of our knowledge, currently no other tools can do comprehensive de novo discovery of bacterial DNA methylation motifs and methylation typing. As Alan replied, we are highly motivated to continue supporting nanodisco for R10, and are actively doing our best to get it ready for users ASAP. Thank you for your interest. Gang

…

On Feb 14, 2023, at 2:32 PM, Emily Weiss ***@***.***> wrote:  @touala follow-up question- do you know of any currently existing tools similar to nanodisco that do work with R10? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[touala]

Great to hear @Pedroo,

No, you do not have to change version. This is a warning we implemented early on in case new Guppy version widely affect signal properties but the initial characterisation model still holds.

Alan