v1.7.0: Add quality upgrade during download

New feature: Quality upgrade

When downloading a stream, FC2 may not expose the 3Mbps URL at the start of the download (because the streamer was streaming below 3MBps, for example). At the time, the solution proposed by HoloArchivists is to use waitForQualityMaxTries (equivalent to wait-for-quality-timeout in the original project), which waits for the download.

However, there's a problem: the 3Mbps URL may appear later, perhaps 1, 5, 10 minutes... depending on the streamer's bitrate. Because of this, the waitForQuality feature may delete the beginning of the video.

A new feature is added to handle this issue: quality upgrade during download.

allowQualityUpgrade and pollQualityUpgradeInterval have been added to the configuration file:

   ## If the requested quality is not available, keep retrying before falling
   ## back to the next best quality. (default: 60)
   ##
   ## There is a 1 second delay between each retry. The value must be big enough
   ## so that the best quality (3Mbps) is available. If your streamer takes more
   ## than expected to prepare, you should increase this value.
   waitForQualityMaxTries: 60
+  ## EXPERIMENTAL: Allow quality upgrade during download if the requested
+  ## quality is not "yet" available. (default: false)
+  ##
+  ## If the requested quality is not available, the downloader will fallback to
+  ## the best quality available. However, it is possible that the streamer will
+  ## upgrade the quality during the stream. FC2 often "waits" for the stream to
+  ## be stable before upgrading the quality.
+  ##
+  ## If this option is enabled, the downloader will periodically check if the
+  ## quality has been upgraded. If the quality has been upgraded, the downloader
+  ## will switch to the new quality. **A cut off will be present in the recording.**
+  ##
+  ## If this option is enabled, it is recommended to:
+  ##
+  ## - Reduce waitForQualityMaxTries to 10s.
+  ## - Enable Remux or Concat to fix mpegts discontinuities.
+  allowQualityUpgrade: false
+  ## How many seconds between checks to see if the quality can be upgraded. (default: 10s)
+  ##
+  ## allowQualityUpgrade needs to be enabled for this to work.
+  pollQualityUpgradeInterval: '10s'

Please read the comments before enabling this feature. A cut will be present in the recording. (similar to switching qualities on Youtube).

Fixes

• Mixed format (aac stream vs aac) will use intermediate fifos during concatenation to avoid failing aac_adtstoasc which can only be applied on the output file. FIFOs does not impact performance. However, Windows users will use intermediate files, which does. (#39)
• Memory leaks are now checked seriously with Valgrind.

SHA256 Checksums

de005106a94a7a1bfba26139e3cc8bd329ce0fdda1c1ad47c911818d22a9cdbd *fc2-live-dl-go-darwin-amd64
fa8bd9f1c57c6b258e6a9b177049c7afae92a08d3f9c721d1ebf88567b0a5cd4 *fc2-live-dl-go-darwin-arm64
11115f1e4311f2bfafaf6b49894bcc132b38d0abc264ad935b0381e1edf4b6b9 *fc2-live-dl-go-linux-amd64
9f356eafffac1053106f02591653cc171fff246aa034a0ab3b5f46b95fd6dd7f *fc2-live-dl-go-linux-arm64
e9ecc85b3dc6e108413ef849dbeabf5334ca955f9ef47551fcc58dd35105efdc *fc2-live-dl-go-linux-riscv64
a4a19cb372e66aaaca61347477fea42e22c1e6dbad31452769ccfc16c08cfaad *fc2-live-dl-go-windows-amd64.exe

Full Changelog: v1.6.5...v1.7.0

v1.7.0-rc.1: Add quality upgrade

New feature: Quality upgrade

When downloading a stream, FC2 may not expose the 3Mbps URL at the start of the download (because the streamer was streaming below 3MBps, for example). The solution proposed by HoloArchivists is to use waitForQualityMaxTries (wait-for-quality-timeout), which waits for the download.

However, there's a problem: the 3Mbps URL may appear later, perhaps 1, 5, 10 minutes... depending on the streamer's bitrate. Because of this, the waitForQuality feature may delete the beginning of the video.

A new feature is added to handle this issue: quality upgrade during download.

allowQualityUpgrade and pollQualityUpgradeInterval have been added to the configuration file:

   ## If the requested quality is not available, keep retrying before falling
   ## back to the next best quality. (default: 60)
   ##
   ## There is a 1 second delay between each retry. The value must be big enough
   ## so that the best quality (3Mbps) is available. If your streamer takes more
   ## than expected to prepare, you should increase this value.
   waitForQualityMaxTries: 60
+  ## EXPERIMENTAL: Allow quality upgrade during download if the requested
+  ## quality is not "yet" available. (default: false)
+  ##
+  ## If the requested quality is not available, the downloader will fallback to
+  ## the best quality available. However, it is possible that the streamer will
+  ## upgrade the quality during the stream. FC2 often "waits" for the stream to
+  ## be stable before upgrading the quality.
+  ##
+  ## If this option is enabled, the downloader will periodically check if the
+  ## quality has been upgraded. If the quality has been upgraded, the downloader
+  ## will switch to the new quality. **A cut off will be present in the recording.**
+  ##
+  ## If this option is enabled, it is recommended to:
+  ##
+  ## - Reduce waitForQualityMaxTries to 10s.
+  ## - Enable Remux or Concat to fix mpegts discontinuities.
+  allowQualityUpgrade: false
+  ## How many seconds between checks to see if the quality can be upgraded. (default: 10s)
+  ##
+  ## allowQualityUpgrade needs to be enabled for this to work.
+  pollQualityUpgradeInterval: '10s'

Please read the comments before enabling this feature. A cut will be present in the recording. (similar to switching qualities on Youtube).

Fixes

• Mixed format (aac stream vs aac) will use intermediate fifos during concatenation to avoid failing aac_adtstoasc which can only be applied on the output file. FIFOs does not impact performance. However, Windows users will use intermediate files. (#39)
• Memory leaks are now checked seriously with Valgrind.

SHA256 Checksums

de005106a94a7a1bfba26139e3cc8bd329ce0fdda1c1ad47c911818d22a9cdbd *fc2-live-dl-go-darwin-amd64
fa8bd9f1c57c6b258e6a9b177049c7afae92a08d3f9c721d1ebf88567b0a5cd4 *fc2-live-dl-go-darwin-arm64
11115f1e4311f2bfafaf6b49894bcc132b38d0abc264ad935b0381e1edf4b6b9 *fc2-live-dl-go-linux-amd64
9f356eafffac1053106f02591653cc171fff246aa034a0ab3b5f46b95fd6dd7f *fc2-live-dl-go-linux-arm64
e9ecc85b3dc6e108413ef849dbeabf5334ca955f9ef47551fcc58dd35105efdc *fc2-live-dl-go-linux-riscv64
a4a19cb372e66aaaca61347477fea42e22c1e6dbad31452769ccfc16c08cfaad *fc2-live-dl-go-windows-amd64.exe

Full Changelog: v1.6.5...v1.7.0-rc.1

v1.7.0-alpha.2 Fixed mixed format concatenation

Fixes

• Mixed format (aac stream vs aac) will use intermediate fifos during concatenation to avoid failing aac_adtstoasc which can only be applied on the output file. FIFOs does not impact performance. However, Windows users will use intermediate files. (#39)

SHA256 Checksums

98900e33bbfb999aba3b9c6dc9b64bdfec7715236b74decfc827d5482d0a6d42 *fc2-live-dl-go-darwin-amd64
dc3d306aee187bcba52647fc263d2e6fc0364db488fe434b526bc350930630dd *fc2-live-dl-go-darwin-arm64
59c52e0c067a90c83da4590daefd4017892ab93d1a1111b137e2ada93d7e8cd4 *fc2-live-dl-go-linux-amd64
64effc5bb1d0670e7acc9f4a900d1d126d012e86d53cc90b401fe36e1dda0125 *fc2-live-dl-go-linux-arm64
d4220499bdd326ffd3cb55e617cbcd5fcae8cf986d4419d31db0733c7fcb23ca *fc2-live-dl-go-linux-riscv64
885d77d3822c05a3de472488d0d3be585925abc3a1fecac25f3ec6e4acf70873 *fc2-live-dl-go-windows-amd64.exe

Full Changelog: v1.7.0-alpha.1...v1.7.0-alpha.2

v1.7.0-alpha.1: Add quality upgrade

New feature: Quality upgrade

When downloading a stream, FC2 may not expose the 3Mbps URL at the start of the download (because the streamer was streaming below 3MBps, for example). The solution proposed by HoloArchivists is to use waitForQualityMaxTries (wait-for-quality-timeout), which waits for the download.

However, there's a problem: the 3Mbps URL may appear later, perhaps 1, 5, 10 minutes... depending on the streamer's bitrate. Because of this, the waitForQuality feature may delete the beginning of the video.

A new feature is added to handle this issue: quality upgrade during download.

allowQualityUpgrade and pollQualityUpgradeInterval have been added to the configuration file:

   ## If the requested quality is not available, keep retrying before falling
   ## back to the next best quality. (default: 60)
   ##
   ## There is a 1 second delay between each retry. The value must be big enough
   ## so that the best quality (3Mbps) is available. If your streamer takes more
   ## than expected to prepare, you should increase this value.
   waitForQualityMaxTries: 60
+  ## EXPERIMENTAL: Allow quality upgrade during download if the requested
+  ## quality is not "yet" available. (default: false)
+  ##
+  ## If the requested quality is not available, the downloader will fallback to
+  ## the best quality available. However, it is possible that the streamer will
+  ## upgrade the quality during the stream. FC2 often "waits" for the stream to
+  ## be stable before upgrading the quality.
+  ##
+  ## If this option is enabled, the downloader will periodically check if the
+  ## quality has been upgraded. If the quality has been upgraded, the downloader
+  ## will switch to the new quality. **A cut off will be present in the recording.**
+  ##
+  ## If this option is enabled, it is recommended to:
+  ##
+  ## - Reduce waitForQualityMaxTries to 10s.
+  ## - Enable Remux or Concat to fix mpegts discontinuities.
+  allowQualityUpgrade: false
+  ## How many seconds between checks to see if the quality can be upgraded. (default: 10s)
+  ##
+  ## allowQualityUpgrade needs to be enabled for this to work.
+  pollQualityUpgradeInterval: '10s'

Please read the comments before enabling this feature. A cut will be present in the recording. (similar to switching qualities on Youtube).

Known issues:

• concat: Concatenating with extract audio with a mp4 and ts results in invalid data (#39)

SHA256 Checksums

b7b9f1ec8ab19f9f4cedfb62c6314fbf1c5cecf1ed9a040f5a19d8167e0af54d *fc2-live-dl-go-darwin-amd64
c12b2dee2850ac14e6270221bb9741cc1679f23ef7c40801fee2a5dc001806bf *fc2-live-dl-go-darwin-arm64
60bc382432c2bf5ce9916bd2765655487568f1c7fa70d95a9c90945d216c8138 *fc2-live-dl-go-linux-amd64
0025e7eca539838f5260ca58db0edac39ab31e2d964894f363e0945e78c6fda1 *fc2-live-dl-go-linux-arm64
c3b0fa0f2d76cda90075ec72b7d9ea1f2220d54c4838a9bc07b3fb85678d0745 *fc2-live-dl-go-linux-riscv64
0d138c1af20265b0669d767bd44900f0e8a6a2cb14b1c7c7449968f9d2206eab *fc2-live-dl-go-windows-amd64.exe

Full Changelog: v1.6.5...v1.7.0-alpha.1

v1.6.5: Fix audio extraction on concatenation

Fixes

• Fixed audio extraction on concatenation (#37)
• Increased waitForQualityMaxTries to 60 ~= 60 seconds to avoid the fallback to 1.5Mbps.

SHA256 Checksums

288ec59f3e4b938e5954bd3bffb30d27113dc89ed1e067e2423a5ca266c661a0 *fc2-live-dl-go-darwin-amd64
09f124c7b4257608f36d96ceada76d2944006bebecdf62bff611efa9a390b57d *fc2-live-dl-go-darwin-arm64
17cc60c1ca8ec97e0af234a9fcc1253efed571f82af6321f9fa0eb5aba68a952 *fc2-live-dl-go-linux-amd64
c266d3a864ceb772459f5336a3b9379ef56033e74e54a889d2e4661d7dc47264 *fc2-live-dl-go-linux-arm64
c7bb8067394e2f52848085d6792d24fc2c1eb00b4f8735202212eb879694f303 *fc2-live-dl-go-linux-riscv64
8b9e4f0d9b6637f958caf439d9ba53f2a24b713a405d4fda8e05fd40c0a36728 *fc2-live-dl-go-windows-amd64.exe

v1.6.4: Fix wrong dts concatenation

Fixes

• Fix DTS (decoding timestamp) computation when concatenating: instead of adding +1 to avoid dts conflict, we add the previous packet duration.

SHA256 Checksums

6d028937b33757181dd444e0d83e487d7a1afcd19a506e014a5110499548587e *fc2-live-dl-go-darwin-amd64
57f52d0ed0cd91f1f1ad67005e91760d22ff09d2b595a061213326ca78881f1e *fc2-live-dl-go-darwin-arm64
763755e2afb2cadde502174f3c28a1c14e1d18f234a0844b3dba9448e3e6e31a *fc2-live-dl-go-linux-amd64
0d62807214c7162c07ccc39eef2e760bb12f73c32ad71221e7dd2492f6c45e6d *fc2-live-dl-go-linux-arm64
3e7da691b1776324d2f37aa8ac9a7971698b30bc3bec060eb999e4690ebbd680 *fc2-live-dl-go-linux-riscv64
1c9e0f1f274c2cc9c3a26105660ef811572a2f81859612d53abaa780c816ba87 *fc2-live-dl-go-windows-amd64.exe

Full Changelog: v1.6.3...v1.6.4

v1.6.3: Fixed major HLS corruption

Fixes

• Fixed critical bug where the HLS is not able to sort fragments by timestamp, causing major stream corruption. (#24)
  • Fixed by defaulting to name sorting.
• Fixed a bug where concatenating no file would cause a crash.

SHA256 Checksums

defcbfce44541b71775e41e958bc47b41e69e4d3f1e66f4677961b5ac379ae04 *fc2-live-dl-go-darwin-amd64
f2fb2d0c18bce7a79b674bd52143acfca9c3fc52ca1aa4a19078458fde15665a *fc2-live-dl-go-darwin-arm64
5be3ef04e4e1d4029dbe19eb137307955865bf678f8724fb408c9e6ae386f785 *fc2-live-dl-go-linux-amd64
bf71a812a845174d42ef9dc1337b6091b25a31f447e84000558d6c27f0598467 *fc2-live-dl-go-linux-arm64
2b810a28d15a176578a13bb8769cc5a07a25100998f5d7acc4a52891c48bfc2f *fc2-live-dl-go-linux-riscv64
1cc874f4b74fd8bac1cb402578f73e5687eaafeb05ae1f7bd9e5a1bfb77d8623 *fc2-live-dl-go-windows-amd64.exe

Full Changelog: v1.6.1...v1.6.3

v1.6.1: Bug fixes

Changes

Caution

This release contains a critical bug in the HLS downloader. Some streams may get corrupted (10x size).
Please use version >= v1.6.2.

• concat has been tagged experimental.
• Reduce logs, added logs to concat.
• Fixes possible race condition when reloading config.

SHA256 Checksums

cc0ba2cc24cd8fd48c139e6f84247b016998b6248dcc197fdbb14474f50c5e2b *fc2-live-dl-go-darwin-amd64
3ef1a690b7ca5993b621c9db12931c09b43a6af045d2bf43656001be876ed4e3 *fc2-live-dl-go-darwin-arm64
36817278ac567e2dfc4369fb2cb03bacb52e8639c639fc405c48e43ab30d41d4 *fc2-live-dl-go-linux-amd64
298eeae492e8f36f7dcd1127f24aa3f1ddd643e2dc160e2850592bf22a616c67 *fc2-live-dl-go-linux-arm64
095041faf5d5d217a2fc487cc599c6d69e8354e8366d206cf8923fefbf8ed713 *fc2-live-dl-go-linux-riscv64
814a7683aab5d9bb5e2a686b6c15610ae48c6fa4cc36a7abaff291aac1d99581 *fc2-live-dl-go-windows-amd64.exe

Full Changelog: v1.6.0...v1.6.1

v1.6.0: Add periodic cleaning for `concat`

Changes

Caution

This release contains a critical bug in the HLS downloader. Some streams may get corrupted (10x size).
Please use version >= v1.6.2.

• Added deletion of old .ts that has an existing .combined.* file:

  defaultParams:
    ## Keep the raw .ts recordings after it has been remuxed. (default: false)
    ##
    ## If this option is set to false and concat is true, before every "waiting for stream to be online", a scan will be executed to detect *.combined.* files.
    ## The scan will be done on the directory of `scanDirectory`.
    ## If a non-corrupted .combined. file is detected, it will remove .ts older than `eligibleForCleaningAge`.
    ## After the cleaning, the .combined files will be renamed without the ".combined" part (if a file already exists due to remux, it won't be renamed).
    keepIntermediates: false
    ## Directory to be scanned for .ts files to be deleted after concatenation. (default: '')
    ##
    ## Scan is recursive.
    ##
    ## Empty value means no scanning.
    scanDirectory: ''
    ## Minimum age of .combined files to be eligible for cleaning. (default: 48h)
    ##
    ## The minimum should be the expected duration of a stream to avoid any race condition.
    eligibleForCleaningAge: '48h'

  The cleaner is ran every hour.

• Added corrupted file deletion feature.

  defaultParams:
    ## Delete corrupted .ts recordings. (default: true)
    deleteCorrupted: true

• Fix: Fixed various memory leaks with concat.

• Added clean command

• Fix: Instead of checking if a file exists to run remux, probe the file to see if the file is a video (equivalent to ffprobe).

• Fix: Concat has a better ordering with numbers.

SHA256 Checksums

2d8aaeeda04f1c11c4eef46a980354e510676d37658fc4f3b6460d8f5946cc84 *fc2-live-dl-go-darwin-amd64
94428d66c1882f64d05ab5c5624ceec2f7ae0655ee310e3063edd37580494573 *fc2-live-dl-go-darwin-arm64
35e11e080617be2844be10986188d6c77e6465fa738d63ed2356b3c87d09c219 *fc2-live-dl-go-linux-amd64
00909c99652607afac6699e502797db857f508d798f6c11e06cbee570123609b *fc2-live-dl-go-linux-arm64
5aec6e5fd143834b641b2b1cf8831864d16dc8daf064887ba464b5a63814291e *fc2-live-dl-go-linux-riscv64
e20dfa65119e8dd038cc7141c0b92b2aa22efc490d3337d35b1f315498dfdb84 *fc2-live-dl-go-windows-amd64.exe

Full Changelog: v1.5.1...v1.6.0

v1.6.0-rc.4

Fixes

• Run cleaner periodically.
• Fix memory leaks (one related to args, the other related the a missing av_packet_unref.

SHA256 Checksums

26294d3839ca8967e543c46b2205d6ad6b7ee368f5c834e69003c7f16568a6ba *fc2-live-dl-go-darwin-amd64
855d7082a7ee08df33fd049516d64865a88804b80a46d57fad46d9bb00350c6e *fc2-live-dl-go-darwin-arm64
7dd9192433e682b78850b1f841aefafe492ec7ec59d2640dbe3c34cc480ac96a *fc2-live-dl-go-linux-amd64
6f1a0ab38dd00bfb1300734112748175f4f8ed48af1c80c6a36c7390f675f3cc *fc2-live-dl-go-linux-arm64
19b1be764cea2b9ef2037af793c0467c9bcd6cc9455fade123bf1f9052c71e51 *fc2-live-dl-go-linux-riscv64
3112488783dfdb06178a3e682b8c77b18928d29487450544e0a43c49b69d5941 *fc2-live-dl-go-windows-amd64.exe

Full Changelog: v1.6.0-rc.2...v1.6.0-rc.4