Maven scan-repository fails in air-gapped environment

[kradebahs]

Describe the bug

frogbot scan-repository on maven gitlab repository in an air-gapped environment fails with an error:
[ERROR] The specified user settings file does not exist: /tmp/jfrog.cli.temp.-1705390710-539606271/settings.xml

This happens only if a vulnerability is found and frogbot attempts to resolve the vulnerable dependency.
The settings.xml file exist in repositories root directory

Current behavior

Debug Log Output:

07:38:21 [🔵Info] Frogbot version: 2.19.7
07:38:21 [Debug] Reading config from file system. Looking for .frogbot/frogbot-config.yml
07:38:21 [Debug] frogbot-config.yml wasn't found in /root/.frogbot/frogbot-config.yml. Searching for it in upstream directories
07:38:21 [Debug] Attempting to download frogbot-config.yml from build-examples/maven
07:38:21 [Debug] The frogbot-config.yml will be downloaded from test-frogbot-scan-repository branch
07:38:21 [Debug] The .frogbot/frogbot-config.yml file wasn't recognized in <build-examples/maven>
07:38:21 [Debug] Locking config file to run config AddOrEdit command.
07:38:21 [Debug] Creating lock in: /tmp/jfrog.cli.temp.-1705390701-1747402071/locks/config
07:38:21 [Debug] Releasing lock: /tmp/jfrog.cli.temp.-1705390701-1747402071/locks/config/jfrog-cli.conf.lck.53234.1705390701278164459
07:38:21 [Debug] Config AddOrEdit command completed successfully. config file is released.
07:38:21 [Debug] Usage Report: Sending info...
07:38:21 [🔵Info] Running Frogbot "scan-repository" command
07:38:21 [Debug] Sending HTTP GET request to: https://artifactory/artifactory/api/system/version
07:38:21 [Debug] Sending HTTP HEAD request to: 'https://github.com/jfrog/frogbot'
07:38:21 [Debug] Sending HTTP POST request to: https://usage-ecosystem.jfrog.io/api/usage/report
07:38:21 [Debug] Sending HTTP GET request to: https://artifactory/xray/api/v1/system/version
07:38:21 [Debug] JFrog Xray version is: 3.79.11
07:38:21 [Debug] Artifactory response: 200 OK
07:38:21 [Debug] JFrog Artifactory version is: 7.63.10
07:38:21 [Debug] Sending HTTP POST request to: https://artifactory/artifactory/api/system/usage
07:38:21 [Debug] Setting timeout for go-git to 120 seconds ...
07:38:21 [Debug] Created temp working directory: /tmp/jfrog.cli.temp.-1705390701-2280860946
07:38:21 [Debug] Running git clone https://gitlab/build-examples/maven.git (test-frogbot-scan-repository branch)...
07:38:22 [Debug] Project cloned from https://gitlab/build-examples/maven.git to /tmp/jfrog.cli.temp.-1705390701-2280860946
07:38:22 [Debug] Sending HTTP GET request to: https://artifactory/xray/api/v1/system/version
07:38:22 [Debug] Sending HTTP GET request to: https://artifactory/xray/api/v1/entitlements/feature/contextual_analysis
07:38:22 [Debug] The path '/tmp/jfrog.cli.temp.-1705390701-2280860946/.gitlab-ci.yml' is excluded
07:38:22 [Debug] mapped 1 working directories with indicators/descriptors:
{
  "/tmp/jfrog.cli.temp.-1705390701-2280860946": [
    "/tmp/jfrog.cli.temp.-1705390701-2280860946/pom.xml"
  ]
}
07:38:22 [Debug] Detected 1 technologies at /tmp/jfrog.cli.temp.-1705390701-2280860946: [maven].
07:38:22 [🔵Info] Preforming 1 SCA scans:
[
  {
    "Technology": "maven",
    "WorkingDirectory": "/tmp/jfrog.cli.temp.-1705390701-2280860946",
    "Descriptors": [
      "/tmp/jfrog.cli.temp.-1705390701-2280860946/pom.xml"
    ]
  }
]
07:38:22 [🔵Info] Running SCA scan for maven vulnerable dependencies in /tmp/jfrog.cli.temp.-1705390701-2280860946 directory...
07:38:22 [🔵Info] Calculating Maven dependencies...
07:38:25 [Debug] Created 'Maven' dependency tree with 3 nodes. Elapsed time: 2.9 seconds.
07:38:25 [Debug] Unique dependencies list:
[
    "gav://junit:junit:4.12",
    "gav://org.hamcrest:hamcrest-core:1.3",
    "gav://com.test.application:test-project:1.2"
  ]
07:38:25 [🔵Info] Scanning 3 maven dependencies...
07:38:25 [Debug] Sending HTTP GET request to: https://artifactory/xsc/api/v1/system/version
07:38:25 [Debug] Sending HTTP POST request to: https://artifactory/xray/api/v1/scan/graph?scan_type=dependency
07:38:25 [🔵Info] Waiting for scan to complete on JFrog Xray...
07:38:25 [Debug] Sending HTTP GET request to: https://artifactory/xray/api/v1/scan/graph/66fd3bd8-0be9-472a-4ba5-2403a78d5efe?include_vulnerabilities=true
07:38:25 [Debug] Get Dependencies Scan results... (Attempt 1)
07:38:30 [Debug] Sending HTTP GET request to: https://artifactory/xray/api/v1/scan/graph/66fd3bd8-0be9-472a-4ba5-2403a78d5efe?include_vulnerabilities=true
07:38:30 [🔵Info] Xray scan completed
07:38:30 [Debug] Frogbot will attempt to resolve the following vulnerable dependencies:
 junit:junit
07:38:30 [Debug] Attempting to fix junit:junit:4.12 with 4.13.1
07:38:30 [Debug] Creating branch frogbot-junit_junit-0024823062e5e0506067e31c30b818ed ...
07:38:31 [Debug] Running 'mvn -U -B org.codehaus.mojo:versions-maven-plugin:use-dep-version -Dincludes=junit:junit -DdepVersion=4.13.1 -DgenerateBackupPoms=false -DprocessDependencies=true -DprocessDependencyManagement=false'
07:38:32 [🔵Info] [ERROR] Error executing Maven.
[ERROR] The specified user settings file does not exist: /tmp/jfrog.cli.temp.-1705390710-539606271/settings.xml

07:38:32 [Debug] Running git checkout to branch: test-frogbot-scan-repository
07:38:32 [🚨Error] the following errors occured while fixing vulnerabilities in /tmp/jfrog.cli.temp.-1705390701-2280860946:
failed running command 'mvn -U -B org.codehaus.mojo:versions-maven-plugin:use-dep-version -Dincludes=junit:junit -DdepVersion=4.13.1 -DgenerateBackupPoms=false -DprocessDependencies=true -DprocessDependencyManagement=false -s /tmp/jfrog.cli.temp.-1705390710-539606271/settings.xml': exit status 1

While frogbot scan-repository was executed I did monitor the /tmp directory and can confirm that the /tmp/jfrog.cli.temp.-1705390710-539606271/settings.xml did exist:

Tue Jan 16 07:38:31 UTC 2024
ls -la /tmp/jfrog.cli.temp.-1705390710-539606271
total 8
drwx------ 2 root root   26 Jan 16 07:38 .
drwxrwxrwt 1 root root 4096 Jan 16 07:38 ..
-rw------- 1 root root 1406 Jan 16 07:38 settings.xml

Reproduction steps

Setup frogbot to run against internal artifactory registry by adding JF_RELEASE_REPO, JF_DEPS_REPO and settings.xml

Expected behavior

Merge request should be created by frogbot

JFrog Frogbot version

2.19.7

Package manager info

Maven 3.8.6

Git provider

GitLab

JFrog Frogbot configuration yaml file

No response

Operating system type and version

Debian 11 (bullseye)

JFrog Xray version

3.79.11

[nicgrobler]

if it helps, using frogtbot 2.14.1, with no other changes, worked as expected. Further investigation shows:

• all versions after this fail as above.
• although each version has issues with different files not being found in /tmp/.., the one commonality is that all of the errors relate to files not being found within the /tmp/ directory structure frogtbot creates. very much as if multiple goroutines are standing on each other, or paths are being reported inconsistently in different parts of the code.

[eranturgeman]

Hi @nicgrobler, thank you for reporting this issue and for using Frogbot.
Our team will carefully review the issue, and we'll keep you informed about any progress or fixes.