Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(sbt-plugin): add new default registry URL #29648

Merged
merged 23 commits into from
Jul 1, 2024
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
b3fe75b
Replace the default SBT plugin repo URL
george-wilson-rea Jun 13, 2024
cc126e6
Merge branch 'main' into update-sbt-plugin-url
george-wilson-rea Jun 13, 2024
c0a0c42
Merge branch 'main' into update-sbt-plugin-url
george-wilson-rea Jun 14, 2024
382e47a
Add back in the old sbt plugin repo
george-wilson-rea Jun 14, 2024
01b45f3
Make sure the plugins repo is in the defaultRegistryUrls
george-wilson-rea Jun 14, 2024
67e5dbe
Add readme.md for sbt-package and sbt-plugin datasources
george-wilson-rea Jun 19, 2024
5f298e9
Merge branch 'main' into update-sbt-plugin-url
george-wilson-rea Jun 19, 2024
044ed1b
Merge branch 'main' into update-sbt-plugin-url
george-wilson-rea Jun 20, 2024
8ecbaa2
Merge branch 'main' into update-sbt-plugin-url
george-wilson-rea Jun 23, 2024
d614086
Improve readmes for sbt-plugin and sbt-package
george-wilson-rea Jun 24, 2024
43bfeac
Merge remote-tracking branch 'refs/remotes/george-wilson/update-sbt-p…
george-wilson-rea Jun 24, 2024
7ba61fa
typo
george-wilson-rea Jun 24, 2024
efeae3b
Changes to PR
zharinov Jun 24, 2024
f513651
fix
zharinov Jun 24, 2024
2d9442e
Merge pull request #1 from zharinov/29648-changes
george-wilson-rea Jun 24, 2024
0239f97
Merge branch 'main' into update-sbt-plugin-url
george-wilson-rea Jun 24, 2024
94bd1c5
Fix SBT tests
george-wilson-rea Jun 25, 2024
c46a94b
Revert tests
zharinov Jun 25, 2024
5e0c71d
Merge pull request #2 from zharinov/29648-more-changes
george-wilson-rea Jun 25, 2024
706dc90
Apply suggestions from code review
george-wilson-rea Jun 26, 2024
beeb436
prettier
george-wilson-rea Jun 26, 2024
e5b1e19
Apply suggestions from code review
george-wilson-rea Jun 28, 2024
b5583ec
prettier
george-wilson-rea Jun 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions lib/modules/datasource/sbt-package/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,11 @@ import { compare } from '../../versioning/maven/compare';
import { MavenDatasource } from '../maven';
import { MAVEN_REPO } from '../maven/common';
import { downloadHttpProtocol } from '../maven/util';
import type { GetReleasesConfig, ReleaseResult } from '../types';
import type {
GetReleasesConfig,
RegistryStrategy,
ReleaseResult,
} from '../types';
import {
getLatestVersion,
normalizeRootRelativeUrls,
Expand All @@ -22,7 +26,7 @@ export class SbtPackageDatasource extends MavenDatasource {

override readonly defaultVersioning = ivyVersioning.id;

override readonly registryStrategy = 'hunt';
override readonly registryStrategy: RegistryStrategy = 'hunt';
rarkins marked this conversation as resolved.
Show resolved Hide resolved

override readonly sourceUrlSupport = 'package';
override readonly sourceUrlNote =
Expand Down
13 changes: 13 additions & 0 deletions lib/modules/datasource/sbt-package/readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
This datasource finds SBT package updates from Maven repositories.

By default, Renovate checks <https://repo.maven.apache.org/maven2> for SBT packages. You can override this behavior by overriding the `registryUrls` setting. For example:
george-wilson-rea marked this conversation as resolved.
Show resolved Hide resolved

```json
{
"matchDatasources": ["sbt-package"],
"registryUrls": [
"https://repo.maven.apache.org/maven2",
"https://oss.sonatype.org/content/repositories/snapshots"
]
}
```
123 changes: 64 additions & 59 deletions lib/modules/datasource/sbt-plugin/index.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { codeBlock } from 'common-tags';
george-wilson-rea marked this conversation as resolved.
Show resolved Hide resolved
import { getPkgReleases } from '..';
import { Fixtures } from '../../../../test/fixtures';
import * as httpMock from '../../../../test/http-mock';
Expand Down Expand Up @@ -37,10 +38,12 @@ describe('modules/datasource/sbt-plugin/index', () => {
.get('/maven2/org/scalatest/')
.reply(
200,
'<a href="scalatest/" title=\'scalatest/\'>scalatest_2.12/</a>\n' +
'<a href="scalatest_2.12/" title=\'scalatest_2.12/\'>scalatest_2.12/</a>\n' +
"<a href='scalatest_sjs2.12/'>scalatest_2.12/</a>" +
"<a href='scalatest_native2.12/'>scalatest_2.12/</a>",
codeBlock`
george-wilson-rea marked this conversation as resolved.
Show resolved Hide resolved
<a href="scalatest/">scalatest/</a>
<a href="scalatest_2.12/">scalatest_2.12/</a>
<a href="scalatest_sjs2.12/">scalatest_sjs2.12/</a>
<a href="scalatest_native2.12/">scalatest_native2.12/</a>
`,
);
httpMock
.scope('https://repo.maven.apache.org')
Expand All @@ -52,74 +55,74 @@ describe('modules/datasource/sbt-plugin/index', () => {
.reply(200, "<a href='1.2.3/'>4.5.6/</a>");

httpMock
.scope('https://repo.scala-sbt.org')
.get('/scalasbt/sbt-plugin-releases/com.github.gseitz/')
.reply(200, '');
httpMock
.scope('https://repo.scala-sbt.org')
.get('/scalasbt/sbt-plugin-releases/org.foundweekends/sbt-bintray/')
.scope('https://repo.maven.apache.org')
.get('/maven2/org/foundweekends/sbt-bintray/')
.reply(
200,
'<html>\n' +
'<head>\n' +
'</head>\n' +
'<body>\n' +
'<pre><a href="scala_2.12/">scala_2.12/</a></pre>\n' +
'</body>\n' +
'</html>',
codeBlock`
george-wilson-rea marked this conversation as resolved.
Show resolved Hide resolved
<html>
<head>
</head>
<body>
<pre><a href="scala_2.12/">scala_2.12/</a></pre>
</body>
</html>
`,
);
httpMock
.scope('https://repo.scala-sbt.org')
.get(
'/scalasbt/sbt-plugin-releases/org.foundweekends/sbt-bintray/scala_2.12/',
)
.scope('https://repo.maven.apache.org')
.get('/maven2/org/foundweekends/sbt-bintray/scala_2.12/')
.reply(
200,
'\n' +
'<html>\n' +
'<head>\n' +
'</head>\n' +
'<body>\n' +
'<pre><a href="sbt_1.0/">sbt_1.0/</a></pre>\n' +
'</body>\n' +
'</html>\n',
codeBlock`
george-wilson-rea marked this conversation as resolved.
Show resolved Hide resolved
<html>
<head>
</head>
<body>
<pre><a href="sbt_1.0/">sbt_1.0/</a></pre>
</body>
</html>
`,
);
httpMock
.scope('https://repo.scala-sbt.org')
.get(
'/scalasbt/sbt-plugin-releases/org.foundweekends/sbt-bintray/scala_2.12/sbt_1.0/',
)
.scope('https://repo.maven.apache.org')
.get('/maven2/org/foundweekends/sbt-bintray/scala_2.12/sbt_1.0/')
.reply(
200,
'\n' +
'<html>\n' +
'<head>\n' +
'</head>\n' +
'<body>\n' +
'<pre><a href="0.5.5/">0.5.5/</a></pre>\n' +
'</body>\n' +
'</html>\n',
codeBlock`
george-wilson-rea marked this conversation as resolved.
Show resolved Hide resolved
<html>
<head>
</head>
<body>
<pre><a href="0.5.5/">0.5.5/</a></pre>
</body>
</html>
`,
);

httpMock
.scope('https://repo.maven.apache.org')
.get('/maven2/io/get-coursier/')
.reply(
200,
'<a href="sbt-coursier_2.10_0.13/">sbt-coursier_2.10_0.13/</a>\n' +
'<a href="sbt-coursier_2.12_1.0/">sbt-coursier_2.12_1.0/</a>\n' +
'<a href="sbt-coursier_2.12_1.0.0-M5/">sbt-coursier_2.12_1.0.0-M5/</a>\n' +
'<a href="sbt-coursier_2.12_1.0.0-M6/">sbt-coursier_2.12_1.0.0-M6/</a>\n',
codeBlock`
george-wilson-rea marked this conversation as resolved.
Show resolved Hide resolved
<a href="sbt-coursier_2.10_0.13/">sbt-coursier_2.10_0.13/</a>
<a href="sbt-coursier_2.12_1.0/">sbt-coursier_2.12_1.0/</a>
<a href="sbt-coursier_2.12_1.0.0-M5/">sbt-coursier_2.12_1.0.0-M5/</a>
<a href="sbt-coursier_2.12_1.0.0-M6/">sbt-coursier_2.12_1.0.0-M6/</a>
`,
);
httpMock
.scope('https://repo.maven.apache.org')
.get('/maven2/io/get-coursier/sbt-coursier_2.12_1.0/')
.reply(
200,
'<a href="2.0.0-RC2/">2.0.0-RC2/</a>\n' +
'<a href="2.0.0-RC6-1/">2.0.0-RC6-1/</a>\n' +
'<a href="2.0.0-RC6-2/">2.0.0-RC6-2/</a>\n' +
'<a href="2.0.0-RC6-6/">2.0.0-RC6-6/</a>\n',
codeBlock`
george-wilson-rea marked this conversation as resolved.
Show resolved Hide resolved
<a href="2.0.0-RC2/">2.0.0-RC2/</a>
<a href="2.0.0-RC6-1/">2.0.0-RC6-1/</a>
<a href="2.0.0-RC6-2/">2.0.0-RC6-2/</a>
<a href="2.0.0-RC6-6/">2.0.0-RC6-6/</a>
`,
);
httpMock
.scope('https://repo.maven.apache.org')
Expand All @@ -128,12 +131,14 @@ describe('modules/datasource/sbt-plugin/index', () => {
)
.reply(
200,
'<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">\n' +
'<url>https://get-coursier.io/</url>\n' +
'<scm>\n' +
'<url>https://github.com/coursier/sbt-coursier</url>\n' +
'</scm>\n' +
'</project>\n',
codeBlock`
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<url>https://get-coursier.io/</url>
<scm>
<url>https://github.com/coursier/sbt-coursier</url>
</scm>
</project>
`,
);
});

Expand Down Expand Up @@ -169,8 +174,8 @@ describe('modules/datasource/sbt-plugin/index', () => {
}),
).toEqual({
dependencyUrl:
'https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/org.foundweekends/sbt-bintray',
registryUrl: 'https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases',
'https://repo.maven.apache.org/maven2/org/foundweekends/sbt-bintray',
registryUrl: 'https://repo.maven.apache.org/maven2',
releases: [{ version: '0.5.5' }],
});
});
Expand All @@ -185,8 +190,8 @@ describe('modules/datasource/sbt-plugin/index', () => {
}),
).toEqual({
dependencyUrl:
'https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/org.foundweekends/sbt-bintray',
registryUrl: 'https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases',
'https://repo.maven.apache.org/maven2/org/foundweekends/sbt-bintray',
registryUrl: 'https://repo.maven.apache.org/maven2',
releases: [{ version: '0.5.5' }],
});
});
Expand Down
15 changes: 11 additions & 4 deletions lib/modules/datasource/sbt-plugin/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,27 @@ import { regEx } from '../../../util/regex';
import { ensureTrailingSlash } from '../../../util/url';
import * as ivyVersioning from '../../versioning/ivy';
import { compare } from '../../versioning/maven/compare';
import { MAVEN_REPO } from '../maven/common';
import { downloadHttpProtocol } from '../maven/util';
import { SbtPackageDatasource } from '../sbt-package';
import { getLatestVersion, parseIndexDir } from '../sbt-package/util';
viceice marked this conversation as resolved.
Show resolved Hide resolved
import type { GetReleasesConfig, ReleaseResult } from '../types';
import type {
GetReleasesConfig,
RegistryStrategy,
ReleaseResult,
} from '../types';

export const SBT_PLUGINS_REPO =
'https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases';

export const defaultRegistryUrls = [SBT_PLUGINS_REPO];
export const defaultRegistryUrls = [SBT_PLUGINS_REPO, MAVEN_REPO];

export class SbtPluginDatasource extends SbtPackageDatasource {
static override readonly id = 'sbt-plugin';

override readonly defaultRegistryUrls = defaultRegistryUrls;

override readonly registryStrategy = 'hunt';
override readonly registryStrategy: RegistryStrategy = 'merge';

override readonly defaultVersioning = ivyVersioning.id;

Expand Down Expand Up @@ -98,7 +103,9 @@ export class SbtPluginDatasource extends SbtPackageDatasource {
const repoRoot = ensureTrailingSlash(registryUrl);
const searchRoots: string[] = [];
// Optimize lookup order
searchRoots.push(`${repoRoot}${groupIdSplit.join('.')}`);
if (!registryUrl.startsWith(MAVEN_REPO)) {
searchRoots.push(`${repoRoot}${groupIdSplit.join('.')}`);
}
searchRoots.push(`${repoRoot}${groupIdSplit.join('/')}`);

for (let idx = 0; idx < searchRoots.length; idx += 1) {
Expand Down
14 changes: 14 additions & 0 deletions lib/modules/datasource/sbt-plugin/readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
This datasource finds SBT plugin updates from Maven repositories.

By default, Renovate checks <https://repo1.maven.org/maven2/> for SBT plugins, and then falls back to the legacy URL <https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases>. You can override this behavior by overriding the `registryUrls` setting. For example:
george-wilson-rea marked this conversation as resolved.
Show resolved Hide resolved

```json
{
"matchDatasources": ["sbt-plugin"],
"registryUrls": [
"https://repo1.maven.org/maven2/",
"https://oss.sonatype.org/content/repositories/snapshots",
"https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases"
]
}
```
6 changes: 3 additions & 3 deletions lib/modules/datasource/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -102,9 +102,9 @@ export interface DatasourceApi extends ModuleApi {

/**
* Strategy to use when multiple registryUrls are available to the datasource.
* first: only the first registryUrl will be tried and others ignored
* hunt: registryUrls will be tried in order until one returns a result
* merge: all registryUrls will be tried and the results merged if more than one returns a result
* - `first`: only the first registryUrl will be tried and others ignored
* - `hunt`: registryUrls will be tried in order until one returns a result
* - `merge`: all registryUrls will be tried and the results merged if more than one returns a result
*/
registryStrategy?: RegistryStrategy | undefined;

Expand Down
16 changes: 8 additions & 8 deletions lib/modules/manager/sbt/extract.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -490,7 +490,7 @@ describe('modules/manager/sbt/extract', () => {
registryUrls: [
'http://example.org/repo',
'https://example.org/ivy-repo/',
'https://repo1.maven.org/maven2',
'https://repo.maven.apache.org/maven2',
],
},
{
Expand All @@ -499,7 +499,7 @@ describe('modules/manager/sbt/extract', () => {
registryUrls: [
'http://example.org/repo',
'https://example.org/ivy-repo/',
'https://repo1.maven.org/maven2',
'https://repo.maven.apache.org/maven2',
],
},
{
Expand All @@ -508,7 +508,7 @@ describe('modules/manager/sbt/extract', () => {
registryUrls: [
'http://example.org/repo',
'https://example.org/ivy-repo/',
'https://repo1.maven.org/maven2',
'https://repo.maven.apache.org/maven2',
],
},
{
Expand All @@ -517,7 +517,7 @@ describe('modules/manager/sbt/extract', () => {
registryUrls: [
'http://example.org/repo',
'https://example.org/ivy-repo/',
'https://repo1.maven.org/maven2',
'https://repo.maven.apache.org/maven2',
],
},
{
Expand All @@ -526,7 +526,7 @@ describe('modules/manager/sbt/extract', () => {
registryUrls: [
'http://example.org/repo',
'https://example.org/ivy-repo/',
'https://repo1.maven.org/maven2',
'https://repo.maven.apache.org/maven2',
],
},
{
Expand All @@ -535,7 +535,7 @@ describe('modules/manager/sbt/extract', () => {
registryUrls: [
'http://example.org/repo',
'https://example.org/ivy-repo/',
'https://repo1.maven.org/maven2',
'https://repo.maven.apache.org/maven2',
],
},
],
Expand All @@ -551,7 +551,7 @@ describe('modules/manager/sbt/extract', () => {
for (const dep of pkg.deps.filter((d) => d.depType === 'plugin')) {
expect(dep.registryUrls).toStrictEqual([
'https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases',
'https://repo1.maven.org/maven2',
'https://repo.maven.apache.org/maven2',
'https://example.com/repos/1/',
'https://example.com/repos/2/',
'https://example.com/repos/3/',
Expand All @@ -563,7 +563,7 @@ describe('modules/manager/sbt/extract', () => {
for (const pkg of packages) {
for (const dep of pkg.deps.filter((d) => d.depType !== 'plugin')) {
expect(dep.registryUrls).toStrictEqual([
'https://repo1.maven.org/maven2',
'https://repo.maven.apache.org/maven2',
'https://example.com/repos/1/',
'https://example.com/repos/2/',
'https://example.com/repos/3/',
Expand Down
Loading