Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(hex): use registry v2 for packages rather than hex.pm http API #29756

Open
wants to merge 19 commits into
base: main
Choose a base branch
from

Conversation

bryannaegele
Copy link
Contributor

Changes

  • use hex registry v2 for package information
  • use hex.pm package API for supplemental package metadata when package is hosted there.
  • support registryAliases for private repo/registry mapping
  • run mix.repo add in mix manager for private repos

Context

#29622

This changes the hex datasource to use hex registry v2 for package management. Hex registries are not required to implement an http API like hex.pm has, so the prior implementation prevented updating of packages in private repos such as oban pro. Additionally, due to the way mix works when updating dependencies, mix.lock files could not be updated when such a dependency existed and the mix manager ran mix deps.update <<updated packages>>.

In order to keep as much parity with the prior implementation, hex.pm is used for supplemental package metadata where possible.

Documentation (please check one with an [x])

  • I have updated the documentation, or
  • No documentation update is required

How I've tested my work (please select one)

I have verified these changes via:

  • Code inspection only, or
  • Newly added/modified unit tests, or
  • No unit tests but ran on a real repository, or
  • Both unit tests + ran on a real repository

@bryannaegele bryannaegele marked this pull request as draft June 19, 2024 00:42
@bryannaegele bryannaegele changed the title Use hex registries for packages rather than hex.pm http API Use hex registry v2 for packages rather than hex.pm http API Jun 19, 2024
@bryannaegele bryannaegele changed the title Use hex registry v2 for packages rather than hex.pm http API feat/Use hex registry v2 for packages rather than hex.pm http API Jun 19, 2024
@bryannaegele bryannaegele changed the title feat/Use hex registry v2 for packages rather than hex.pm http API feat(hex) use registry v2 for packages rather than hex.pm http API Jun 19, 2024
@bryannaegele bryannaegele changed the title feat(hex) use registry v2 for packages rather than hex.pm http API feat(hex): use registry v2 for packages rather than hex.pm http API Jun 19, 2024
@bryannaegele
Copy link
Contributor Author

Leaving this as a draft until there's an answer for this #29622 (comment)

I am no TS expert, so please feel free to jump in and fix up whatever can be done better.

Copy link
Member

@viceice viceice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reduce fixtures, we don't need all versions for testing, ~3 are enough. ensure the tgz file is at minimum too

lib/modules/datasource/hex/package.spec.ts Outdated Show resolved Hide resolved
@@ -0,0 +1,17 @@
// import { mockDeep } from 'jest-mock-extended';
import { gunzipSync } from 'zlib';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use async functions

lib/modules/manager/mix/artifacts.ts Outdated Show resolved Hide resolved
@@ -0,0 +1,154 @@
// Code generated by protoc-gen-ts_proto. DO NOT EDIT.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be generated on build and not checked into git. or fix it to a readable format for proper linting.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you help with how you would go about that? I am not a TS developer and this is what was generated by https://www.npmjs.com/package/ts-proto#usage, unedited in any way. There are a lot of options that can be passed to it but it's all foreign to me. The comment and eslint ignore are output by ts-proto.

I'm not sure how generating the files on build ends up with a different result. It would still be the same code, just lint ignored? And the code has to be there for unit tests, so I'm not clear on how a build step improves things.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you need to add that package to dev dependencies, then add a generate step to package.json scripts. there are already some generation steps as samples.

@bryannaegele
Copy link
Contributor Author

reduce fixtures, we don't need all versions for testing, ~3 are enough. ensure the tgz file is at minimum too

Is this strictly necessary? The tgz has the registry version of the package wrapped in a signed wrapper. I don't have any way to create a version with less releases in it that would have a valid signature.

I can try to find a different package as an example but this is one of the rare ones with retirements.

@bryannaegele
Copy link
Contributor Author

Mix changes split to #29775

@bryannaegele bryannaegele marked this pull request as ready for review June 24, 2024 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants