Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement the calculate spower offchain feature #944

Merged
merged 102 commits into from
Jun 29, 2024

Conversation

wuhaixian1984
Copy link
Contributor

Implement the feature to support spower calculation offchain, this needs to work together with the crust-spower service https://github.com/crustio/crust-spower.

Main changes:

  1. swork::report_works doesn't invoke market::upsert_replicas and market::delete_replicas anymore.
  2. The crust-spower service will index the work reports from chain, and aggregate multiple work reports, then call the newly added market::update_replicas extrinsic to update the replicas data in batch.
  3. Update the market::calculate_rewards extrinsic to only liquidate, renew, or close file, but do not update replicas and spower anymore.
  4. The crust-spower service will index the market::FilesV2 data from chain, and perform spower calculation for changed files in batch, then call the newly added swork::update_spower extrinsic to update the sworker spower and file spower.
  5. The newly added market::update_replicas extrinsic and swork::update_spower extrinsic can only be called by specific register spower superior account, the account need to be set
  6. Unit tests have been updated per these changes.

wuhaixian1984 and others added 30 commits May 17, 2024 18:06
…hain crust-spower service.

2. Add the WorkReportMetadata to this queue when the report_works call is successful.
3. Refactor the update_files function to only calculate the changed_spower and changed_files_count based on the raw value directly. Do not invoke the market::upsert_replica and delete_replica anymore.
…pdated files replicas info at the specific report_block

2. Add work_reports args to the update_replicas extrinsic call, which will then call the T::SworkerInterface::clear_process_work_reports to purge the successfully processed work reports from the WorkReportsToProcess db item
…eplicas and update_spower extrinsic call

2. Refactor some data structure names
…e, so that we can replay this data in crust-spower service

2. Refactor some detailed implementations for the upsert_replica and delete_replica to handle some detailed design considerations
2. Perform the illegal file replicas count substraction directly during the market::update_replicas call
3. Remove some unused function parameter
2. Refactor the WorkReportsToProcess design (include db and events)
…ate_identities function will be called starting from 500th block in a report slot, while the original value is from 400th block in a slot.

The 400th ~500 th block is reserved for the spower update.
… store the UpdatedFilesToProcess right now.

2. Perform the spower changed based on file_size during update_replicas
3. Add FileClosed event
4. Refactor the update_spower,  which doesn't update spower per blocks of files now
…Vec.

The reason is that the order of items in BTreeMap are not determined, when sending tx from polkadot.js, it may fail with error 'Invalid Transaction: Transaction has a bad signature'
…nt management extrinsic

2. Fix bug for update_replicas with non-exist cids, need to add to the illegal_file_replicas_map as well under this scenario
3. Use valid_at as the value of created_at
4. Update the calculated_at field during maybe_reward_liquidator
5. Update the calculate_spower to use the latest spower calculation logic
…e get_current_reported_slot() instead of current_report_slot()

2. Charge no fee for swork::update_spower extrinsic, since this is a frequent management call
3. Remove unused commented out codes
Crust Update Bot and others added 29 commits June 28, 2024 15:22
…hain crust-spower service.

2. Add the WorkReportMetadata to this queue when the report_works call is successful.
3. Refactor the update_files function to only calculate the changed_spower and changed_files_count based on the raw value directly. Do not invoke the market::upsert_replica and delete_replica anymore.
…eplicas and update_spower extrinsic call

2. Refactor some data structure names
2. Perform the illegal file replicas count substraction directly during the market::update_replicas call
3. Remove some unused function parameter
@wuhaixian1984 wuhaixian1984 merged commit b5b7cd8 into mainnet Jun 29, 2024
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant