Bump datasets from 2.18.0 to 2.20.0

[dependabot]

Bumps datasets from 2.18.0 to 2.20.0.

Release notes

Sourced from datasets's releases.

2.20.0

Important

• Remove default trust_remote_code=True by @​lhoestq in huggingface/datasets#6954
  • datasets with a python loading script now require passing trust_remote_code=True to be used

Datasets features

• [Resumable IterableDataset] Add IterableDataset state_dict by @​lhoestq in huggingface/datasets#6658

  • checkpoint and resume an iterable dataset (e.g. when streaming):

    >>> iterable_dataset = Dataset.from_dict({"a": range(6)}).to_iterable_dataset(num_shards=3)
    >>> for idx, example in enumerate(iterable_dataset):
    ...     print(example)
    ...     if idx == 2:
    ...         state_dict = iterable_dataset.state_dict()
    ...         print("checkpoint")
    ...         break
    >>> iterable_dataset.load_state_dict(state_dict)
    >>> print(f"restart from checkpoint")
    >>> for example in iterable_dataset:
    ...     print(example)

    Returns:

    {'a': 0}
    {'a': 1}
    {'a': 2}
    checkpoint
    restart from checkpoint
    {'a': 3}
    {'a': 4}
    {'a': 5}
    

General improvements and bug fixes

• Add docs about the CLI by @​albertvillanova in huggingface/datasets#6831
• Remove token arg from CLI examples by @​albertvillanova in huggingface/datasets#6839
• Allow deleting a subset/config from a no-script dataset by @​albertvillanova in huggingface/datasets#6820
• Fix line-endings in tests on Windows by @​albertvillanova in huggingface/datasets#6857
• Fix CI by temporarily pinning huggingface-hub < 0.23.0 by @​albertvillanova in huggingface/datasets#6861
• Fix dataset name for community Hub script-datasets by @​albertvillanova in huggingface/datasets#6855
• Update tqdm >= 4.66.3 to fix vulnerability by @​albertvillanova in huggingface/datasets#6870
• Fix download for dict of dicts of URLs by @​albertvillanova in huggingface/datasets#6871
• Set dev version by @​albertvillanova in huggingface/datasets#6873
• Shorten long logs by @​lhoestq in huggingface/datasets#6875
• Support jax 0.4.27 in CI tests by @​albertvillanova in huggingface/datasets#6885
• Close gzipped files properly by @​lhoestq in huggingface/datasets#6893
• Make CLI convert_to_parquet not raise error if no rights to create script branch by @​albertvillanova in huggingface/datasets#6902

... (truncated)

Commits

• 98fdc9e Release: 2.20.0 (#6969)
• af3acfd fix(ci): remove unnecessary permissions (#6962)
• 37a6036 Move info_utils errors to exceptions module (#6952)
• 9510252 Better error handling in dataset_module_factory (#6959)
• 97513be feat(ci): add trufflehog secrets detection (#6960)
• 686f5df Add support for categorical/dictionary types (#6892)
• a2dc287 Remove default trust_remote_code=True (#6954)
• 09ebf51 Validate config name and data_files in packaged modules (#6915)
• 5bbbf1b Validate config name and data_files in packaged modules (#6915)
• 6548e0e Fix typos in docs (#6957)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

[github-actions]

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

Package	Version	Score	Details
pip/datasets	2.20.0	🟢 5.9	Details Check Score Reason Code-Review 🟢 4 Found 14/30 approved changesets -- score normalized to 4 Maintained 🟢 10 30 commit(s) and 10 issue activity found in the last 90 days -- score normalized to 10 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected License 🟢 10 license file detected Branch-Protection ⚠️ -1 internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Packaging ⚠️ -1 packaging workflow not detected Security-Policy 🟢 10 security policy file detected Signed-Releases ⚠️ -1 no releases found Binary-Artifacts 🟢 10 no binaries found in the repo Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions Vulnerabilities 🟢 10 0 existing vulnerabilities detected Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0 Fuzzing ⚠️ 0 project is not fuzzed SAST ⚠️ 0 SAST tool is not run on all commits -- score normalized to 0
pip/datasets	2.18.0	🟢 5.9	Details Check Score Reason Code-Review 🟢 4 Found 14/30 approved changesets -- score normalized to 4 Maintained 🟢 10 30 commit(s) and 10 issue activity found in the last 90 days -- score normalized to 10 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected License 🟢 10 license file detected Branch-Protection ⚠️ -1 internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Packaging ⚠️ -1 packaging workflow not detected Security-Policy 🟢 10 security policy file detected Signed-Releases ⚠️ -1 no releases found Binary-Artifacts 🟢 10 no binaries found in the repo Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions Vulnerabilities 🟢 10 0 existing vulnerabilities detected Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0 Fuzzing ⚠️ 0 project is not fuzzed SAST ⚠️ 0 SAST tool is not run on all commits -- score normalized to 0

Scanned Manifest Files

setup.py

• [email protected]
• [email protected]