Add num_workers to extract_archive(s)

[hagenw]

Speeds up archive extraction by providing num_workers.

Summary by Sourcery

Enable configurable parallelism in archive extraction by introducing num_workers and leveraging run_tasks for concurrent file and archive processing.

Enhancements:

• Add num_workers parameter to extract_archive and extract_archives to enable parallel extraction
• Replace iterative zip and tar extraction loops with run_tasks-based parallel tasks
• Conditionally parallelize archive-level extraction when num_workers permits

Documentation:

• Update docstrings for extract_archive(s) to document num_workers parameter

[sourcery-ai]

Reviewer's Guide

This PR adds a num_workers parameter to both extract_archive and extract_archives and replaces manual extraction loops with the run_tasks utility to enable parallel extraction of members and archives based on the configured worker count.

Sequence diagram for parallel extraction in extract_archive

sequenceDiagram
    participant Caller
    participant extract_archive
    participant run_tasks
    Caller->>extract_archive: call extract_archive(archive, destination, num_workers)
    extract_archive->>run_tasks: run_tasks(extract, members, num_workers)
    run_tasks-->>extract_archive: extracted member list
    extract_archive-->>Caller: return member list

Loading

Sequence diagram for parallel extraction in extract_archives

sequenceDiagram
    participant Caller
    participant extract_archives
    participant run_tasks
    participant extract_archive
    Caller->>extract_archives: call extract_archives(archives, destination, num_workers)
    alt num_workers < len(archives)
        extract_archives->>extract_archive: sequentially call extract_archive for each archive
        extract_archive-->>extract_archives: return member list
    else num_workers >= len(archives)
        extract_archives->>run_tasks: run_tasks(extract_archive, archives, num_workers)
        run_tasks->>extract_archive: call extract_archive in parallel
        extract_archive-->>run_tasks: return member list
        run_tasks-->>extract_archives: return combined member list
    end
    extract_archives-->>Caller: return member list

Loading

Class diagram for updated extract_archive and extract_archives functions

classDiagram
    class extract_archive {
        +destination: str
        +keep_archive: bool = True
        +num_workers: int = 1
        +verbose: bool = False
        +extract_zip(archive: str) list
        +extract_tar(archive: str) list
    }
    class extract_archives {
        +archives: list[str]
        +destination: str
        +keep_archive: bool = True
        +num_workers: int = 1
        +verbose: bool = False
    }
    extract_archives --> extract_archive : calls
    extract_archive ..> run_tasks : uses
    extract_archives ..> run_tasks : uses

Loading

File-Level Changes

Change	Details	Files
Add num_workers parameter to extraction functions	Introduce num_workers argument in extract_archive signature and docstring Introduce num_workers argument in extract_archives signature and docstring	audeer/core/io.py
Parallelize member extraction using run_tasks	Replace zipfile loop and progress_bar with run_tasks for extract_zip Replace tarfile loop and progress_bar with run_tasks for extract_tar Remove disable flag for progress_bar in extract_archive	audeer/core/io.py
Support parallel extraction of multiple archives	Add a conditional in extract_archives to choose sequential or parallel mode based on num_workers Implement sequential extraction branch using existing progress_bar loop Implement parallel extraction branch with run_tasks over extract_archive calls	audeer/core/io.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[hagenw]

Closing in favor of #186