Parallelize repository cloning during dev environment setup

[Copilot]

Sequential git clone calls during composer install/composer update made initial dev environment setup unnecessarily slow. The refresh step already used xargs for parallelism; cloning did not.

Changes

• New .maintenance/clone-all-repositories.sh: Bash script that orchestrates the full clone + refresh flow. Checks that jq is available at startup with a clear error message. Detects CPU cores via nproc (Linux) or sysctl -n hw.logicalcpu (macOS), defaulting to 4 if neither is available, and caps parallelism at 8 to avoid saturating GitHub connections or IO. The cap can be overridden via the CLONE_JOBS or WPCLI_DEV_JOBS environment variables. Fetches the repository list from the GitHub API using curl -fsS (failing fast on HTTP errors), validates the response is a JSON array (with distinct messages for rate-limit errors vs. unexpected shapes), parses with jq, filters the skip list, applies the destination map, then clones missing repos and refreshes all repos in parallel using xargs -P$CORES.
• New .maintenance/clone-repository.sh: Bash helper that clones a single repository, accepting exactly <destination> and <clone_url> as arguments (rejects anything other than exactly 2 args).
• Removed .maintenance/clone-all-repositories.php and .maintenance/clone-repository.php.
• Updated composer.json: pre-install-cmd and pre-update-cmd now call bash .maintenance/clone-all-repositories.sh.

# Check prerequisites
if ! command -v jq &>/dev/null; then
    echo "Required command 'jq' is not installed or not available in PATH." >&2
    exit 1
fi

# Detect CPU cores, cap at 8, allow env override
CORES="${CLONE_JOBS:-${WPCLI_DEV_JOBS:-${DETECTED_CORES}}}"
[[ "${CORES}" -gt 8 ]] && CORES=8

# Fetch and validate API response
if ! RESPONSE=$(curl -fsS ... ); then exit 1; fi
if ! jq -e 'type == "array"' >/dev/null <<< "${RESPONSE}"; then exit 1; fi

# Clone and refresh in parallel, bounded by core count
printf '%s\n' "${CLONE_LIST[@]}" | xargs -n2 -P"${CORES}" bash clone-repository.sh
printf '%s\n' "${UPDATE_FOLDERS[@]}" | xargs -P"${CORES}" -I% php refresh-repository.php %

[Copilot]

Pull request overview

This PR speeds up initial wp-cli-dev environment setup by replacing the previous PHP-based cloning workflow with Bash scripts that clone and refresh repositories in parallel, and wires the new script into Composer’s pre-install/update hooks.

Changes:

• Replace .maintenance/clone-all-repositories.php / .maintenance/clone-repository.php with Bash equivalents.
• Parallelize cloning and refreshing using xargs -P with detected CPU core count.
• Update composer.json hooks to run the new Bash entrypoint.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
composer.json	Switch Composer pre-install/update to call the new Bash orchestrator.
.maintenance/clone-all-repositories.sh	New orchestration script: fetch org repos, filter/map, then parallel clone + refresh.
.maintenance/clone-repository.sh	New helper script to clone a single repo given destination + URL.
.maintenance/clone-all-repositories.php	Removed old PHP orchestrator.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[swissspidy]

@copilot apply changes based on the comments in this thread

[Copilot]

@copilot apply changes based on the comments in this thread

Done in a7630d5. Applied all suggestions from the review thread:

• Added jq availability check at the top with a clear error message
• Changed curl -s → curl -fsS with explicit failure handling
• Validate the API response is a JSON array before processing; differentiate between rate-limit errors and other unexpected responses
• Capped auto-detected cores to 8 (max), with CLONE_JOBS/WPCLI_DEV_JOBS env vars to override
• Dropped redundant -n1 from the refresh xargs call
• Changed -lt 2 → -ne 2 in clone-repository.sh