Skip to content

fix(cli): report failed skills installs instead of claiming success - #1290

Merged
AmanVarshney01 merged 7 commits into
mainfrom
fix/skills-install-errors
Oct 22, 2026
Merged

AmanVarshney01 merged 7 commits into
mainfrom
fix/skills-install-errors

Conversation

@AmanVarshney01

@AmanVarshney01 AmanVarshney01 commented Oct 22, 2026 •

Copy link
Copy Markdown
Owner

Summary

When skills@latest add exited non-zero (e.g. an unknown skill name), the skills addon only logged a dim warning, the spinner still said "Skills installed", and add-json returned "success": true with exit code 0.

  • setupSkills now collects per-source failures, stops the spinner with "Failed to install skills" and returns an AddonSetupError containing the underlying command error (including its stderr).
  • runOptionalStep records failed steps (alongside the existing interrupt tracking), so silent/JSON callers can see them instead of the error being dropped.
  • add / add-json: a failed addon setup step now fails the command (exit 1, error text in the CLI error / success: false + error for programmatic callers), and bts.jsonc is not updated with the failed addon. This matches how other add setup errors already behave, and add has no other purpose than setting up the requested addons.
  • create / create-json: the project is still created (addon setup stays an optional post-scaffold step, as for the other addons), but failed steps are now listed in warnings.

Fixes #1096

Outdated curated skills

Many of these failures came from our own defaults. I checked every curated skill against skills add <source> --list:

  • Convex: 7 of the 9 skills in waynesutton/convexskills no longer exist, so every Convex project with the skills addon failed. The source is now Convex's official get-convex/agent-skills, with convex, convex-design, convex-docs, convex-reviewer, convex-authz, convex-crons, convex-migrate, convex-env and convex-test.
  • Expo: expo-tailwind-setup was removed upstream.
  • Prisma: prisma-database-setup and prisma-postgres are now deprecated aliases. They're replaced by prisma-orm-setup and prisma-postgres-setup.

Every other source still publishes all of its curated names. With the default selection, add-json installs all skills cleanly into a Next.js + Expo + Convex project and a Prisma project.

Verification

  • bun run check
  • cd apps/cli && AGENT=1 BTS_TELEMETRY=0 bun run test:ci (112 pass)
  • Built CLI, generated a project in a scratch folder, then:
    • add-json with selections: [{ source: "vercel-labs/agent-skills", skills: ["does-not-exist-xyz"] }] -> exit 1, error includes Failed to install skills from vercel-labs/agent-skills: Command failed with exit code 1 ... No matching skills found for: does-not-exist-xyz; no .claude folder, bts.jsonc unchanged.
    • Same with web-design-guidelines -> "success": true, .claude/skills/web-design-guidelines installed.
    • create-json with the bad skill -> success: true with the install error in warnings.
    • Interactive add --addons skills (driven through a pty, with a failing npx on PATH) -> spinner shows "Failed to install skills", each source's error is printed, ends with "Addon setup failed", exit 1.
  • No contract test added: the test setup forces BTS_SKIP_EXTERNAL_COMMANDS=1, so setupSkills never runs the skills CLI in tests, and exercising it would need network access.

Summary by CodeRabbit

  • Bug Fixes
    • Skill installation failures are reported after all sources have been attempted.
    • Addon setup failures appear in CLI results. Newly reserved workspace packages are removed when setup fails; if cleanup also fails, the CLI advises removing the directory before retrying.
    • Project creation results include warnings for failed optional setup steps. Cancellations and interruptions remain warnings; other optional-step failures are reported as errors.
  • Updates
    • Updated available Convex and Prisma skills and migrated saved Convex skill selections to the new source and skill names.
    • Expo’s curated list no longer conditionally includes the Tailwind setup skill.

@vercel

vercel Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
create-better-t-stack-web Ready Ready Preview Oct 22, 2026 5:25pm UTC

Request Review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-22T08:02:10.072684Z c252d49 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c252d49ff0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/cli/src/utils/interrupt.ts Resolved
@coderabbitai

coderabbitai Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

The CLI records failures from optional setup steps and includes them in project creation results. It collects skill installation errors, cleans up reserved package directories after addon setup failures, and updates skill sources and recommendations.

Changes

CLI setup failure reporting

Layer / File(s) Summary
Record optional-step failures
apps/cli/src/utils/context.ts, apps/cli/src/utils/interrupt.ts, apps/cli/src/utils/optional-step.ts
Interrupt scopes clear recorded failures. Non-cancellation failures from optional steps are recorded in the CLI context and can be retrieved as a copy.
Include failures in project creation results
apps/cli/src/helpers/core/create-project.ts, apps/cli/src/helpers/core/command-handlers.ts
Successful project creation outcomes include recorded step failures. The command handler adds them to the warnings.
Report addon installation failures
apps/cli/src/helpers/addons/skills-setup.ts, apps/cli/src/helpers/core/add-handler.ts, apps/cli/src/utils/bts-config.ts, packages/types/src/schemas.ts, apps/cli/test/add-project.test.ts
Skill installation errors are collected and returned together after all sources are attempted. The add handler removes the reserved package directory and returns an error when setup-step failures are recorded. Convex source identifiers and legacy configuration mappings change. Expo and Prisma skill selections also change.

Priority: ➖ Normal

Severity of issue fixed: Medium

Merge Risk: 🔵 Low · up to c5250

Unsupported legacy skill selections can be silently changed, but the impact is localized and the fix is straightforward.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 21.43% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 14 functions across 10 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the primary change: reporting failed skill installations instead of indicating success.
Linked Issues check Passed Issue #1096 reports that skill installation failures appear only as warnings. The PR records failed optional steps in runOptionalStep and collects per-source failures in setupSkills. The PR fails …
Out of Scope Changes check Passed The changes stay within issue #1096. Error propagation, failure reporting, spinner status, configuration handling, warning output, and curated skill updates all support reliable skills installation. T…
Full details: Linked Issues check

Explanation

Issue #1096 reports that skill installation failures appear only as warnings. The PR records failed optional steps in runOptionalStep and collects per-source failures in setupSkills. The PR fails add and add-json with the command error, preserves the failed addon's configuration, and reports failures in create and create-json warnings while allowing project creation to finish. The curated skill updates address known invalid selections that caused installation failures. The supplied verification reports passing checks and the required CLI scenarios.

Full details: Out of Scope Changes check

Explanation

The changes stay within issue #1096. Error propagation, failure reporting, spinner status, configuration handling, warning output, and curated skill updates all support reliable skills installation. The configuration migration for legacy Convex selections supports the curated source update. No unrelated change is demonstrated.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b3c2e9d6-eff5-4d44-9a86-7b80fdb47dd6
📥 Commits

Reviewing files that changed from the base of the PR and between 09edd06 and c252d49.

📒 Files selected for processing: 6
  • apps/cli/src/helpers/addons/skills-setup.ts
  • apps/cli/src/helpers/core/add-handler.ts
  • apps/cli/src/helpers/core/command-handlers.ts
  • apps/cli/src/helpers/core/create-project.ts
  • apps/cli/src/utils/interrupt.ts
  • apps/cli/src/utils/optional-step.ts

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

Comment thread apps/cli/src/utils/interrupt.ts Resolved

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 14bac925-3c2a-4a7a-9598-a08b617e0c03
📥 Commits

Reviewing files that changed from the base of the PR and between c252d49 and f4d213c.

📒 Files selected for processing: 3
  • apps/cli/src/helpers/core/add-handler.ts
  • apps/cli/src/utils/context.ts
  • apps/cli/src/utils/interrupt.ts
🚧 Files skipped from review as they are similar to previous changes: 1
  • apps/cli/src/utils/interrupt.ts

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

If the reserved package directory can't be removed after a failed add,
the error now says so and asks the user to delete it, since the leftover
directory makes the next add with the same name fail.
Checked every curated skill against `skills add <source> --list`:

- Convex: 7 of the 9 skills in waynesutton/convexskills no longer exist.
  Switch to Convex's official get-convex/agent-skills: convex,
  convex-design, convex-docs, convex-reviewer, convex-authz, convex-crons,
  convex-migrate, convex-env and convex-test.
- Expo: expo-tailwind-setup was removed upstream.
- Prisma: prisma-database-setup and prisma-postgres are now deprecated
  aliases; use prisma-orm-setup and prisma-postgres-setup.

All other sources still publish every curated name.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 02a1be40-8062-401e-9613-96c3f7186d5f
📥 Commits

Reviewing files that changed from the base of the PR and between 8afaaa3 and bd4cfff.

📒 Files selected for processing: 2
  • apps/cli/src/helpers/addons/skills-setup.ts
  • packages/types/src/schemas.ts

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 5 remain after this review.

Comment thread packages/types/src/schemas.ts Resolved
…kills

Projects created before the Convex source change store
waynesutton/convexskills in bts.jsonc, which the schema no longer
accepts, so add reported no project found. Map the legacy source and
its skill names to the current ones when reading the config, the same
way the old Hono source is handled; add then saves the migrated
selection.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: d88fc0d0-073b-4633-9a68-0b64380552cb
📥 Commits

Reviewing files that changed from the base of the PR and between bd4cfff and c525033.

📒 Files selected for processing: 2
  • apps/cli/src/utils/bts-config.ts
  • apps/cli/test/add-project.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 5 remain after this review.

Comment thread apps/cli/src/utils/bts-config.ts Resolved
The CLI targets the latest scaffold only, so bts.jsonc is read as is
instead of rewriting the old Hono and Convex skill sources.
@AmanVarshney01
AmanVarshney01 merged commit 6b47ce1 into main Oct 22, 2026
9 of 11 checks passed
@AmanVarshney01
AmanVarshney01 deleted the fix/skills-install-errors branch October 22, 2026 17:30
@coderabbitai coderabbitai Bot mentioned this pull request Oct 24, 2026

This branch was successfully deployed

1 active deployment
Preview — 64643054 Deployed Oct 22, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Could not install skills

1 participant