Skip to content

feat(devtui): track usage telemetry for setup flows and dashboard#1162

Open
Soner (shyim) wants to merge 3 commits into
nextfrom
feat/devtui-telemetry
Open

feat(devtui): track usage telemetry for setup flows and dashboard#1162
Soner (shyim) wants to merge 3 commits into
nextfrom
feat/devtui-telemetry

Conversation

@shyim

Copy link
Copy Markdown
Member

Summary

Adds anonymous usage telemetry to the interactive dev TUI (shopware-cli project dev) so we can answer: where do the setup wizards lose users, do Docker environments start reliably, and which dashboard features are actually used.

All events go through the existing internal/tracking pipeline (pseudonymous ID, UDP fire-and-forget, DO_NOT_TRACK opt-out) and are fully documented in docs/TELEMETRY.md. Privacy posture is unchanged: no URLs, no sales-channel/theme names, no credentials, no free-text input — only enumerated choices, outcomes, and durations.

Events

shopware_cli.project.dev.install — first-run Shopware installation

Sent when the "Shopware is not initialized yet" wizard reaches a terminal state. Choice tags only appear once the user made that choice.

Tag Values
result success / failure / cancelled / skipped
abandoned_at ask / language / currency / credentials / installing (cancelled only)
failed_step last deployment-helper step that had started, e.g. system:install (failure only)
duration_ms install runtime once the install actually started
language / currency selected defaults, e.g. de-DE / EUR
custom_credentials true/false — whether the default admin user/password were changed; the values themselves are never sent

shopware_cli.project.dev.migration_wizard — dev-environment setup wizard

Replaces the old migration_wizard_completed event, which only reported successful runs, so no conversion rate could be computed.

Tag Values
result completed / cancelled / failed
abandoned_at welcome / admin_user / docker_php / review (cancelled only)
duration_ms time since the welcome screen was confirmed
php_version selected PHP version (completed/failed)
deployment_helper_added whether composer.json was missing shopware/deployment-helper (completed only)

shopware_cli.project.dev.docker_start — container startup

Sent when a docker compose up -d run by the TUI finishes (or the user gives up waiting).

Tag Values
trigger initial / config_change
result success / failure / cancelled (user quit while waiting)
duration_ms startup time

shopware_cli.project.dev.action — dashboard actions

Sent per command-palette action. Instant actions carry only the name; task actions also report outcome and runtime.

Tag Values
action open-shop / open-admin / cache-clear / admin-build / sf-build
result success / failure / cancelled (task actions only)
duration_ms task runtime (task actions only)

shopware_cli.project.dev.watcher — admin/storefront watchers

Sent once per watcher run, when it ends. Crash rate vs. user-stop rate tells us if the watchers are reliable; uptime tells us if people develop inside the TUI or just use it as a launcher.

Tag Values
watcher admin / storefront
result prep_failed / crashed / user_stopped / session_end
uptime_ms how long the watcher ran

shopware_cli.project.dev.health — setup health snapshot

Sent once per session when the Overview tab's setup-health report first loads: one tag per check with its level as value. Shows how common misconfigurations are in the wild — and which checks are worth turning into automatic fixes. Only the level is sent, never the underlying values.

Tag Values
php_version ok / warn / critical
memory_limit ok / warn
admin_worker ok / warn
flow_builder_log_level ok / warn

shopware_cli.project.dev.session — dashboard session shape

Sent when the user leaves the dashboard. Total command duration is already covered by shopware_cli.command; this adds what happened inside the TUI.

Tag Values
executor docker / local
duration_ms TUI session length
tabs_visited sorted, comma-separated, e.g. config,instance,overview
actions number of palette actions invoked
watchers_used admin,storefront (omitted if none)
exit stop_containers / keep_running / quit

Implementation notes

  • All bookkeeping lives in a new telemetryState, held by pointer on Model so Bubble Tea's value copies share it; every method is nil-safe because tests construct Model{} directly.
  • Outcome events latch (installOnce, healthOnce, sessionTags, per-run watcher start times), so overlapping quit paths — e.g. pressing q on the install-failure screen, or a trailing logDoneMsg after a user stop — cannot double-report.
  • Events on quit paths send synchronously with a 300 ms cap (a goroutine would race process exit); everything else sends async so the UI never blocks on DNS.
  • A package TestMain sets DO_NOT_TRACK, so the devtui test suite can never emit real events.
  • docs/TELEMETRY.md is updated with all events (and now also mentions the previously undocumented migration-wizard tracking).

Test plan

  • go test ./... — new unit tests cover tag building (choice omission, credential redaction, once-latching, watcher double-report protection, health flattening)
  • go vet, gofmt, go build

🤖 Generated with Claude Code

Soner (shyim) and others added 2 commits July 6, 2026 09:40
Add anonymous usage events to the project dev TUI so we can see where
the setup wizards lose users, whether Docker environments start
reliably, and which dashboard features are used:

- project.dev.install: install wizard outcome incl. abandoned step,
  failed deployment-helper step, and chosen language/currency
- project.dev.migration_wizard: replaces migration_wizard_completed,
  now also reporting cancelled (with step) and failed runs
- project.dev.docker_start: compose up duration and result, for both
  initial startup and config-change restarts
- project.dev.action: command-palette actions with task result/duration
- project.dev.watcher: per-watcher-run uptime and end reason
- project.dev.health: one-per-session setup-health snapshot (levels only)
- project.dev.session: session duration, tabs visited, action counts,
  and stop-containers vs keep-running exit choice

All state lives in a nil-safe telemetryState shared across Model copies;
outcome events latch so quit paths cannot double-report. No URLs,
credentials, or free-text input are ever sent. Events are documented in
docs/TELEMETRY.md, and a package TestMain sets DO_NOT_TRACK so tests
never emit real events.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Check the os.Setenv error in TestMain, make the healthLevel switch
exhaustive, and extract watcher message handling from Model.Update into
updateWatcherMsg to bring its cyclomatic complexity back under the limit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7ba3e2e401

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".

Comment thread internal/devtui/model_commands.go
if msg.err != nil {
if m.telemetry.installOnce() {
tags := m.telemetry.installTags("failure", m.install)
tags["failed_step"] = installFailedStep(m.installProg.currentStep)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would extract all the string identifiers for events into constants

…nstants

Address PR review feedback:

- A config-change container restart still in flight when the user leaves
  the dashboard is now reported as a cancelled docker_start event from
  shutdown() instead of being silently dropped (matching the
  initial-start quit path).
- Extract the result / watcher-end / exit tag string values into
  constants in telemetry.go so call sites can't drift apart. Tests keep
  asserting the literal wire values on purpose.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants