feat(devtui): track usage telemetry for setup flows and dashboard#1162
Merged
Conversation
Add anonymous usage events to the project dev TUI so we can see where the setup wizards lose users, whether Docker environments start reliably, and which dashboard features are used: - project.dev.install: install wizard outcome incl. abandoned step, failed deployment-helper step, and chosen language/currency - project.dev.migration_wizard: replaces migration_wizard_completed, now also reporting cancelled (with step) and failed runs - project.dev.docker_start: compose up duration and result, for both initial startup and config-change restarts - project.dev.action: command-palette actions with task result/duration - project.dev.watcher: per-watcher-run uptime and end reason - project.dev.health: one-per-session setup-health snapshot (levels only) - project.dev.session: session duration, tabs visited, action counts, and stop-containers vs keep-running exit choice All state lives in a nil-safe telemetryState shared across Model copies; outcome events latch so quit paths cannot double-report. No URLs, credentials, or free-text input are ever sent. Events are documented in docs/TELEMETRY.md, and a package TestMain sets DO_NOT_TRACK so tests never emit real events. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Check the os.Setenv error in TestMain, make the healthLevel switch exhaustive, and extract watcher message handling from Model.Update into updateWatcherMsg to bring its cyclomatic complexity back under the limit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7ba3e2e401
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
somethings (lasomethingsomething)
requested a review
from Tomasz Turkowski (tturkowski)
…nstants Address PR review feedback: - A config-change container restart still in flight when the user leaves the dashboard is now reported as a cancelled docker_start event from shutdown() instead of being silently dropped (matching the initial-start quit path). - Extract the result / watcher-end / exit tag string values into constants in telemetry.go so call sites can't drift apart. Tests keep asserting the literal wire values on purpose. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Tomasz Turkowski (tturkowski)
approved these changes
Jul 6, 2026
Tomasz Turkowski (tturkowski)
left a comment
Tomasz Turkowski (tturkowski)
left a comment
Contributor
There was a problem hiding this comment.
LGTM
Move all telemetry event names, tag keys, and shared result values into internal/tracking (tracking.Event*, tracking.Tag*, tracking.Result*) so every Track caller — cmd/root.go, project create/upgrade-check, and the dev TUI — shares one vocabulary instead of scattered string literals. The devtui package keeps only its domain-specific tag values (watcher end reasons, session exit choices). Tests still assert the raw wire strings on purpose. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…olumns The analytics events table materializes result and duration_ms columns from tags, so events benefit from reusing those keys: - watcher: uptime_ms -> duration_ms - session: exit -> result (stop_containers / keep_running / quit) - health: instead of one event with dynamic per-check keys (whose php_version key collided with the migration wizard's php_version value tag), send one event per check tagged check + result Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds anonymous usage telemetry to the interactive dev TUI (
shopware-cli project dev) so we can answer: where do the setup wizards lose users, do Docker environments start reliably, and which dashboard features are actually used.All events go through the existing
internal/trackingpipeline (pseudonymous ID, UDP fire-and-forget,DO_NOT_TRACKopt-out) and are fully documented indocs/TELEMETRY.md. Privacy posture is unchanged: no URLs, no sales-channel/theme names, no credentials, no free-text input — only enumerated choices, outcomes, and durations.Tag keys are deliberately shared across events (
result,duration_ms) so they align with the analytics ClickHouse table's materialized columns — every event's outcome and runtime is aggregatable through the same two columns.Events
shopware_cli.project.dev.install— first-run Shopware installationSent when the "Shopware is not initialized yet" wizard reaches a terminal state. Choice tags only appear once the user made that choice.
resultsuccess/failure/cancelled/skippedabandoned_atask/language/currency/credentials/installing(cancelled only)failed_stepsystem:install(failure only)duration_mslanguage/currencyde-DE/EURcustom_credentialstrue/false— whether the default admin user/password were changed; the values themselves are never sentshopware_cli.project.dev.migration_wizard— dev-environment setup wizardReplaces the old
migration_wizard_completedevent, which only reported successful runs, so no conversion rate could be computed.resultcompleted/cancelled/failedabandoned_atwelcome/admin_user/docker_php/review(cancelled only)duration_msphp_versiondeployment_helper_addedshopware/deployment-helper(completed only)shopware_cli.project.dev.docker_start— container startupSent when a
docker compose up -drun by the TUI finishes (or the user gives up waiting). A config-change restart still in flight when the user quits is reported ascancelledinstead of being dropped.triggerinitial/config_changeresultsuccess/failure/cancelled(user quit while waiting)duration_msshopware_cli.project.dev.action— dashboard actionsSent per command-palette action. Instant actions carry only the name; task actions also report outcome and runtime.
actionopen-shop/open-admin/cache-clear/admin-build/sf-buildresultsuccess/failure/cancelled(task actions only)duration_msshopware_cli.project.dev.watcher— admin/storefront watchersSent once per watcher run, when it ends. Crash rate vs. user-stop rate tells us if the watchers are reliable; runtime tells us if people develop inside the TUI or just use it as a launcher.
watcheradmin/storefrontresultprep_failed/crashed/user_stopped/session_endduration_msshopware_cli.project.dev.health— setup health snapshotSent once per session when the Overview tab's setup-health report first loads: one event per check. Shows how common misconfigurations are in the wild — and which checks are worth turning into automatic fixes. Only the check's level is sent, never the underlying values. New checks are aggregatable without any schema or dashboard changes.
checkphp_version/memory_limit/admin_worker/flow_builder_log_levelresultok/warn/criticalshopware_cli.project.dev.session— dashboard session shapeSent when the user leaves the dashboard. Total command duration is already covered by
shopware_cli.command; this adds what happened inside the TUI.executordocker/localduration_mstabs_visitedconfig,instance,overviewactionswatchers_usedadmin,storefront(omitted if none)resultstop_containers/keep_running/quitImplementation notes
internal/tracking/events.go(tracking.Event*,tracking.Tag*,tracking.Result*), used by every Track caller — the dev TUI,cmd/root.go, and the project create/upgrade-check commands, which previously used bare string literals.telemetryState, held by pointer onModelso Bubble Tea's value copies share it; every method is nil-safe because tests constructModel{}directly.installOnce,healthOnce,sessionTags, per-run watcher start times), so overlapping quit paths — e.g. pressingqon the install-failure screen, or a trailinglogDoneMsgafter a user stop — cannot double-report.TestMainsetsDO_NOT_TRACK, so the devtui test suite can never emit real events.docs/TELEMETRY.mdis updated with all events (and now also mentions the previously undocumented migration-wizard tracking).Test plan
go test ./...— new unit tests cover tag building (choice omission, credential redaction, once-latching, watcher double-report protection, per-check health events)go vet,gofmt,golangci-lint,go build🤖 Generated with Claude Code