Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 106 additions & 1 deletion docs/TELEMETRY.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,9 @@ The goal of telemetry is to answer questions like:
CI pipelines?
- For key flows (project creation, upgrade checks), which configuration options
are popular (e.g. which Shopware version, which CI provider, Docker yes/no)?
- For the interactive development TUI (`project dev`): do the setup wizards
succeed or where do users abandon them, do Docker environments start
reliably, and which dashboard features (watchers, builds, tabs) are used?

## How we identify users

Expand Down Expand Up @@ -87,7 +90,8 @@ All events share a common envelope:
| `timestamp` | RFC 3339 timestamp of when the event was sent |
| `tags` | Event-specific key/value pairs (all string values), described below|

There are currently **three** tracked events.
There are three command-level events, plus a set of events sent by the
interactive development TUI (`shopware-cli project dev`).

### `shopware_cli.command` — every command run

Expand Down Expand Up @@ -135,6 +139,107 @@ how often blockers are encountered.
| `target_version` | The version the user wants to upgrade to | `6.6.0` |
| `has_blockers` | Whether any blocking incompatibilities were found| `true` |

## Development TUI events (`shopware-cli project dev`)

The interactive development dashboard sends the events below so we can see
where the setup flows lose users, whether Docker environments start reliably,
and which dashboard features are actually used. As everywhere else: no URLs,
no sales-channel or theme names, no credentials, and no free-text input are
ever transmitted — only enumerated choices, outcomes, and durations.

### `shopware_cli.project.dev.install` — first-run Shopware installation

Sent when the "Shopware is not initialized yet" wizard reaches a terminal
state. Choice tags are only present once the user made that choice (an event
for a run cancelled on the language step carries no `language` tag).

| Tag | Meaning | Example |
|----------------------|---------------------------------------------------------------|------------------|
| `result` | Outcome of the wizard | `success` / `failure` / `cancelled` / `skipped` |
| `abandoned_at` | Step shown when the user quit (only for `cancelled`) | `ask` / `language` / `currency` / `credentials` / `installing` |
| `failed_step` | Last install step that had started (only for `failure`) | `system:install` |
| `duration_ms` | Install runtime, once the install actually started | `84213` |
| `language` | Selected default language | `de-DE` |
| `currency` | Selected default currency | `EUR` |
| `custom_credentials` | Whether the default admin username/password were changed. The credentials themselves are **never** sent. | `false` |

### `shopware_cli.project.dev.migration_wizard` — dev-environment setup wizard

Sent when the wizard that migrates an existing project to the Docker dev
environment reaches a terminal state. This replaces the earlier
`shopware_cli.migration_wizard_completed` event, which only reported
successful runs.

| Tag | Meaning | Example |
|---------------------------|------------------------------------------------------|--------------|
| `result` | Outcome of the wizard | `completed` / `cancelled` / `failed` |
| `abandoned_at` | Step shown when the user quit (only for `cancelled`) | `welcome` / `admin_user` / `docker_php` / `review` |
| `duration_ms` | Time since the welcome screen was confirmed | `45120` |
| `php_version` | Selected PHP version (`completed` / `failed` only) | `8.3` |
| `deployment_helper_added` | Whether composer.json was missing `shopware/deployment-helper` and it was added (`completed` only) | `true` |

### `shopware_cli.project.dev.docker_start` — container startup

Sent when a `docker compose up -d` run by the TUI finishes.

| Tag | Meaning | Example |
|---------------|--------------------------------------------------------------|-----------------|
| `trigger` | Why the containers were (re)started | `initial` / `config_change` |
| `result` | Outcome (`cancelled` when the user quit while waiting) | `success` / `failure` / `cancelled` |
| `duration_ms` | How long the startup took | `12894` |

### `shopware_cli.project.dev.action` — dashboard actions

Sent when a command-palette action runs. Instant actions (opening the shop or
admin in the browser) carry only the action name; task actions also report
their outcome and runtime.

| Tag | Meaning | Example |
|---------------|--------------------------------------------|----------------|
| `action` | The invoked action | `open-shop` / `open-admin` / `cache-clear` / `admin-build` / `sf-build` |
| `result` | Task outcome (task actions only) | `success` / `failure` / `cancelled` |
| `duration_ms` | Task runtime (task actions only) | `31022` |

### `shopware_cli.project.dev.watcher` — admin/storefront watchers

Sent once per watcher run, when the watcher ends.

| Tag | Meaning | Example |
|-------------|-------------------------------------------------------------|------------|
| `watcher` | Which watcher ran | `admin` / `storefront` |
| `result` | How the run ended: preparation failed, the dev-server process exited on its own, the user stopped it, or the TUI session ended | `prep_failed` / `crashed` / `user_stopped` / `session_end` |
| `uptime_ms` | How long the watcher ran | `1830211` |

### `shopware_cli.project.dev.health` — setup health snapshot

Sent once per session when the Overview tab's setup-health report first
loads: one tag per check, with the check's level as the value. This tells us
how common misconfigurations are in real projects — and which checks are
worth turning into automatic fixes. Only the level is sent, never the
underlying values.

| Tag | Meaning | Example |
|--------------------------|--------------------------------------------|---------|
| `php_version` | Running PHP version vs. Shopware constraint| `ok` / `warn` / `critical` |
| `memory_limit` | PHP memory_limit vs. recommendation | `ok` / `warn` |
| `admin_worker` | Browser admin worker disabled? | `ok` / `warn` |
| `flow_builder_log_level` | Flow Builder log level vs. recommendation | `ok` / `warn` |

### `shopware_cli.project.dev.session` — dashboard session shape

Sent when the user leaves the dashboard (the overall command duration is
already covered by `shopware_cli.command`; this event adds what happened
inside the TUI).

| Tag | Meaning | Example |
|-----------------|----------------------------------------------------|-----------------------------|
| `executor` | Environment the project runs in | `docker` / `local` |
| `duration_ms` | TUI session length | `1912734` |
| `tabs_visited` | Which tabs were opened (sorted, comma-separated) | `config,instance,overview` |
| `actions` | Number of palette actions invoked | `3` |
| `watchers_used` | Which watchers were started (omitted if none) | `admin,storefront` |
| `exit` | How the session ended: via the "stop containers?" dialog or a plain quit | `stop_containers` / `keep_running` / `quit` |

## What we explicitly do **not** collect

- No names, emails, usernames, or account / license identifiers.
Expand Down
9 changes: 9 additions & 0 deletions internal/devtui/install_wizard.go
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,11 @@ var installStepPatterns = []struct {

func (m Model) updateInstallPrompt(msg tea.KeyPressMsg) (tea.Model, tea.Cmd) {
if k := msg.String(); k == keyQ || k == keyCtrlC {
if m.telemetry.installOnce() {
tags := m.telemetry.installTags(resultCancelled, m.install)
tags["abandoned_at"] = installStepTagName(m.install.step)
trackEventNow(eventDevInstall, tags)
}
return m, tea.Quit
}

Expand Down Expand Up @@ -141,6 +146,9 @@ func (m Model) updateInstallStepAsk(msg tea.KeyPressMsg) (tea.Model, tea.Cmd) {
m.install.cursor = 0
return m, nil
}
if m.telemetry.installOnce() {
trackEvent(eventDevInstall, m.telemetry.installTags(resultSkipped, m.install))
}
m.phase = phaseDashboard
return m, m.startDashboard()
}
Expand Down Expand Up @@ -197,6 +205,7 @@ func (m Model) handleInstallCredentialsEnter() (tea.Model, tea.Cmd) {
return m, nil
}
m.install.blur()
m.telemetry.beginInstall()
m.phase = phaseInstalling
m.overlayLines = nil
m.installProg = installProgress{
Expand Down
11 changes: 11 additions & 0 deletions internal/devtui/lifecycle.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,9 @@ func (m Model) updateLifecycle(msg tea.Msg) (tea.Model, tea.Cmd) {
return m, nil

case dockerStartedMsg:
if tags, ok := m.telemetry.dockerStartTags(msg.err); ok {
trackEvent(eventDevDockerStart, tags)
}
if msg.err != nil {
m.dockerShowLogs = true
m.overlayLines = append(m.overlayLines, errorStyle.Render("Failed: "+msg.err.Error()))
Expand Down Expand Up @@ -74,13 +77,21 @@ func (m Model) updateLifecycle(msg tea.Msg) (tea.Model, tea.Cmd) {

case shopwareInstallDoneMsg:
if msg.err != nil {
if m.telemetry.installOnce() {
tags := m.telemetry.installTags(resultFailure, m.install)
tags["failed_step"] = installFailedStep(m.installProg.currentStep)
Comment thread
shyim marked this conversation as resolved.
Outdated
trackEvent(eventDevInstall, tags)
}
m.installProg.showLogs = true
m.overlayLines = append(m.overlayLines, "", errorStyle.Render("Installation failed: "+msg.err.Error()))
m.overlayLines = append(m.overlayLines, "", helpStyle.Render("Press q to exit"))
return m, nil
}
m.installProg.done = true
m.installProg.currentStep = len(installStepPatterns)
if m.telemetry.installOnce() {
trackEvent(eventDevInstall, m.telemetry.installTags(resultSuccess, m.install))
}

username := m.install.username.Value()
password := m.install.password.Value()
Expand Down
93 changes: 74 additions & 19 deletions internal/devtui/model.go
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ type Model struct {
taskErr error
watchers map[string]*watcherHandle
migrationWizard migrationWizard
telemetry *telemetryState
}

type dockerAlreadyRunningMsg struct{}
Expand All @@ -100,6 +101,7 @@ func New(opts Options) Model {
config: opts.Config,
envConfig: opts.EnvConfig,
watchers: make(map[string]*watcherHandle),
telemetry: newTelemetryState(opts.Executor.Type() == executor.TypeDocker),
}
m.rebuildTabs()
return m
Expand Down Expand Up @@ -161,9 +163,23 @@ func (m *Model) shutdown() {
defer cancel()

for name, h := range m.watchers {
if tags, ok := m.telemetry.watcherEndTags(name, watcherEndSessionEnd); ok {
trackEventNow(eventDevWatcher, tags)
}
h.stop(ctx)
delete(m.watchers, name)
}

// A config-change container restart may still be in flight when the user
// leaves the dashboard; report it as cancelled instead of dropping it.
if tags, ok := m.telemetry.configRestartTags(nil); ok {
tags["result"] = resultCancelled
trackEventNow(eventDevDockerStart, tags)
}

if tags, ok := m.telemetry.sessionTags(); ok {
trackEventNow(eventDevSession, tags)
}
}

func (m *Model) startDashboard() tea.Cmd {
Expand Down Expand Up @@ -191,6 +207,9 @@ func (m Model) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
case taskDoneMsg:
m.taskDone = true
m.taskErr = msg.err
if tags, ok := m.telemetry.taskTags(resultTag(msg.err)); ok {
trackEvent(eventDevAction, tags)
}
if msg.err != nil {
m.overlayLines = append(m.overlayLines, "", errorStyle.Render("Failed: "+msg.err.Error()))
} else {
Expand All @@ -201,11 +220,54 @@ func (m Model) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
case configRestartDoneMsg:
return m.handleConfigRestartDone(msg)

case watcherStartedMsg, watcherRunningMsg, stopWatcherRequestMsg,
startStorefrontWatchRequestMsg, watcherStoppedMsg, logDoneMsg:
return m.updateWatcherMsg(msg)

case setupHealthLoadedMsg:
if len(msg.checks) > 0 && m.telemetry.healthOnce() {
trackEvent(eventDevHealth, healthTags(msg.checks))
}
return m.updateFallback(msg)

case paletteResultMsg:
return m.handlePaletteResult(msg)

case pickerResultMsg:
if _, ok := msg.Key.(salesChannelPickerKey); ok {
break
}
return m.handlePickerResult(msg)

case salesChannelPickerResultMsg:
return m.handleSalesChannelPickerResult(msg)

case stopConfirmResultMsg:
return m.handleStopConfirmResult(msg)

case tea.KeyPressMsg:
return m.updateKeyPress(msg)
}

return m.updateFallback(msg)
}

// updateWatcherMsg handles the watcher lifecycle messages: start, prep
// done/failed, stop requests, stopped, and the log stream ending (the watcher
// process exiting on its own).
func (m Model) updateWatcherMsg(msg tea.Msg) (tea.Model, tea.Cmd) {
switch msg := msg.(type) {
case watcherStartedMsg:
m.watchers[msg.name] = msg.handle
m.telemetry.watcherStarted(msg.name)
return m, m.instance.AddStreamingSource(msg.name, msg.lines)

case watcherRunningMsg:
if msg.err != nil {
if tags, ok := m.telemetry.watcherEndTags(msg.name, watcherEndPrepFailed); ok {
trackEvent(eventDevWatcher, tags)
}
}
_, exists := m.watchers[msg.name]
switch msg.name {
case watcherAdmin:
Expand Down Expand Up @@ -249,29 +311,14 @@ func (m Model) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
case watcherStorefront:
m.overview.sfWatchRunning = false
}
if tags, ok := m.telemetry.watcherEndTags(msg.source, watcherEndCrashed); ok {
trackEvent(eventDevWatcher, tags)
}
delete(m.watchers, msg.source)
return m.updateChildren(msg)

case paletteResultMsg:
return m.handlePaletteResult(msg)

case pickerResultMsg:
if _, ok := msg.Key.(salesChannelPickerKey); ok {
break
}
return m.handlePickerResult(msg)

case salesChannelPickerResultMsg:
return m.handleSalesChannelPickerResult(msg)

case stopConfirmResultMsg:
return m.handleStopConfirmResult(msg)

case tea.KeyPressMsg:
return m.updateKeyPress(msg)
}

return m.updateFallback(msg)
return m, nil
}

// updateFallback handles non-key messages that aren't matched by Update's
Expand Down Expand Up @@ -360,6 +407,11 @@ func (m Model) handleStopConfirmResult(msg stopConfirmResultMsg) (tea.Model, tea
if msg.Cancel {
return m, nil
}
if msg.Stop {
m.telemetry.setExitChoice(exitStopContainers)
} else {
m.telemetry.setExitChoice(exitKeepRunning)
}
m.shutdown()
if msg.Stop {
m.phase = phaseStopping
Expand Down Expand Up @@ -418,6 +470,9 @@ func (m Model) updateChildren(msg tea.Msg) (tea.Model, tea.Cmd) {
}

func (m Model) handleConfigRestartDone(msg configRestartDoneMsg) (tea.Model, tea.Cmd) {
if tags, ok := m.telemetry.configRestartTags(msg.err); ok {
trackEvent(eventDevDockerStart, tags)
}
m.configTab.restarting = false
if msg.err != nil {
m.configTab.err = msg.err
Expand Down
2 changes: 2 additions & 0 deletions internal/devtui/model_commands.go
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,7 @@ func runComposeCommand(ctx context.Context, projectRoot string, args []string, r
}

func (m *Model) startContainers() tea.Cmd {
m.telemetry.beginDockerStart()
ch, outputCmd, doneCmd := runComposeCommand(
context.Background(),
m.projectRoot,
Expand All @@ -201,6 +202,7 @@ func (m *Model) startContainers() tea.Cmd {
}

func (m *Model) restartContainersForConfig() tea.Cmd {
m.telemetry.beginConfigRestart()
Comment thread
shyim marked this conversation as resolved.
projectRoot := m.projectRoot
cfg := m.config
return func() tea.Msg {
Expand Down
Loading
Loading