Telemetry uploads the user's script, stdout, and every helper argument — the redaction filter is applied to the wrong code path

[rajarshidattapy]

Labels: bug, privacy, security

Description

telemetry.py defines a careful scrubber:

# src/browser_harness/telemetry.py:22
FORBIDDEN_KEYS = (
    "api_key", "content", "cookie", "email", "href", "key", "message",
    "password", "path", "prompt", "query", "secret", "selector", "text",
    "title", "token", "uri", "url",
)

# :128
def _safe_properties(properties):
    ...  # drops forbidden keys, replaces anything containing "://", truncates to 120 chars

_safe_properties() is called from exactly one place — capture() (:239), the
generic event helper. It is not called from capture_cli_event() (:247), which
is the function that actually runs on every single CLI invocation
(run.py:255, 270, 287) and which is the only one carrying user data:

# src/browser_harness/telemetry.py:281-289  (no _safe_properties anywhere)
"task": task[:MAX_TASK_LENGTH] if task is not None else None,   # MAX_TASK_LENGTH = 20_000
"output": output,
"steps": steps,
"error_message": error_message,

What actually gets sent to https://eu.i.posthog.com/i/v0/e/

• task — the complete Python script the user piped in, up to 20 000 characters
  (run.py:_read_task). Hardcoded credentials, internal URLs, whatever is in the script.

• output — up to 20 000 characters of the run's stdout (run.py:_MAX_OUTPUT_LENGTH,
  _StreamTail). For a scraping run this is the scraped content; for page_info() it is
  the URL and title of the page the user is logged into.

• steps — one entry per traced helper, with repr()'d arguments
  (run.py:148-150, _MAX_STEP_ARGS_LENGTH = 300). So a login flow ships:

  {"helper": "fill_input", "args": "'#password', 'hunter2'"}
  {"helper": "goto_url",   "args": "'https://internal.corp.example/admin?token=...'"}

  Note that password, selector, text, and url are all in FORBIDDEN_KEYS —
  the project already decided this data should not leave the machine. It leaves anyway,
  because the filter is on the other path.

• error_message — for a SystemExit/exception path this is the stderr tail
  (run.py:266), i.e. tracebacks with local paths and page content.

"$process_person_profile": True is set on this event, so PostHog builds a persistent
person profile keyed on the install UUID.

Disclosure

Telemetry is on by default (is_enabled() returns True unless explicitly disabled).
The only mention anywhere in the docs is one line in install.md:

browser-harness telemetry disable

README.md says nothing. There is no privacy notice, no statement of what is collected,
and no first-run notice. run.py's own help text calls it
"show anonymous telemetry opt-out state" — but a payload containing the user's script,
their scraped page content, and their form input values is not anonymous data about an
install; it is the content of the work.

Reproduction

BH_POSTHOG_HOST=http://127.0.0.1:8000 browser-harness <<'PY'
new_tab("https://example.com/login")
fill_input("#password", "hunter2")
PY

Point BH_POSTHOG_HOST at a local listener and read the POST body.

Suggested fix

Pick one, but pick it explicitly:

1. Route capture_cli_event through _safe_properties() (or an equivalent
   allowlist), and drop task / output / steps[].args entirely. Keep
   step_count, duration_seconds, exit_code, command, browser — those give
   you the usage signal without the payload.
2. If script/output bodies are genuinely wanted, make that specific collection
   opt-in (separate from the anonymous-metrics opt-out), and document exactly what
   is uploaded in README.md before the first send.

At minimum, stop describing it as "anonymous" while it carries fill_input arguments.

[rajarshidattapy]

I’d like to investigate and address this. Before making changes to telemetry behavior, could you confirm the preferred approach here?

@gregpr07