Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions .github/workflows/fetch-telemetry.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Fetch Telemetry Data

on:
schedule:
# Run daily at 08:00 UTC (00:00 Pacific during PST, 01:00 during PDT)
- cron: '0 8 * * *'
workflow_dispatch:

permissions:
contents: write

jobs:
fetch-telemetry:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'

- name: Fetch and transform telemetry data
run: python3 hack/fetch_telemetry.py

- name: Check for changes
id: changes
run: |
if git diff --quiet static/oss-health-data/telemetry.json; then
echo "changed=false" >> "$GITHUB_OUTPUT"
else
echo "changed=true" >> "$GITHUB_OUTPUT"
fi

- name: Commit and push
if: steps.changes.outputs.changed == 'true'
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add static/oss-health-data/telemetry.json
git commit -m "chore(oss-health): update telemetry snapshot"
git push
7 changes: 7 additions & 0 deletions assets/scss/_oss_health.scss
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,13 @@
font-size: clamp(2rem, 3vw, 2.8rem);
line-height: 1;
}

&__hint {
color: #637595;
display: block;
font-size: 0.85rem;
margin-top: 0.6rem;
}
}

.oss-health-mini {
Expand Down
90 changes: 90 additions & 0 deletions assets/scss/_telemetry.scss
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
/* telemetry page */

.telemetry-page {
margin-top: 4rem;

@include media-breakpoint-down(sm) {
margin-top: 2rem;
}

.nav-tabs {
border-bottom: 2px solid $primary;

.nav-link {
color: $cozy-mid-gray;
font-weight: 600;
border: none;
border-bottom: 3px solid transparent;
padding: 0.75rem 1.5rem;

&:hover {
color: $primary;
border-bottom-color: rgba($primary, 0.3);
}

&.active {
color: $primary;
border-bottom-color: $primary;
background: transparent;
}
}
}

.telemetry-card {
border: none;
border-radius: 0.75rem;
transition: transform 0.15s ease;

&:hover {
transform: translateY(-2px);
}

.telemetry-icon {
font-size: 1.75rem;
color: $primary;
margin-bottom: 0.5rem;
}

.telemetry-value {
font-size: 2.5rem;
font-weight: 700;
color: $cozy-black;
line-height: 1.2;
}

.telemetry-label {
font-size: 0.95rem;
font-weight: 600;
color: $cozy-mid-gray;
text-transform: uppercase;
letter-spacing: 0.05em;
margin-top: 0.25rem;
}

.telemetry-secondary {
font-size: 0.85rem;
color: $cozy-light-gray;
margin-top: 0.25rem;
}
}

.table {
th {
font-weight: 600;
}

.table-primary {
--bs-table-bg: #{rgba($primary, 0.08)};
--bs-table-border-color: #{rgba($primary, 0.15)};
color: $cozy-black;
}

code {
color: $primary;
font-weight: 500;
background: rgba($primary, 0.06);
padding: 0.15rem 0.4rem;
border-radius: 0.25rem;
}
}
}
1 change: 1 addition & 0 deletions assets/scss/main.scss
Original file line number Diff line number Diff line change
Expand Up @@ -161,3 +161,4 @@ a {
@import "announcement-banner";
@import "tabs_alerts";
@import "override-docsy-tabs";
@import "telemetry";
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Move the telemetry import above all style rules.

Line 152 places @import "telemetry"; after rule blocks, which violates the current Stylelint rule (no-invalid-position-at-import-rule) and will keep lint failing.

♻️ Proposed fix
 // Import Docsy styles_project (allows further customization)
 `@import` "docsy/styles_project";
+@import "telemetry";
@@
 `@import` "tabs_alerts";
 `@import` "override-docsy-tabs";
-@import "telemetry";
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
@import "telemetry";
// Import Docsy styles_project (allows further customization)
`@import` "docsy/styles_project";
`@import` "telemetry";
`@import` "tabs_alerts";
`@import` "override-docsy-tabs";
🧰 Tools
🪛 Stylelint (17.6.0)

[error] 152-152: Unexpected invalid position @import rule (no-invalid-position-at-import-rule)

(no-invalid-position-at-import-rule)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@assets/scss/main.scss` at line 152, The `@import` "telemetry"; statement is
placed after style rule blocks causing a Stylelint
no-invalid-position-at-import-rule failure; move the import so it appears before
any CSS/SCSS rule definitions (i.e., place `@import` "telemetry"; at the very top
of main.scss, above all selectors, variables, and mixins) to satisfy the linter.

1 change: 1 addition & 0 deletions content/en/oss-health/_index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: "OSS Health"
description: "Open source project health snapshots for Cozystack."
type: oss-health
---

Project health snapshots for Cozystack across community activity, repository trends, and security posture.
10 changes: 10 additions & 0 deletions content/en/oss-health/telemetry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: "Telemetry"
layout: "oss-health-app"
draft: false
description: "Anonymous usage statistics reported by live Cozystack clusters."
oss_health_key: "telemetry"
oss_health_kind: "telemetry"
source_url: "https://telemetry.cozystack.io/"
lede: "Monthly, quarterly, and yearly snapshots of Cozystack fleet size and application usage across opted-in installations."
---
186 changes: 186 additions & 0 deletions hack/fetch_telemetry.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,186 @@
#!/usr/bin/env python3
"""
Fetch Cozystack telemetry and produce a JSON payload for the OSS Health shell.

What it does:
1. Query https://telemetry.cozystack.io/api/overview?year=YYYY&month=MM
2. Filter apps to entries visible on the Cozystack dashboard.
3. Merge case-insensitive / Pax* / legacy-name aliases into one canonical entry
per application, keeping the maximum instance count (zero-count entries
left after the merge are dropped from the table).
4. Pull `Tenant` out of the apps map and surface it as the top-level Tenants
summary card (the raw `total_tenants` field from the API is always zero).
5. Emit the payload in the shape consumed by `oss-health-app.html` +
`renderTelemetry`, including `summary_cards`, `apps`, `range`.

Used by both `.github/workflows/fetch-telemetry.yml` (daily cron) and the
developer who needs to refresh the seed file locally.
"""
from __future__ import annotations

import datetime as dt
import json
import os
import sys
import urllib.error
import urllib.request

API_URL = "https://telemetry.cozystack.io/api/overview"
OUTPUT_PATH = os.environ.get(
"TELEMETRY_OUTPUT_PATH",
"static/oss-health-data/telemetry.json",
)

# Canonical display name per normalized key. Anything not in this table is
# dropped (internal entities like `Info`, `Pax*` experimental variants that
# don't map to a dashboard app, duplicate lowercase CR kind names, etc.).
# Keys are lower-cased and stripped of hyphens so we can match PascalCase,
# lowercase, kebab-case and Pax-prefixed variants against the same canonical.
ALIASES: dict[str, str] = {
# Managed applications (docs/v1.2/applications/_include/*)
"clickhouse": "ClickHouse",
"paxclickhouse": "ClickHouse",
"foundationdb": "FoundationDB",
"harbor": "Harbor",
"kafka": "Kafka",
"mariadb": "MariaDB",
"mongodb": "MongoDB",
"nats": "NATS",
"openbao": "OpenBAO",
"opensearch": "OpenSearch",
"postgres": "Postgres",
"postgresql": "Postgres",
"paxpostgres": "Postgres",
"qdrant": "Qdrant",
"rabbitmq": "RabbitMQ",
"redis": "Redis",
"paxredis": "Redis",
"clearml": "ClearML",
# Services (docs/v1.2/operations/services/*)
"etcd": "Etcd",
"ingress": "Ingress",
"monitoring": "Monitoring",
"bucket": "Bucket",
"seaweedfs": "SeaweedFS",
"nfs": "NFS",
# Networking (docs/v1.2/networking/_include/*)
"httpcache": "HTTPCache",
"tcpbalancer": "TCPBalancer",
"virtualprivatecloud": "VirtualPrivateCloud",
"vpc": "VirtualPrivateCloud",
"vpn": "VPN",
# Virtualization (docs/v1.2/virtualization/_include/*)
"vminstance": "VMInstance",
"paxvminstance": "VMInstance",
"vmdisk": "VMDisk",
# Managed Kubernetes
"kubernetes": "Kubernetes",
}


def normalize_key(raw: str) -> str:
return raw.lower().replace("-", "").replace("_", "")


def clean_apps(apps: dict[str, int]) -> list[dict[str, object]]:
"""Filter, dedupe (max), drop zeros, sort desc by count."""
merged: dict[str, int] = {}
for raw_name, count in apps.items():
canonical = ALIASES.get(normalize_key(raw_name))
if not canonical:
continue
if count > merged.get(canonical, 0):
merged[canonical] = count
non_zero = [(name, n) for name, n in merged.items() if n > 0]
non_zero.sort(key=lambda item: (-item[1], item[0].lower()))
return [{"name": name, "value": str(count)} for name, count in non_zero]


def transform_period(raw_period: dict, label_fallback: str) -> dict | None:
if not raw_period:
return None
apps_raw = raw_period.get("apps", {}) or {}
tenants = int(apps_raw.get("Tenant", 0))
clusters = int(raw_period.get("clusters", 0))
total_nodes = int(raw_period.get("total_nodes", 0))
avg_nodes = raw_period.get("avg_nodes_per_cluster")
summary = [
{"label": "Clusters", "value": str(clusters)},
{
"label": "Total Nodes",
"value": str(total_nodes),
"hint": (
f"avg {avg_nodes:.1f} per cluster"
if clusters and isinstance(avg_nodes, (int, float))
else ""
),
},
{
"label": "Tenants",
"value": str(tenants),
"hint": (
f"avg {tenants / clusters:.1f} per cluster"
if clusters
else ""
),
},
]
period = {
"label": raw_period.get("label") or label_fallback,
"summary_cards": summary,
"apps": clean_apps(apps_raw),
}
start = raw_period.get("start")
end = raw_period.get("end")
if start and end:
period["range"] = {"from": start, "to": end}
return period


def fetch(year: int, month: int) -> dict:
url = f"{API_URL}?year={year}&month={month:02d}"
req = urllib.request.Request(url, headers={"User-Agent": "cozystack-website/telemetry-fetch"})
with urllib.request.urlopen(req, timeout=30) as resp:
if resp.status != 200:
raise RuntimeError(f"telemetry API returned HTTP {resp.status}")
return json.loads(resp.read().decode("utf-8"))


def build_payload(raw: dict) -> dict:
periods_raw = raw.get("periods", {}) or {}
periods_out: dict[str, dict] = {}
for key in ("month", "quarter", "year"):
transformed = transform_period(periods_raw.get(key, {}) or {}, label_fallback=key.title())
if transformed:
periods_out[key] = transformed
return {
"updated_at": raw.get("generated_at"),
"title": "Telemetry",
"source": {"label": "Cozystack Telemetry Server"},
"periods": periods_out,
}


def main() -> int:
today = dt.datetime.now(dt.timezone.utc)
year = int(os.environ.get("TELEMETRY_YEAR", today.year))
month = int(os.environ.get("TELEMETRY_MONTH", today.month))
try:
raw = fetch(year, month)
except (urllib.error.URLError, urllib.error.HTTPError, RuntimeError, ValueError) as err:
print(f"fetch failed: {err}", file=sys.stderr)
return 1
payload = build_payload(raw)
if not payload["periods"]:
print("fetched payload has no usable periods; refusing to write empty file", file=sys.stderr)
return 1
os.makedirs(os.path.dirname(OUTPUT_PATH), exist_ok=True)
with open(OUTPUT_PATH, "w", encoding="utf-8") as fh:
json.dump(payload, fh, indent=2, ensure_ascii=False)
fh.write("\n")
print(f"wrote {OUTPUT_PATH} ({len(payload['periods'])} periods, {sum(len(p['apps']) for p in payload['periods'].values())} app rows total)")
return 0


if __name__ == "__main__":
sys.exit(main())
10 changes: 7 additions & 3 deletions hugo.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -206,18 +206,22 @@ menus:
- name: OSS Health
identifier: oss-health
weight: 35
- name: Telemetry
url: /oss-health/telemetry/
parent: oss-health
weight: 1
- name: DevStats
url: /oss-health/devstats/
parent: oss-health
weight: 1
weight: 2
- name: OpenSSF
url: /oss-health/openssf/
parent: oss-health
weight: 2
weight: 3
- name: OSS Insight
url: /oss-health/oss-insight/
parent: oss-health
weight: 3
weight: 4
- name: Enterprise support
url: /support
weight: 5
Expand Down
Loading