Skip to content
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,50 @@ Client source code:

Read more about how telemetry is collected from docs:
- https://cozystack.io/docs/telemetry/

## Public Statistics API

The server exposes a `GET /api/overview` endpoint that returns aggregated usage statistics in JSON format. This data is used by the [Cozystack website](https://cozystack.io/) to display public telemetry on the [Telemetry page](https://cozystack.io/oss-health/telemetry/).

### How it works

1. On the **1st of each month at 00:01 Pacific Time**, the server queries VictoriaMetrics for the current state of all reporting clusters and stores a monthly snapshot to disk.
2. On first startup (if no snapshot exists for the current month), an initial snapshot is collected automatically.
3. The app list is fetched dynamically from [cozystack/cozystack packages/apps](https://github.com/cozystack/cozystack/tree/main/packages/apps) to ensure newly added applications are always included.
4. The `/api/overview` endpoint aggregates stored snapshots into three time periods: **last month**, **last quarter** (3 months), and **last 12 months**.

### Response format

```json
{
"generated_at": "2026-04-01T07:01:00Z",
"periods": {
"month": {
"label": "March 2026",
"start": "2026-03-01",
"end": "2026-03-31",
"clusters": 42,
"total_nodes": 210,
"avg_nodes_per_cluster": 5.0,
"total_tenants": 84,
"avg_tenants_per_cluster": 2.0,
"apps": {
"postgres": 120,
"redis": 85,
"kubernetes": 30
}
},
"quarter": { "..." : "averaged over 3 months" },
"year": { "..." : "averaged over 12 months" }
}
}
```

### Configuration flags

| Flag | Default | Description |
|------|---------|-------------|
| `--forward-url` | `http://vminsert-cozy-telemetry:8480/insert/0/prometheus/api/v1/import/prometheus` | URL to forward ingested metrics to |
| `--listen-addr` | `:8081` | Address to listen on |
| `--vmselect-url` | `http://vmselect-cozy-telemetry:8481` | VictoriaMetrics vmselect base URL for queries |
| `--snapshot-dir` | `/data/snapshots` | Directory to store monthly snapshot JSON files |
13 changes: 12 additions & 1 deletion charts/cozy-telemetry/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@ metadata:
labels:
app: cozy-telemetry
spec:
replicas: {{ .Values.replicaCount }}
replicas: 1
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This and PVC will make telepetry-server unable to run in multiple replicas, if you want to cache the state, it might be worth of using emptyDir: {} storage.

strategy:
type: Recreate
selector:
matchLabels:
app: cozy-telemetry
Expand All @@ -21,10 +23,15 @@ spec:
args:
- "--forward-url={{ .Values.config.forwardURL }}"
- "--listen-addr={{ .Values.config.listenAddr }}"
- "--vmselect-url={{ .Values.config.vmSelectURL }}"
- "--snapshot-dir={{ .Values.config.snapshotDir }}"
ports:
- containerPort: {{ .Values.service.port }}
resources:
{{- toYaml .Values.resources | nindent 10 }}
volumeMounts:
- name: snapshots
mountPath: /data/snapshots
readinessProbe:
tcpSocket:
port: 8081
Expand All @@ -35,3 +42,7 @@ spec:
port: 8081
initialDelaySeconds: 15
periodSeconds: 10
volumes:
- name: snapshots
persistentVolumeClaim:
claimName: cozy-telemetry-snapshots
12 changes: 12 additions & 0 deletions charts/cozy-telemetry/templates/pvc.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: cozy-telemetry-snapshots
labels:
app: cozy-telemetry
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: {{ .Values.snapshot.storage }}
5 changes: 5 additions & 0 deletions charts/cozy-telemetry/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ service:
config:
forwardURL: "http://vminsert-cozy-telemetry:8480/insert/0/prometheus/api/v1/import/prometheus"
listenAddr: ":8081"
vmSelectURL: "http://vmselect-cozy-telemetry:8481"
snapshotDir: "/data/snapshots"

resources:
requests:
Expand All @@ -22,6 +24,9 @@ resources:
cpu: 500m
memory: 512Mi

snapshot:
storage: 1Gi

# Ingress spec
ingress:
enabled: true
Expand Down
9 changes: 9 additions & 0 deletions main.go
Original file line number Diff line number Diff line change
Expand Up @@ -149,20 +149,29 @@ func main() {
// Define flags
forwardURL := flag.String("forward-url", "http://vminsert-cozy-telemetry:8480/insert/0/prometheus/api/v1/import/prometheus", "URL to forward the metrics to")
listenAddr := flag.String("listen-addr", ":8081", "Address to listen on for incoming metrics")
vmSelectURL := flag.String("vmselect-url", "http://vmselect-cozy-telemetry:8481", "VictoriaMetrics vmselect base URL for queries")
snapshotDir := flag.String("snapshot-dir", "/data/snapshots", "Directory to store monthly snapshots")
flag.Parse()

// Initialize overview manager
overview := NewOverviewManager(*vmSelectURL, *snapshotDir)
overview.Start()

server := &http.Server{
Addr: *listenAddr,
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
}

http.HandleFunc("/api/overview", overview.HandleOverview)
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
handleTelemetry(w, r, *forwardURL)
})

log.Printf("Starting server on %s", *listenAddr)
log.Printf("Forwarding metrics to %s", *forwardURL)
log.Printf("VictoriaMetrics select URL: %s", *vmSelectURL)
log.Printf("Snapshot directory: %s", *snapshotDir)

if err := server.ListenAndServe(); err != nil {
log.Fatalf("Server error: %v", err)
Expand Down
Loading