Skip to content

[consensus] Fix proposer_delay_proposal histogram buckets #19474

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
danielxiangzl merged 3 commits into from
Apr 20, 2026
+5 −4
Merged

[consensus] Fix proposer_delay_proposal histogram buckets #19474

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2c62345
[consensus] Fix proposer_delay_proposal histogram buckets
danielxiangzl Apr 16, 2026
48efa5f
[consensus] Rename metric to aptos_proposer_delay_proposal_seconds
danielxiangzl Apr 20, 2026
78e531a
Merge branch 'main' into daniel/fix-proposer-delay-histogram
danielxiangzl Apr 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions consensus/src/counters.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -405,12 +405,13 @@ pub static CONSENSUS_PROPOSAL_PENDING_DURATION: Lazy<DurationHistogram> = Lazy::
)
});

/// Amount of time (in seconds) proposal is delayed due to backpressure/backoff
pub static PROPOSER_DELAY_PROPOSAL: Lazy<Histogram> = Lazy::new(|| {
register_avg_counter(
"aptos_proposer_delay_proposal",
register_histogram!(
Copy link
Copy Markdown
Contributor

@ibalajiarun ibalajiarun Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest creating a new metric altogether otherwise rollout will be a mess.

Copy link
Copy Markdown
Contributor Author

@danielxiangzl danielxiangzl Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

"aptos_proposer_delay_proposal_seconds",
Copy link
Copy Markdown
Contributor

@cursor cursor Bot Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Metric name changed despite PR claiming it's unchanged

High Severity

The metric name changed from "aptos_proposer_delay_proposal" to "aptos_proposer_delay_proposal_seconds", contradicting the PR description's claim that "Metric name is unchanged." This is a breaking change: any existing external dashboards, alerts, or Prometheus queries referencing aptos_proposer_delay_proposal (including _sum, _count, _bucket variants) will silently stop receiving data. During rolling deployments, nodes will emit different metric names, fragmenting aggregation queries.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 48efa5f. Configure here.

"Amount of time (in seconds) proposal is delayed due to backpressure/backoff",
vec![0.001, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0]
)
.unwrap()
});

/// Histogram for max number of transactions (after filtering for dedup, expirations, etc) proposer uses when creating block.
Expand Down
2 changes: 1 addition & 1 deletion dashboards/end-to-end-txn-latency.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -705,7 +705,7 @@
{
"datasource": { "type": "prometheus", "uid": "${Datasource}" },
"editorMode": "code",
"expr": "aptos_proposer_delay_proposal{chain_name=~\"$chain_name\", cluster=~\"$cluster\", metrics_source=~\"$metrics_source\", namespace=~\"$namespace\", kubernetes_pod_name=~\"$kubernetes_pod_name\", role=~\"$role\"} > 0",
"expr": "aptos_proposer_delay_proposal_seconds{chain_name=~\"$chain_name\", cluster=~\"$cluster\", metrics_source=~\"$metrics_source\", namespace=~\"$namespace\", kubernetes_pod_name=~\"$kubernetes_pod_name\", role=~\"$role\"} > 0",
"format": "time_series",
"intervalFactor": 1,
"legendFormat": "__auto",
Expand Down
Loading