Skip to content

Add lb checks - #385

Open
Kiarash-Parvizi wants to merge 7 commits into
motis-project:masterfrom
Kiarash-Parvizi:add-lb-checks
Open

Add lb checks#385
Kiarash-Parvizi wants to merge 7 commits into
motis-project:masterfrom
Kiarash-Parvizi:add-lb-checks

Conversation

@Kiarash-Parvizi

Copy link
Copy Markdown
Contributor

Benchmark

  • Compiler: GCC
  • Dataset: Germany
  • Algorithm: Range Raptor

Response Time

Metric Value
Average 1,850.28
Max 35,210
P99 18,571
P90 5,257
P80 2,073
Median (P50) 366
Min 70

Baseline

Metric Value
Average 1,947.55
Max 38,789
P99 18,211
P90 5,305
P80 2,256
Median (P50) 424
Min 70

Delta vs. Baseline

Metric Change
Average -5.0%
Max -9.2%
P99 +2.0%
P90 -0.9%
P80 -8.1%
Median (P50) -13.7%
Min 0.0%

@felixguendling felixguendling left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dataset: Germany

We need Transitous numbers.

CI is red.

@Kiarash-Parvizi

Copy link
Copy Markdown
Contributor Author

CI is red.

Is this GPU Benchmark error related to my work?

@felixguendling

Copy link
Copy Markdown
Member

It's green on master and has compiler errors on your branch. The PR can't be merged unless this is solved.

@Kiarash-Parvizi

Copy link
Copy Markdown
Contributor Author

Please don't merge this PR for now. there appears to be some mismatch in the results which I must investigate later.

@Kiarash-Parvizi

Copy link
Copy Markdown
Contributor Author

Benchmark 1

  • Compiler: GCC
  • Dataset: Transitous
  • Algorithm: Range Raptor
Metric 4492e7e Baseline (dac9393) Change
Average 3,338.32 3,490.35 -4.4%
Max 110,634 112,123 -1.3%
P99 34,304 34,318 -0.0%
P90 6,260 6,661 -6.0%
P80 3,314 3,412 -2.9%
Median (P50) 1,313 1,365 -3.8%
Min 434 436 -0.5%

Benchmark 2

  • Compiler: GCC
  • Dataset: Transitous
  • Algorithm: Pong
Metric 4492e7e Baseline (dac9393) Change
Average 1,637.34 1,660.24 -1.4%
Max 16,021 15,419 +3.9%
P99 8,136 8,383 -2.9%
P90 2,868 3,115 -7.9%
P80 1,898 1,919 -1.1%
Median (P50) 1,281 1,258 +1.8%

@Kiarash-Parvizi

Copy link
Copy Markdown
Contributor Author

Please don't merge this PR for now. there appears to be some mismatch in the results which I must investigate later.

I was comparing to the wrong baseline commit. It seems to be fine now. Ready for review and merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants