thresholds.adoc

title	Load Testing Thresholds
page-title	Setting Thresholds for Load Tests
description	How to set and control failure thresholds for the load tests
meta-description	Learn how to set and control failure thresholds for the load tests
order	60
version	since:com.vaadin:vaadin@V25.2

Load Testing Thresholds

Overview

k6 thresholds define pass/fail criteria for your load tests.

When a threshold is breached, k6 marks the test as failed, and the Maven build fails if k6.failOnThreshold=true (the default).

The TestBench k6 converter plugin generates some default thresholds automatically in every k6 script produced. You can configure the default thresholds, disable individual ones, or add custom thresholds for any k6 metric.

Default Thresholds

Every generated k6 script includes these thresholds out of the box:

Metric	Expression	Default
http_req_duration	95th percentile response time	p(95)<2000 (2 seconds)
http_req_duration	99th percentile response time	p(99)<5000 (5 seconds)
checks	All checks must pass	rate==1 with abortOnFail: true

This generates the following block in the k6 script:

export const options = {
  thresholds: {
    checks: [{ threshold: 'rate==1', abortOnFail: true, delayAbortEval: '5s' }],
    http_req_duration: ['p(95)<2000', 'p(99)<5000'],
  },
}

Editing Default Thresholds

You can change the default threshold values with Maven properties or through configuration in the POM.

Via Command Line

# Tighter response time requirements
mvn k6:convert -Dk6.harFile=recording.har \
    -Dk6.threshold.httpReqDurationP95=1000 \
    -Dk6.threshold.httpReqDurationP99=3000

# Allow checks to fail without aborting the test
mvn k6:convert -Dk6.harFile=recording.har \
    -Dk6.threshold.checksAbortOnFail=false

# Disable the p95 threshold entirely (set to 0)
mvn k6:convert -Dk6.harFile=recording.har \
    -Dk6.threshold.httpReqDurationP95=0

Via POM Configuration

<plugin>
    <groupId>com.vaadin</groupId>
    <artifactId>testbench-converter-plugin</artifactId>
    <configuration>
        <!-- 95th percentile response time in ms (0 to disable) -->
        <httpReqDurationP95>1000</httpReqDurationP95>
        <!-- 99th percentile response time in ms (0 to disable) -->
        <httpReqDurationP99>3000</httpReqDurationP99>
        <!-- Set to false to continue the test even when checks fail -->
        <checksAbortOnFail>false</checksAbortOnFail>
    </configuration>
</plugin>

Default Threshold Parameters

Parameter	Default	Description
k6.threshold.httpReqDurationP95	2000	95th percentile response time threshold in ms. Set to 0 to disable.
k6.threshold.httpReqDurationP99	5000	99th percentile response time threshold in ms. Set to 0 to disable.
k6.threshold.checksAbortOnFail	true	When true, the test aborts immediately on the first check failure. When false, failures are recorded but the test continues.

Adding Custom Thresholds

You can add pass/fail thresholds for any k6 built-in metric using the k6.threshold.custom property.

The format is a comma-separated list of metric:expression pairs:

metric1:expression1,metric2:expression2,...

Via Command Line

# Fail if more than 1% of requests fail
mvn k6:run -Dk6.testFile=test.js \
    -Dk6.threshold.custom="http_req_failed:rate<0.01"

# Multiple custom thresholds
mvn k6:run -Dk6.testFile=test.js \
    -Dk6.threshold.custom="http_req_failed:rate<0.01,http_req_waiting:p(95)<500,http_reqs:count>100"

Via POM Configuration

<plugin>
    <groupId>com.vaadin</groupId>
    <artifactId>testbench-converter-plugin</artifactId>
    <configuration>
        <!-- Custom thresholds in metric:expression format -->
        <customThresholds>http_req_failed:rate&lt;0.01,http_req_waiting:p(95)&lt;500</customThresholds>
    </configuration>
</plugin>

Note	In XML, use < instead of < inside threshold expressions.

Combining Defaults and Custom Thresholds

Custom thresholds are added alongside the defaults. If you add a custom http_req_duration expression, it is merged with the default p95/p99 thresholds:

# Adds a median threshold alongside the existing p95 and p99
mvn k6:convert -Dk6.harFile=recording.har \
    -Dk6.threshold.custom="http_req_duration:p(50)<1000"

This produces:

export const options = {
  thresholds: {
    checks: [{ threshold: 'rate==1', abortOnFail: true, delayAbortEval: '5s' }],
    http_req_duration: ['p(95)<2000', 'p(99)<5000', 'p(50)<1000'],
  },
}

Common Threshold Examples

Below are commonly used threshold configurations for Vaadin load tests.

Strict Performance Requirements

mvn k6:run -Dk6.testFile=test.js \
    -Dk6.threshold.httpReqDurationP95=500 \
    -Dk6.threshold.httpReqDurationP99=1000 \
    -Dk6.threshold.custom="http_req_failed:rate<0.001,http_req_waiting:p(95)<300"

Generated output:

thresholds: {
  checks: [{ threshold: 'rate==1', abortOnFail: true, delayAbortEval: '5s' }],
  http_req_duration: ['p(95)<500', 'p(99)<1000'],
  http_req_failed: ['rate<0.001'],
  http_req_waiting: ['p(95)<300'],
},

Relaxed Thresholds for Stress Testing

mvn k6:run -Dk6.testFile=test.js \
    -Dk6.threshold.httpReqDurationP95=10000 \
    -Dk6.threshold.httpReqDurationP99=0 \
    -Dk6.threshold.checksAbortOnFail=false \
    -Dk6.threshold.custom="http_req_failed:rate<0.05"

Generated output:

thresholds: {
  checks: ['rate==1'],
  http_req_duration: ['p(95)<10000'],
  http_req_failed: ['rate<0.05'],
},

Throughput Minimum

mvn k6:run -Dk6.testFile=test.js \
    -Dk6.threshold.custom="http_reqs:count>1000"

k6 Metric Reference

The following k6 built-in metrics can be used with custom thresholds. For the full reference, see the k6 metrics documentation.

Metric	Type	Description
http_req_duration	Trend	Total time for the request (send + wait + receive)
http_req_waiting	Trend	Time spent waiting for response (Time to First Byte)
http_req_connecting	Trend	Time spent establishing TCP connection
http_req_sending	Trend	Time spent sending data
http_req_receiving	Trend	Time spent receiving data
http_req_failed	Rate	Rate of failed requests
http_reqs	Counter	Total number of requests
checks	Rate	Rate of successful checks

Trend metrics support: p(N)<value, avg<value, min<value, max<value, med<value
Rate metrics support: rate<value, rate>value
Counter metrics support: count<value, count>value