Skip to content

Commit 429bedc

Browse files
feat: score rendered-test interactions by main-thread work to commit (#901)
* feat(stats-generator): score interaction trace by main-thread work to commit * feat: record interaction work to commit alongside the Lighthouse breakdown * Reverted ordering --------- Co-authored-by: Alexander Karan <47707063+AlexanderKaran@users.noreply.github.com>
1 parent 184d5b2 commit 429bedc

31 files changed

Lines changed: 1213 additions & 64 deletions

‎packages/docs/src/benchmarks/lighthouse-client-side-rendered-tests/Description.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
These CSR browser performance tests use Lighthouse to measure paint timings and a controlled navigation interaction.
1+
These CSR browser performance tests use Lighthouse to measure paint timings and a controlled navigation interaction. The interaction is also scored as [work to commit](/methodology/#interaction-work-to-commit) from the same trace.
22

33
Methodology: [Client Side Rendered Tests (Lighthouse)](/methodology/#client-side-rendered-tests-lighthouse).
44

‎packages/docs/src/benchmarks/lighthouse-client-side-rendered-tests/LighthouseFrameworkClientRenderedVersionCharts.astro‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,9 @@ import {
44
prepareLighthouseClientSideRenderedTestsFcpMsHistory,
55
prepareLighthouseClientSideRenderedTestsFirstPaintMsHistory,
66
prepareLighthouseClientSideRenderedTestsInteractionHistory,
7+
prepareClientSideRenderedInteractionWorkHistory,
78
} from '../../lib/prepare-chart'
9+
import { INTERACTION_WORK } from '../../lib/interaction-phases'
810
import VersionLineChart from '../../components/VersionLineChart.astro'
911
import InteractionHistoryChart from '../shared/InteractionHistoryChart.astro'
1012
@@ -16,6 +18,8 @@ const { versions } = Astro.props
1618
1719
const interactionData =
1820
prepareLighthouseClientSideRenderedTestsInteractionHistory(versions)
21+
const interactionWorkData =
22+
prepareClientSideRenderedInteractionWorkHistory(versions)
1923
---
2024

2125
<VersionLineChart
@@ -24,6 +28,15 @@ const interactionData =
2428
<VersionLineChart
2529
{...prepareLighthouseClientSideRenderedTestsFcpMsHistory(versions)}
2630
/>
31+
{
32+
interactionWorkData.length > 0 ? (
33+
<InteractionHistoryChart
34+
title="Client Interaction Work to Commit by Version"
35+
data={interactionWorkData}
36+
metric={INTERACTION_WORK}
37+
/>
38+
) : null
39+
}
2740
{
2841
interactionData.length > 0 ? (
2942
<InteractionHistoryChart
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
1-
These SSR browser performance tests use Lighthouse to measure paint timings and a controlled navigation interaction.
1+
These SSR browser performance tests use Lighthouse to measure paint timings and a controlled navigation interaction. The interaction is also scored as [work to commit](/methodology/#interaction-work-to-commit) from the same trace.
22

33
Methodology: [Server Side Rendered Tests (Lighthouse)](/methodology/#server-side-rendered-tests-lighthouse).

‎packages/docs/src/benchmarks/lighthouse-server-side-rendered-tests/LighthouseFrameworkServerRenderedVersionCharts.astro‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,9 @@ import {
44
prepareLighthouseServerSideRenderedTestsFcpMsHistory,
55
prepareLighthouseServerSideRenderedTestsFirstPaintMsHistory,
66
prepareLighthouseServerSideRenderedTestsInteractionHistory,
7+
prepareServerSideRenderedInteractionWorkHistory,
78
} from '../../lib/prepare-chart'
9+
import { INTERACTION_WORK } from '../../lib/interaction-phases'
810
import VersionLineChart from '../../components/VersionLineChart.astro'
911
import InteractionHistoryChart from '../shared/InteractionHistoryChart.astro'
1012
@@ -16,6 +18,8 @@ const { versions } = Astro.props
1618
1719
const interactionData =
1820
prepareLighthouseServerSideRenderedTestsInteractionHistory(versions)
21+
const interactionWorkData =
22+
prepareServerSideRenderedInteractionWorkHistory(versions)
1923
---
2024

2125
<VersionLineChart
@@ -24,6 +28,15 @@ const interactionData =
2428
<VersionLineChart
2529
{...prepareLighthouseServerSideRenderedTestsFcpMsHistory(versions)}
2630
/>
31+
{
32+
interactionWorkData.length > 0 ? (
33+
<InteractionHistoryChart
34+
title="Server Interaction Work to Commit by Version"
35+
data={interactionWorkData}
36+
metric={INTERACTION_WORK}
37+
/>
38+
) : null
39+
}
2740
{
2841
interactionData.length > 0 ? (
2942
<InteractionHistoryChart

‎packages/docs/src/benchmarks/shared/InteractionBreakdownChart.astro‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,10 @@
22
import { prepareInteractionBreakdownTable } from '../../lib/prepare-table'
33
import StatsTable from '../../components/StatsTable.astro'
44
import ViewSwitcher from '../../components/ViewSwitcher.astro'
5-
import { INTERACTION_PHASES } from '../../lib/interaction-phases'
5+
import {
6+
LIGHTHOUSE_INTERACTION,
7+
type InteractionMetric,
8+
} from '../../lib/interaction-phases'
69
import InteractionChartCaption from './InteractionChartCaption.astro'
710
import StackedBarChart from '../../components/StackedBarChart.astro'
811
@@ -14,10 +17,11 @@ interface InteractionBreakdownDatum {
1417
interface Props {
1518
title: string
1619
data: InteractionBreakdownDatum[]
20+
metric?: InteractionMetric
1721
}
1822
19-
const { title, data } = Astro.props
20-
const series = INTERACTION_PHASES.map(({ key, label, color }) => ({
23+
const { title, data, metric = LIGHTHOUSE_INTERACTION } = Astro.props
24+
const series = metric.parts.map(({ key, label, color }) => ({
2125
key,
2226
label,
2327
color,
@@ -37,7 +41,7 @@ const chart1Table = prepareInteractionBreakdownTable(chart1)
3741
defaultValue="chart"
3842
>
3943
<StackedBarChart slot="chart" {...chart1}>
40-
<InteractionChartCaption slot="caption" />
44+
<InteractionChartCaption slot="caption" {metric} />
4145
</StackedBarChart>
4246
<StatsTable slot="table" {...chart1Table} />
4347
</ViewSwitcher>

‎packages/docs/src/benchmarks/shared/InteractionChartCaption.astro‎

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,23 @@
11
---
2-
import { INTERACTION_PHASES } from '../../lib/interaction-phases'
2+
import {
3+
LIGHTHOUSE_INTERACTION,
4+
type InteractionMetric,
5+
} from '../../lib/interaction-phases'
36
47
interface Props {
58
title?: string
9+
metric?: InteractionMetric
610
}
711
8-
const { title } = Astro.props
12+
const { title, metric = LIGHTHOUSE_INTERACTION } = Astro.props
913
---
1014

1115
<figcaption>
1216
{title && <p class="interaction-chart-title">{title}</p>}
13-
<p>
14-
Total latency is split into the three consecutive phases reported by
15-
Lighthouse.
16-
</p>
17+
<p>{metric.summary}</p>
1718
<ul class="phase-descriptions">
1819
{
19-
INTERACTION_PHASES.map(({ label, description }) => (
20+
metric.parts.map(({ label, description }) => (
2021
<li>
2122
The <strong>{label.toLowerCase()}</strong>, {description}
2223
</li>
@@ -25,7 +26,7 @@ const { title } = Astro.props
2526
</ul>
2627
<ul class="legend" aria-hidden="true">
2728
{
28-
INTERACTION_PHASES.map(({ label, color }) => (
29+
metric.parts.map(({ label, color }) => (
2930
<li>
3031
<span class="swatch" style={`background: ${color}`} />
3132
{label}
Lines changed: 18 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,33 +1,42 @@
11
---
22
import StackedAreaChart from '../../components/StackedAreaChart.astro'
3-
import { INTERACTION_PHASES } from '../../lib/interaction-phases'
4-
import type { InteractionHistoryDatum } from '../../lib/types'
3+
import {
4+
LIGHTHOUSE_INTERACTION,
5+
type InteractionMetric,
6+
} from '../../lib/interaction-phases'
7+
import type {
8+
InteractionHistoryDatum,
9+
InteractionWorkHistoryDatum,
10+
} from '../../lib/types'
511
import InteractionChartCaption from './InteractionChartCaption.astro'
612
713
interface Props {
814
title: string
9-
data: InteractionHistoryDatum[]
15+
data: (InteractionHistoryDatum | InteractionWorkHistoryDatum)[]
16+
metric?: InteractionMetric
1017
}
1118
12-
const { title, data } = Astro.props
19+
const { title, data, metric = LIGHTHOUSE_INTERACTION } = Astro.props
20+
const valueOf = (datum: Props['data'][number], key: string) =>
21+
(datum as unknown as Record<string, number>)[key]!
1322
---
1423

1524
<StackedAreaChart
1625
{title}
1726
data={data.map((datum) => ({
1827
name: datum.name,
19-
total: datum.interactionLatency,
28+
total: valueOf(datum, metric.totalKey),
2029
values: Object.fromEntries(
21-
INTERACTION_PHASES.map(({ key }) => [key, datum[key]]),
30+
metric.parts.map(({ key }) => [key, valueOf(datum, key)]),
2231
),
2332
}))}
24-
series={INTERACTION_PHASES.map(({ key, label, color }) => ({
33+
series={metric.parts.map(({ key, label, color }) => ({
2534
key,
2635
label,
2736
color,
2837
}))}
2938
valueFormat="ms"
30-
yAxisLabel="Interaction latency (ms)"
39+
yAxisLabel={metric.yAxisLabel}
3140
>
32-
<InteractionChartCaption slot="caption" {title} />
41+
<InteractionChartCaption slot="caption" {title} {metric} />
3342
</StackedAreaChart>

‎packages/docs/src/benchmarks/shared/LighthouseRenderedInteractionCharts.astro‎

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,34 @@
11
---
2-
import { getRenderedInteractionBreakdownData } from '../../lib/prepare-chart'
2+
import {
3+
getRenderedInteractionBreakdownData,
4+
getRenderedInteractionWorkBreakdownData,
5+
} from '../../lib/prepare-chart'
6+
import { INTERACTION_WORK } from '../../lib/interaction-phases'
37
import InteractionBreakdownChart from './InteractionBreakdownChart.astro'
48
59
interface Props {
610
testsKey: 'clientSideRendered' | 'serverSideRendered'
711
}
812
913
const { testsKey } = Astro.props
14+
const work = getRenderedInteractionWorkBreakdownData(testsKey).sort(
15+
(a, b) => a.workToCommit - b.workToCommit || a.name.localeCompare(b.name),
16+
)
1017
const data = getRenderedInteractionBreakdownData(testsKey).sort(
1118
(a, b) =>
1219
a.interactionLatency - b.interactionLatency || a.name.localeCompare(b.name),
1320
)
1421
---
1522

23+
{
24+
work.length > 0 ? (
25+
<InteractionBreakdownChart
26+
title="Interaction Work to Commit"
27+
data={work}
28+
metric={INTERACTION_WORK}
29+
/>
30+
) : null
31+
}
1632
{
1733
data.length > 0 ? (
1834
<InteractionBreakdownChart

‎packages/docs/src/content/docs/methodology.md‎

Lines changed: 54 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,8 @@ Test use larger, 16-CPU runners, as described in their sections below.
4444
disables its screen emulation, with no simulated CPU or network throttling.
4545
Requests use a local connection to a production server. The Lighthouse flow
4646
runs only the audits required for the published First Paint, First Contentful
47-
Paint, and interaction-latency measurements.
47+
Paint, and interaction-latency measurements, and records the interaction's
48+
trace for work to commit.
4849
- **Recorded environment:** Results include runner details and the measured
4950
framework version; browser results also include the Chrome version used. Any benchmark that runs multiple itterations record mean, median, minimum, and maximum.
5051

@@ -385,14 +386,18 @@ Current pinned dependencies are listed in each project's `package.json`:
385386
- Interaction latency sums the input delay, processing duration, and
386387
presentation delay from Lighthouse's INP breakdown. It measures this
387388
controlled interaction, not page-lifetime INP.
389+
- The same interaction is also scored as
390+
[work to commit](/methodology/#interaction-work-to-commit), from the trace
391+
Lighthouse records for it. Each comparison ranks frameworks by its own
392+
metric, lowest first.
388393
- CI builds once per framework and test, then keeps one production server running
389394
for all five measurements. Each measurement launches a fresh Chrome process.
390395
- Before all measurements, one HTTP request warms `/client-side-rendered`.
391396
The response is fully consumed with a ten-second timeout. This warms the
392397
server, not the measured browsers, and is excluded from results.
393-
- Valid slow results are retained. Paint values and total interaction latency
394-
must be finite and greater than zero; individual interaction components may
395-
be zero.
398+
- Valid slow results are retained. Paint values, total interaction latency, and
399+
work to commit must be finite and greater than zero; individual interaction
400+
components may be zero.
396401
- Each framework uses its supported production routing and rendering controls.
397402
- Next.js disables SSR for the measured table and detail content.
398403
- Astro uses React client-only islands. Its detail link performs a full-document
@@ -414,15 +419,19 @@ Current pinned dependencies are listed in each project's `package.json`:
414419
- Interaction latency sums the input delay, processing duration, and
415420
presentation delay from Lighthouse's INP breakdown. It measures this
416421
controlled interaction, not page-lifetime INP.
422+
- The same interaction is also scored as
423+
[work to commit](/methodology/#interaction-work-to-commit), from the trace
424+
Lighthouse records for it. Each comparison ranks frameworks by its own
425+
metric, lowest first.
417426
- CI builds once per framework and test, then keeps one production server running
418427
for all five measurements. Each measurement launches a fresh Chrome process.
419428
- Before all measurements, HTTP requests warm `/server-side-rendered` and a
420429
detail URL with a random UUID. Both responses are fully consumed under one
421430
ten-second timeout. This warms the server, not the measured browsers, and is
422431
excluded from results.
423-
- Valid slow results are retained. Paint values and total interaction latency
424-
must be finite and greater than zero; individual interaction components may
425-
be zero.
432+
- Valid slow results are retained. Paint values, total interaction latency, and
433+
work to commit must be finite and greater than zero; individual interaction
434+
components may be zero.
426435
- Links use each framework's production navigation component or ordinary
427436
anchors, allowing default prefetching. The table is loaded and rendered on
428437
demand without caching or prerendering its output. Static surrounding shells
@@ -432,6 +441,44 @@ Current pinned dependencies are listed in each project's `package.json`:
432441
- Astro's table and detail routes use `export const prerender = false` for
433442
request-time rendering.
434443

444+
### Interaction Work to Commit
445+
446+
The client and server side rendered tests also score the benchmark's click from
447+
the Chrome trace that Lighthouse records for the interaction step.
448+
449+
- Work to commit is the sum of three parts:
450+
- **Input delay:** from the input's timestamp to the start of the first event
451+
callback for the interaction.
452+
- **Processing duration:** from the start of the first event callback to the
453+
end of the last, across the interaction's pointer and click events.
454+
- **Rendering work:** time the renderer main thread is busy, from the end of
455+
processing until the interaction's frame commits. This includes framework
456+
work scheduled by the event, style, layout, and paint. Overlapping tasks are
457+
counted once.
458+
- Input delay and processing duration match Lighthouse's INP breakdown for
459+
client-routed navigations. Rendering work takes the place of presentation
460+
delay.
461+
- Idle main-thread time while waiting for the next frame is excluded. That wait
462+
depends on where the click lands within the frame interval rather than on the
463+
framework.
464+
- Work after commit is not measured. This includes raster, draw, presentation,
465+
and browser UI frames, such as the frame software-rendered headless Chrome
466+
waits for after `pushState`
467+
([#573](https://github.com/e18e/framework-performance/issues/573)).
468+
- The commit boundary is the Event Timing `commitFinishTime` for the
469+
interaction, capped at the end of its Event Timing duration. A client-routed
470+
run without a `commitFinishTime` fails the benchmark.
471+
- Tables and breakdown charts show the run with the median work to commit, so
472+
its parts add up to the total. Results also record the mean, median, minimum,
473+
and maximum of each part.
474+
- For full-document navigations (Astro), the document usually unloads before
475+
it presents a frame. The Event Timing duration then ends with processing and
476+
rendering work is zero. Astro's score stops at unload and does not include
477+
loading or rendering the detail page, so it is not directly comparable with
478+
the client-routed frameworks.
479+
- Results recorded before work to commit was measured only have the Lighthouse
480+
breakdown, so its version history starts later.
481+
435482
### Initial Load
436483

437484
- Each production app serves the same small, styled landing page at

0 commit comments

Comments
 (0)