Main graph v2 backend

[RobertJoonas]

Changes

This PR gets rid of the/main-graph endpoint in favor of the new API v2 based /query endpoint. Doing so, a few gaps needed to be filled in the query API. See below.

NOTE: This PR will be kept as draft until #6159 (new main graph FE) gets merged into it.

1. Change the result format of comparison results (when time dimension breakdown)

With this PR, the query API starts returning comparison_results as a separate response field if dimensions == [time:*], while breakdown and aggregate responses keep comparisons nested for each row in the original results list.

In hindsight, the new comparison results format could have been avoided by filling in all the missing buckets instead. This would have let us avoid returning a bunch of meta fields (time_labels, comparison_time_labels, time_label_indices, comparison_time_label_indices, and empty_metrics).

The main reason I went with this approach was to prepare comparisons for a public API release (playing well with time_labels), but since it’s still far from perfect (a noisy query.include + meta; difficult for the client to read the data), we’ll still need to re-evaluate this. We should also carefully consider what to do with include.total_rows and multi-dimensional (time + non-time) breakdowns, where zipping comparison <> main results gets tricky.

2. Add new response.meta and query.include fields

Note: all these flags are internal-use only. Nothing gets released into the public API

• meta.comparison_time_labels
  • same as meta.time_labels but for generating the comparison plot
  • added when include.time_labels && include.compare
• meta.present_index
  • index of the time label that is currently ongoing
  • added when include.present_index
  • NOTE: with the recent decision to change the dotted line meaning to "any incomplete bucket", this might not be necessary. However, I would keep this around until we're certain that the new definition of the dotted line is OK.
• meta.time_label_result_indices
  • a list (of the same length as meta.time_labels) of indices where each item corresponds to an index in the results list or null when there is no result for that bucket. Makes it much easier for the frontend to understand the data.
  • added when include.time_label_result_indices
• meta.comparison_time_label_result_indices
  • same as meta.time_label_result_indices but for comparison results
  • added when include.time_label_result_indices && include.compare
• meta.partial_time_labels
  • a list of length 0, 1, or 2 where items correspond to labels of time buckets that are incomplete. Only the first and last can be incomplete, which is why the max length is 2.
  • added when include.partial_time_labels
• meta.comparison_partial_time_labels
  • same as meta.partial_time_labels but for comparison results
  • added when include.partial_time_labels && include.compare
• meta.empty_metrics
  • a list representing an "empty value" for every queried metric (in the same order). E.g. when query.metrics = ['visitors', 'time_on_page'], then meta.empty_metrics = [0, null]. Makes it possible for the frontend to fill in missing buckets without re-defining empty metrics.
  • added when include.empty_metrics && "event:goal" not in query.dimensions - there shouldn't be any need to ask for empty metrics in the goal breakdown, but this makes it clear that there's a conflict (possibly multiple revenue currencies).

3. Remove old code

• Delete the main-graph endpoint
• Remove comparisons logic from legacy timeseries (see commit with description 7ad120a)

4. Transform main_graph_test.exs

A major rewrite of main_graph_test.exs - populate_stats and other tests setup remained the same, but the request is now made against the new endpoint and the assertions on the returned data structures have changed significantly.

Tests

• Automated tests have been added

Changelog

• Entry has been added to changelog

Documentation

• This change does not need a documentation update

Dark mode

• The UI has been tested both in dark and light mode