Don't manipulate time in oximeter database usage tests#10405
Conversation
- The database usage tests paused and advanced time to ensure we trigger another query against the DB to compute usage. This interacts poorly with a timeout in the connection type for talking to ClickHouse. Tokio's test-utils allow pausing time, but when there is literally no other work to do, the runtime "auto-advances" time to the next pending timer. That happens to be our connection timeout, and so the tests fail. Instead, just lower the polling interval for this one test and live with it taking a few seconds. - Fix #10398
hawkw
left a comment
There was a problem hiding this comment.
Overall, this seems like an unfortunate but reasonable solution. I don't think any of us like writing tests that will always spend multiple seconds waiting for stuff, but I don't think that this test is going to be the longest-running test and we run them in parallel, so it's probably not the absolute worst thing.
There is an alternative solution 13 might consider, though: the Tokio docs suggest that a spawn_blocking task can be used to temporarily suppress the auto-advance behavior with paused time. I think that we could potentially do something like spawning a blocking task that waits on a blocking channel that we don't unblock until the test has completed, or something. Then, we could continue using paused time. On the other hand, this feels like a bit of a hack to me, and we might be better off just avoiding the paused timer entirely. Ultimately, it's up to you whether you'd rather try that out or continue with the current approach.
omicron-clickhouse-admin::context::tests::test_database_usage#10398