Fix VRAM leak when monitors are off

[phuongdpham]

Per discussion in #3295: with monitors off, several paths kept driving the redraw loop and leaking memory.

This patch breaks the loop by:

• Scheduling the same estimated-vblank throttle Tty::render schedules on NoDamage, so commit-driven queue_redraw doesn't busy-loop.
• Skipping send_frame_callbacks while monitors_active == false - we never present, so inviting clients to commit just queues buffers that go nowhere.
• Skipping the 1s fallback frame-callback timer while monitors are off - ticking smithay's throttle bookkeeping serves no purpose when nothing is being presented.
• Clearing unfinished_animations_remain in the inactive-monitors branch so the estimated-vblank timer doesn't keep re-queuing redraws at refresh-rate cadence.

The throttle helpers moved from Tty to impl Niri so the inactive-monitors call site doesn't need a backend-type check.

Credit to @sys-rq for empirical testing on i915 + Lenovo T490 that surfaced the three additional drivers; the original patch only addressed the per-commit busy-loop.

Testing

• cargo check / cargo clippy --all --all-targets / cargo +nightly fmt --all -- --check clean.
• cargo test --all --exclude niri-visual-tests - 217 tests pass.
• I'm on NVIDIA RTX 3060 Ti and can't reproduce locally. @sys-rq has reproduced + tested the partial patch; would appreciate a re-test of this updated version.

Refs #3295.

[YaLTeR]

well this clearly looks kinda cursed with that check and pub(crate)

[phuongdpham]

Refactored: both functions moved to impl Niri, no more pub(crate) or Backend::Tty check.

Heads up: with the throttle now backend-agnostic, monitors_active=false can leave Winit's redraw_state at WaitingForEstimatedVBlank{,AndQueued}, which Winit::render still marks unreachable!() (winit.rs:289-290). Not reachable in practice (no screencast on Winit), but flagging since the precondition shifted.

[YaLTeR]

I'm gonna have to take a good look at this and at surrounding code to verify that it's right. Sometime in the future

[phuongdpham]

Thank you.

[bczhc]

This also fixes #3691. Though in that discussion the issue is RAM climbing but not VRAM, which is considered a bit different symptom to issue #3295. But anyway this also fixes 3691!

[sys-rq]

Tested on lenovo t490/i915 + arch + niri-git + 3910. Still observing the leak leading to eventual overnight OOMs. First time observed on Dec 12 (though it might be that I wasn't updating the system for up to 2 weeks at the time, take it as an approx date). Powering off monitors frequently (Mod+Shift+P { power-off-monitors; }). Tried to downgrade to the oldest available locally cached version (25.08-2), did not helped, free -h reports +2Gi increase in shared every 15 mins with monitors off.

[sys-rq]

What helped:
• prevented send_frame_callbacks() while monitors_active == false
• disabled the fallback frame callback timer while monitors are off.
• clearing unfinished_animations_remain in the monitors-off redraw branch so the estimated-vblank timer doesn’t keep scheduling animation redraw

[phuongdpham]

@sys-rq Thanks for the testing and the diagnosis - all three paths you identified check out against the code, and they explain why my original patch didn't close the leak (it only stopped the per-commit busy-loop, not the timer-driven re-queuing).

Folded all three into this PR (force-pushed, see updated description):

• Skip send_frame_callbacks while monitors_active == false
• Skip the 1s fallback frame-callback timer while monitors are off
• Clear unfinished_animations_remain in the inactive-monitors branch

If you have time, a re-test on the new revision would be much appreciated.

[as3ii]

Per my own tests with AMD iGPU, commit 3cfd6b2 fixed the VRAM leak related to monitor powered off. In the next hours/days I'll test the latest commit

[phuongdpham]

Per my own tests with AMD iGPU, commit 3cfd6b2 fixed the VRAM leak related to monitor powered off. In the next hours/days I'll test the latest commit

Thank you @as3ii.

[sys-rq]

looks promising, tested for 1h with monitor off and, just in case, locked but turned on, free -h is not showing increase in shared

[phuongdpham]

Thanks for re-testing - good to have confirmation on the hardware where the leak was reproducible. Appreciate the legwork on the original diagnosis too.