From 9a4b95c46277383045ea6ecdb4ee590c4dfa0023 Mon Sep 17 00:00:00 2001 From: Erik Darling <2136037+erikdarlingdata@users.noreply.github.com> Date: Mon, 13 Jul 2026 21:10:23 -0400 Subject: [PATCH 1/2] Add 3 Tier-2 issue-classification arms to the shared FactScorer Restore three classifications the deprecated Dashboard's report.* / config.critical_issues SQL made but the ported shared analysis engine had dropped. All three are pure FactScorer additions over facts the collectors already emit (no collector/schema/install change), so they light up in Lite, the Dashboard, and Darling recommendations at once; each cites its source SQL line in a code comment. - Query Store disabled: INFO advisory folded into DB_CONFIG, detected as query_store_on_count < database_count (install/50:83). - Security-cache (TokenAndPermUserStore) growth: flat WARNING when the USERSTORE_TOKENPERM clerk >= 1 GB, off the context-only MEMORY_CLERKS fact (install/50:562,583). - Runnable-queue / worker pressure: standalone ScoreCpuFact arm on total_runnable_tasks_count tiered > 50/20/10 plus the small-box runnable_tasks_warning fallback (install/47:1839-1844); additive to and distinct from the #1494 THREADPOOL amplifier, which is unchanged. Advice wired for the new/expanded root keys (MEMORY_CLERKS + RUNNABLE_TASKS blocks; Query Store folded into ComposeDbConfig) so the cards are not contentless, sourced from the Dashboard's own message/recommendation text. FactScorer tests added in both Lite.Tests and Dashboard.Tests (tiers fire at the right band, stay silent below the bar, existing arms unchanged). The Lite clean-server seeder now defaults Query Store ON (the healthy value) so the QS-off arm stays dormant in fixtures not exercising it. Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 2 + Dashboard.Tests/FactScorerTests.cs | 173 ++++++++++++++++++++++ Lite.Tests/FactScorerTests.cs | 173 ++++++++++++++++++++++ Lite.Tests/TestDataSeeder.cs | 6 +- PerformanceMonitor.Analysis/FactAdvice.cs | 28 ++++ PerformanceMonitor.Analysis/FactScorer.cs | 58 +++++++- 6 files changed, 437 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0fab464d..56071947 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- **Analysis engine: three more issue-classification arms in the shared FactScorer (Tier-2 parity)** ([#PRNUM]) — the shared `PerformanceMonitor.Analysis` scorer/inference brain (ONE code path for Lite, the deprecated Dashboard, AND Darling) now classifies three problems the deprecated Dashboard's `report.*` / `config.critical_issues` SQL flagged but the ported engine had dropped. All three are pure `FactScorer` additions over facts the collectors ALREADY emit — no collector, schema, or `install/*.sql` change — so they light up in all three apps' recommendations at once, each citing its source SQL line in a code comment. (1) **Query Store disabled** — an INFO advisory folded into the existing `DB_CONFIG` finding, detected as `query_store_on_count < database_count` (user DBs only, system DBs excluded on both counts), mirroring `install/50_configuration_issues_analyzer.sql`'s INFO Query-Store-off issue (line 83); the DB-config recommendation now lists "N with Query Store off" and appends enable-QS guidance. (2) **Security-cache (TokenAndPermUserStore) growth** — a flat WARNING when the `USERSTORE_TOKENPERM` memory clerk reaches >= 1 GB, read from the otherwise context-only `MEMORY_CLERKS` fact (the clerk size rides in metadata keyed by clerk_type), mirroring `install/50`'s WARNING at `pages_kb / 1024 / 1024 >= 1.0` (lines 562, 583); a new recommendation names the `DBCC FREESYSTEMCACHE('TokenAndPermUserStore')` relief plus the ad-hoc / dynamic-SQL churn cause. (3) **Runnable-queue / worker pressure** — a STANDALONE arm rooting the collected `cpu_scheduler_stats` snapshot on `total_runnable_tasks_count`, tiered to `report.cpu_scheduler_pressure`'s `> 50 / > 20 / > 10` CASE plus the collector's `runnable_tasks_warning` small-box fallback (`install/47_create_reporting_views.sql` lines 1839-1844), additive to and distinct from the #1494 runnable-queue -> THREADPOOL amplifier (kept unchanged). Base severity maxes at 1.0 (WARNING) like every base fact — the CRITICAL band stays reserved for amplifier-corroborated cases (the runnable-queue -> THREADPOOL escalation). `FactScorer` tests in BOTH `Lite.Tests` and `Dashboard.Tests` pin every tier (fires when it should, stays silent below the bar, at the right band) and that a co-present worse `DB_CONFIG` issue is left unchanged. Behavior of the existing arms is unchanged. - **Darling viewer: live Current Active Queries sub-tab (service-mediated)** ([#1515]) — a LIVE, on-demand DMV snapshot of what is running on a monitored server RIGHT NOW, distinct from the stored **Active Queries** sub-tab (the collector's hourly snapshots). Darling's viewer is store-only, so it is delivered through the Stage-2 command plane exactly like `execute_actual_plan` / `fetch_plan`: the viewer enqueues a new `fetch_active_queries` command, the SERVICE (which holds a live connection to every monitored server) runs the query on the target's live connection and returns the rows in `result_json`, and the viewer renders them in a new **Current Active Queries** sub-tab (Queries sub-index 2, right after the stored Active Queries tab — every routing reference uses the named constant, so only the constant values shift) with a **Refresh** button (ON-DEMAND only — never auto-fetched on tab selection or a range refresh, since a live server hit is an explicit action). The DMV query is REUSED, not reinvented: a new read-only `DarlingCollectorRunner.FetchRowsAsync` (the "fetch phase only" twin of `RunAsync` — same `CollectorContext`, no store write) runs the shared `QuerySnapshotsCollector` against the live connection, so the live snapshot carries the SAME columns as the stored one and renders in the same `ViewerQuerySnapshotRow` grid (the Estimated / Actual plan buttons reuse the stored handlers on the plan the DMV read carried inline). New `CommandKind.FetchActiveQueries` is pinned viewer↔executor (`ViewerDataService.CommandFetchActiveQueries` ↔ `DarlingCommandExecutor.ResolvePlan`), requires `target_server_id` and takes no args, dispatches through `IDarlingCommandHost.FetchActiveQueriesLiveAsync` → `DarlingWorker.RunFetchActiveQueriesLiveAsync` (resolves the runtime under the servers lock, mirroring `RunFetchPlanAsync`), and returns legible `server not monitored` / `server not connected` / `timed out` / `permission denied` / `sql error` outcomes; the service caps the SQL read at 30s (comfortably under the viewer's 60s poll budget so a slow read reports a real outcome, not a poll miss), deletes the (large) command row after the viewer reads it, and — because enqueuing is a config write — a read-only `viewer` seat cannot fetch it (same rule as Pause / `fetch_plan`). It is a read-only DMV read (`sys.dm_exec_requests`/`sessions` + the sql_text / query_plan DMFs), so unlike `execute_actual_plan` it is NOT consent-class. Tests pin the command constant, the `ResolvePlan` case, and the FULL service-serialize → viewer-parse wire contract (no live SQL Server needed), plus the timeout / failure mappings and the poll-budget-vs-SQL-cap ordering; the live executor round-trip is gated on `DARLING_TEST_PG` like the `fetch_plan` one. Darling-only — no Lite, Full Dashboard, or `install/*.sql` changes. - **Lite viewer: Blocking Stats sub-tab (Tier-1 parity BUILD)** ([#1514]) — the Lite port of the Darling viewer's **Blocking Stats** sub-tab: a 2x2 chart grid + a window summary strip showing blocking AND deadlock SEVERITY over the window, appended as the LAST sub-tab of the **Blocking** tab (sub-index 4, after Deadlocks — nothing else reindexed; `RefreshBlockingAsync`'s subTabOnly switch gains `case 4` while the top-level Blocking tab stays index 8). It is the severity companion to Lite's existing Blocking **Trends** sub-tab (which plots blocking/deadlock COUNTS): the four charts plot block Max/Avg duration + block Total duration, then deadlock Max/Avg wait + deadlock Total wait (Total split onto its own axis per pair so the aggregate magnitude doesn't swamp the per-incident one), and the summary strip carries events / total / max / avg block duration, the deadlock count, and the deadlock victim count + total deadlock wait — the Lite equivalent of the Dashboard's pre-aggregated `collect.blocking_deadlock_stats` (`total_blocking_duration_ms` / `victim_count` / `total_deadlock_wait_time_ms`), computed ON-THE-FLY from rows the store already keeps, so no collector or migration is needed. The reader (`LocalDataService.BlockingStats.cs`) adapts Darling's `GetBlockingDurationStatsAsync` + `GetDeadlockSeverityStatsAsync` to DuckDB: the block-duration aggregate SUM/MAX/AVGs `wait_time_ms` per minute over the SAME XE-preferred (`v_blocked_process_reports`) + DMV-fallback (`v_dmv_blocking_snapshots`, `WHERE NOT EXISTS`) union the blocking-incident count trend uses — so the severity chart and the count chart never disagree on which rows exist — and honors the #1319 database filter on both arms; the deadlock-severity aggregate reads the raw `deadlock_graph_xml` over the SAME `v_deadlocks` / `collection_time` window as the deadlock count (uncapped, so it can't drop rows the count keeps), parses each graph OFF the UI thread with Lite's own `DeadlockProcessDetail.ParseFromRows` (the SAME parse the Deadlocks grid uses, so the two agree in-app), and buckets the per-process victim/wait rows by minute (`victim_count` = SUM of victims, total/max/avg over every process's wait). **Collapse check**: Darling's deadlock-severity computation is NOT a viewer↔service twin — the service/MCP side hosts only the count trends and deliberately omits a `get_blocking_deadlock_stats` (no raw victim-wait source), and the Darling viewer's aggregate already parses via the shared `PerformanceMonitor.Common.DeadlockGraphParser` — so per the brief Lite reuses its own existing `ParseFromRows` (keeping Lite's severity internally consistent with its Deadlocks grid) rather than introducing a third copy; nothing on the Darling side was touched. The four charts (`ServerTab.BlockingStats.cs`) mirror Darling's rendering with Lite's chart idioms (shared `ChartStyle` / `ChartHoverHelper`, `SeriesColors` / `ChartPalette` identities, `UtcOffsetMinutes` display conversion, Y-floor-at-0, window-pinned `SetLimitsX`, connected-scatter padded to the window ends), wired into the Blocking `TabControl` + `RefreshBlockingAsync` (both the subTabOnly `case 4` and the full-refresh path) + `ServerTab.xaml.cs` init/dispose. Real-DuckDB round-trip reader tests (`BlockingStatsReaderTests`) plant blocked_process_reports / dmv_blocking_snapshots / deadlocks rows and assert the per-minute duration aggregate (count + total/max/avg), the XE→DMV fallback union, the #1319 filter, window bounding, and the deadlock-severity aggregate (victim_count + total/max/avg wait) reconciling with the deadlock count over the same window. - **Lite viewer: Expensive Queries sub-tab (Tier-1 parity BUILD)** ([#1513]) — the Lite port of the Darling viewer's **Expensive Queries** view: a unified cross-source ranked list appended as the LAST sub-tab of the **Queries** tab (sub-index 6, after Query Heatmap — nothing else reindexed; the Queries inner switch gains `case 6` while the top-level Queries tab stays index 2). One grid UNIONs the three query engines Lite already collects — `query_stats` (Query Stats), `procedure_stats` (Stored Procedure / Trigger / Function), and `query_store_stats` (Query Store) — each grouped and TOP-20-per-source ranked by AVERAGE worker (CPU) time, then merged, re-ranked by average worker time and capped at 20, mirroring the Dashboard's `report.expensive_queries_today` and Darling's `GetUnifiedExpensiveQueriesAsync`. The reader (`LocalDataService.ExpensiveQueries.cs`) adapts the Postgres/T-SQL UNION to DuckDB against the `v_query_stats` / `v_procedure_stats` / `v_query_store_stats` archive views: the query_stats/query_store arms SUM the collected deltas / weight per-interval averages by `execution_count` (never the raw cumulative DMV totals), a LATERAL fetches each group's latest text + stored plan, every arm is both-sides window-bound on `collection_time` and honors the #1319 database filter, and the µs→ms/sec + KB/page→MB unit conversions run in SQL exactly like the Dashboard view's outer projection. The stored plan rides IN-ROW, so "View Plan" / "Copy Repro Script" open it with zero extra reads via the shared `DataGridContextMenu` — mirroring the Top Queries sub-tab exactly (View Plan on the in-row plan; Get Actual Plan re-executes the runnable statement sources; both no-op cleanly for the module sources and for rows with no plan). Store-driven deviations from the Dashboard view, each faithful to Lite's store: Query Stats objects are the constant `Adhoc` (Lite's query_stats carries no object columns), all functions collapse to a single `Function` label (the shared collector labels them `FUNCTION`), the Query Store grant is pages→MB (×8/1024, unit-correct, matching Lite's Query Store tab), and the Query Store arm carries neither a stored plan nor a plan_handle (Lite's query_store_stats stores neither) so its View Plan / Get Actual Plan gate off. Every Darling grid column is reproduced. Real-DuckDB round-trip reader tests (`ExpensiveQueriesReaderTests`) plant query_stats/procedure_stats/query_store rows and assert the cross-source ranking by avg worker time, the unit conversions, the source labels + object-name shapes, the in-row plan (and the Query Store no-plan/no-handle deviation), delta summing across snapshots, the #1319 filter, and window bounding. @@ -303,6 +304,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 [#1484]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1484 [#1488]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1488 [#1486]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1486 +[#PRNUM]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/PRNUM [#1510]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1510 [#1507]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1507 [#1506]: https://github.com/erikdarlingdata/PerformanceMonitor/issues/1506 diff --git a/Dashboard.Tests/FactScorerTests.cs b/Dashboard.Tests/FactScorerTests.cs index 909b4bac..032edd81 100644 --- a/Dashboard.Tests/FactScorerTests.cs +++ b/Dashboard.Tests/FactScorerTests.cs @@ -368,6 +368,179 @@ public void Score_Threadpool_RunnableQueueAmplifier_DoesNotFireBelowThreshold() Assert.False(rqAmp.Matched); } + // ── Tier-2 parity arms (batch 1): runnable-queue / security-cache / QS-off ────────────────── + // All three are additions to the SHARED FactScorer, so they fire identically for Lite, Dashboard, + // and Darling findings. Sourced thresholds are cited per arm against the deprecated Dashboard SQL. + + // ARM 3 — runnable-task-queue depth roots a STANDALONE finding off the collected cpu_scheduler_stats + // snapshot, tiered to the Dashboard's report.cpu_scheduler_pressure CASE (install/47:1839-1841): + // > 50 CRITICAL-tier (base 1.0), > 20 HIGH (0.75), > 10 MEDIUM (0.5 — the 0.5 root entry point). + // Base maxes at 1.0 (top of WARNING) like every base fact; CRITICAL band is reached only via the + // #1494 runnable-queue -> THREADPOOL amplifier, which still fires independently. + [Theory] + [InlineData(60, 1.0)] // > 50 -> CRITICAL tier (install/47:1839) + [InlineData(30, 0.75)] // > 20 -> HIGH tier (install/47:1840) + [InlineData(15, 0.5)] // > 10 -> MEDIUM tier (install/47:1841) + public void Score_RunnableTasks_TiersMatchDashboard(double total, double expected) + { + var facts = new List + { + new() { Source = "cpu", Key = "RUNNABLE_TASKS", Value = total, + Metadata = new() { ["total_runnable_tasks"] = total, ["runnable_tasks_warning"] = 1 } }, + }; + + new FactScorer().ScoreAll(facts); + + var rt = facts.First(f => f.Key == "RUNNABLE_TASKS"); + // No amplifiers on RUNNABLE_TASKS and it is not tuning-class, so Severity == base. + Assert.Equal(expected, rt.Severity, precision: 4); + Assert.True(rt.Severity >= 0.5, "a non-NORMAL runnable queue must clear the 0.5 root threshold"); + } + + // A shallow queue below the > 10 bar with the warning flag CLEAR does not score (context-only). + [Fact] + public void Score_RunnableTasks_BelowThreshold_DoesNotScore() + { + var facts = new List + { + new() { Source = "cpu", Key = "RUNNABLE_TASKS", Value = 5, + Metadata = new() { ["total_runnable_tasks"] = 5, ["runnable_tasks_warning"] = 0 } }, + }; + + new FactScorer().ScoreAll(facts); + + Assert.Equal(0.0, facts.First(f => f.Key == "RUNNABLE_TASKS").Severity, precision: 4); + } + + // Small-box fallback: below the absolute > 10 bar, the collector's runnable_tasks_warning flag + // (SUM(runnable_tasks_count) >= cpu_count) still scores HIGH (install/47:1844). + [Fact] + public void Score_RunnableTasks_WarningFlagOnSmallBox_ScoresHigh() + { + var facts = new List + { + new() { Source = "cpu", Key = "RUNNABLE_TASKS", Value = 6, + Metadata = new() { ["total_runnable_tasks"] = 6, ["runnable_tasks_warning"] = 1 } }, + }; + + new FactScorer().ScoreAll(facts); + + Assert.Equal(0.75, facts.First(f => f.Key == "RUNNABLE_TASKS").Severity, precision: 4); + } + + // ARM 2 — TokenAndPermUserStore (security cache) growth scores a flat WARNING at >= 1 GB off the + // otherwise context-only MEMORY_CLERKS fact (install/50:562 severity=WARNING, :583 threshold 1 GB). + // The clerk size rides in metadata keyed by clerk_type (= sys.dm_os_memory_clerks.type). + [Fact] + public void Score_MemoryClerks_SecurityCacheOver1Gb_ScoresWarning() + { + var facts = new List + { + new() { Source = "memory", Key = "MEMORY_CLERKS", Value = 52_048, + Metadata = new() + { + ["MEMORYCLERK_SQLBUFFERPOOL"] = 50_000, + ["USERSTORE_TOKENPERM"] = 2_048, // 2 GB + ["total_top_clerks_mb"] = 52_048, + ["clerk_count"] = 2 + } }, + }; + + new FactScorer().ScoreAll(facts); + + var mc = facts.First(f => f.Key == "MEMORY_CLERKS"); + Assert.True(mc.Severity >= 0.75 && mc.Severity < 1.5, "security-cache growth bands WARNING"); + } + + // Under 1 GB does not score; MEMORY_CLERKS stays context-only. + [Fact] + public void Score_MemoryClerks_SecurityCacheUnder1Gb_DoesNotScore() + { + var facts = new List + { + new() { Source = "memory", Key = "MEMORY_CLERKS", Value = 50_512, + Metadata = new() + { + ["MEMORYCLERK_SQLBUFFERPOOL"] = 50_000, + ["USERSTORE_TOKENPERM"] = 512, // 0.5 GB — below the 1 GB bar + } }, + }; + + new FactScorer().ScoreAll(facts); + + Assert.Equal(0.0, facts.First(f => f.Key == "MEMORY_CLERKS").Severity, precision: 4); + } + + // A normal clerk set with no TokenAndPermUserStore entry scores 0 (context-only preserved). + [Fact] + public void Score_MemoryClerks_NoSecurityCacheClerk_DoesNotScore() + { + var facts = new List + { + new() { Source = "memory", Key = "MEMORY_CLERKS", Value = 50_000, + Metadata = new() { ["MEMORYCLERK_SQLBUFFERPOOL"] = 50_000, ["clerk_count"] = 1 } }, + }; + + new FactScorer().ScoreAll(facts); + + Assert.Equal(0.0, facts.First(f => f.Key == "MEMORY_CLERKS").Severity, precision: 4); + } + + // ARM 1 — Query Store off on a user database is an INFO advisory carried by the DB_CONFIG fact, + // detected as query_store_on_count < database_count (install/50:83 severity=INFO). Low 0.3 base = + // INFO band, and DB_CONFIG roots at any positive severity (a ConfigAdvisoryRootKey). + [Fact] + public void Score_DbConfig_QueryStoreOff_ScoresInfoAdvisory() + { + var facts = new List + { + new() { Source = "database_config", Key = "DB_CONFIG", Value = 3, + Metadata = new() { ["database_count"] = 3, ["query_store_on_count"] = 1 } }, + }; + + new FactScorer().ScoreAll(facts); + + var db = facts.First(f => f.Key == "DB_CONFIG"); + Assert.Equal(0.3, db.Severity, precision: 4); + Assert.True(db.Severity > 0 && db.Severity < 0.75, "QS-off is an INFO-band advisory"); + } + + // Every user DB has Query Store on and nothing else drifted -> DB_CONFIG does not score. + [Fact] + public void Score_DbConfig_QueryStoreAllOn_NoContribution() + { + var facts = new List + { + new() { Source = "database_config", Key = "DB_CONFIG", Value = 3, + Metadata = new() { ["database_count"] = 3, ["query_store_on_count"] = 3 } }, + }; + + new FactScorer().ScoreAll(facts); + + Assert.Equal(0.0, facts.First(f => f.Key == "DB_CONFIG").Severity, precision: 4); + } + + // QS-off never lowers a worse co-present DB_CONFIG issue (existing arms unchanged): AUTO_SHRINK on + // 3 DBs stays 0.9 even with Query Store also off on all of them. + [Fact] + public void Score_DbConfig_QueryStoreOff_DoesNotLowerWorseIssue() + { + var facts = new List + { + new() { Source = "database_config", Key = "DB_CONFIG", Value = 3, + Metadata = new() + { + ["database_count"] = 3, + ["query_store_on_count"] = 0, // all off + ["auto_shrink_on_count"] = 3 // worse issue -> 3 * 0.3 = 0.9 + } }, + }; + + new FactScorer().ScoreAll(facts); + + Assert.Equal(0.9, facts.First(f => f.Key == "DB_CONFIG").Severity, precision: 4); + } + // Confidence-hardening: the low-quality fallback floors at 0.5 AFTER the confidence multiply, so a // fired thin-baseline anomaly stays >= InferenceEngine's 0.5 root entry-point even if confidence // ever drops below 1.0 (it is a hardcoded 1.0 today; this guards the future). Same shared scorer. diff --git a/Lite.Tests/FactScorerTests.cs b/Lite.Tests/FactScorerTests.cs index 8740139d..87015b44 100644 --- a/Lite.Tests/FactScorerTests.cs +++ b/Lite.Tests/FactScorerTests.cs @@ -521,6 +521,179 @@ public void Score_Threadpool_RunnableQueueAmplifier_DoesNotFireBelowThreshold() Assert.False(rqAmp.Matched); } + // ── Tier-2 parity arms (batch 1): runnable-queue / security-cache / QS-off ────────────────── + // All three are additions to the SHARED FactScorer, so they fire identically for Lite, Dashboard, + // and Darling findings. Sourced thresholds are cited per arm against the deprecated Dashboard SQL. + + // ARM 3 — runnable-task-queue depth roots a STANDALONE finding off the collected cpu_scheduler_stats + // snapshot, tiered to the Dashboard's report.cpu_scheduler_pressure CASE (install/47:1839-1841): + // > 50 CRITICAL-tier (base 1.0), > 20 HIGH (0.75), > 10 MEDIUM (0.5 — the 0.5 root entry point). + // Base maxes at 1.0 (top of WARNING) like every base fact; CRITICAL band is reached only via the + // #1494 runnable-queue -> THREADPOOL amplifier, which still fires independently. + [Theory] + [InlineData(60, 1.0)] // > 50 -> CRITICAL tier (install/47:1839) + [InlineData(30, 0.75)] // > 20 -> HIGH tier (install/47:1840) + [InlineData(15, 0.5)] // > 10 -> MEDIUM tier (install/47:1841) + public void Score_RunnableTasks_TiersMatchDashboard(double total, double expected) + { + var facts = new List + { + new() { Source = "cpu", Key = "RUNNABLE_TASKS", Value = total, + Metadata = new() { ["total_runnable_tasks"] = total, ["runnable_tasks_warning"] = 1 } }, + }; + + new FactScorer().ScoreAll(facts); + + var rt = facts.First(f => f.Key == "RUNNABLE_TASKS"); + // No amplifiers on RUNNABLE_TASKS and it is not tuning-class, so Severity == base. + Assert.Equal(expected, rt.Severity, precision: 4); + Assert.True(rt.Severity >= 0.5, "a non-NORMAL runnable queue must clear the 0.5 root threshold"); + } + + // A shallow queue below the > 10 bar with the warning flag CLEAR does not score (context-only). + [Fact] + public void Score_RunnableTasks_BelowThreshold_DoesNotScore() + { + var facts = new List + { + new() { Source = "cpu", Key = "RUNNABLE_TASKS", Value = 5, + Metadata = new() { ["total_runnable_tasks"] = 5, ["runnable_tasks_warning"] = 0 } }, + }; + + new FactScorer().ScoreAll(facts); + + Assert.Equal(0.0, facts.First(f => f.Key == "RUNNABLE_TASKS").Severity, precision: 4); + } + + // Small-box fallback: below the absolute > 10 bar, the collector's runnable_tasks_warning flag + // (SUM(runnable_tasks_count) >= cpu_count) still scores HIGH (install/47:1844). + [Fact] + public void Score_RunnableTasks_WarningFlagOnSmallBox_ScoresHigh() + { + var facts = new List + { + new() { Source = "cpu", Key = "RUNNABLE_TASKS", Value = 6, + Metadata = new() { ["total_runnable_tasks"] = 6, ["runnable_tasks_warning"] = 1 } }, + }; + + new FactScorer().ScoreAll(facts); + + Assert.Equal(0.75, facts.First(f => f.Key == "RUNNABLE_TASKS").Severity, precision: 4); + } + + // ARM 2 — TokenAndPermUserStore (security cache) growth scores a flat WARNING at >= 1 GB off the + // otherwise context-only MEMORY_CLERKS fact (install/50:562 severity=WARNING, :583 threshold 1 GB). + // The clerk size rides in metadata keyed by clerk_type (= sys.dm_os_memory_clerks.type). + [Fact] + public void Score_MemoryClerks_SecurityCacheOver1Gb_ScoresWarning() + { + var facts = new List + { + new() { Source = "memory", Key = "MEMORY_CLERKS", Value = 52_048, + Metadata = new() + { + ["MEMORYCLERK_SQLBUFFERPOOL"] = 50_000, + ["USERSTORE_TOKENPERM"] = 2_048, // 2 GB + ["total_top_clerks_mb"] = 52_048, + ["clerk_count"] = 2 + } }, + }; + + new FactScorer().ScoreAll(facts); + + var mc = facts.First(f => f.Key == "MEMORY_CLERKS"); + Assert.True(mc.Severity >= 0.75 && mc.Severity < 1.5, "security-cache growth bands WARNING"); + } + + // Under 1 GB does not score; MEMORY_CLERKS stays context-only. + [Fact] + public void Score_MemoryClerks_SecurityCacheUnder1Gb_DoesNotScore() + { + var facts = new List + { + new() { Source = "memory", Key = "MEMORY_CLERKS", Value = 50_512, + Metadata = new() + { + ["MEMORYCLERK_SQLBUFFERPOOL"] = 50_000, + ["USERSTORE_TOKENPERM"] = 512, // 0.5 GB — below the 1 GB bar + } }, + }; + + new FactScorer().ScoreAll(facts); + + Assert.Equal(0.0, facts.First(f => f.Key == "MEMORY_CLERKS").Severity, precision: 4); + } + + // A normal clerk set with no TokenAndPermUserStore entry scores 0 (context-only preserved). + [Fact] + public void Score_MemoryClerks_NoSecurityCacheClerk_DoesNotScore() + { + var facts = new List + { + new() { Source = "memory", Key = "MEMORY_CLERKS", Value = 50_000, + Metadata = new() { ["MEMORYCLERK_SQLBUFFERPOOL"] = 50_000, ["clerk_count"] = 1 } }, + }; + + new FactScorer().ScoreAll(facts); + + Assert.Equal(0.0, facts.First(f => f.Key == "MEMORY_CLERKS").Severity, precision: 4); + } + + // ARM 1 — Query Store off on a user database is an INFO advisory carried by the DB_CONFIG fact, + // detected as query_store_on_count < database_count (install/50:83 severity=INFO). Low 0.3 base = + // INFO band, and DB_CONFIG roots at any positive severity (a ConfigAdvisoryRootKey). + [Fact] + public void Score_DbConfig_QueryStoreOff_ScoresInfoAdvisory() + { + var facts = new List + { + new() { Source = "database_config", Key = "DB_CONFIG", Value = 3, + Metadata = new() { ["database_count"] = 3, ["query_store_on_count"] = 1 } }, + }; + + new FactScorer().ScoreAll(facts); + + var db = facts.First(f => f.Key == "DB_CONFIG"); + Assert.Equal(0.3, db.Severity, precision: 4); + Assert.True(db.Severity > 0 && db.Severity < 0.75, "QS-off is an INFO-band advisory"); + } + + // Every user DB has Query Store on and nothing else drifted -> DB_CONFIG does not score. + [Fact] + public void Score_DbConfig_QueryStoreAllOn_NoContribution() + { + var facts = new List + { + new() { Source = "database_config", Key = "DB_CONFIG", Value = 3, + Metadata = new() { ["database_count"] = 3, ["query_store_on_count"] = 3 } }, + }; + + new FactScorer().ScoreAll(facts); + + Assert.Equal(0.0, facts.First(f => f.Key == "DB_CONFIG").Severity, precision: 4); + } + + // QS-off never lowers a worse co-present DB_CONFIG issue (existing arms unchanged): AUTO_SHRINK on + // 3 DBs stays 0.9 even with Query Store also off on all of them. + [Fact] + public void Score_DbConfig_QueryStoreOff_DoesNotLowerWorseIssue() + { + var facts = new List + { + new() { Source = "database_config", Key = "DB_CONFIG", Value = 3, + Metadata = new() + { + ["database_count"] = 3, + ["query_store_on_count"] = 0, // all off + ["auto_shrink_on_count"] = 3 // worse issue -> 3 * 0.3 = 0.9 + } }, + }; + + new FactScorer().ScoreAll(facts); + + Assert.Equal(0.9, facts.First(f => f.Key == "DB_CONFIG").Severity, precision: 4); + } + // Confidence-hardening: the low-quality fallback floors at 0.5 AFTER the confidence multiply, so a // fired thin-baseline anomaly stays >= InferenceEngine's 0.5 root entry-point even if confidence // ever drops below 1.0 (it is a hardcoded 1.0 today; this guards the future). diff --git a/Lite.Tests/TestDataSeeder.cs b/Lite.Tests/TestDataSeeder.cs index 346d9fc0..21e540a8 100644 --- a/Lite.Tests/TestDataSeeder.cs +++ b/Lite.Tests/TestDataSeeder.cs @@ -1529,7 +1529,9 @@ INSERT INTO memory_clerks } /// - /// Seeds database_config with per-database configuration flags. + /// Seeds database_config with per-database configuration flags. Query Store defaults ON (the healthy + /// value, like rcsi-on / CHECKSUM here) so the QS-off advisory arm stays dormant in fixtures not + /// exercising it; the QS-off scorer arm is covered directly in FactScorerTests with hand-built facts. /// internal async Task SeedDatabaseConfigAsync( params (string dbName, bool rcsiOn, bool autoShrink, bool autoClose, string pageVerify)[] databases) @@ -1547,7 +1549,7 @@ INSERT INTO database_config recovery_model, is_auto_shrink_on, is_auto_close_on, is_read_committed_snapshot_on, is_auto_create_stats_on, is_auto_update_stats_on, page_verify_option, is_query_store_on) -VALUES ($1, $2, $3, $4, $5, 'FULL', $6, $7, $8, true, true, $9, false)"; +VALUES ($1, $2, $3, $4, $5, 'FULL', $6, $7, $8, true, true, $9, true)"; cmd.Parameters.Add(new DuckDBParameter { Value = _nextId-- }); cmd.Parameters.Add(new DuckDBParameter { Value = TestPeriodEnd }); diff --git a/PerformanceMonitor.Analysis/FactAdvice.cs b/PerformanceMonitor.Analysis/FactAdvice.cs index 3980fe50..fd49ab9f 100644 --- a/PerformanceMonitor.Analysis/FactAdvice.cs +++ b/PerformanceMonitor.Analysis/FactAdvice.cs @@ -1480,6 +1480,13 @@ void Add(string metaKey, string desc) Add("page_verify_not_checksum_count", "page verify not set to CHECKSUM"); Add("auto_create_stats_off_count", "auto-create statistics off"); Add("auto_update_stats_off_count", "auto-update statistics off"); + // Query Store off = user DBs (database_count) minus those with it on (query_store_on_count); + // the QS-off scorer arm (INFO, install/50:83) roots off the same two counts. + var qsOff = f.Metadata.TryGetValue("database_count", out var dbc) && dbc > 0 + && f.Metadata.TryGetValue("query_store_on_count", out var qOn) + ? (long)(dbc - qOn) : 0L; + if (qsOff > 0) + items.Add($"{qsOff:N0} with Query Store off"); if (items.Count == 0) return fallback; @@ -1489,6 +1496,11 @@ void Add(string metaKey, string desc) "indexes and churns I/O re-growing what it shrank), AUTO_CLOSE off, page verify set to CHECKSUM so " + "corruption is detected, auto-create and auto-update statistics on, and RCSI on where the blocking " + "is readers-versus-writers. The Database Configuration view lists exactly which databases each applies to."; + if (qsOff > 0) + rem += + " Where Query Store is off, enable it (ALTER DATABASE SET QUERY_STORE = ON; then " + + "OPERATION_MODE = READ_WRITE) so query performance history is captured for troubleshooting " + + "and plan-regression detection."; return fallback with { @@ -1956,6 +1968,14 @@ private static Dictionary BuildAdviceTable() Remediation: "If a plan regression co-fired, force the historically faster plan as the fast fix. If parameter sensitivity co-fired, do NOT force a plan — that locks in the wrong plan for the other parameter values; use OPTION (RECOMPILE) on the affected statement, or branch the procedure by parameter value. For an ad-hoc reporting spike matching neither, Resource Governor or moving the report off-peak is the durable fix."); + t["RUNNABLE_TASKS"] = new AdviceBlock( + Headline: + "Runnable-task queue is backed up — tasks are ready to run but waiting for a CPU scheduler", + Investigation: + "Every runnable task is a query that has everything it needs EXCEPT a free scheduler — the CPUs are the bottleneck, not I/O or locks. The count is the latest cpu_scheduler_stats snapshot's total across all schedulers (the runnable_tasks_warning flag trips when the queue reaches roughly one task per core). Open the CPU tab and the Wait Stats tab: SOS_SCHEDULER_YIELD co-elevation confirms scheduler starvation, and a THREADPOOL wait means it has escalated to worker-thread exhaustion. Cross-check the top CPU-consuming queries for the window.", + Remediation: + "CPU pressure — tune the CPU-heavy queries first (almost always cheaper than adding cores); the top consumers for the window are attached. Excessive parallelism is a frequent driver: if CXPACKET or high-DOP queries co-fired, lower MAXDOP / raise the Cost Threshold for Parallelism so small queries stop going parallel and monopolizing schedulers. If the queries are already tuned and the queue stays deep, the box is genuinely under-provisioned for the load — add CPU."); + // ───────────────────────────────────────────────────────────────── // Memory pressure // ───────────────────────────────────────────────────────────────── @@ -1984,6 +2004,14 @@ private static Dictionary BuildAdviceTable() Remediation: "The usual cause is over-granting: a query gets a grant much larger than it actually uses because cardinality estimation was wrong. Pull the offending queries' plans and compare the operator-level row estimates against actuals; update statistics with FULLSCAN on the affected tables, or add filtered indexes if a subset of the data drives the bad estimate. Per-query stopgap: `OPTION (MAX_GRANT_PERCENT = X)` caps a single offender's grant without affecting others. Resource Governor workload-group caps are the durable answer if one workload chronically starves the others."); + t["MEMORY_CLERKS"] = new AdviceBlock( + Headline: + "The TokenAndPermUserStore security cache has grown large — it can cause memory pressure and CPU spent on cache lookups", + Investigation: + "TokenAndPermUserStore caches security tokens for permission checks. Under lots of ad-hoc queries, dynamic SQL, or frequent security-context switching (EXECUTE AS, cross-database ownership chaining) it can balloon into gigabytes, crowding out useful plans and making every permission check walk a huge cache. The USERSTORE_TOKENPERM clerk size for this server is attached; the Memory → Memory Clerks view shows it alongside the other top clerks.", + Remediation: + "Clear it during a maintenance window with DBCC FREESYSTEMCACHE('TokenAndPermUserStore') for immediate relief. For a durable fix, cut the churn that grows it: parameterize ad-hoc queries so plans and security tokens are reused, and reduce security-context switching (EXECUTE AS, cross-database ownership chaining). If the growth recurs, investigate what is generating the volume of distinct security contexts."); + t["RESOURCE_SEMAPHORE_QUERY_COMPILE"] = new AdviceBlock( Headline: "Query compile gateway waits — too many concurrent compilations of expensive queries", diff --git a/PerformanceMonitor.Analysis/FactScorer.cs b/PerformanceMonitor.Analysis/FactScorer.cs index db5e4058..fcb71637 100644 --- a/PerformanceMonitor.Analysis/FactScorer.cs +++ b/PerformanceMonitor.Analysis/FactScorer.cs @@ -185,10 +185,36 @@ private static double ScoreCpuFact(Fact fact) // CPU spike: value is max CPU %. Concerning at 80%, critical at 95%. // Only emitted when max is significantly above average (bursty). "CPU_SPIKE" => ApplyThresholdFormula(fact.Value, 80, 95), + // Runnable-task queue depth — a STANDALONE scheduler-pressure signal that roots the collected + // cpu_scheduler_stats snapshot directly. Distinct from (and additive to) the #1494 THREADPOOL + // runnable-queue amplifier, which still fires independently off the same RUNNABLE_TASKS fact. + "RUNNABLE_TASKS" => ScoreRunnableTasks(fact), _ => 0.0 }; } + /// + /// Scores the runnable-task-queue pressure signal (RUNNABLE_TASKS context fact; Value = + /// total_runnable_tasks_count from the latest cpu_scheduler_stats snapshot). Tiers mirror the + /// Dashboard's report.cpu_scheduler_pressure pressure_level CASE + /// (install/47_create_reporting_views.sql lines 1839-1844): > 50 CRITICAL, > 20 HIGH, > 10 MEDIUM, + /// else the collector's own runnable_tasks_warning flag (SUM(runnable_tasks_count) >= cpu_count) as a + /// small-box HIGH fallback the absolute > 10 bar misses. Base maxes at 1.0 (WARNING band) exactly as + /// every other base fact does — the CRITICAL band (>= 1.5) is reached only with corroboration, which + /// is precisely the runnable-queue -> THREADPOOL amplifier path (#1494). A bare runnable queue with no + /// thread/CPU corroboration is a strong WARNING, not an outage. + /// + private static double ScoreRunnableTasks(Fact fact) + { + var total = fact.Value; // total_runnable_tasks_count (latest snapshot) + if (total > 50) return 1.0; // CRITICAL - High runnable task queue (install/47:1839) + if (total > 20) return 0.75; // HIGH - Moderate runnable task queue (install/47:1840) + if (total > 10) return 0.5; // MEDIUM - Some runnable tasks queued (install/47:1841) + // Small-box per-scheduler pressure below the absolute bar (install/47:1844: runnable_tasks_warning). + if (fact.Metadata.GetValueOrDefault("runnable_tasks_warning") >= 1.0) return 0.75; + return 0.0; + } + /// /// Scores I/O latency facts. Value is average latency in ms. /// @@ -218,7 +244,7 @@ private static double ScoreTempDbFact(Fact fact) } /// - /// Scores memory grant facts. Only MEMORY_GRANT_PENDING (from resource semaphore) for now. + /// Scores memory facts: grant waiters (MEMORY_GRANT_PENDING) and security-cache growth (MEMORY_CLERKS). /// private static double ScoreMemoryFact(Fact fact) { @@ -226,10 +252,28 @@ private static double ScoreMemoryFact(Fact fact) { // Grant waiters: concerning at 1, critical at 5 "MEMORY_GRANT_PENDING" => ApplyThresholdFormula(fact.Value, 1, 5), + // Security cache (TokenAndPermUserStore) growth — WARNING at >= 1 GB. See ScoreSecurityCache. + "MEMORY_CLERKS" => ScoreSecurityCache(fact), _ => 0.0 }; } + /// + /// Scores TokenAndPermUserStore (security cache) growth off the otherwise context-only MEMORY_CLERKS + /// fact. That fact carries each top-clerk's size in MB keyed by its clerk_type (MemoryClerksCollector + /// stores clerk_type = sys.dm_os_memory_clerks.type), so the security cache is the USERSTORE_TOKENPERM + /// entry. The Dashboard fires a single WARNING at >= 1 GB with no size escalation + /// (install/50_configuration_issues_analyzer.sql line 562 severity=WARNING, line 583 threshold + /// pages_kb / 1024 / 1024 >= 1.0), so this is a flat WARNING-band base (0.9). Absent when the clerk is + /// not among the top-10 collected, which for a >= 1 GB clerk is effectively never. Non-security clerk + /// sets (buffer pool, etc.) score 0, preserving MEMORY_CLERKS as context-only for those. + /// + private static double ScoreSecurityCache(Fact fact) + { + var securityCacheMb = fact.Metadata.GetValueOrDefault("USERSTORE_TOKENPERM"); + return securityCacheMb >= 1024.0 ? 0.9 : 0.0; // >= 1 GB -> flat WARNING (install/50:562,583) + } + /// /// Scores query-level aggregate facts. /// @@ -345,6 +389,18 @@ private static double ScoreDatabaseConfigFact(Fact fact) if (rcsiOff > 0) score = Math.Max(score, 0.3); + // Query Store disabled on a user database — INFO advisory (install/50_configuration_issues_analyzer.sql + // line 83 severity=INFO). Detected purely from the aggregate counts every collector already emits: + // query_store_on_count (user DBs with QS on; system DBs are excluded from both counts) < database_count + // (user DB total) means at least one user database has Query Store off. Low 0.3 base — DB_CONFIG is a + // ConfigAdvisoryRootKey so it roots as a standing INFO advisory at any positive severity, and 0.3 keeps + // it in the INFO band (< 0.75) matching the Dashboard. Requires BOTH counts present so a fact carrying + // partial metadata never trips it. + if (fact.Metadata.TryGetValue("database_count", out var dbCount) && dbCount > 0 + && fact.Metadata.TryGetValue("query_store_on_count", out var queryStoreOn) + && queryStoreOn < dbCount) + score = Math.Max(score, 0.3); + return score; } From 64f26f30992ad3acd70d9697991a3f286a7e985f Mon Sep 17 00:00:00 2001 From: Erik Darling <2136037+erikdarlingdata@users.noreply.github.com> Date: Mon, 13 Jul 2026 21:11:18 -0400 Subject: [PATCH 2/2] CHANGELOG: fill in PR #1517 link Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 56071947..45be3a2f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,7 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added -- **Analysis engine: three more issue-classification arms in the shared FactScorer (Tier-2 parity)** ([#PRNUM]) — the shared `PerformanceMonitor.Analysis` scorer/inference brain (ONE code path for Lite, the deprecated Dashboard, AND Darling) now classifies three problems the deprecated Dashboard's `report.*` / `config.critical_issues` SQL flagged but the ported engine had dropped. All three are pure `FactScorer` additions over facts the collectors ALREADY emit — no collector, schema, or `install/*.sql` change — so they light up in all three apps' recommendations at once, each citing its source SQL line in a code comment. (1) **Query Store disabled** — an INFO advisory folded into the existing `DB_CONFIG` finding, detected as `query_store_on_count < database_count` (user DBs only, system DBs excluded on both counts), mirroring `install/50_configuration_issues_analyzer.sql`'s INFO Query-Store-off issue (line 83); the DB-config recommendation now lists "N with Query Store off" and appends enable-QS guidance. (2) **Security-cache (TokenAndPermUserStore) growth** — a flat WARNING when the `USERSTORE_TOKENPERM` memory clerk reaches >= 1 GB, read from the otherwise context-only `MEMORY_CLERKS` fact (the clerk size rides in metadata keyed by clerk_type), mirroring `install/50`'s WARNING at `pages_kb / 1024 / 1024 >= 1.0` (lines 562, 583); a new recommendation names the `DBCC FREESYSTEMCACHE('TokenAndPermUserStore')` relief plus the ad-hoc / dynamic-SQL churn cause. (3) **Runnable-queue / worker pressure** — a STANDALONE arm rooting the collected `cpu_scheduler_stats` snapshot on `total_runnable_tasks_count`, tiered to `report.cpu_scheduler_pressure`'s `> 50 / > 20 / > 10` CASE plus the collector's `runnable_tasks_warning` small-box fallback (`install/47_create_reporting_views.sql` lines 1839-1844), additive to and distinct from the #1494 runnable-queue -> THREADPOOL amplifier (kept unchanged). Base severity maxes at 1.0 (WARNING) like every base fact — the CRITICAL band stays reserved for amplifier-corroborated cases (the runnable-queue -> THREADPOOL escalation). `FactScorer` tests in BOTH `Lite.Tests` and `Dashboard.Tests` pin every tier (fires when it should, stays silent below the bar, at the right band) and that a co-present worse `DB_CONFIG` issue is left unchanged. Behavior of the existing arms is unchanged. +- **Analysis engine: three more issue-classification arms in the shared FactScorer (Tier-2 parity)** ([#1517]) — the shared `PerformanceMonitor.Analysis` scorer/inference brain (ONE code path for Lite, the deprecated Dashboard, AND Darling) now classifies three problems the deprecated Dashboard's `report.*` / `config.critical_issues` SQL flagged but the ported engine had dropped. All three are pure `FactScorer` additions over facts the collectors ALREADY emit — no collector, schema, or `install/*.sql` change — so they light up in all three apps' recommendations at once, each citing its source SQL line in a code comment. (1) **Query Store disabled** — an INFO advisory folded into the existing `DB_CONFIG` finding, detected as `query_store_on_count < database_count` (user DBs only, system DBs excluded on both counts), mirroring `install/50_configuration_issues_analyzer.sql`'s INFO Query-Store-off issue (line 83); the DB-config recommendation now lists "N with Query Store off" and appends enable-QS guidance. (2) **Security-cache (TokenAndPermUserStore) growth** — a flat WARNING when the `USERSTORE_TOKENPERM` memory clerk reaches >= 1 GB, read from the otherwise context-only `MEMORY_CLERKS` fact (the clerk size rides in metadata keyed by clerk_type), mirroring `install/50`'s WARNING at `pages_kb / 1024 / 1024 >= 1.0` (lines 562, 583); a new recommendation names the `DBCC FREESYSTEMCACHE('TokenAndPermUserStore')` relief plus the ad-hoc / dynamic-SQL churn cause. (3) **Runnable-queue / worker pressure** — a STANDALONE arm rooting the collected `cpu_scheduler_stats` snapshot on `total_runnable_tasks_count`, tiered to `report.cpu_scheduler_pressure`'s `> 50 / > 20 / > 10` CASE plus the collector's `runnable_tasks_warning` small-box fallback (`install/47_create_reporting_views.sql` lines 1839-1844), additive to and distinct from the #1494 runnable-queue -> THREADPOOL amplifier (kept unchanged). Base severity maxes at 1.0 (WARNING) like every base fact — the CRITICAL band stays reserved for amplifier-corroborated cases (the runnable-queue -> THREADPOOL escalation). `FactScorer` tests in BOTH `Lite.Tests` and `Dashboard.Tests` pin every tier (fires when it should, stays silent below the bar, at the right band) and that a co-present worse `DB_CONFIG` issue is left unchanged. Behavior of the existing arms is unchanged. - **Darling viewer: live Current Active Queries sub-tab (service-mediated)** ([#1515]) — a LIVE, on-demand DMV snapshot of what is running on a monitored server RIGHT NOW, distinct from the stored **Active Queries** sub-tab (the collector's hourly snapshots). Darling's viewer is store-only, so it is delivered through the Stage-2 command plane exactly like `execute_actual_plan` / `fetch_plan`: the viewer enqueues a new `fetch_active_queries` command, the SERVICE (which holds a live connection to every monitored server) runs the query on the target's live connection and returns the rows in `result_json`, and the viewer renders them in a new **Current Active Queries** sub-tab (Queries sub-index 2, right after the stored Active Queries tab — every routing reference uses the named constant, so only the constant values shift) with a **Refresh** button (ON-DEMAND only — never auto-fetched on tab selection or a range refresh, since a live server hit is an explicit action). The DMV query is REUSED, not reinvented: a new read-only `DarlingCollectorRunner.FetchRowsAsync` (the "fetch phase only" twin of `RunAsync` — same `CollectorContext`, no store write) runs the shared `QuerySnapshotsCollector` against the live connection, so the live snapshot carries the SAME columns as the stored one and renders in the same `ViewerQuerySnapshotRow` grid (the Estimated / Actual plan buttons reuse the stored handlers on the plan the DMV read carried inline). New `CommandKind.FetchActiveQueries` is pinned viewer↔executor (`ViewerDataService.CommandFetchActiveQueries` ↔ `DarlingCommandExecutor.ResolvePlan`), requires `target_server_id` and takes no args, dispatches through `IDarlingCommandHost.FetchActiveQueriesLiveAsync` → `DarlingWorker.RunFetchActiveQueriesLiveAsync` (resolves the runtime under the servers lock, mirroring `RunFetchPlanAsync`), and returns legible `server not monitored` / `server not connected` / `timed out` / `permission denied` / `sql error` outcomes; the service caps the SQL read at 30s (comfortably under the viewer's 60s poll budget so a slow read reports a real outcome, not a poll miss), deletes the (large) command row after the viewer reads it, and — because enqueuing is a config write — a read-only `viewer` seat cannot fetch it (same rule as Pause / `fetch_plan`). It is a read-only DMV read (`sys.dm_exec_requests`/`sessions` + the sql_text / query_plan DMFs), so unlike `execute_actual_plan` it is NOT consent-class. Tests pin the command constant, the `ResolvePlan` case, and the FULL service-serialize → viewer-parse wire contract (no live SQL Server needed), plus the timeout / failure mappings and the poll-budget-vs-SQL-cap ordering; the live executor round-trip is gated on `DARLING_TEST_PG` like the `fetch_plan` one. Darling-only — no Lite, Full Dashboard, or `install/*.sql` changes. - **Lite viewer: Blocking Stats sub-tab (Tier-1 parity BUILD)** ([#1514]) — the Lite port of the Darling viewer's **Blocking Stats** sub-tab: a 2x2 chart grid + a window summary strip showing blocking AND deadlock SEVERITY over the window, appended as the LAST sub-tab of the **Blocking** tab (sub-index 4, after Deadlocks — nothing else reindexed; `RefreshBlockingAsync`'s subTabOnly switch gains `case 4` while the top-level Blocking tab stays index 8). It is the severity companion to Lite's existing Blocking **Trends** sub-tab (which plots blocking/deadlock COUNTS): the four charts plot block Max/Avg duration + block Total duration, then deadlock Max/Avg wait + deadlock Total wait (Total split onto its own axis per pair so the aggregate magnitude doesn't swamp the per-incident one), and the summary strip carries events / total / max / avg block duration, the deadlock count, and the deadlock victim count + total deadlock wait — the Lite equivalent of the Dashboard's pre-aggregated `collect.blocking_deadlock_stats` (`total_blocking_duration_ms` / `victim_count` / `total_deadlock_wait_time_ms`), computed ON-THE-FLY from rows the store already keeps, so no collector or migration is needed. The reader (`LocalDataService.BlockingStats.cs`) adapts Darling's `GetBlockingDurationStatsAsync` + `GetDeadlockSeverityStatsAsync` to DuckDB: the block-duration aggregate SUM/MAX/AVGs `wait_time_ms` per minute over the SAME XE-preferred (`v_blocked_process_reports`) + DMV-fallback (`v_dmv_blocking_snapshots`, `WHERE NOT EXISTS`) union the blocking-incident count trend uses — so the severity chart and the count chart never disagree on which rows exist — and honors the #1319 database filter on both arms; the deadlock-severity aggregate reads the raw `deadlock_graph_xml` over the SAME `v_deadlocks` / `collection_time` window as the deadlock count (uncapped, so it can't drop rows the count keeps), parses each graph OFF the UI thread with Lite's own `DeadlockProcessDetail.ParseFromRows` (the SAME parse the Deadlocks grid uses, so the two agree in-app), and buckets the per-process victim/wait rows by minute (`victim_count` = SUM of victims, total/max/avg over every process's wait). **Collapse check**: Darling's deadlock-severity computation is NOT a viewer↔service twin — the service/MCP side hosts only the count trends and deliberately omits a `get_blocking_deadlock_stats` (no raw victim-wait source), and the Darling viewer's aggregate already parses via the shared `PerformanceMonitor.Common.DeadlockGraphParser` — so per the brief Lite reuses its own existing `ParseFromRows` (keeping Lite's severity internally consistent with its Deadlocks grid) rather than introducing a third copy; nothing on the Darling side was touched. The four charts (`ServerTab.BlockingStats.cs`) mirror Darling's rendering with Lite's chart idioms (shared `ChartStyle` / `ChartHoverHelper`, `SeriesColors` / `ChartPalette` identities, `UtcOffsetMinutes` display conversion, Y-floor-at-0, window-pinned `SetLimitsX`, connected-scatter padded to the window ends), wired into the Blocking `TabControl` + `RefreshBlockingAsync` (both the subTabOnly `case 4` and the full-refresh path) + `ServerTab.xaml.cs` init/dispose. Real-DuckDB round-trip reader tests (`BlockingStatsReaderTests`) plant blocked_process_reports / dmv_blocking_snapshots / deadlocks rows and assert the per-minute duration aggregate (count + total/max/avg), the XE→DMV fallback union, the #1319 filter, window bounding, and the deadlock-severity aggregate (victim_count + total/max/avg wait) reconciling with the deadlock count over the same window. - **Lite viewer: Expensive Queries sub-tab (Tier-1 parity BUILD)** ([#1513]) — the Lite port of the Darling viewer's **Expensive Queries** view: a unified cross-source ranked list appended as the LAST sub-tab of the **Queries** tab (sub-index 6, after Query Heatmap — nothing else reindexed; the Queries inner switch gains `case 6` while the top-level Queries tab stays index 2). One grid UNIONs the three query engines Lite already collects — `query_stats` (Query Stats), `procedure_stats` (Stored Procedure / Trigger / Function), and `query_store_stats` (Query Store) — each grouped and TOP-20-per-source ranked by AVERAGE worker (CPU) time, then merged, re-ranked by average worker time and capped at 20, mirroring the Dashboard's `report.expensive_queries_today` and Darling's `GetUnifiedExpensiveQueriesAsync`. The reader (`LocalDataService.ExpensiveQueries.cs`) adapts the Postgres/T-SQL UNION to DuckDB against the `v_query_stats` / `v_procedure_stats` / `v_query_store_stats` archive views: the query_stats/query_store arms SUM the collected deltas / weight per-interval averages by `execution_count` (never the raw cumulative DMV totals), a LATERAL fetches each group's latest text + stored plan, every arm is both-sides window-bound on `collection_time` and honors the #1319 database filter, and the µs→ms/sec + KB/page→MB unit conversions run in SQL exactly like the Dashboard view's outer projection. The stored plan rides IN-ROW, so "View Plan" / "Copy Repro Script" open it with zero extra reads via the shared `DataGridContextMenu` — mirroring the Top Queries sub-tab exactly (View Plan on the in-row plan; Get Actual Plan re-executes the runnable statement sources; both no-op cleanly for the module sources and for rows with no plan). Store-driven deviations from the Dashboard view, each faithful to Lite's store: Query Stats objects are the constant `Adhoc` (Lite's query_stats carries no object columns), all functions collapse to a single `Function` label (the shared collector labels them `FUNCTION`), the Query Store grant is pages→MB (×8/1024, unit-correct, matching Lite's Query Store tab), and the Query Store arm carries neither a stored plan nor a plan_handle (Lite's query_store_stats stores neither) so its View Plan / Get Actual Plan gate off. Every Darling grid column is reproduced. Real-DuckDB round-trip reader tests (`ExpensiveQueriesReaderTests`) plant query_stats/procedure_stats/query_store rows and assert the cross-source ranking by avg worker time, the unit conversions, the source labels + object-name shapes, the in-row plan (and the Query Store no-plan/no-handle deviation), delta summing across snapshots, the #1319 filter, and window bounding. @@ -304,7 +304,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 [#1484]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1484 [#1488]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1488 [#1486]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1486 -[#PRNUM]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/PRNUM +[#1517]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1517 [#1510]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1510 [#1507]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1507 [#1506]: https://github.com/erikdarlingdata/PerformanceMonitor/issues/1506