Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- **Analysis engine: two tempdb-deep issue-classification arms in the shared FactScorer (Tier-2 parity)** ([#1519]) — the #1517 companion: the shared `PerformanceMonitor.Analysis` scorer (ONE code path for Lite, the deprecated Dashboard, AND Darling) now classifies two tempdb problems the deprecated Dashboard's `report.tempdb_pressure` / `report.tempdb_contention_analysis` SQL flagged but the ported engine only partially covered. Both are PURE `FactScorer` additions over facts the collectors ALREADY emit — no collector, schema, or `install/*.sql` change — each citing its source SQL line in a code comment. (1) **Version-store pressure by ABSOLUTE size** — `TEMPDB_USAGE` previously scored the space-FRACTION only, blind to a multi-GB version store sitting in a barely-full tempdb (a long-running RCSI/snapshot transaction pinning row versions). It now scores the WORSE of space-fraction and version-store size, reading `max_version_store_mb` (already in the fact metadata — the same key `ComposeTempdbUsage` reads) and tiering it to `report.tempdb_pressure`'s `pressure_level` CASE (`install/47_create_reporting_views.sql` lines 1431-1433): > 5000 MB CRITICAL-tier, > 2000 MB HIGH, > 1000 MB MEDIUM; `tempdb_contention_analysis` corroborates the 1 GB bar (`version_store_high_warning`, install/47:2504). The tempdb recommendation card now names the version store as the driver whenever it clears the 1 GB bar — even if another consumer is nominally larger — mirroring the source view keying `pressure_level` purely on version-store size. (2) **Allocation / PFS-GAM-SGAM PAGELATCH contention** — `ScoreWaitFact` had PAGEIOLATCH + LATCH_EX/SH but no PAGELATCH arm. It now scores the `PAGELATCH_UP` wait fact by ABSOLUTE `wait_time_ms` (the same absolute-ms shape as the THREADPOOL gate), tripping at > 10000 ms -> MEDIUM per `tempdb_contention_analysis`'s `contention_level` CASE (install/47:2515) — read from the SAME server-wide `wait_stats` the view reads, so it is a faithful port; a new advice block (routed through the wait-family `ComposeWaitByKey`, so the card states the actual wait total) names the add-tempdb-files / TF 1118 fix (install/47:2522, 2535). Base severity maxes at 1.0 (WARNING) like every base fact — the CRITICAL band stays reserved for corroboration. NOTE: the view's stronger CRITICAL "allocation contention" signal (`allocation_contention_warning`, a tempdb-scoped `dm_os_waiting_tasks` snapshot computed in `install/34`) is NOT carried in any fact, so per the pure-scorer scope it was left to the PAGELATCH_UP wait rather than adding collection. `FactScorer` tests in BOTH `Lite.Tests` and `Dashboard.Tests` pin every tier (fires at the right band, stays silent below the bar, and the version-store arm never lowers a worse space-fraction score). No test-seeder fix was needed (unlike #1517's Query-Store default): the clean-server fixture seeds a 10 MB version store and no PAGELATCH waits, and the largest version store across all seeders is 900 MB — below the 1 GB bar. Behavior of the existing arms is unchanged.
- **Analysis engine: three more issue-classification arms in the shared FactScorer (Tier-2 parity)** ([#1517]) — the shared `PerformanceMonitor.Analysis` scorer/inference brain (ONE code path for Lite, the deprecated Dashboard, AND Darling) now classifies three problems the deprecated Dashboard's `report.*` / `config.critical_issues` SQL flagged but the ported engine had dropped. All three are pure `FactScorer` additions over facts the collectors ALREADY emit — no collector, schema, or `install/*.sql` change — so they light up in all three apps' recommendations at once, each citing its source SQL line in a code comment. (1) **Query Store disabled** — an INFO advisory folded into the existing `DB_CONFIG` finding, detected as `query_store_on_count < database_count` (user DBs only, system DBs excluded on both counts), mirroring `install/50_configuration_issues_analyzer.sql`'s INFO Query-Store-off issue (line 83); the DB-config recommendation now lists "N with Query Store off" and appends enable-QS guidance. (2) **Security-cache (TokenAndPermUserStore) growth** — a flat WARNING when the `USERSTORE_TOKENPERM` memory clerk reaches >= 1 GB, read from the otherwise context-only `MEMORY_CLERKS` fact (the clerk size rides in metadata keyed by clerk_type), mirroring `install/50`'s WARNING at `pages_kb / 1024 / 1024 >= 1.0` (lines 562, 583); a new recommendation names the `DBCC FREESYSTEMCACHE('TokenAndPermUserStore')` relief plus the ad-hoc / dynamic-SQL churn cause. (3) **Runnable-queue / worker pressure** — a STANDALONE arm rooting the collected `cpu_scheduler_stats` snapshot on `total_runnable_tasks_count`, tiered to `report.cpu_scheduler_pressure`'s `> 50 / > 20 / > 10` CASE plus the collector's `runnable_tasks_warning` small-box fallback (`install/47_create_reporting_views.sql` lines 1839-1844), additive to and distinct from the #1494 runnable-queue -> THREADPOOL amplifier (kept unchanged). Base severity maxes at 1.0 (WARNING) like every base fact — the CRITICAL band stays reserved for amplifier-corroborated cases (the runnable-queue -> THREADPOOL escalation). `FactScorer` tests in BOTH `Lite.Tests` and `Dashboard.Tests` pin every tier (fires when it should, stays silent below the bar, at the right band) and that a co-present worse `DB_CONFIG` issue is left unchanged. Behavior of the existing arms is unchanged.
- **Lite MCP: the 18 Darling-only tools ported (drains the MCP inventory ratchet to ZERO)** ([#1518]) — the mechanical `[McpServerTool]` companion to the #1504 / #1505 / #1509 viewer BUILDs: every tool the #1501 `CrossAppMcpToolInventoryPinTests` ratchet listed as Darling-only is now exposed by Lite's MCP host, so `KnownLiteMissingMcpTools` is **empty** (only the `get_blocked_process_reports` <-> `get_blocking` naming-drift pair remains, handled separately). The 18: `get_latch_stats`, `get_spinlock_stats`, `get_plan_cache_bloat`, `get_cpu_scheduler_pressure`, `get_resource_semaphore`, `get_default_trace_events`, `get_daily_summary`, `get_server_config_changes`, `get_database_config_changes`, `get_trace_flag_changes`, and the eight `get_health_parser_*` tools (system_health / severe_errors / io_issues / scheduler_issues / memory_conditions / cpu_tasks / memory_broker / memory_node_oom). Each is a thin wrapper over the reader Lite ALREADY had from the viewer builds — no new collection, no live-server hit: latch/spinlock wrap the latest-snapshot readers, plan-cache-bloat folds the true-total summary + composition grid + the shared `ClassifyPlanCacheBloat` banding, cpu-scheduler-pressure surfaces the latest scheduler snapshot with the collector's warning flags, daily-summary wraps the Performance Calendar aggregate (composite health band), the three config-change tools wrap the shared `ConfigChangeDiff` histories, and default-trace + the eight health-parser tools wrap the System Events parse-on-read readers (gated by the shared `SystemHealthSignificance` / `DefaultTraceEventSignificance` — the SAME significant set the viewer shows). The ONE reader added is `GetResourceSemaphoreSnapshotAsync` (a latest-snapshot read of the FULL semaphore ceiling columns — target / max-target / total workspace memory + cumulative timeout/forced-grant counts — which the existing per-pool `GetMemoryGrantChartDataAsync` chart reader doesn't surface), so Lite's `get_resource_semaphore` matches the Dashboard/Darling shape rather than duplicating `get_memory_grants`. A raw naive-UTC `EventTime` / `ChangeTime` accessor was added to the System Events + Config Changes viewer row DTOs so the tools emit ISO-8601 timestamps (matching every other Lite MCP tool) instead of the grids' display-local strings. All 18 are registered in `McpHostService` and documented in the MCP server instructions. Lite-only — no Darling, Full Dashboard, or `install/*.sql` changes.
- **Darling viewer: Query Store Regressions sub-tab (Dashboard parity BUILD)** ([#1516]) — the missing Dashboard **Query Store Regressions** view ported to the Darling viewer: the regressed-query GRID over the Query Store data Darling already collects (its `Query Store by Duration` tab + `QueryStoreHistoryWindow` read the same `query_store_stats` store). A new **Queries** sub-tab inserted at index 6 — right after `Query Store by Duration`, matching the Dashboard's Query Store → Query Store Regressions adjacency (every routing reference uses the NAMED constant, so only Expensive Queries / Query Heatmap shift, and the XAML `<TabItem>` order tracks the constants exactly — the same insert pattern as [#1515]). The read (`ViewerDataService.QueryStoreRegressions.cs`) is the Postgres port of the Dashboard's `report.query_store_regressions` inline TVF (`install/47_create_reporting_views.sql`) against `query_store_stats`: two windowed CTEs — BASELINE (every capture BEFORE the window) vs RECENT (the in-window captures) — averaged and joined per `(database, query_id)`, computing the duration / CPU / I/O regression percents, the execution-count-weighted `additional_duration_ms` (extra total time = per-exec duration delta × recent exec count), the plan-count deltas, and a DURATION-driven severity band, keeping only rows whose **CPU regressed > 25%** (the TVF's single-metric gate), ranked by added duration and capped at the TVF's TOP (50). Faithful store adaptations, all documented in the read: the baseline/recent split windows on `collection_time` (Darling's UTC window column — the frame the toolbar's bounds are in) rather than the TVF's `server_last_execution_time` (Darling's column is the server's LOCAL wall clock, so windowing it against UTC bounds would be a timezone bug); the recent query-text sample is `MAX(query_text)` from Darling's already-decompressed column (the TVF `DECOMPRESS()`es a compressed one); and it ports the TVF's ACTUAL behavior (unbounded baseline, plain AVG, CPU-only gate), NOT the stale intent comment on the Dashboard's C# caller (bounded/mirrored baseline, weighted averages, multi-metric, minimums) that the TVF does not actually run. The row VM reproduces **every Dashboard grid column** (Last Execution, Severity, Database, Query ID, Total Impact, Base/Recent Execs, Base/Recent Plans, Duration Δ%, Base/Recent Dur, CPU Δ%, Base/Recent CPU, I/O Δ%, Base/Recent Reads, Query Text); the grid honors the global #1319 database filter (`$4::text[] IS NULL OR database_name = ANY($4)`), carries a `DataGridFilterManager`, and wires the per-row **double-click execution-history** window (reusing `QueryStoreHistoryWindow` query-scoped with plan_id 0, since the regression aggregates over plans — mirroring the Dashboard grid's double-click). Like the Dashboard grid it has NO slicer / comparison / plan surface (the read is a baseline-vs-recent contrast and carries no plan XML — like the sibling Query Store tab, whose View-Plan is deferred), so it uses the plan-less query-grid menu (`QueryGridRowStyle`) whose **Copy Repro** builds a plan-less repro from the query-text sample. Tests pin the read's load-bearing clauses (base table, the `collection_time` baseline/recent split, the CPU gate, the added-duration ranking + `LIMIT 50`, the severity thresholds, the bigint/integer casts, the µs→ms conversion, the DB filter, and the PG positional dialect via the shared Queries dialect theory), the row display, the double-click key mapper, and Copy Repro, plus a `DARLING_TEST_PG`-gated live round-trip proving baseline-vs-recent detection, the CPU gate, the INNER-JOIN exclusion of NEW (no-baseline) queries, and the CRITICAL severity + added-duration math end-to-end. Darling-only — no Lite, Full Dashboard, or `install/*.sql` changes.
Expand Down Expand Up @@ -315,6 +316,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
[#1513]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1513
[#1514]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1514
[#1515]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1515
[#1519]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1519
[#1518]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1518
[#1516]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1516
[#1505]: https://github.com/erikdarlingdata/PerformanceMonitor/pull/1505
Expand Down
98 changes: 98 additions & 0 deletions Dashboard.Tests/FactScorerTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -541,6 +541,104 @@ public void Score_DbConfig_QueryStoreOff_DoesNotLowerWorseIssue()
Assert.Equal(0.9, facts.First(f => f.Key == "DB_CONFIG").Severity, precision: 4);
}

// ── Tier-2 parity arms (batch 2): tempdb-deep — version-store size + PAGELATCH_UP allocation ─────
// Both are additions to the SHARED FactScorer, so they fire identically for Lite, Dashboard, and
// Darling findings. Every trip point is cited to the deprecated Dashboard's install SQL.

// ARM 1 — tempdb VERSION-STORE pressure by ABSOLUTE size (max_version_store_mb in the TEMPDB_USAGE
// fact metadata), scored as the WORSE of space-fraction and version-store size. Tiers mirror
// report.tempdb_pressure's pressure_level CASE (install/47:1431-1433): > 5000 MB CRITICAL-tier
// (base 1.0), > 2000 MB HIGH (0.75), > 1000 MB MEDIUM (0.5 — the 0.5 root entry point). Space is only
// 20% full, whose sub-0.5 fraction score is dominated by the version-store arm via Math.Max, so the
// asserted severity IS the version-store tier — proving the arm scores independently of fill.
[Theory]
[InlineData(6000, 1.0)] // > 5000 -> CRITICAL tier (install/47:1431)
[InlineData(3000, 0.75)] // > 2000 -> HIGH tier (install/47:1432)
[InlineData(1500, 0.5)] // > 1000 -> MEDIUM tier (install/47:1433)
public void Score_TempDbVersionStore_TiersMatchDashboard(double versionMb, double expected)
{
var facts = new List<Fact>
{
new() { Source = "tempdb", Key = "TEMPDB_USAGE", Value = 0.20, // space only 20% full
Metadata = new() { ["max_version_store_mb"] = versionMb } },
};

new FactScorer().ScoreAll(facts);

var t = facts.First(f => f.Key == "TEMPDB_USAGE");
Assert.Equal(expected, t.Severity, precision: 4);
Assert.True(t.Severity >= 0.5, "a version store over 1 GB must clear the 0.5 root threshold");
}

// A version store below the > 1000 MB bar does not score. Space fraction 0 isolates the version-store
// arm (the space-fraction arm yields a small sub-0.5 base for any positive fill), proving a 900 MB
// store trips nothing on its own.
[Fact]
public void Score_TempDbVersionStore_BelowThreshold_DoesNotScore()
{
var facts = new List<Fact>
{
new() { Source = "tempdb", Key = "TEMPDB_USAGE", Value = 0.0,
Metadata = new() { ["max_version_store_mb"] = 900 } }, // 900 MB — below the 1 GB bar
};

new FactScorer().ScoreAll(facts);

Assert.Equal(0.0, facts.First(f => f.Key == "TEMPDB_USAGE").Severity, precision: 4);
}

// WORSE-of: the version-store arm never LOWERS a higher space-fraction score. Space 95% full (the
// fraction arm hits its 0.90 critical -> 1.0) with a small version store stays 1.0.
[Fact]
public void Score_TempDbVersionStore_DoesNotLowerWorseSpaceFraction()
{
var facts = new List<Fact>
{
new() { Source = "tempdb", Key = "TEMPDB_USAGE", Value = 0.95,
Metadata = new() { ["max_version_store_mb"] = 200 } }, // small VS, but space is critical
};

new FactScorer().ScoreAll(facts);

Assert.Equal(1.0, facts.First(f => f.Key == "TEMPDB_USAGE").Severity, precision: 4);
}

// ARM 2 — tempdb allocation / PFS-GAM-SGAM contention scored off the PAGELATCH_UP wait fact by
// ABSOLUTE wait_time_ms (server-wide wait_stats, the SAME data the source view reads), tripping at
// > 10000 ms -> MEDIUM (install/47:2515: pagelatch_up_ms > 10000 -> "MEDIUM - PAGELATCH_UP
// contention"). Flat 0.5 — the view has no higher PAGELATCH_UP band. Value (fraction-of-period) only
// has to be > 0 to clear the wait guard; the absolute wait_time_ms is what scores.
[Fact]
public void Score_PageLatchUp_Over10Sec_ScoresMedium()
{
var facts = new List<Fact>
{
new() { Source = "waits", Key = "PAGELATCH_UP", Value = 0.01,
Metadata = new() { ["wait_time_ms"] = 15_000, ["waiting_tasks_count"] = 800 } },
};

new FactScorer().ScoreAll(facts);

var p = facts.First(f => f.Key == "PAGELATCH_UP");
Assert.Equal(0.5, p.Severity, precision: 4);
Assert.True(p.Severity >= 0.5, "PAGELATCH_UP over the 10s bar must clear the 0.5 root threshold");
}

// At/under the 10000 ms bar PAGELATCH_UP stays context-only (a little allocation latching is normal).
[Fact]
public void Score_PageLatchUp_Under10Sec_DoesNotScore()
{
var facts = new List<Fact>
{
new() { Source = "waits", Key = "PAGELATCH_UP", Value = 0.01,
Metadata = new() { ["wait_time_ms"] = 8_000, ["waiting_tasks_count"] = 300 } },
};

new FactScorer().ScoreAll(facts);

Assert.Equal(0.0, facts.First(f => f.Key == "PAGELATCH_UP").Severity, precision: 4);
}

// Confidence-hardening: the low-quality fallback floors at 0.5 AFTER the confidence multiply, so a
// fired thin-baseline anomaly stays >= InferenceEngine's 0.5 root entry-point even if confidence
// ever drops below 1.0 (it is a hardcoded 1.0 today; this guards the future). Same shared scorer.
Expand Down
Loading
Loading