Scoring rules for filters

docs/api_reference/developer/inference/filter_configs.md

@@ -2,7 +2,8 @@
 
 The single `Filter()` handler is directed to the appropriate filtering algorithm via the provided `FilterConfig`.
 
+Several shared fields on `BaseFilterConfig` now control predicted-observation recordings (`record_predicted_observations_mean`, `record_predicted_observations_cov`, and `record_predicted_observations_ensemble`). These backend summaries are distinct from the scoring path documented on the companion [Scoring](scoring.md) page: scoring can run without recording them into the trace.
+
 ::: dynestyx.inference.filter_configs
     options:
       filters: []
-

docs/api_reference/developer/inference/filters.md

@@ -1,8 +1,9 @@
 # Filters
 
-One of the principal functions of a dynamical systems inference engine is *filtering*, i.e., computation of the distribution \(p(x_t \mid y_{1:T}, \theta)\). In the computation of a filtering distribution, we also obtain estimates of the marginal likelihood, \(p(y_{1:T} | \theta)\), used for parameter inference/system identification. To tell `dynestyx` that a dynamical system should be processed via a filtering algorithm, we use the `Filter` class.
+One of the principal functions of a dynamical systems inference engine is *filtering*, i.e., computation of the distribution \(p(x_t \mid y_{1:t}, \theta)\). In the computation of a filtering distribution, we also obtain estimates of the marginal likelihood, \(p(y_{1:T} | \theta)\), used for parameter inference/system identification. To tell `dynestyx` that a dynamical system should be processed via a filtering algorithm, we use the `Filter` class.
+
+This module also hosts the public handler entry point for predicted-observation scoring via `scoring_config`. The scoring surface is documented on the [Scoring](scoring.md) page, while backend translation of predictive summaries currently lives in `dynestyx.inference.observation_predictions`.
 
 ::: dynestyx.inference.filters
     options:
       filters: []
-

docs/api_reference/developer/inference/scoring.md

@@ -0,0 +1,15 @@
+# Scoring
+
+`dynestyx.inference.scoring` defines the public scoring-rule configuration objects consumed by `Filter(..., scoring_config=...)`.
+
+From the developer perspective, this module is intentionally narrow: it defines score objects and configuration policy for the predictive observation distribution, while backend-specific predictive summary extraction remains in `dynestyx.inference.observation_predictions`.
+
+::: dynestyx.inference.scoring
+    options:
+      members:
+        - BaseObservationScore
+        - GaussianLogProbScore
+        - DawidSebastianiScore
+        - ObservationWiseCRPSScore
+        - EnergyScore
+        - ObservationScoringConfig

docs/api_reference/public/inference/filter_configs.md

@@ -2,6 +2,8 @@
 
 The single `Filter()` handler is directed to the appropriate filtering algorithm via the provided `FilterConfig`. We provide a summary below, as well as an exhaustive list of classes.
 
+The shared `record_predicted_observations_*` fields on filter configs control whether backend predictive observation means, covariances, or ensembles are recorded to the NumPyro trace. Those recordings are optional and separate from observation scoring: `scoring_config` can score the predictive observation distribution without enabling these trace sites.
+
 ## Available filter configurations
 
 | Config class               | Time domain         | When it fits best |

docs/api_reference/public/inference/filters.md

@@ -1,9 +1,10 @@
 # Filters
 
-One of the principal functions of a dynamical systems inference engine is *filtering*, i.e., computation of the distribution \(p(x_t \mid y_{1:T}, \theta)\). In the computation of a filtering distribution, we also obtain estimates of the marginal likelihood, \(p(y_{1:T} | \theta)\), used for parameter inference/system identification. To tell `dynestyx` that a dynamical system should be processed via a filtering algorithm, we use the `Filter` class.
+One of the principal functions of a dynamical systems inference engine is *filtering*, i.e., computation of the distribution \(p(x_t \mid y_{1:t}, \theta)\). In the computation of a filtering distribution, we also obtain estimates of the marginal likelihood, \(p(y_{1:T} | \theta)\), used for parameter inference/system identification. To tell `dynestyx` that a dynamical system should be processed via a filtering algorithm, we use the `Filter` class.
+
+`Filter` can also expose one-step-ahead predictive observation diagnostics through `scoring_config` and the `record_predicted_observations_*` fields on the chosen filter config. Scoring always refers to the predictive observation distribution, while the `record_predicted_observations_*` flags only control whether backend predictive summaries are also written into the trace. That scoring path currently supports only the continuous-time CD-Dynamax Gaussian filters (`ContinuousTimeKFConfig`, `ContinuousTimeEKFConfig`, `ContinuousTimeUKFConfig`, and `ContinuousTimeEnKFConfig`). See the [Scoring](scoring.md) page for the score definitions and configuration surface.
 
 ::: dynestyx.inference.filters
     options:
       members:
         - Filter
-

docs/api_reference/public/inference/scoring.md

@@ -0,0 +1,15 @@
+# Scoring
+
+`dynestyx.inference.scoring` defines the proper scoring-rule surface used by `Filter(..., scoring_config=...)`.
+
+These scores operate on the one-step-ahead predictive observation distributions produced by the filter. Recording predicted observation means, covariances, or ensembles into the trace is a separate concern controlled by the filter config. At the moment, Dynestyx supports scoring only for the continuous-time CD-Dynamax Gaussian filters (`ContinuousTimeKFConfig`, `ContinuousTimeEKFConfig`, `ContinuousTimeUKFConfig`, and `ContinuousTimeEnKFConfig`).
+
+::: dynestyx.inference.scoring
+    options:
+      members:
+        - BaseObservationScore
+        - GaussianLogProbScore
+        - DawidSebastianiScore
+        - ObservationWiseCRPSScore
+        - EnergyScore
+        - ObservationScoringConfig

docs/tutorials/gentle_intro/00_index.ipynb

@@ -41,7 +41,9 @@
     "\n",
     "13. **[Part 11b: Missing observations with `Simulator` + MCMC](../11b_missing_observations_simulator_mcmc/)** — Joint posterior inference over parameters and latent states under Gaussian missing-data simulator conditioning.\n",
     "\n",
-    "14. **[Part 11c: Missing observations in HMMs](../11c_missing_observations_hmms/)** — Exact HMM missingness handling for `MultivariateNormal` and `Independent(..., 1)` observation families."
+    "14. **[Part 11c: Missing observations in HMMs](../11c_missing_observations_hmms/)** — Exact HMM missingness handling for `MultivariateNormal` and `Independent(..., 1)` observation families.\n",
+    "\n",
+    "15. **[Part 12: Observation scoring with filters](../12_observation_scoring_with_filters/)** — Use filter-predicted observation distributions to compute Gaussian log-probs, Dawid-Sebastiani scores, CRPS, and energy scores."
    ]
   },
   {

docs/tutorials/gentle_intro/11c_missing_observations_hmms.ipynb

@@ -806,7 +806,7 @@
    "source": [
     "**Previous:** [Part 11b — Missing observations with `Simulator` + MCMC](../11b_missing_observations_simulator_mcmc/)\n",
     "\n",
-    "**Next:** [Part 12 — Hierarchical modeling patterns](../12_hierarchical_modeling_patterns/)\n"
+    "**Next:** [Part 12 — Observation scoring with filters](../12_observation_scoring_with_filters/)\n"
    ]
   }
  ],