Feat/ssm ext#51
Conversation
Three tutorials in examples/ (basics, GLM emissions, ACDC model selection) in the style of HiddenMarkovModels.jl: Literate.jl scripts whose @test lines are hidden from the rendered docs with #src but run as part of the package test suite. docs/make.jl converts them into a Tutorials section, and test/runtests.jl includes each example as a testset.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #51 +/- ##
==========================================
+ Coverage 99.12% 99.36% +0.23%
==========================================
Files 7 9 +2
Lines 1257 1408 +151
==========================================
+ Hits 1246 1399 +153
+ Misses 11 9 -2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Benchmark Results (Julia v1)Time benchmarks
Memory benchmarks
|
|
cc @itsdfish @kiante-fernandez can one / both of y'all review this? I guess one question is if any of the code related to the Signed / Coherence DDM should live upstream. Happy to move things around. As we discussed one of these could also just live as an example in a Literate tutorial as well |
@rsenne, I will look over your code and and markdown over the next couple days and try to provide some high level feedback. I'm still having issues with typing. So I can't get into the code in a lot of depth and interact with it. But I look forward to learning about your model and how it hooks into SSM.jl. |
|
No worries, thanks for all the help, even with a hand injury :) happy to answer any questions / if anything looks funky let me know |
There was a problem hiding this comment.
how does this perform if the number of states are correct but the mission distributions are misspecified?
There was a problem hiding this comment.
Great question. I’m not the original author of this method, but let me tag @nguyenston, who adapted it for use with HMMs.
My understanding is that ACDC is designed with the expectation that some degree of model misspecification will almost always be present. Rather than assuming the fitted model is exactly correct, it aims to select
It may be helpful to distinguish between two levels of misspecification. In a common case, the observation model may capture the broad structure of the data but miss some distributional details. For example, the data may be approximately Gaussian overall, while one or more latent-state distributions are closer to gamma distributions and exhibit right skew. In settings like this, ACDC can help reduce the tendency of mixture models and HMMs to compensate for the misspecified emission distribution by introducing extra components or states. In other words, it may help prevent over-segmentation caused by using multiple Gaussian components to approximate a single non-Gaussian distribution.
The situation is different when there is a substantial mismatch between the assumed observation model and the true data-generating process. In that case, ACDC may still identify the value of K that minimizes discrepancy within the specified model class, but it cannot correct the model class itself. The selected number of components or states could therefore remain difficult to interpret, since additional states may be used to absorb systematic features that the observation model cannot represent.
2 participants
EmissionModelsSequentialSamplingModelsExtStimulusCodedDDM" andCoherenceDDM`ACDCdrivers for robust model selectionControlledEmissionHMMworked with ACDCResolves #34