Add Sonic Similarity plugin for audio-based track similarity#1
Closed
chrisuthe wants to merge 6 commits intotask/sonic-analysis-providerfrom
Closed
Add Sonic Similarity plugin for audio-based track similarity#1chrisuthe wants to merge 6 commits intotask/sonic-analysis-providerfrom
chrisuthe wants to merge 6 commits intotask/sonic-analysis-providerfrom
Conversation
8205e59 to
e46a759
Compare
729ed8a to
3d93152
Compare
e46a759 to
2f5dd0a
Compare
3d93152 to
b04676e
Compare
1c86642 to
19a32f3
Compare
b04676e to
fa65f09
Compare
19a32f3 to
dfb6426
Compare
…earch indexing, and debug UI Similarity search plugin with: - 17-dim vector assembly from AudioAnalysisData (9 required + 3 optional ML + key/mode + dynamics) - USearch HNSW index with per-provider separate indexes - Presets (balanced, vibe, party, genre_era, discover) with 7 feature groups including mood - Per-group distance breakdown, MMR diversity, recursive depth expansion - Provider comparison endpoint, analysis export with MBID/ISRC - Debug page with weight sliders, backfill, provider switching, CSV export - Configurable aa_provider_domain for switching between sonic/essentia analysis Adapted API paths to upstream: mass.streams.audio_analysis.*
fa65f09 to
7f5d82d
Compare
dfb6426 to
6db8ac4
Compare
Upstream merged arousal field on AudioAnalysisData. Add it to OPTIONAL_FIELDS in vectors.py alongside instrumentalness, valence, acousticness. Mood group now covers 4 dims (9-13). Defaults to 0.5 when absent (librosa/lite providers don't produce it).
…mpare/export/switching capabilities The rebase copied an early version of the plugin missing: - Audio Analysis Providers panel with config editing - Compare Providers endpoint and debug UI - Export analysis with MBID/ISRC, random pick, ML fields - Provider switching with separate indexes - AA provider config save API - Mood/genre display in export and compare views All features restored. API paths updated for upstream.
Arousal was in the vector (OPTIONAL_FIELDS) but missing from: - export_fields (export API) - compare_fields (compare providers API) - JS EXPORT_COLS (debug page export table/CSV) - JS compare fields (debug page compare view)
6db8ac4 to
83e77ad
Compare
chrisuthe
added a commit
that referenced
this pull request
Apr 28, 2026
Both bulk-read sites (_load_overlay_overrides and _rebuild_search_index) now call mass.streams.audio_analysis.list_rows_by_domain() instead of hitting mass.music.database directly. Per upstream rule that providers go through controllers, not the DB. Stacks on PR #1 (feat/sonic-analysis-provider-pr) for the controller helpers; once PR #1 lands and this rebases onto dev, the dependency becomes implicit.
chrisuthe
added a commit
that referenced
this pull request
Apr 28, 2026
_rebuild_index_from_database now calls mass.streams.audio_analysis.list_rows_by_domain() instead of hitting mass.music.database directly. Per upstream rule that providers go through controllers, not the DB. Stacks on PR #1 (feat/sonic-analysis-provider-pr) for the controller helpers; once PR #1 lands and this rebases onto dev, the dependency becomes implicit.
chrisuthe
added a commit
that referenced
this pull request
Apr 29, 2026
Both bulk-read sites (_load_overlay_overrides and _rebuild_search_index) now call mass.streams.audio_analysis.list_rows_by_domain() instead of hitting mass.music.database directly. Per upstream rule that providers go through controllers, not the DB. Stacks on PR #1 (feat/sonic-analysis-provider-pr) for the controller helpers; once PR #1 lands and this rebases onto dev, the dependency becomes implicit.
chrisuthe
added a commit
that referenced
this pull request
Apr 29, 2026
_rebuild_index_from_database now calls mass.streams.audio_analysis.list_rows_by_domain() instead of hitting mass.music.database directly. Per upstream rule that providers go through controllers, not the DB. Stacks on PR #1 (feat/sonic-analysis-provider-pr) for the controller helpers; once PR #1 lands and this rebases onto dev, the dependency becomes implicit.
chrisuthe
added a commit
that referenced
this pull request
Apr 30, 2026
Both bulk-read sites (_load_overlay_overrides and _rebuild_search_index) now call mass.streams.audio_analysis.get_audio_analysis_rows() instead of hitting mass.music.database directly. Per upstream rule that providers go through controllers, not the DB. Stacks on PR #1 (feat/sonic-analysis-provider-pr) for the controller helpers; once PR #1 lands and this rebases onto dev, the dependency becomes implicit.
chrisuthe
added a commit
that referenced
this pull request
Apr 30, 2026
_rebuild_index_from_database now calls mass.streams.audio_analysis.get_audio_analysis_rows() instead of hitting mass.music.database directly. Per upstream rule that providers go through controllers, not the DB. Stacks on PR #1 (feat/sonic-analysis-provider-pr) for the controller helpers; once PR #1 lands and this rebases onto dev, the dependency becomes implicit.
chrisuthe
added a commit
that referenced
this pull request
Apr 30, 2026
_rebuild_index_from_database now calls mass.streams.audio_analysis.get_audio_analysis_rows() instead of hitting mass.music.database directly. Per upstream rule that providers go through controllers, not the DB. Stacks on PR #1 (feat/sonic-analysis-provider-pr) for the controller helpers; once PR #1 lands and this rebases onto dev, the dependency becomes implicit.
chrisuthe
added a commit
that referenced
this pull request
Apr 30, 2026
Both bulk-read sites (_load_overlay_overrides and _rebuild_search_index) now call mass.streams.audio_analysis.get_audio_analysis_rows() instead of hitting mass.music.database directly. Per upstream rule that providers go through controllers, not the DB. Stacks on PR #1 (feat/sonic-analysis-provider-pr) for the controller helpers; once PR #1 lands and this rebases onto dev, the dependency becomes implicit.
chrisuthe
added a commit
that referenced
this pull request
Apr 30, 2026
Both bulk-read sites (_load_overlay_overrides and _rebuild_search_index) now call mass.streams.audio_analysis.get_audio_analysis_rows() instead of hitting mass.music.database directly. Per upstream rule that providers go through controllers, not the DB. Stacks on PR #1 (feat/sonic-analysis-provider-pr) for the controller helpers; once PR #1 lands and this rebases onto dev, the dependency becomes implicit.
chrisuthe
added a commit
that referenced
this pull request
May 4, 2026
Both bulk-read sites (_load_overlay_overrides and _rebuild_search_index) now call mass.streams.audio_analysis.get_audio_analysis_rows() instead of hitting mass.music.database directly. Per upstream rule that providers go through controllers, not the DB. Stacks on PR #1 (feat/sonic-analysis-provider-pr) for the controller helpers; once PR #1 lands and this rebases onto dev, the dependency becomes implicit.
chrisuthe
added a commit
that referenced
this pull request
May 4, 2026
_rebuild_index_from_database now calls mass.streams.audio_analysis.get_audio_analysis_rows() instead of hitting mass.music.database directly. Per upstream rule that providers go through controllers, not the DB. Stacks on PR #1 (feat/sonic-analysis-provider-pr) for the controller helpers; once PR #1 lands and this rebases onto dev, the dependency becomes implicit.
chrisuthe
added a commit
that referenced
this pull request
May 4, 2026
Both bulk-read sites (_load_overlay_overrides and _rebuild_search_index) now call mass.streams.audio_analysis.get_audio_analysis_rows() instead of hitting mass.music.database directly. Per upstream rule that providers go through controllers, not the DB. Stacks on PR #1 (feat/sonic-analysis-provider-pr) for the controller helpers; once PR #1 lands and this rebases onto dev, the dependency becomes implicit.
chrisuthe
added a commit
that referenced
this pull request
May 4, 2026
_rebuild_index_from_database now calls mass.streams.audio_analysis.get_audio_analysis_rows() instead of hitting mass.music.database directly. Per upstream rule that providers go through controllers, not the DB. Stacks on PR #1 (feat/sonic-analysis-provider-pr) for the controller helpers; once PR #1 lands and this rebases onto dev, the dependency becomes implicit.
chrisuthe
added a commit
that referenced
this pull request
May 5, 2026
Both bulk-read sites (_load_overlay_overrides and _rebuild_search_index) now call mass.streams.audio_analysis.get_audio_analysis_rows() instead of hitting mass.music.database directly. Per upstream rule that providers go through controllers, not the DB. Stacks on PR #1 (feat/sonic-analysis-provider-pr) for the controller helpers; once PR #1 lands and this rebases onto dev, the dependency becomes implicit.
chrisuthe
added a commit
that referenced
this pull request
May 5, 2026
_rebuild_index_from_database now calls mass.streams.audio_analysis.get_audio_analysis_rows() instead of hitting mass.music.database directly. Per upstream rule that providers go through controllers, not the DB. Stacks on PR #1 (feat/sonic-analysis-provider-pr) for the controller helpers; once PR #1 lands and this rebases onto dev, the dependency becomes implicit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sonic Similarity Plugin
This PR adds a similarity search plugin that builds a fast vector index over audio analysis data and exposes a general-purpose API for finding similar tracks.
Stacks on top of the sonic analysis provider (upstream PR music-assistant#3516).
How It Works
AudioAnalysisDatarows from the databasesonic_analysis/similarAPI for nearest-neighbor searchVector Schema (14 dimensions)
The plugin owns the vector definition — the provider just stores semantic fields, the plugin decides how to turn them into a searchable vector.
API:
sonic_analysis/similarSingle endpoint that covers every use case through parameters:
item_id="42", limit=1item_ids=["42"], limit=25, resolve=Trueitem_ids=[...last 5], depth=2, diversity=0.2item_ids=[a,b,c], blend_mode="union", limit=50filter_genres=["jazz"], filter_providers=["prov_local"]include_group_distances=TrueFeatures:
Code Organization
vectors.py— Vector schema definition, assembly fromAudioAnalysisData, normalization, corpus stats, weighted distance computation. Zero MA dependencies.similarity.py— Pure math: centroid blending, union merge, MMR diversity, recursive expansion. Zero MA dependencies.__init__.py— MA integration: USearch index management, API handlers, backfill, label mapping, metadata re-ranking, debug UITesting
test_vector_assembly.py— 38 tests for vector assembly, key encoding, normalization, corpus stats, weighted distancetest_group_distances.py— 8 tests for per-group distance breakdowntest_similarity.py— 17 tests for pure similarity functions (centroid, union, MMR, recursive expansion)test_plugin_api.py— 25 tests for parameter validation, weight parsing, filters, backward compatDebug UI
Browser-based debug console at
/sonic_analysis/debugwith WebSocket controls for searching similar tracks, triggering backfill, rebuilding the index, and tuning all parameters (depth, blend mode, diversity, weight sliders, presets).Dependencies
AudioAnalysisDatafields