Repositories list

• LP-CFM

  Public
  Official implementation of "LP-CFM: Perceptual Invariance-Aware Conditional Flow Matching for Speech Modeling" (ICASSP 2026)

  Python

  •00 forks in kaistmm/LP-CFM•00 stars in kaistmm/LP-CFM•00 issues in kaistmm/LP-CFM•00 pull requests in kaistmm/LP-CFM•Updated Oct 25, 2026Oct 25, 2026

• JAVIS

  Public

  kaistmm/JAVIS’s past year of commit activity
  Official code for JAVIS: Joint Audio-Visual Interaction and Streaming for Real-Time Conversational Agents

  Python

  •

  Apache License 2.0

  •00 forks in kaistmm/JAVIS•11 star in kaistmm/JAVIS•00 issues in kaistmm/JAVIS•00 pull requests in kaistmm/JAVIS•Updated Oct 22, 2026Oct 22, 2026

• Plug-and-Steer

  Public

  kaistmm/Plug-and-Steer’s past year of commit activity
  [Interspeech 2026] Plug-and-Steer: Decoupling Separation and Selection in Audio-Visual Target Speaker Extraction

  Python

  •00 forks in kaistmm/Plug-and-Steer•44 stars in kaistmm/Plug-and-Steer•11 issue in kaistmm/Plug-and-Steer•00 pull requests in kaistmm/Plug-and-Steer•Updated Oct 13, 2026Oct 13, 2026

• VoxMM

  Public

  kaistmm/VoxMM’s past year of commit activity
  Python

  •00 forks in kaistmm/VoxMM•2323 stars in kaistmm/VoxMM•11 issue in kaistmm/VoxMM•00 pull requests in kaistmm/VoxMM•Updated Oct 6, 2026Oct 6, 2026

• Who-Says-What

  Public

  kaistmm/Who-Says-What’s past year of commit activity
  [NeurIPS 2026] Who Says What: Symbolic Trimodal Binding Mechanisms in Audio-Visual LLMs

  Python

  •00 forks in kaistmm/Who-Says-What•55 stars in kaistmm/Who-Says-What•00 issues in kaistmm/Who-Says-What•00 pull requests in kaistmm/Who-Says-What•Updated Oct 4, 2026Oct 4, 2026

• FiTS

  Public

  kaistmm/FiTS’s past year of commit activity
  [NeurIPS 2026 Spotlight] FiTS: Interpretable Spiking Neurons via Frequency Selectivity and Temporal Shaping

  Python

  •

  Apache License 2.0

  •00 forks in kaistmm/FiTS•22 stars in kaistmm/FiTS•00 issues in kaistmm/FiTS•00 pull requests in kaistmm/FiTS•Updated Sep 30, 2026Sep 30, 2026

• crossmodal-hub

  Public
  [ICML 2026] Probing Cross-modal Information Hubs in Audio-Visual LLMs

  interpretabilitycausal-analysisavllm

  interpretabilitycausal-analysisavllm

  Python

  •00 forks in kaistmm/crossmodal-hub•1010 stars in kaistmm/crossmodal-hub•00 issues in kaistmm/crossmodal-hub•00 pull requests in kaistmm/crossmodal-hub•Updated Sep 28, 2026Sep 28, 2026

• OutRo

  Public
  [NeurIPS 2026] On the Nature of Attention Sink that Shapes Decoding Strategy in Omni-LLMs

  Python

  •

  Apache License 2.0

  •00 forks in kaistmm/OutRo•33 stars in kaistmm/OutRo•00 issues in kaistmm/OutRo•00 pull requests in kaistmm/OutRo•Updated Sep 26, 2026Sep 26, 2026

• AlignDiT

  Public
  [ACM MM 2025] AlignDiT: Multimodal Aligned Diffusion Transformer for Synchronized Speech Generation

  Python

  •

  MIT License

  •22 forks in kaistmm/AlignDiT•2424 stars in kaistmm/AlignDiT•22 issues in kaistmm/AlignDiT•00 pull requests in kaistmm/AlignDiT•Updated Sep 17, 2026Sep 17, 2026

• OmniKVQuant

  Public

  kaistmm/OmniKVQuant’s past year of commit activity
  OmniKVQuant: KV Cache Quantization for Omni-LLMs

  Python

  •

  Apache License 2.0

  •00 forks in kaistmm/OmniKVQuant•66 stars in kaistmm/OmniKVQuant•11 issue in kaistmm/OmniKVQuant•00 pull requests in kaistmm/OmniKVQuant•Updated Sep 11, 2026Sep 11, 2026

• SeeandSniff

  Public
  [ECCV 2026 Oral] Official Pytorch implementation for See & Sniff: Learning Visuo-Olfactory Representations

  visionsmellmultimodal+ 1

  visionsmellmultimodalvisuo-olfactory

  Python

  •

  MIT License

  •44 forks in kaistmm/SeeandSniff•3535 stars in kaistmm/SeeandSniff•00 issues in kaistmm/SeeandSniff•00 pull requests in kaistmm/SeeandSniff•Updated Aug 13, 2026Aug 13, 2026

• Dub-S2ST

  Public
  [EMNLP 2025 Findings] Dub-S2ST: Textless Speech-to-Speech Translation for Seamless Dubbing

  Python

  •

  MIT License

  •00 forks in kaistmm/Dub-S2ST•11 star in kaistmm/Dub-S2ST•00 issues in kaistmm/Dub-S2ST•00 pull requests in kaistmm/Dub-S2ST•Updated Jul 12, 2026Jul 12, 2026

• SeeingThroughTouch

  Public

  kaistmm/SeeingThroughTouch’s past year of commit activity
  [CVPR 2026] Official Pytorch implementation for Seeing Through Touch: Tactile-Driven Visual Localization of Material Regions

  tactilemultimodalvisuo-tactile

  tactilemultimodalvisuo-tactile

  Python

  •

  Apache License 2.0

  •00 forks in kaistmm/SeeingThroughTouch•1010 stars in kaistmm/SeeingThroughTouch•11 issue in kaistmm/SeeingThroughTouch•00 pull requests in kaistmm/SeeingThroughTouch•Updated Jun 29, 2026Jun 29, 2026

• ITS-AVGen

  Public

  kaistmm/ITS-AVGen’s past year of commit activity
  [TMLR] Inference-Time Scaling for Joint Audio–Video Generation

  Python

  •00 forks in kaistmm/ITS-AVGen•44 stars in kaistmm/ITS-AVGen•00 issues in kaistmm/ITS-AVGen•00 pull requests in kaistmm/ITS-AVGen•Updated Jun 8, 2026Jun 8, 2026

• ITS-AVGen-LTX2

  Public
  [TMLR] Inference-Time Scaling for Joint Audio–Video Generation

  Python

  •

  Other

  •00 forks in kaistmm/ITS-AVGen-LTX2•44 stars in kaistmm/ITS-AVGen-LTX2•00 issues in kaistmm/ITS-AVGen-LTX2•00 pull requests in kaistmm/ITS-AVGen-LTX2•Updated Jun 8, 2026Jun 8, 2026

• mamtra_inference

  Public

  kaistmm/mamtra_inference’s past year of commit activity
  [Interspeech 2026] The official inference code of MamTra

  Python

  •11 fork in kaistmm/mamtra_inference•1212 stars in kaistmm/mamtra_inference•00 issues in kaistmm/mamtra_inference•00 pull requests in kaistmm/mamtra_inference•Updated May 28, 2026May 28, 2026

• IITP_SmartHome_LightEmoMLLM

  Public

  kaistmm/IITP_SmartHome_LightEmoMLLM’s past year of commit activity
  Python

  •

  Other

  •00 forks in kaistmm/IITP_SmartHome_LightEmoMLLM•33 stars in kaistmm/IITP_SmartHome_LightEmoMLLM•00 issues in kaistmm/IITP_SmartHome_LightEmoMLLM•00 pull requests in kaistmm/IITP_SmartHome_LightEmoMLLM•Updated Mar 9, 2026Mar 9, 2026

• IITP_SmartHome_VideoCollection

  Public
  Python

  •

  GNU Affero General Public License v3.0

  •00 forks in kaistmm/IITP_SmartHome_VideoCollection•44 stars in kaistmm/IITP_SmartHome_VideoCollection•00 issues in kaistmm/IITP_SmartHome_VideoCollection•00 pull requests in kaistmm/IITP_SmartHome_VideoCollection•Updated Dec 23, 2025Dec 23, 2025

• seed-pytorch

  Public
  [INTERSPEECH 2025] Official code for "SEED: Speaker Embedding Enhancement Diffusion Model"

  audiodeep-learningrepresentation-learning+ 7

  audiodeep-learningrepresentation-learningspeaker-recognitionspeaker-verificationspeech-processingaudio-processingin-the-wilddiffusion-modelsembedding-enhancement

  Python

  •33 forks in kaistmm/seed-pytorch•6262 stars in kaistmm/seed-pytorch•11 issue in kaistmm/seed-pytorch•00 pull requests in kaistmm/seed-pytorch•Updated Nov 3, 2025Nov 3, 2025

• AVCD

  Public
  [NeurIPS 2025] AVCD: Mitigating Hallucinations in Audio-Visual Large Language Models through Contrastive Decoding

  Python

  •

  Apache License 2.0

  •11 fork in kaistmm/AVCD•2727 stars in kaistmm/AVCD•11 issue in kaistmm/AVCD•00 pull requests in kaistmm/AVCD•Updated Nov 3, 2025Nov 3, 2025

• Diffusion-Link

  Public
  00 forks in kaistmm/Diffusion-Link•00 stars in kaistmm/Diffusion-Link•00 issues in kaistmm/Diffusion-Link•00 pull requests in kaistmm/Diffusion-Link•Updated Oct 14, 2025Oct 14, 2025

• voxsim_trainer

  Public
  [INTERSPEECH 2024] Official code for VoxSim: A perceptual voice similarity dataset

  speech-synthesisspeaker-recognitionvoice-conversion

  speech-synthesisspeaker-recognitionvoice-conversion

  Python

  •00 forks in kaistmm/voxsim_trainer•2424 stars in kaistmm/voxsim_trainer•00 issues in kaistmm/voxsim_trainer•00 pull requests in kaistmm/voxsim_trainer•Updated Sep 29, 2025Sep 29, 2025

• SlowFastSign

  Public
  [ICASSP 2024] Official code for Slowfast Network for Continuous Sign Language Recognition

  sign-language-recognitioncontinuous-sign-language

  sign-language-recognitioncontinuous-sign-language

  Python

  •1717 forks in kaistmm/SlowFastSign•6666 stars in kaistmm/SlowFastSign•1212 issues in kaistmm/SlowFastSign•00 pull requests in kaistmm/SlowFastSign•Updated Jul 4, 2025Jul 4, 2025

• V2SFlow

  Public
  [ICASSP 2025] V2SFlow: Video-to-Speech Generation with Speech Decomposition and Rectified Flow

  Python

  •22 forks in kaistmm/V2SFlow•2121 stars in kaistmm/V2SFlow•22 issues in kaistmm/V2SFlow•00 pull requests in kaistmm/V2SFlow•Updated Jun 3, 2025Jun 3, 2025

• AdaptVC

  Public
  Python

  •55 forks in kaistmm/AdaptVC•1717 stars in kaistmm/AdaptVC•11 issue in kaistmm/AdaptVC•00 pull requests in kaistmm/AdaptVC•Updated Jun 2, 2025Jun 2, 2025

• SSLalignment

  Public
  Python

  •33 forks in kaistmm/SSLalignment•3939 stars in kaistmm/SSLalignment•11 issue in kaistmm/SSLalignment•00 pull requests in kaistmm/SSLalignment•Updated May 28, 2025May 28, 2025

• AVDiffuSS

  Public
  Official implementation of AVDiffuSS (ICASSP 2024).

  Python

  •11 fork in kaistmm/AVDiffuSS•1515 stars in kaistmm/AVDiffuSS•00 issues in kaistmm/AVDiffuSS•00 pull requests in kaistmm/AVDiffuSS•Updated May 19, 2025May 19, 2025

• TalkNCE

  Public
  Official implementation of TalkNCE (ICASSP 2024).

  Python

  •

  MIT License

  •11 fork in kaistmm/TalkNCE•1919 stars in kaistmm/TalkNCE•22 issues in kaistmm/TalkNCE•00 pull requests in kaistmm/TalkNCE•Updated Apr 30, 2025Apr 30, 2025

• VoiceDiT

  Public
  [ICASSP2025] Official code for VoiceDiT: Dual-Condition Diffusion Transformer for Environment-Aware Speech Synthesis

  Python

  •77 forks in kaistmm/VoiceDiT•5353 stars in kaistmm/VoiceDiT•22 issues in kaistmm/VoiceDiT•00 pull requests in kaistmm/VoiceDiT•Updated Apr 9, 2025Apr 9, 2025

• Audio-Mamba-AuM

  Public
  Official Implementation of the work "Audio Mamba: Bidirectional State Space Model for Audio Representation Learning"

  audiodeep-learningpytorch+ 7

  audiodeep-learningpytorchrepresentation-learningaudio-classificationmambastate-space-modelspeaker-identificationspeech-classificationaudio-mamba

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •2323 forks in kaistmm/Audio-Mamba-AuM•176176 stars in kaistmm/Audio-Mamba-AuM•22 issues in kaistmm/Audio-Mamba-AuM•00 pull requests in kaistmm/Audio-Mamba-AuM•Updated Nov 24, 2024Nov 24, 2024