MamTra: A Hybrid Mamba-Transformer Backbone for Speech Synthesis

   

---

Authors

Tan Dat Nguyen1	Sangmin Bae1	Joon Son Chung1	Ji-Hoon Kim2†

¹ Korea Advanced Institute of Science and Technology (KAIST), South Korea
² Chung-Ang University, South Korea
^† Corresponding author  ·  tandat.kaist@kaist.ac.kr

---

Overview

MamTra introduces a hybrid Mamba-Transformer backbone for text-to-speech synthesis, combining the linear-time sequence modeling of Mamba with the global context modeling of Transformer attention. The result is a system that achieves strong synthesis quality while remaining efficient and hardware-friendly.

Note: MamTra depends on Mamba and associated CUDA extensions, which may require careful environment setup — particularly on machines with non-standard CUDA configurations. If you encounter installation issues, please open a GitHub issue and we will do our best to help.

---

🚀 Quickstart

1. Environment Setup

Python 3.10 is required due to a dependency on CosyVoice's ttsfrd. We recommend isolating the environment to avoid package conflicts:

conda create -n mamtra python=3.10 -y
conda activate mamtra

CUDA 11.6 or later is required to build the Mamba CUDA kernels. If your system CUDA version is too old, you can install a compatible version via conda:

conda install nvidia/label/cuda-11.8.0::cuda-nvcc
conda install nvidia/label/cuda-11.8.0::cuda

Install CuDNN alongside:

conda install anaconda::cudnn
# alternative if the above fails:
conda install -c conda-forge cudnn

When using the conda-managed CUDA, make sure to set LD_LIBRARY_PATH accordingly — see scripts/uv_inference_hybrid_11.sh for an example.

2. Install Dependencies

We recommend uv for dependency management. It handles the version conflicts that commonly arise when installing Mamba, causal_conv1d, and CosyVoice together:

pip install -U uv
uv sync
uv sync --extra compile   # required for the hybrid model

---

Inference

0. Download Checkpoint

The MamTra 1:1 checkpoint is available on HuggingFace. Download it with:

huggingface-cli download blueskyheaven/mamtra11 \
  --local-dir huggingface_model_dir \
  --local-dir-use-symlinks False

Additional model variants will be released in the near future.

Once downloaded, set the path as the MODEL_DIR variable in the inference script of your choice.

1. Single Sample Inference

bash scripts/single_inference.sh

2. Batch Inference

bash scripts/uv_inference_hybrid_11.sh

---

Citation

If you find MamTra useful in your research, please consider citing:

@article{nguyen2025mamtra,
  title   = {MamTra: A Hybrid Mamba-Transformer Backbone for Speech Synthesis},
  author  = {Nguyen, Tan Dat and Bae, Sangmin and Chung, Joon Son and Kim, Ji-Hoon},
  journal = {arXiv preprint arXiv:2603.12342},
  year    = {2025}
}