🐸 Coqui TTS is a library for advanced Text-to-Speech generation.

🚀 Pretrained models in +1100 languages.

🛠️ Tools for training new models and fine-tuning existing models in any language.

📚 Utilities for dataset analysis and curation.

📣 Notice

This is a fork of the new Coqui TTS which is a fork of the old Coqui TTS that is now unmaintained repository that shutdown.

• New PyPI package: coqui-tts
• Make sure you aren't using the Python package TTS. You should be using Coqui-TTS.

From here on, I will not reference the old Coqui TTS (unless explicitly stated), so if I mention the "previous Coqui TTS" or the "old Coqui TTS", I am referring to idiap/coqui-ai-TTS.

My goal with this fork is to document my attempts to clone a voice using Coqui-TTS to be used on Home Assistant. I am running the cloned voice on my local desktop. My Home Assistant is running on a Raspberry Pi 5 with 8 GB of RAM. As of right now, they both run on the same network, but, in the future, I will attempt to have these operations be performed over the internet.

As a result, I have emphasized human-like speech synthesis, so any results or features that did not aid in this endeavor may have been removed from the old version of Coqui-TTS. I also did not use Docker, so a lot of that was removed. If you are interested in running Docker, feel free to refer to the previous Coqui-TTS repo.

My local desktop is running on the following components:

• OS: Ubuntu 24.04
• GPU: Sapphire AMD Radeon RX 7900 XT - 20 GB GDR6, AMD RDNA 3
• CPU: Intel i7 14th generation - 14700K with 20 cores (8 P-Cores + 12 E-Cores)
• CPU Cooler: Corsair iCUE Link Titan 360 RX RGB Liquid CPU Cooler - 360 mm AIO
• PSU: Corsair HX1000i Fully Modular Ultra-Low Noise
• Storage: WD_BLACK 4 TB SN850X NVMe M.2 SSD with Heatsink
• Memory: Corsair Vengeance RGB DDR5 RAM 96 GB (2x48GB) 5600 MHz CL40 XMP
• Motherboard: ASUS TUF Gaming Z790-PLus Wi-Fi LGA 1700 ATX

💬 Where to ask questions (idiap/coqui-ai-TTS)

If it's about my modifications of Coqui-TTS, you can message me directly on GitHub. If it's not about my modifications, please refer to idiap's version of coqui-ai-TTS.

🔗 Links and Resources

Type	Links
💼 Documentation	ReadTheDocs
💾 Installation	TTS/README.md
👩‍💻 Contributing	CONTRIBUTING.md
🚀 Released Models	Standard models and Fairseq models in ~1100 languages

For my specific changes, you can read about them in their corresponding *.md files on this fork.

Features

• High-performance text-to-speech and voice conversion models, see list below.
• Fast and efficient model training with detailed training logs on the terminal and Tensorboard.
• Support for multi-speaker and multilingual TTS.
• Released and ready-to-use models.
• Tools to curate TTS datasets under dataset_analysis/.
• Command line and Python APIs to use and test your models.
• Modular (but not too much) code base enabling easy implementation of new ideas.

Model Implementations

Spectrogram models

• Tacotron, Tacotron2
• Glow-TTS, SC-GlowTTS
• Speedy-Speech
• Align-TTS
• FastPitch
• FastSpeech, FastSpeech2
• Capacitron
• OverFlow
• Neural HMM TTS
• Delightful TTS

End-to-End Models

• XTTS
• VITS
• 🐸YourTTS
• 🐢Tortoise
• 🐶Bark

Vocoders

• MelGAN
• MultiBandMelGAN
• ParallelWaveGAN
• GAN-TTS discriminators
• WaveRNN
• WaveGrad
• HiFiGAN
• UnivNet

Voice Conversion

• FreeVC
• kNN-VC
• OpenVoice

Others

• Attention methods: Guided Attention, Forward Backward Decoding, Graves Attention, Double Decoder Consistency, Dynamic Convolutional Attention, Alignment Network
• Speaker encoders: GE2E, Angular Loss

You can also help us implement more models.

Installation

🐸TTS is tested on Ubuntu 24.04 with python >= 3.10, < 3.13, but should also work on Mac and Windows.

If you plan to code or train models, clone 🐸TTS and install it locally.

git clone https://github.com/idiap/coqui-ai-TTS

cd coqui-ai-TTS

pip install -e .

When I was starting up the repo, I received errors with the previous steps alone, so if you have issues as well, or if you are only interested in synthesizing speech with the pretrained 🐸TTS models, installing from PyPI is the easiest option.

pip install coqui-tts

Optional dependencies

The following extras allow the installation of optional dependencies:

Name	Description
all	All optional dependencies
notebooks	Dependencies only used in notebooks
server	Dependencies to run the TTS server
bn	Bangla G2P
ja	Japanese G2P
ko	Korean G2P
zh	Chinese G2P
languages	All language-specific dependencies

You can install extras with one of the following commands:

pip install coqui-tts[server,ja]

or

pip install -e .[server,ja]

Platforms

If you are on Ubuntu (Debian), you can also run the following commands for installation.

make system-deps

make install

A Note on GPU Usage

The original Coqui-TTS includes GPU capabilities for Nvidia, but, as you can see above, I am running an AMD GPU. As of right now, I haven't figured out how to have the software use my AMD GPU, but the software does recognize my AMD GPU, so that's a start. I will work on the implementation of the AMD GPU once everything else is up and running since the CPU support is sufficient for now.

GPU Usage is noted in the following files (maybe more):

• Markdown files:

  • /root/README.md
    • Mentions GPU usage in the context of running TTS models, e.g., using "cuda" as the device in Python API examples and references to GPU in hardware requirements.
  • /root/recipes/bel-alex73/README.md
    • Describes running training with GPU, e.g., using CUDA_VISIBLE_DEVICES and multi-GPU training.
  • /root/docs/source/training/training_a_model.md
    • Explains how to check available GPUs (nvidia-smi) and how to run multi-GPU training with CUDA_VISIBLE_DEVICES.
  • /root/docs/source/docker_images.md
    • Shows how to run Docker containers with GPU support using --gpus all and mentions checking CUDA version with nvidia-smi.

• Python scripts:

  • /root/TTS/bin/collect_env_info.py
    • Collects and prints CUDA (GPU) info, including device names and availability.
  • /root/TTS/tts/layers/xtts/gpt.py
    • Has parameters for number of GPUs and DeepSpeed inference.
  • /root/TTS/utils/distribute.py
    • Contains functions for initializing distributed training across multiple GPUs using PyTorch.
  • /root/TTS/tts/models/vits.py
    • Handles distributed sampling and batch samplers for multi-GPU training.
  • /root/TTS/bin/compute_statistics.py
    • Uses torch.cuda.is_available() to determine device for processing.
  • /root/TTS/demos/xtts_ft_demo/xtts_demo.py
    • Calls clear_gpu_cache() to manage GPU memory during dataset preprocessing.
  • /root/notebooks (e.g., ExtractTTSpectrogram.ipynb, notebooks/TestAttention.ipynb)
    • Use torch.cuda.is_available() and set CUDA_VISIBLE_DEVICES for GPU selection.

This is a good starting point to guide someone that is wanting to add AMD GPU support.

Synthesizing speech by 🐸TTS

🐍 Python API

Multi-speaker and multi-lingual model

import torch
from TTS.api import TTS

# Get device
device = "cuda" if torch.cuda.is_available() else "cpu"

# List available 🐸TTS models
print(TTS().list_models())

# Initialize TTS
tts = TTS("tts_models/multilingual/multi-dataset/xtts_v2").to(device)

# List speakers
print(tts.speakers)

# Run TTS
# ❗ XTTS supports both, but many models allow only one of the `speaker` and
# `speaker_wav` arguments

# TTS with list of amplitude values as output, clone the voice from `speaker_wav`
wav = tts.tts(
  text="Hello world!",
  speaker_wav="my/cloning/audio.wav",
  language="en"
)

# TTS to a file, use a preset speaker
tts.tts_to_file(
  text="Hello world!",
  speaker="Craig Gutsy",
  language="en",
  file_path="output.wav"
)

From version 0.27.0 you can cache cloned voices with a custom speaker ID, so you only need to pass audio files in speaker_wav once.

Single speaker model

# Initialize TTS with the target model name
tts = TTS("tts_models/de/thorsten/tacotron2-DDC").to(device)

# Run TTS
tts.tts_to_file(text="Ich bin eine Testnachricht.", file_path=OUTPUT_PATH)

Voice conversion (VC)

Converting the voice in source_wav to the voice of target_wav:

tts = TTS("voice_conversion_models/multilingual/vctk/freevc24").to("cuda")
tts.voice_conversion_to_file(
  source_wav="my/source.wav",
  target_wav="my/target.wav",
  file_path="output.wav"
)

Other available voice conversion models:

• voice_conversion_models/multilingual/multi-dataset/knnvc
• voice_conversion_models/multilingual/multi-dataset/openvoice_v1
• voice_conversion_models/multilingual/multi-dataset/openvoice_v2

For more details, see this dedicated page.

Voice cloning by combining single speaker TTS model with the default VC model

This way, you can clone voices by using any model in 🐸TTS. The FreeVC model is used for voice conversion after synthesizing speech.

tts = TTS("tts_models/de/thorsten/tacotron2-DDC")
tts.tts_with_vc_to_file(
    "Wie sage ich auf Italienisch, dass ich dich liebe?",
    speaker_wav="target/speaker.wav",
    file_path="output.wav"
)

TTS using Fairseq models in ~1100 languages 🤯

For Fairseq models, use the following name format: tts_models/<lang-iso_code>/fairseq/vits. You can find the language ISO codes here and learn about the Fairseq models here.

# TTS with fairseq models
api = TTS("tts_models/deu/fairseq/vits")
api.tts_to_file(
    "Wie sage ich auf Italienisch, dass ich dich liebe?",
    file_path="output.wav"
)

Command-line interface tts

Synthesize speech on the command line.

You can either use your trained model or choose a model from the provided list.

• List provided models:

  tts --list_models

• Get model information. Use the names obtained from --list_models.

  tts --model_info_by_name "<model_type>/<language>/<dataset>/<model_name>"

  For example:

  tts --model_info_by_name tts_models/tr/common-voice/glow-tts

  tts --model_info_by_name vocoder_models/en/ljspeech/hifigan_v2

Single speaker models

• Run TTS with the default model (tts_models/en/ljspeech/tacotron2-DDC):

  tts --text "Text for TTS" --out_path output/path/speech.wav

• Run TTS and pipe out the generated TTS wav file data:

  tts --text "Text for TTS" --pipe_out --out_path output/path/speech.wav | aplay

• Run a TTS model with its default vocoder model:

  tts --text "Text for TTS" \
      --model_name "<model_type>/<language>/<dataset>/<model_name>" \
      --out_path output/path/speech.wav

  For example:

  tts --text "Text for TTS" \
      --model_name "tts_models/en/ljspeech/glow-tts" \
      --out_path output/path/speech.wav

• Run with specific TTS and vocoder models from the list. Note that not every vocoder is compatible with every TTS model.

  tts --text "Text for TTS" \
      --model_name "<model_type>/<language>/<dataset>/<model_name>" \
      --vocoder_name "<model_type>/<language>/<dataset>/<model_name>" \
      --out_path output/path/speech.wav

  For example:

  tts --text "Text for TTS" \
      --model_name "tts_models/en/ljspeech/glow-tts" \
      --vocoder_name "vocoder_models/en/ljspeech/univnet" \
      --out_path output/path/speech.wav

• Run your own TTS model (using Griffin-Lim Vocoder):

  tts --text "Text for TTS" \
      --model_path path/to/model.pth \
      --config_path path/to/config.json \
      --out_path output/path/speech.wav

• Run your own TTS and Vocoder models:

  tts --text "Text for TTS" \
      --model_path path/to/model.pth \
      --config_path path/to/config.json \
      --out_path output/path/speech.wav \
      --vocoder_path path/to/vocoder.pth \
      --vocoder_config_path path/to/vocoder_config.json

Multi-speaker models

• List the available speakers and choose a <speaker_id> among them:

  tts --model_name "<language>/<dataset>/<model_name>"  --list_speaker_idxs

• Run the multi-speaker TTS model with the target speaker ID:

  tts --text "Text for TTS." --out_path output/path/speech.wav \
      --model_name "<language>/<dataset>/<model_name>"  --speaker_idx <speaker_id>

• Run your own multi-speaker TTS model:

  tts --text "Text for TTS" --out_path output/path/speech.wav \
      --model_path path/to/model.pth --config_path path/to/config.json \
      --speakers_file_path path/to/speaker.json --speaker_idx <speaker_id>

Voice conversion models

tts --out_path output/path/speech.wav --model_name "<language>/<dataset>/<model_name>" \
    --source_wav <path/to/speaker/wav> --target_wav <path/to/reference/wav>