Fish Audio
Ultra-fast open-source TTS and zero-shot voice cloning foundation model
About Fish Audio
Fish Audio is an open-source text-to-speech (TTS) and voice cloning platform built on state-of-the-art auto-regressive transformer models. Engineered to deliver sub-150ms voice generation latencies with natural human inflection, Fish Audio allows developers, creators, and voice agents to generate studio-grade audio across dozens of global languages from a single reference sample. The platform's zero-shot voice cloning engine requires as little as 10 to 30 seconds of clean reference audio to accurately replicate pitch, accent, emotional timbre, and speaking rhythm. With support for bilingual code-switching, emotional tone control (excited, whisper, solemn), and real-time streaming WebSockets, Fish Audio powers conversational voice AI applications, video game character voiceovers, and dynamic audiobook narration. Fish Audio provides both a managed cloud platform with intuitive web interfaces and self-hostable open-weight model checkpoints, giving enterprise developers full control over data privacy, on-premise compute deployment, and model fine-tuning.
Fish Audio is architected around a dual-component neural pipeline comprising an auto-regressive acoustic model and a high-fidelity vocoder. By tokenizing reference audio through a continuous vector-quantized latent space, the model preserves acoustic subtleties without requiring extensive multi-speaker training datasets. Developers interact with Fish Audio via high-performance REST APIs, Python/TypeScript SDKs, and streaming WebSockets designed for live conversational AI pipelines. The API provides granular parameters for temperature, top-p sampling, pitch scaling, speech rate modulation, and custom pronunciation lexicons via SSML tags. For high-throughput production environments, Fish Audio supports GPU-accelerated inference across NVIDIA TensorRT and vLLM-compatible backends, enabling parallel synthesis across thousands of concurrent agent dialogue streams.
Upload or record a 10–30 second clean voice sample into the Fish Audio dashboard or API.
Input your text script and select optional language, emotion, and pace modifiers.
The neural acoustic transformer processes text tokens and synthesizes raw audio waveforms.
Preview the generated speech in real-time or export lossless WAV/MP3 files.
Integrate into live applications via streaming WebSocket APIs for sub-200ms latency dialogue.
Capabilities & Features
Common Use Cases
Real-Time Conversational Voice AI Agents
Zero-Shot Voice Cloning for Video Dubbing
Dynamic Game NPC Voice Synthesis
Automated Audiobook and Podcast Production
Multi-Language Speech Translation
Frequently Asked Questions
What is Fish Audio?
Fish Audio is an open-source text-to-speech and zero-shot voice cloning engine designed for real-time speech generation and conversational AI agents.
How long of a sample is needed for voice cloning?
Fish Audio requires as little as 10 to 30 seconds of clean, background-noise-free audio to clone a speaker's unique vocal characteristics.
Can I self-host Fish Audio on my own servers?
Yes, Fish Audio provides open-source model weights on GitHub and Hugging Face, allowing teams to run full inference locally or on private cloud GPUs.
Free Plan
Free tier with 50,000 monthly synthesis credits, web playground access, and standard voice generation.
Paid Plan
Pro tiers from $12/month with unlimited zero-shot voice clones, commercial usage rights, priority GPU generation, and dedicated API access.
Direct link · Verified & reader-supported
Pros & Cons
Ultra-low latency sub-150ms speech synthesis ideal for conversational voice agents
High-accuracy zero-shot voice cloning from just 10–30 seconds of reference audio
Open-source model weights available for private on-premise infrastructure deployment
Native support for 30+ languages with seamless multi-language code-switching
Granular controls for emotion, pitch, cadence, and speech velocity
Developer-friendly REST API and streaming WebSocket interfaces
Audio quality depends heavily on the clarity of the reference sample provided
Self-hosting requires dedicated GPU hardware (NVIDIA RTX 3090 / A10G minimum)
Looking for the best alternatives to Fish Audio?
Side-by-side feature matrix, pricing models, and decision frameworks.
Alternatives to Fish Audio
Deep comparison hubElevenLabs
Ultra-realistic AI voices and voice cloning
A leading AI voice platform for generating ultra-realistic speech, voice cloning, and multilingual audio content.
Cartesia
Ultra-low-latency real-time voice generation for AI agents
Cartesia is an AI audio and voice intelligence company that powers conversational AI agents and interactive applications with ultra-low-latency, hyper-realistic voice synthesis. Its flagship State Space Model (SSM) architecture, Sonic, delivers human-like voice generation with sub-90ms time-to-first-audio latency. While traditional transformer-based text-to-speech models struggle with high latency and compute overhead, Cartesia's lightweight SSM architecture allows developers to build fluid, conversational voice bots that feel instantaneous and natural, without awkward pauses. Cartesia is used by developers, contact centers, game studios, and AI agent builders across the world to power real-time phone assistants, gaming NPCs, live transcription translation, and voice-enabled enterprise interfaces.
Deepgram
Real-time AI speech-to-text, text-to-speech, and voice agent API
Deepgram is an enterprise AI speech platform that provides world-class speech-to-text (STT), text-to-speech (TTS), and real-time voice agent APIs. Powered by end-to-end deep learning models like Nova-2 and Aura, Deepgram delivers industry-leading accuracy, sub-250ms latency, and high cost-efficiency for processing live conversational audio. Deepgram is the infrastructure backbone for next-generation conversational AI applications, customer service bots, meeting transcription engines, and autonomous voice agents. Its streaming WebSocket architecture allows models to transcribe noisy, multi-speaker phone audio in real time while simultaneously synthesizing natural, expressive voices with zero perceptible delay.
Moshi by Kyutai
Real-time full-duplex conversational voice AI model with sub-200ms latency
Moshi is an open-source real-time conversational voice AI foundation model developed by Kyutai, the non-profit AI research lab based in Paris. Engineered to revolutionize human-AI verbal communication, Moshi operates on a full-duplex architecture capable of listening, thinking, and speaking simultaneously with sub-200ms end-to-end latency. Unlike traditional voice assistants that chain separate Speech-to-Text (STT), Large Language Model (LLM), and Text-to-Speech (TTS) pipelines together, Moshi processes raw multi-stream audio natively as continuous speech tokens. This allows Moshi to understand emotional nuances, interrupt and be interrupted naturally, chuckle, whisper, and express genuine conversational timing. Moshi is fully open-source with openly accessible weights, training recipes, and inference code, serving as a foundational milestone for research in real-time spoken language modeling and multi-modal conversational systems.
LiveKit Agents
Open-source real-time WebRTC infrastructure for building ultra-low-latency voice and multimodal AI agents
LiveKit Agents is an open-source real-time communication framework engineered to build conversational voice, video, and multimodal AI agents with sub-500ms latency. Leveraging WebRTC, it connects speech-to-text (Deepgram, Whisper), LLMs (OpenAI, Anthropic), and text-to-speech (Cartesia, ElevenLabs) in a tightly synchronized bidirectional stream. From customer service avatars to interactive language tutors and hands-free coding copilots, LiveKit Agents provides the enterprise infrastructure for real-time human-AI interaction.
Udio
The most musical AI track generator
An AI music creation platform known for its exceptional audio fidelity and sophisticated musical arrangements across every imaginable genre.
Compare Fish Audio with Alternatives
Side-by-side feature, pricing, and pros & cons breakdowns