Fish Audio

Fish AudioSince 2024-03

Fish Audio is an open-source text-to-speech (TTS) and voice cloning platform built on state-of-the-art auto-regressive transformer models. Engineered to deliver sub-150ms voice generation latencies with natural human inflection, Fish Audio allows developers, creators, and voice agents to generate studio-grade audio across dozens of global languages from a single reference sample. The platform's zero-shot voice cloning engine requires as little as 10 to 30 seconds of clean reference audio to accurately replicate pitch, accent, emotional timbre, and speaking rhythm. With support for bilingual code-switching, emotional tone control (excited, whisper, solemn), and real-time streaming WebSockets, Fish Audio powers conversational voice AI applications, video game character voiceovers, and dynamic audiobook narration. Fish Audio provides both a managed cloud platform with intuitive web interfaces and self-hostable open-weight model checkpoints, giving enterprise developers full control over data privacy, on-premise compute deployment, and model fine-tuning.

Fish Audio is architected around a dual-component neural pipeline comprising an auto-regressive acoustic model and a high-fidelity vocoder. By tokenizing reference audio through a continuous vector-quantized latent space, the model preserves acoustic subtleties without requiring extensive multi-speaker training datasets. Developers interact with Fish Audio via high-performance REST APIs, Python/TypeScript SDKs, and streaming WebSockets designed for live conversational AI pipelines. The API provides granular parameters for temperature, top-p sampling, pitch scaling, speech rate modulation, and custom pronunciation lexicons via SSML tags. For high-throughput production environments, Fish Audio supports GPU-accelerated inference across NVIDIA TensorRT and vLLM-compatible backends, enabling parallel synthesis across thousands of concurrent agent dialogue streams.

LiveKit AgentsSince 2026-08

LiveKit Agents is an open-source real-time communication framework engineered to build conversational voice, video, and multimodal AI agents with sub-500ms latency. Leveraging WebRTC, it connects speech-to-text (Deepgram, Whisper), LLMs (OpenAI, Anthropic), and text-to-speech (Cartesia, ElevenLabs) in a tightly synchronized bidirectional stream. From customer service avatars to interactive language tutors and hands-free coding copilots, LiveKit Agents provides the enterprise infrastructure for real-time human-AI interaction.

Building real-time voice agents requires solving audio interruption, packet jitter, and latency stacking. LiveKit Agents abstracts these challenges with native Voice Activity Detection (VAD), turn-taking management, and edge-routed audio pipelines. Available in Python and Node.js with client SDKs across React, iOS, Android, and Flutter, LiveKit enables developers to self-host their agent backend or deploy onto LiveKit Cloud.