Fish Audio

Fish AudioSince 2024-03

Fish Audio is an open-source text-to-speech (TTS) and voice cloning platform built on state-of-the-art auto-regressive transformer models. Engineered to deliver sub-150ms voice generation latencies with natural human inflection, Fish Audio allows developers, creators, and voice agents to generate studio-grade audio across dozens of global languages from a single reference sample. The platform's zero-shot voice cloning engine requires as little as 10 to 30 seconds of clean reference audio to accurately replicate pitch, accent, emotional timbre, and speaking rhythm. With support for bilingual code-switching, emotional tone control (excited, whisper, solemn), and real-time streaming WebSockets, Fish Audio powers conversational voice AI applications, video game character voiceovers, and dynamic audiobook narration. Fish Audio provides both a managed cloud platform with intuitive web interfaces and self-hostable open-weight model checkpoints, giving enterprise developers full control over data privacy, on-premise compute deployment, and model fine-tuning.

Fish Audio is architected around a dual-component neural pipeline comprising an auto-regressive acoustic model and a high-fidelity vocoder. By tokenizing reference audio through a continuous vector-quantized latent space, the model preserves acoustic subtleties without requiring extensive multi-speaker training datasets. Developers interact with Fish Audio via high-performance REST APIs, Python/TypeScript SDKs, and streaming WebSockets designed for live conversational AI pipelines. The API provides granular parameters for temperature, top-p sampling, pitch scaling, speech rate modulation, and custom pronunciation lexicons via SSML tags. For high-throughput production environments, Fish Audio supports GPU-accelerated inference across NVIDIA TensorRT and vLLM-compatible backends, enabling parallel synthesis across thousands of concurrent agent dialogue streams.

UdioSince 2024-04

An AI music creation platform known for its exceptional audio fidelity and sophisticated musical arrangements across every imaginable genre.

Founded by former Google DeepMind researchers. Focuses on high-quality stereo sound and creative control.