Deepgram

DeepgramSince 2021-06

Deepgram is an enterprise AI speech platform that provides world-class speech-to-text (STT), text-to-speech (TTS), and real-time voice agent APIs. Powered by end-to-end deep learning models like Nova-2 and Aura, Deepgram delivers industry-leading accuracy, sub-250ms latency, and high cost-efficiency for processing live conversational audio. Deepgram is the infrastructure backbone for next-generation conversational AI applications, customer service bots, meeting transcription engines, and autonomous voice agents. Its streaming WebSocket architecture allows models to transcribe noisy, multi-speaker phone audio in real time while simultaneously synthesizing natural, expressive voices with zero perceptible delay.

Deepgram replaced legacy heuristic speech pipelines with pure end-to-end deep learning neural networks trained on over 100,000 hours of diverse multi-lingual audio. Its Nova-2 STT model achieves lower Word Error Rates (WER) than traditional cloud providers while operating up to 40x faster and at a fraction of the cost. The platform offers specialized features including automatic language detection, smart formatting (punctuating numbers, dates, and acronyms), multichannel diarization, topic detection, and PII redaction. Deepgram also provides the Deepgram Voice Agent API, which bundles speech recognition, LLM reasoning, and ultra-low-latency voice synthesis into a single unified WebSocket connection.

Moshi by KyutaiSince 2024-07

Moshi is an open-source real-time conversational voice AI foundation model developed by Kyutai, the non-profit AI research lab based in Paris. Engineered to revolutionize human-AI verbal communication, Moshi operates on a full-duplex architecture capable of listening, thinking, and speaking simultaneously with sub-200ms end-to-end latency. Unlike traditional voice assistants that chain separate Speech-to-Text (STT), Large Language Model (LLM), and Text-to-Speech (TTS) pipelines together, Moshi processes raw multi-stream audio natively as continuous speech tokens. This allows Moshi to understand emotional nuances, interrupt and be interrupted naturally, chuckle, whisper, and express genuine conversational timing. Moshi is fully open-source with openly accessible weights, training recipes, and inference code, serving as a foundational milestone for research in real-time spoken language modeling and multi-modal conversational systems.

Moshi’s architecture is built on Helium, a 7-billion parameter language model coupled with Mimi, a cutting-edge neural audio codec that compresses 24kHz audio into multi-stream discrete tokens at just 1.1 kbps. By operating on a joint text-audio token stream, Moshi predicts both conversational text tokens and acoustic speech tokens in parallel. This end-to-end audio modeling eliminates the latency bottlenecks and acoustic information loss inherent in cascading STT-LLM-TTS pipelines. The Kyutai team provides full PyTorch and Rust-based inference engines optimized for local GPU execution, enabling real-time full-duplex conversations on consumer-grade hardware (NVIDIA RTX 4090 or Apple Silicon Mac).