The GPT-3 Moment for Voice Is Here

CEO Tobi Lütke of Shopify recently tweeted, “Sesame's voice model is absolutely insane.” Sesame’s AI voices are turning heads—and for good reason. Shopify CEO Tobi Lütke called them “absolutely insane,” and anyone who’s tried the demo seems to agree. Unlike traditional AI speech, which often feels robotic and flat, Sesame’s models breathe, pause, and even throw in casual filler words like “like” and “you know.” The result? Conversations that sound human.

Founded by Brendan Iribe, Ankit Kumar, and Ryan Brown, Sesame backed by Andreessen Horowitz, is betting big on a bold idea that the future of AI isn’t in screens, it’s in sound.

Sesame’s AI technology is the Conversational Speech Model (CSM-1B), a 1-billion-parameter system designed to generate natural-sounding speech that sets it apart. Unlike traditional text-to-speech models that synthesize audio in a rigid, step-by-step manner, CSM-1B operates end-to-end, incorporating both text and voice context into a single model.

CSM-1B is built using a Meta Llama backbone, paired with an advanced audio decoder. It employs residual vector quantization (RVQ) to convert input into discrete audio tokens, which are then reconstructed into human-like speech. This technique is also utilized in Google’s SoundStream and Meta’s Encodec, indicating a growing trend in AI-generated voice synthesis.