Image: ElevenLabs — official logo (via Wikimedia Commons, CC BY-SA 4.0).
TL;DR: ElevenLabs Interaction Engine explained: a new system that picks which voice and language model handles each part of a call — a cheap model for routine triage, a powerful one for the hard steps. Previewed at ElevenLabs' New York Summit on 11 November 2026 with 13 new products, but the Engine is a preview, not a finished product yet.
Key Highlights
- ElevenLabs unveiled 13 new products and upgrades at its New York Summit on 11 November 2026, repositioning itself from a voice-generation company into an enterprise conversational AI platform.
- The Interaction Engine, the headline announcement, is a previewed system that orchestrates voice models and large language models across a conversation, managing turn-taking, interruptions and which model answers each part.
- ElevenAI, a new workplace app, opened free early access on 11 November — it takes meeting notes, transcribes dictation and hands context between people and AI agents.
- Scribe v3, launched 10 November, cuts word errors by 27% against Scribe v2 across 102 languages on ElevenLabs' own tests, including Hindi, Tamil and Hinglish code-switching.
- About half of what was announced is preview, early access or roadmap rather than generally available — the honest read is a platform pitch to enterprise buyers, not a finished catalogue.
What happened at the ElevenLabs Summit?
ElevenLabs, the AI audio company founded in 2022 by Mati Staniszewski and Piotr Dabkowski, held its New York Summit on 11 November 2026 and announced 13 new products and upgrades in one package. The theme, in chief executive Mati Staniszewski's words, is building "the interaction layer for AI" — the software that sits between a caller and the models, and decides how a conversation should run. "Our goal is to build the interaction layer for AI, and today's announcements make it possible for businesses to use their data to craft experiences that resonate in their unique circumstances," Staniszewski said on stage.
The announcements fall into four groups: new models, agent upgrades, a workplace app, and an enterprise channel. On the model side, the company recapped three voice models shipped over the past month — Scribe v3 for transcription, Eleven v4 for expressive text-to-speech, and Eleven Realtime for speech-to-speech agents — and added Scribe v3 Medical, a transcription model tuned for healthcare terminology, with a financial-services model in development. On the agent side, ElevenAgents gains Memory, Routines and an Architect tool, plus the previewed Interaction Engine. For employees, there is ElevenAI. And for sales, there is a new Partner Program with more than 550 partners across 44 countries, a Marketplace, and an expanded NVIDIA partnership.
The Summit also showed the platform in real use: Yahoo's voice-powered AI assistant and Westgate's new front-desk agent were presented on stage as examples of voice agents already handling customer conversations. Staniszewski framed conversational agents as becoming "critical infrastructure for every enterprise, as essential as cloud or payments" within a few years.
ElevenLabs Interaction Engine Explained: What It Actually Does
The Interaction Engine is the part of the package most likely to change how voice agents are built, so it is worth explaining carefully. Today, a voice agent typically runs one language model for the whole call: the same expensive frontier model answers the greeting, the routine questions and the complicated refund. That wastes money on tokens, because not every step of a conversation needs the most powerful model.
The Interaction Engine sits between the caller and the models and orchestrates the conversation instead. It manages turn-taking and interruptions — knowing when to speak, when to stop, and when the caller has finished — and it assigns different language models to different parts of an exchange. A cheaper model can handle triage ("which department do you need?"), while a frontier model takes over for the complex part, such as working out a billing refund. The design aims to balance performance and reliability with token cost, and ElevenLabs says it can bring in the best voice models and LLMs for a specific use case rather than forcing one model to do everything.
Two honest caveats belong here. First, the Engine is a preview, not a released product — ElevenLabs has given no general-availability date. Second, the company has not disclosed independent benchmarks showing how much businesses would actually save or whether the system improves customer resolution rates compared with existing deployments. The idea is sound and the mechanism is clear; the savings are still a claim, not a measured result.
The 13 announcements at a glance
The full package is wide, so here it is grouped — with each item's real availability status, because that is the detail most coverage skips.
| Announcement | What it is | Status |
|---|---|---|
| Interaction Engine | Orchestrates voice models and LLMs across a conversation; manages turn-taking, interruptions and model routing | Previewed |
| Model customisation | Fine-tune selected open-source LLMs and ElevenLabs voice models on a business's own live deployment data | Begins in coming months; selected enterprises first |
| ElevenAI | Workplace app: meeting notes, summaries, action items, dictation, live call guidance, handoffs between people and agents | Free early access opened 11 Nov |
| ElevenAgents Memory | Agents remember returning authenticated customers — preferences and constraints stored as structured records | Forthcoming |
| ElevenAgents Routines | Automates multi-step work before, during and after calls — callbacks, reminders, CRM updates | Forthcoming |
| ElevenAgents Architect | Tool for designing agents | Announced at Summit |
| Scribe v3 | Speech-to-text for recorded audio; 102 languages, 27% fewer word errors than v2 on company tests | Live since 10 Nov |
| Scribe v3 Live | Real-time transcription with first-ever live speaker labels | Live since 10 Nov |
| Scribe v3 Medical | Transcription tuned for healthcare terminology | Launched at Summit |
| Eleven v4 / Eleven Realtime | Expressive text-to-speech (90+ languages) and speech-to-speech models for live agents | Shipped Sep–Nov; Realtime rollout completed 2 Nov |
| Campaign Manager | Outbound voice campaigns across phone calls, WhatsApp voice and WhatsApp messaging | Launched 9 Nov |
| Partner Program + Marketplace | 550+ partners in 44 countries; curated partner-built apps for healthcare, finance, insurance, education and retail | Live; purchases via existing commitments planned 2027 |
| NVIDIA expansion | Nemotron 3.5 Super VL in ElevenAgents; 15 MW of Vera Rubin compute planned; Poland e-Health landline agent | Announced at Summit |
The customisation programme is the most enterprise-heavy part of the package. Businesses will work directly with ElevenLabs researchers to tune models for their own customers and operating context, choosing optimisation targets such as customer satisfaction or reliability, and the models can keep improving as live interactions accumulate. The catch is access: it begins "in the coming months", rolls out to selected enterprises first, and no pricing has been disclosed — for now it is a roadmap item for large buyers, not something a mid-sized company can sign up for today.
Illustrative image: a studio microphone set up for voice recording. Image: Wikimedia Commons / Shixart1985, CC BY 2.0.
Scribe v3, Eleven v4 and Eleven Realtime: the new models
The three voice models shipped in the month before the Summit are the parts of the package you can actually use today, and Scribe v3 is the most concrete upgrade. Here is how they compare.
| Model | What it does | Key numbers (company tests) |
|---|---|---|
| Scribe v3 | Speech-to-text for recorded audio — meetings, podcasts, voice notes | 102 languages; 27% fewer word errors than v2 (WER 17.0% to 12.4%); up to 32 speakers labelled; $0.20 per audio hour |
| Scribe v3 Live | Real-time transcription for live calls and streams | 43% fewer semantic word errors than v2 Realtime; first live speaker diarization; ~150 ms turn detection; $0.22 per audio hour |
| Eleven v4 | Expressive text-to-speech that reads tone, pacing and context | 90+ languages; Turbo variant at about 100 ms median latency |
| Eleven Realtime | Speech-to-speech model for fast, natural voice agents | Rollout completed 2 Nov 2026 |
Two details in Scribe v3 matter for everyday use. First, it handles code-switching — speakers changing language mid-sentence — far better than v2, which is exactly how many Indian conversations sound. Second, it takes keyterm prompts of up to 1,000 terms, so you can feed it product names or technical words it should get right, and it tags audio events such as laughter and applause in the transcript. One caution on the headline numbers: the 27% figure comes from ElevenLabs' own version of the FLEURS test set — short, clean read speech — not from an independent benchmark, and the company says it will soon open-source a second test set of real meetings, agent calls and voice notes in 16 languages. Voice AI is also a crowded field — for a different take on the same problem, read our explainer on DeepL's Aure voice AI translation, which keeps your tone and voice across 8 languages.
Why it matters for India readers
India is where this announcement lands hardest, for three reasons. First, language coverage: Scribe v3 supports 102 languages including Hindi, Tamil and Urdu, with per-language error reductions of 29% for Tamil on Scribe v3 and 36% for Hindi on Scribe v3 Live, on the company's own tests, and it transcribes mixed English-Hindi in the same sentence without garbling the switch. That makes it directly usable for Indian meetings, podcasts, classrooms and customer-support calls — the same space where Chariot-1-TTS-Multilingual, India's homegrown speech model covering 11 Indian languages, launched the same day.
Second, data residency: Scribe v3 runs in the US by default, but ElevenLabs offers isolated EU, India and Singapore environments, plus a Zero Retention mode that deletes audio and text when a request completes. For Indian companies handling customer calls, that removes one of the usual objections to cloud voice AI.
Third, the price of entry is low. ElevenAI's free early access opened with the announcement, and the transcription models are priced per audio hour ($0.20 for Scribe v3, $0.22 for Live) rather than behind an enterprise contract — so an Indian creator, tutor or small business can try the real models before the enterprise features arrive. What is missing for India is any India-specific pricing in rupees, an India launch date for the enterprise customisation programme, or a local partner announcement; the 550-partner programme spans 44 countries, but no Indian partner was named at the Summit.
Should you care?
✅ Verdict: If you transcribe meetings, build voice agents or run customer support, yes — try Scribe v3 and the free ElevenAI early access now; they are live, cheap per hour, and strong on Indian languages. If you are waiting for the Interaction Engine or enterprise model customisation, wait: both are previews with no dates or pricing, and the savings claims are unproven. The real story of this Summit is direction, not delivery — ElevenLabs is becoming a platform, and the next six months will show whether the previews become products.
For developers, the practical move is small: existing integrations switch to the new models by changing a model ID to scribe_v3 or scribe_v3_live, and both models are already in the ElevenAPI and the ElevenLabs app. For businesses, the question to ask a vendor is not "do you use AI voices?" but "which model answers which part of the call, and what does that cost per conversation?" — which is precisely the question the Interaction Engine is built to answer. And the agent trend behind all of this is bigger than one company: Amazon's AWS emulator was built entirely by AI agents with zero human-written code, a sign of how fast agents are moving from demos to real work.
Price, availability and what is still a preview
Scribe v3 costs $0.20 per hour of audio through the API and Scribe v3 Live costs $0.22, unchanged from the previous generation's pricing — the accuracy gain comes without a price rise. ElevenAI is in free early access. Everything else is staged: the Interaction Engine is a preview, Memory and Routines are forthcoming with no dates, model customisation begins in the coming months for selected enterprises, and Marketplace purchases through existing ElevenLabs commitments are planned to start in 2027.
The NVIDIA angle is worth one careful note. ElevenLabs says its Nemotron 3.5 Super VL integration completed customer-support benchmark tasks at 70% lower cost than other leading open models — but that is the company's own benchmark, not an independently verified result, and no details of the test were published. The same Summit also announced 15 MW of new NVIDIA Vera Rubin compute capacity and an on-premises voice agent for Poland's national e-Health appointment system, which handles 40 million appointments a year with 10–20% no-show rates and now calls landline-only patients to remind them and help them reschedule, with citizen data kept inside government infrastructure via NVIDIA Confidential Computing.
FAQs
What is the ElevenLabs Interaction Engine?
It is a previewed system that orchestrates voice models and large language models across a live conversation. It manages turn-taking and interruptions and assigns different models to different parts of an exchange — a cheaper model for routine triage, a frontier model for complex steps — to balance performance, reliability and token cost.
Is ElevenAI free?
ElevenAI opened in free early access on 11 November 2026. It is a workplace app that takes meeting notes, transcribes dictation, produces summaries and action items, and hands context between people and AI agents. ElevenLabs has not announced what it will cost after early access ends.
What is Scribe v3 and how accurate is it?
Scribe v3 is ElevenLabs' speech-to-text model for recorded audio, launched 10 November 2026. On the company's own tests across 102 languages, it makes 27% fewer word errors than Scribe v2, with word error rate falling from 17.0% to 12.4%. The real-time variant, Scribe v3 Live, cuts semantic word errors by 43% and adds live speaker labels for the first time.
Does ElevenLabs support Hindi and other Indian languages?
Yes. Scribe v3 covers 102 languages including Hindi, Tamil and Urdu, handles mid-sentence switching between English and Hindi, and ElevenLabs offers an isolated India environment for data residency. The company's voice models also support 90+ languages for text-to-speech.
Takeaway
ElevenLabs' Summit was a statement of intent: voice AI is moving from generating speech to running whole conversations, with the Interaction Engine deciding which model does what. The usable parts — Scribe v3, ElevenAI's free early access — are worth trying today, especially for Indian languages; the platform promises are worth watching, not buying yet.
Try Scribe v3 on your next meeting recording and compare it with your current transcription tool — the 27% error reduction is easy to test yourself. And if you work with Indian-language audio, read our Chariot-1-TTS-Multilingual explainer to see how a homegrown Indian model tackles the same problem.
Sources
ElevenLabs launches 13 new products and upgrades — Mumbrella, 12 November 2026 (carries ElevenLabs' own Summit announcement in full)
ElevenLabs builds out the interaction layer for AI — CXO Digital Pulse, 12 November 2026 (company press release via Edelman)
ElevenLabs Unveils Model Customization, ElevenAI and Partner Program — Unite.AI, 11 November 2026
ElevenLabs Expands Its AI Agents With Memory and Automation — CIOL, 12 November 2026
ElevenLabs Scribe v3: 27% fewer errors, same price — DataNorth, 11 November 2026
ElevenLabs Scribe v3 Cuts Word Errors 27%, Live Adds Speaker Labels — Nvdium, 10–11 November 2026
ElevenLabs Unveils AI Interaction Engine and Customization Tools — Blockchain.News, 11 November 2026
Comments
Post a Comment