Devnagri Black Bird tops Voice of India speech benchmark across 15 languages

Devnagri AI says its new automatic speech recognition model, Black Bird, delivered the lowest average error on the Voice of India benchmark across 15 Indian languages. The result matters for banks, government services and commerce companies that need accurate voice automation in noisy, real-world calls.

Why it matters: - Devnagri Black Bird’s lead on a real-call benchmark suggests Indian enterprises may get more reliable speech automation in the languages customers actually use. - The result is especially relevant for banking, government services and digital commerce, where a single transcription error can break onboarding, collections, support or compliance workflows. - Devnagri said the benchmark performance reflects conditions in Indian contact centres and voice-agent deployments, not just clean lab audio.

What happened: - Devnagri AI announced that its new automatic speech recognition model, Devnagri Black Bird, ranked first on the Voice of India benchmark. - The benchmark is an independent Indian ASR test set built by AI4Bharat at IIT Madras with Josh Talks. - Devnagri Black Bird posted an average Open Indic Word Error Rate of 8.7 across 15 Indian languages. - Devnagri said the evaluation was based on its September 2026 testing.

The details: - Devnagri said Black Bird’s 8.7 average error was below Sarvam Saaras V3 at 10.7, Google’s Gemini 3 Pro at 21.1 and Gemini 3 Flash at 23.1. - Devnagri said Black Bird cut average errors by about 25% versus its earlier model, Shivaani ASR. - The company said Black Bird led all nine systems tested and ranked first in all 15 languages, including lower-resource languages such as Bhojpuri and Maithili. - Voice of India was dated April 21, 2026 and was built to reflect how people actually speak in India. - The test set uses 536 hours of unscripted telephonic conversations across 15 Indian languages. - The dataset includes 306,230 utterances from 36,691 native speakers. - The cohorts were balanced for gender and age. - The benchmark accepts valid spelling and transliteration variants rather than penalizing natural speech. - Devnagri said it evaluated all nine systems by calling each production API and scoring outputs against the same references under identical default settings. - Devnagri said the test set is closed to prevent overfitting and that model versions and logs were retained so results can be verified. - Devnagri said the results for competing systems were measured by the company using publicly accessible production APIs and are not vendor-published results.

Between the lines: - The benchmark favors messy, code-mixed, noisy phone calls over polished speech, which is where many enterprise speech products struggle. - A strong result in low-resource languages matters because global systems often perform worse outside Hindi and other high-volume languages. - Devnagri is positioning Black Bird as infrastructure for regulated enterprises rather than a consumer speech product. - The comparison with publicly accessible production APIs gives Devnagri a direct point of reference, but it is still a company-run test rather than a vendor-neutral leaderboard.

What's next: - Devnagri said it will continue improving Black Bird. - The company said enterprises in BFSI, government, retail and digital commerce can use the model for multilingual voice workflows. - Devnagri is pushing the result as a higher expectation for live-call speech AI in the Indian market.

The bottom line: - Devnagri is betting that the next round of enterprise voice adoption in India will be won by models that handle regional languages, code-switching and noisy telephony better than global general-purpose systems.