A support line that speaks Marathi, Hindi and English

Society chairmen, treasurers and residents call with app questions in Marathi, Hindi or English — often mixing them. A text bot does not help someone who would rather talk, and a generic voice bot either answers in the wrong language or confidently invents features the app does not have.

A Pipecat pipeline: audio in, Sarvam Saaras speech-to-text with the detected language, a transcript gate that drops background speech, a knowledge router that picks the one or two relevant topics from 30-odd feature files by keyword scoring (English, Hinglish, Marathi and Devanagari keywords — no vector store), a language switcher, an OpenAI model, then Sarvam Bulbul text-to-speech in the same Shubh voice for all three languages. The caller chooses a language at the start; after that it takes three consecutive transcripts in another language to switch, so one stray English word does not change the voice. When a caller interrupts, history records only what was actually spoken. The same pipeline runs in the browser over WebRTC and on a phone line via Exotel.

• Language choice at call start: Marathi, Hindi or English
• Sarvam Saaras speech-to-text with language detection
• Sarvam Bulbul text-to-speech, one 'Shubh' voice across languages
• Keyword retrieval over 30+ feature topics — no vector store
• Transcript gate that ignores background speech
• Interruption-aware history: records only what was heard
• Answers only from the knowledge base; declines anything else
• Same pipeline over WebRTC (browser) and Exotel (phone)

The agent runs end to end in the browser and answers app questions in all three languages from grounded knowledge, declining anything outside it. The Exotel phone line is being set up; the design was also ported into the ABDOS platform as its voice module. You can hear the same Sarvam voice in the demo on this site.