A multitask tamil speech framework for ASR, TTS, and intelligent conversational agents
Аннотация
Tamil is a complex language with many different dialects and frequent mixing with English, especially in spoken conversations. Because of these unique features, building reliable AI systems that can understand and generate Tamil speech is particularly challenging. Most current Tamil speech technologies—like automatic speech recognition (ASR), conversation processing, and text-to-speech (TTS)—work separately and often don’t handle the variety of dialects or the natural flow from speech to speech. In this work, we introduce a multitask framework that brings these components together into one seamless system. Our model includes a speech recognition system that’s specifically fine-tuned to handle different Tamil dialects and code-mixed speech. For conversation, we use a smart retrieval system that pulls relevant information from domain-specific documents to answer queries accurately. For speech synthesis, we use an advanced neural TTS model (VITS-based) to produce natural-sounding Tamil speech. Our experiments show that this integrated approach leads to better transcription accuracy for different Tamil dialects and code-mixed speech, demonstrating its effectiveness in real-world scenarios.