In the sprawling landscape of India, where a multitude of languages echoes through its diverse population, the challenge of delivering digital services in native tongues looms large. With over 121 languages spoken across the country, ensuring equitable access to digital resources becomes a formidable task. In response to this linguistic diversity, the Indian government has embarked on a groundbreaking initiative, leveraging artificial intelligence (AI) through Bhashini – an innovative AI-led language translation system.
Bhashini serves as a catalyst in constructing open-source datasets tailored to local languages. This strategic effort aims to pave the way for the development of AI tools, ultimately facilitating the delivery of an array of services through digital platforms. In a nation where the linguistic tapestry is rich and complex, this initiative marks a significant stride towards inclusivity in the digital realm.
One of the primary challenges addressed by Bhashini is the limited coverage of natural language processing (NLP) for a majority of the 121 languages. The branch of AI responsible for enabling computers to comprehend text and spoken words is essential for effective communication with users. The exclusion of hundreds of millions of Indians from accessing crucial information highlights the urgency of broadening the scope of NLP to encompass a more extensive array of languages.
Kalika Bali, a principal researcher at Microsoft Research India, emphasizes the importance of AI tools catering to individuals who do not speak widely used global languages such as English, French, or Spanish. Recognizing the monumental task of collecting data for each Indian language, Bali suggests a pragmatic approach of creating layers on top of generative AI models like ChatGPT or Llama.
The training of AI models traditionally relies on datasets comprising written texts. However, the oral tradition predominant in several Indian languages presents a unique challenge, as textual records are not as abundant. Enter Bhashini, which introduces a crowdsourcing initiative, encouraging people to contribute sentences in diverse languages. This collaborative effort involves validating audio or text transcriptions provided by others, translating texts, and labeling images. By harnessing the collective wisdom of the crowd, Bhashini addresses the scarcity of textual data and contributes to the development of robust language datasets.
Pushpak Bhattacharyya, the head of the Computation for Indian Language Technology Lab in Mumbai, underscores the government’s resolute commitment to creating datasets. These datasets play a pivotal role in training large language models specifically designed for Indian languages. The tangible impact of these efforts is already evident in translation tools utilized in education, tourism, and legal proceedings.
In a parallel development, Meta, under the leadership of CEO Mark Zuckerberg, unveiled the SeamlessM4T model earlier this year. This AI-powered speech translation model boasts the capability to translate and transcribe speech across a staggering 100 languages. Zuckerberg envisions the model’s utility in various modes, including speech-to-text, text-to-speech, speech-to-speech, text-to-text translation, and speech recognition.
Meta’s SeamlessM4T model holds particular significance in scenarios where languages lack a widely used writing system or where textual resources are scarce. By harnessing the power of AI, this innovative model serves as a bridge, enabling effective communication and understanding of information in languages unfamiliar to users.
In conclusion, the confluence of government-led initiatives like Bhashini and cutting-edge AI models such as Meta’s SeamlessM4T underscores a transformative phase in India’s digital landscape. The commitment to linguistic diversity is not just a technological endeavor but a societal imperative. As these initiatives unfold, the prospect of a digitally inclusive India, where language is no longer a barrier to information access, comes into clearer focus. The journey towards linguistic equity in the digital age is well underway, guided by the innovative synergy of artificial intelligence and collective human effort.
