IIT Madras-backed AI4Bharat has unveiled the open source Indic LLM Arena, a crowd-sourced platform that benchmarks global AI models for Indian languages.
AI4Bharat, a research initiative backed by IIT Madras, has launched the Indic LLM Arena, an open-source, crowd-sourced benchmarking platform designed to evaluate global large language models (LLMs) for Indian users. The initiative aims to create a trusted and transparent national benchmark for how AI systems understand, respond, and behave across India’s diverse linguistic and cultural spectrum.
Most global AI leaderboards remain English-centric, often overlooking how models perform on Indian languages or code-mixed inputs such as Hinglish and Tanglish. The Indic LLM Arena bridges this gap by evaluating models across three critical pillars, language, context, and safety. It tests whether AI systems comprehend multilingual inputs, respond appropriately in localised settings, and adhere to India’s social sensitivities and fairness norms.
The platform uses a human-in-the-loop system, where users can type, speak, or transliterate prompts in Indian languages, receive responses from two anonymous AI models, and vote for the better one. These collective evaluations feed into statistically robust rankings, identifying the most effective LLMs for Indian use cases.
AI4Bharat describes the Arena as a ‘public utility’ for India’s AI ecosystem, empowering developers to refine Indic models, enterprises to identify the best-fit AI solutions, and users to help define what ‘good AI’ means for India.
Supported by Google Cloud during its initial phase, the open source platform aligns with India’s sovereign AI ambitions under the IndiaAI Mission.
AI researcher Adithya S K, founder of CognitiveLabs, praised the effort, saying, “The UX is spot on and I had the best Kannada typing experience. These are the sort of efforts labs in India should be doing more across domains.” The platform is live at arena.ai4bharat.org.
