A Must for Indic Language Models

A few months ago, AIM pointed out the dire need for creating benchmarks for Indian languages since most of the famous ones, like MMLU and HumanEval, do not necessarily include a good amount of datasets for Indian languages. Now with Indic LLMs coming into the picture, it is time that India got its own LLM benchmark and leaderboard for its models.

At MLDS 2024, Tamil Llama creator Abhinand Balachandran highlighted the importance of creating benchmarks specifically tailored for evaluating Indic language models. Similarly, in an exclusive interview with AIM, Shantipriya Parida, the creator of Odia Llama, revealed that he was planning to create a benchmark for Indic language models.