Research Papers

Test Set Quality in Multilingual LLM Evaluation

This paper focuses on the quality of the datasets used for evaluating multilingual LLMs for Telugu and French languages and argues that test sets should not be considered immutable and should be revisited, checked for correctness, and potentially versioned.

AACL 2025 • 5th Workshop on "Evaluation & Comparison of NLP Systems"
📄 ACL Anthology

MATA (మాట): Mindful Assessment of the Telugu Abilities of Large Language Models

We introduce MATA, a novel evaluation dataset to assess the ability of Large Language Models (LLMs) in Telugu language, comprising 729 carefully curated multiple-choice and open-ended questions that span diverse linguistic dimensions.

LREC 2026 • Main Conference
📄 arXiv

MetricalARGS: A Taxonomy for Studying Metrical Poetry with LLMs

We introduce MetricalARGS, the first taxonomy of poetry-related NLP tasks designed to evaluate LLMs on metrical poetry across four dimensions: Analysis, Retrieval, Generation, and Support.

LREC 2026 • WILDRE8 Workshop
📄 arXiv

Translating Classical Poetry into Modern Prose

We introduce పద్యం2గద్యం (Padyam2Gadyam) a dataset for the task of poem-to-prose translation from 13th-17th Century Telugu Classical Poetry to contemporary Telugu and English prose.

June 2026 • Preprint
📄 arXiv