Research Papers
Test Set Quality in Multilingual LLM Evaluation
This paper focuses on the quality of the datasets used for evaluating multilingual LLMs for Telugu and French languages and argues that test sets should not be considered immutable and should be revisited, checked for correctness, and potentially versioned.
MATA (ā°Žā°žā°): Mindful Assessment of the Telugu Abilities of Large Language Models
We propose a novel approach for detecting and analyzing code-switching patterns in Telugu-English mixed text, with applications in social media analysis and language modeling.
MetricalARGS: A Taxonomy for Studying Metrical Poetry with LLMs
We introduce MetricalARGS, the first taxonomy of poetry-related NLP tasks designed to evaluate LLMs on metrical poetry across four dimensions: Analysis, Retrieval, Generation, and Support.