IMUSA:- A pioneering multimodal shared task for sentiment classification of Punjabi memes, addressing low-resource Indic language understanding at FIRE 2026.
IMUSA is the first shared task focused on multimodal sentiment analysis of Punjabi memes a low-resource Indic language combining visual and textual modalities to understand the rich, culturally-embedded communication in social media content.
Memes convey meaning through the interplay of images and text. Neither modality alone captures the irony, humour, or emotion. IMUSA challenges participants to build systems that jointly model visual and textual features for accurate sentiment classification.
Punjabi is an Indo-Aryan language with a large global diaspora. Despite its widespread use, computational resources for Punjabi remain scarce. IMUSA addresses this gap with a manually annotated multimodal Punjabi meme dataset.
Participants are encouraged to leverage cutting-edge methods including multimodal BERT variants, CNNs for visual feature extraction, LSTMs/Transformers for text, cross-modal fusion mechanisms, and self-attention-based architectures.
Systems developed in IMUSA will have direct applications in content moderation, automated hate-speech detection, mental health monitoring, and cultural analytics across South Asian social media platforms.
Each Punjabi meme in the dataset is annotated into one of four sentiment categories, designed to capture a wide range of emotional tones and communicative intents.
Memes where the intended meaning differs from the literal expression. Relies on irony, exaggeration, or indirect references to convey humour or criticism. Example: commentary on social media behavior with a reversed meaning.
Memes with no strong emotional tone or opinion. Present general information, everyday situations, or observational content without conveying clear positivity, negativity, or intent to influence emotions.
Content that is abusive, insulting, or inappropriate. Targets individuals, communities, or social groups and contributes to harmful narratives. Identifying such content is critical for automated moderation systems.
Memes intended to inspire or encourage positive thinking. Include uplifting messages, life advice, or emotionally supportive content aimed at motivating individuals and promoting well-being.
A manually annotated multimodal Punjabi meme dataset of 3,000 images, labelled with the assistance of linguistic experts to ensure accuracy and contextual consistency.
Track website goes live. Training data made available to registered participants.
Test set distributed to participants for final system evaluation.
All participant system runs must be submitted. Each team may submit 1–3 runs.
Official track results announced and communicated to all participating teams.
Participants submit working notes describing their approach and results.
Final camera-ready copies of working notes and the overview paper are due.
Participant systems will be evaluated using four standard metrics from information retrieval and machine learning, chosen to provide a comprehensive assessment across imbalanced class distributions.
Measures overall correctness — the proportion of correctly predicted samples out of the total. Provides a general sense of system performance.
Evaluated per class. Measures how many instances predicted as a particular class are actually correct — indicating the reliability of positive predictions.
Evaluated per class. Measures how well the system identifies all relevant instances. Critical for Sarcasm and Offensive categories where misclassification has high impact.
The harmonic mean of Precision and Recall. Provides a balanced evaluation especially useful for uneven class distributions, reflecting the trade-off between false positives and negatives.
Official IMUSA leaderboard based on Macro-F1 scores computed on the 500-sample test set. Teams are ranked in descending order of Macro-F1.
| Rank | Team | Macro-F1 | Matched |
|---|---|---|---|
| 🥇 1 | Team Solutions | 0.461960 | 500 / 500 |
| 🥈 2 | Text Guardians | 0.458000 | 500 / 500 |
| 🥉 3 | Lingchao Kong | 0.455817 | 500 / 500 |
| 4 | Delta | 0.442115 | 500 / 500 |
| 5 | Data Alchemist | 0.440392 | 500 / 500 |
| 6 | Big three's | 0.439028 | 500 / 500 |
| 7 | MindMergeAI | 0.438879 | 500 / 500 |
| 8 | UPES Knight | 0.435538 | 500 / 500 |
| 9 | VerSa | 0.432853 | 500 / 500 |
| 10 | TEAM ACE | 0.432098 | 500 / 500 |
| 11 | BASELINE | 0.432067 | 500 / 500 |
| 12 | NexaNLP | 0.427400 | 500 / 500 |
| 13 | Cloud Bolts | 0.422527 | 500 / 500 |
| 14 | Mavericks | 0.421288 | 500 / 500 |
| 15 | Innovrtta | 0.416673 | 447 / 500 |
| 16 | Khushi Malik | 0.410617 | 500 / 500 |
| 17 | Cloud NLP | 0.409210 | 500 / 500 |
| 18 | Janhvi Shukla | 0.404673 | 500 / 500 |
| 19 | TokenX | 0.404456 | 500 / 500 |
| 20 | Team NEXUS | 0.401230 | 500 / 500 |
| 21 | TechAlpha | 0.390536 | 500 / 500 |
| 22 | Insight Nexus | 0.382017 | 500 / 500 |
| 23 | Vagabonds | 0.371886 | 500 / 500 |
| 24 | KARB | 0.355735 | 500 / 500 |
| 25 | Legends | 0.346526 | 500 / 500 |
| 26 | Sentimentalists | 0.339353 | 500 / 500 |
| 27 | Piyush and Hardik | 0.294137 | 500 / 500 |
| 28 | Team Elite | 0.279614 | 500 / 500 |
| 29 | CyberShield | 0.212313 | 500 / 500 |
| 30 | Coders | 0.199985 | 500 / 500 |
| 31 | Team Vitality | N/A | N/A |
| 32 | CORSv1 | 0.295794095 | 500 / 500 |
| 33 | ByteX | 0.428545486 | 500 / 500 |
Note: Team Vitality has no reported Macro-F1 because none of its 500 gold IDs were matched (0 matched samples, 3,002 extra IDs). Innovrtta's score is based on 447 matched samples, with 53 missing IDs. Teams CORSv1 and ByteX were placed at the bottom of the leaderboard due to late submission. All other ranked teams have 500 matched samples.