Leaderboard
We’re inviting you to participate in the natural language understanding research in Indonesia, by submitting your recent results into our IndoNLU benchmark leaderboard. Needed information to participate are provided in this website, and further information about the dataset, evaluation, analysis, and about the IndoNLU benchmark models can be read in our paper listed here. If you need further assistance or consultation, do reach out to us by contacting us through indobenchmark@gmail.com. All the best for Indonesian NLU research communities!

Name | Model | Param | EmoT | SmSA | CASA | HoASA | WReTE | Avg. Score |
---|---|---|---|---|---|---|---|---|
IndoNLU Team | IndoBERT-large-p2 | 335.2M | 79.47 | 92.03 | 94.94 | 93.38 | 80.30 | 88.02 |
IndoNLU Team | IndoBERT-large-p1 | 335.2M | 77.04 | 93.71 | 96.64 | 93.27 | 84.17 | 88.97 |
IndoNLU Team | XLM-R large | 561.0M | 78.51 | 92.35 | 92.40 | 94.27 | 83.82 | 88.27 |
IndoNLU Team | XLM-R base | 278.7M | 71.15 | 91.39 | 91.71 | 91.57 | 79.95 | 85.15 |
IndoNLU Team | IndoBERT-lite-large-p2 | 17.7M | 71.67 | 90.13 | 88.88 | 88.80 | 81.19 | 84.13 |
IndoNLU Team | IndoBERT-lite-large-p1 | 17.7M | 75.19 | 88.66 | 90.99 | 89.53 | 78.98 | 84.67 |
IndoNLU Team | IndoBERT-base-p2 | 124.5M | 76.28 | 87.66 | 93.24 | 92.70 | 78.68 | 85.71 |
IndoNLU Team | IndoBERT-base-p1 | 124.5M | 75.48 | 87.73 | 93.23 | 92.07 | 78.55 | 85.41 |
IndoNLU Team | IndoBERT-lite-base-p2 | 11.7M | 72.27 | 90.29 | 87.63 | 87.62 | 83.62 | 84.29 |
IndoNLU Team | IndoBERT-lite-base-p1 | 11.7M | 73.88 | 90.85 | 89.68 | 88.07 | 82.17 | 84.93 |
IndoNLU Team | mBERT | 167.4M | 67.30 | 84.14 | 72.23 | 84.63 | 84.40 | 78.54 |
IndoNLU Team | XLM-MLM | 573.2M | 65.75 | 86.33 | 82.17 | 88.89 | 64.35 | 77.50 |
IndoNLU Team | fastText Indo4B (6L) | 15.1M | 68.47 | 83.07 | 81.96 | 86.20 | 60.33 | 76.00 |
IndoNLU Team | fastText Indo4B (4L) | 10.7M | 69.23 | 82.13 | 82.20 | 85.88 | 60.42 | 75.97 |
IndoNLU Team | fastText Indo4B (2L) | 6.3M | 70.97 | 83.63 | 78.97 | 80.16 | 57.11 | 74.17 |
IndoNLU Team | fastText CC-ID (6L) | 15.1M | 67.43 | 78.84 | 81.61 | 85.01 | 61.13 | 74.80 |
IndoNLU Team | fastText CC-ID (4L) | 10.7M | 65.36 | 76.92 | 79.02 | 85.32 | 67.36 | 74.79 |
IndoNLU Team | fastText CC-ID (2L) | 6.3M | 64.74 | 76.71 | 75.39 | 78.05 | 65.11 | 72.00 |
IndoNLU Team | scratch (6L) | 15.1M | 52.84 | 67.07 | 69.88 | 76.83 | 58.06 | 64.94 |
IndoNLU Team | scratch (4L) | 10.7M | 57.31 | 67.35 | 67.15 | 76.28 | 64.35 | 66.49 |
IndoNLU Team | scratch (2L) | 6.3M | 58.51 | 64.22 | 65.58 | 78.31 | 59.54 | 65.23 |