Leaderboard

We’re inviting you to participate in the natural language understanding research in Indonesia, by submitting your recent results into our IndoNLU benchmark leaderboard. Needed information to participate are provided in this website, and further information about the dataset, evaluation, analysis, and about the IndoNLU benchmark models can be read in our paper listed here. If you need further assistance or consultation, do reach out to us by contacting us through indobenchmark@gmail.com. All the best for Indonesian NLU research communities!

Image
Classification Seq. Labeling


Name Model Param EmoT SmSA CASA HoASA WReTE Avg. Score
IndoNLU Team IndoBERT-large-p2 335.2M 79.47 92.03 94.94 93.38 80.30 88.02
IndoNLU Team IndoBERT-large-p1 335.2M 77.04 93.71 96.64 93.27 84.17 88.97
IndoNLU Team XLM-R large 561.0M 78.51 92.35 92.40 94.27 83.82 88.27
IndoNLU Team XLM-R base 278.7M 71.15 91.39 91.71 91.57 79.95 85.15
IndoNLU Team IndoBERT-lite-large-p2 17.7M 71.67 90.13 88.88 88.80 81.19 84.13
IndoNLU Team IndoBERT-lite-large-p1 17.7M 75.19 88.66 90.99 89.53 78.98 84.67
IndoNLU Team IndoBERT-base-p2 124.5M 76.28 87.66 93.24 92.70 78.68 85.71
IndoNLU Team IndoBERT-base-p1 124.5M 75.48 87.73 93.23 92.07 78.55 85.41
IndoNLU Team IndoBERT-lite-base-p2 11.7M 72.27 90.29 87.63 87.62 83.62 84.29
IndoNLU Team IndoBERT-lite-base-p1 11.7M 73.88 90.85 89.68 88.07 82.17 84.93
IndoNLU Team mBERT 167.4M 67.30 84.14 72.23 84.63 84.40 78.54
IndoNLU Team XLM-MLM 573.2M 65.75 86.33 82.17 88.89 64.35 77.50
IndoNLU Team fastText Indo4B (6L) 15.1M 68.47 83.07 81.96 86.20 60.33 76.00
IndoNLU Team fastText Indo4B (4L) 10.7M 69.23 82.13 82.20 85.88 60.42 75.97
IndoNLU Team fastText Indo4B (2L) 6.3M 70.97 83.63 78.97 80.16 57.11 74.17
IndoNLU Team fastText CC-ID (6L) 15.1M 67.43 78.84 81.61 85.01 61.13 74.80
IndoNLU Team fastText CC-ID (4L) 10.7M 65.36 76.92 79.02 85.32 67.36 74.79
IndoNLU Team fastText CC-ID (2L) 6.3M 64.74 76.71 75.39 78.05 65.11 72.00
IndoNLU Team scratch (6L) 15.1M 52.84 67.07 69.88 76.83 58.06 64.94
IndoNLU Team scratch (4L) 10.7M 57.31 67.35 67.15 76.28 64.35 66.49
IndoNLU Team scratch (2L) 6.3M 58.51 64.22 65.58 78.31 59.54 65.23