Leaderboard

Leaderboards on Test set

Task SI

RankTeamF1PrecisionRecall
1Hitachi0.515510.565440.47368
2ApplicaAI0.491530.599540.41650
3aschern0.491000.532320.45564
4LTIatCMU0.476630.509700.44758
5UPB0.460600.586120.37936
6Fragarach0.459600.542610.39862
7NoPropaganda0.446810.556220.37337
8CyberWallE0.438580.421580.45699
9Transformers0.436030.498620.38740
10SWEAT0.432160.527680.36593
11YNUtaoxin0.432080.556170.35326
12DREAM0.430970.545350.35625
13newsSweeper0.422090.465180.38631
14PsuedoProp0.412010.415400.40867
15FakeSolomon0.406830.539460.32655
16Solomon0.406830.539460.32655
17YNUHPCC0.406310.365500.45739
18NLFIIT0.405770.509130.33730
19PALI0.405730.531960.32792
20UESTCICSA0.398470.560910.30899
21TTUI0.398410.668830.28371
22BPGC0.387440.493850.31876
23DoNotDistribute0.378610.423550.34230
24UTMNandOCAS0.374900.379670.37025
25Entropy0.372280.416840.33633
26syrapropa0.362000.495300.28523
27SkoltechNLP0.340680.465240.26873
28NTUAAILS0.335960.460520.26444
29UAIC18600.332100.244900.51574
30CCNI0.294840.380890.24051
31NCCUSocialMediaResearchGroup0.284680.172970.80370
323218IR0.234720.226320.24376
33WMD0.200930.471140.12769
34LS0.181800.341420.12388
35HunAlize0.031890.232360.01712
36YOLO0.007150.171980.00365
37Baseline0.003080.130450.00156

Task TC

RankTeamF1F1 Appeal_to_AuthorityF1 Appeal_to_fear-prejudiceF1 Bandwagon,Reductio_ad_hitlerumF1 Black-and-White_FallacyF1 Causal_OversimplificationF1 DoubtF1 Exaggeration,MinimisationF1 Flag-WavingF1 Loaded_LanguageF1 Name_Calling,LabelingF1 RepetitionF1 SlogansF1 Thought-terminating_ClichesF1 Whataboutism,Straw_Men,Red_Herring
1ApplicaAI0.620670.481480.454900.083330.490200.227270.562300.335880.694300.771240.743770.545450.512820.392160.25000
2aschern0.620110.352940.417780.149250.535710.259740.594430.326530.663460.770230.756480.533810.542370.425530.18868
3Hitachi0.617320.400000.389380.048780.479170.366200.634290.345790.680200.756390.741990.378790.456140.294120.26923
4Solomon0.589390.266670.393940.048780.420000.196080.618180.284440.618360.746580.707530.425250.507460.380950.00000
5newsSweeper0.584360.416670.427980.130430.250000.197180.565480.371040.605260.753160.742300.206900.507460.216220.08000
6NoPropaganda0.582680.232560.380230.000000.086960.121210.562690.379930.592960.771730.738980.427120.424240.232560.00000
7Inno0.579890.431370.450910.107140.404040.243240.586510.353850.594060.733110.743040.248850.437500.296300.19355
8CyberWallE0.573740.146340.398440.122450.236840.153850.586510.283400.543780.746750.709240.476820.393940.238100.00000
9PALI0.573180.275860.365850.285710.396230.305880.589670.285710.616160.742940.690850.245610.392160.408160.20896
10Duth0.572070.444440.333330.093020.333330.262300.591550.282350.589470.737090.714060.201010.347830.270270.17778
11DiSaster0.566480.511630.352540.204080.266670.146340.591160.306380.582520.744850.680980.204440.425530.190480.04348
12djichen0.565360.375000.336450.208330.438100.227850.600000.314180.561900.732090.683760.297520.307690.279070.18868
13SocCogCom0.558100.285710.316240.097560.265060.063490.603960.348620.542550.721820.673370.188840.409090.235290.10000
14TTUI0.556430.282350.414810.245610.352940.230770.574020.322030.616820.732190.684930.211760.375000.250000.20290
15JUST0.553070.488890.370970.244900.288890.272730.582580.295650.625640.719580.647270.219410.333330.318180.28571
16NLFIIT0.552510.296300.248850.093020.320990.000000.556600.303030.633170.725480.692970.215520.416670.136360.00000
17UMSIForeseer0.551400.294120.319700.097560.000000.000000.572100.285710.561400.730230.707900.214880.392160.142860.00000
18BPGC0.548050.307690.336450.125000.400000.241760.537760.334690.589290.715820.675090.237440.400000.206900.20896
19UPB0.543020.200000.300000.000000.197180.169490.525470.306220.558660.700940.688580.200000.346150.228570.04878
20syrapropa0.542460.315790.297670.196080.306120.215050.561400.280990.570210.714740.684400.307690.290320.285710.09091
21WMD0.520110.071430.291970.050000.000000.057140.539380.254550.520830.693300.646670.138890.068970.074070.00000
22YNUHPCC0.505030.297870.264000.137930.383840.034480.510390.215380.555560.680820.623330.177220.275860.177780.15000
23UESTCICSA0.499440.127660.227490.000000.112680.037380.449850.254390.531400.682320.668800.279570.413790.285710.03704
24DoNotDistribute0.497210.226420.297520.045450.244900.148940.462540.272340.537630.684440.606470.194440.280700.122450.09677
25NTUAAILS0.463690.363640.283080.097560.000000.000000.487530.296580.464650.657890.545450.184300.137930.114290.04082
26UAIC18600.411730.173910.221200.097560.028990.075950.364060.210080.387760.623280.429660.111590.114290.055560.04255
27UNTLing0.391060.043480.052910.000000.000000.028570.326450.118180.404760.625680.367350.077780.176470.000000.00000
28HunAlize0.370950.064520.118280.000000.080000.078430.317550.238100.299470.585880.158160.020940.045450.000000.00000
29Transformers0.265360.000000.000000.000000.000000.000000.009800.000000.000000.475500.240610.028570.000000.000000.00000
30Baseline0.251960.000000.036810.000000.000000.115610.291430.144200.061950.464770.000000.192620.000000.000000.00000
31Entropy0.203910.039220.063240.000000.022730.074070.128050.063930.049500.377390.154930.058330.000000.000000.06780
32IJSE80.197210.000000.070670.000000.000000.022730.154670.082300.085710.380730.147010.049180.000000.000000.00000
Scroll the table to the right to check the F1 values for each Technique.

Leaderboards on Development set

Task SI

RankTeamF1PrecisionRecall
1PALI0.523490.496440.55365
2ApplicaAI0.521910.471510.58438
3Hitachi0.501180.422620.61562
4aschern0.499900.445280.56978
5LTIatCMU0.490640.433760.56470
6UPB0.467890.424390.52132
7syrapropa0.462480.456360.46878
8NoPropaganda0.461250.406470.53309
9TTUI0.451860.402450.51510
10newsSweeper0.444510.387580.52104
11YNUtaoxin0.443240.428230.45933
12Fragarach0.442720.416810.47207
13UESTCICSA0.441730.432080.45183
14FakeSolomon0.428610.432380.42490
15SWEAT0.425090.429730.42055
16CyberWallE0.423890.334480.57856
17YNUHPCC0.422730.320820.61954
18DREAM0.421500.426620.41650
19Transformers0.419720.389320.45526
20Murgila0.413750.329600.55560
21NLFIIT0.396690.350350.45716
22PsuedoProp0.393180.342720.46108
23TakeLab0.390610.388500.39273
24DoNotDistribute0.377260.324130.45123
25BPGC0.367890.347220.39118
26SkoltechNLP0.367000.349900.38586
27WMD0.363380.332550.40051
28atulcst0.362910.381450.34610
29LS0.354930.224070.85331
30AAA0.346800.306060.40004
31UTMNandOCAS0.343450.236510.62690
32Entropy0.328850.308160.35251
33NTUAAILS0.312130.279460.35346
34UAIC18600.302720.206930.56368
35CCNI0.296140.290400.30211
36CUNLP0.277810.582310.18242
37HunAlize0.244510.377530.18080
38UoB0.240230.222950.26042
39IIITD0.216900.163620.32161
403218IR0.200340.151000.29755
41UniversityOfBirmingham0.169470.230740.13391
42NCCUSocialMediaResearchGroup0.158340.091220.59918
43Baseline0.011020.110040.00580
44SocCogCom0.007860.099660.00409
45Inno0.006380.093610.00330
46YOLO0.006380.093610.00330
47Raghavan0.004030.002030.33445
48California0.003890.059190.00201
Note that the gold labels for the task SI can de extracted from the input files for task TC.
This will not be the case for the test set, since we will release the input data for task TC after closing the submissions for task SI.
Submissions to task SI using the gold labels for task TC will not be accepted and the team might be disqualified.

Task TC

RankTeamF1F1 Appeal_to_AuthorityF1 Appeal_to_fear-prejudiceF1 Bandwagon,Reductio_ad_hitlerumF1 Black-and-White_FallacyF1 Causal_OversimplificationF1 DoubtF1 Exaggeration,MinimisationF1 Flag-WavingF1 Loaded_LanguageF1 Name_Calling,LabelingF1 RepetitionF1 SlogansF1 Thought-terminating_ClichesF1 Whataboutism,Straw_Men,Red_Herring
1ApplicaAI0.704610.428570.484850.800000.200000.470590.635660.609270.825580.804190.741140.706710.626510.434780.41667
2aschern0.681090.285710.452380.400000.324320.304350.661870.593550.742510.800000.738540.673910.638890.333330.38462
3syrapropa0.678270.200000.408160.600000.137930.418600.567380.513890.788890.772390.770000.701200.684930.260870.24242
4FakeSolomon0.671680.095240.427180.888890.187500.312500.612240.507460.743590.792900.743800.660380.700000.434780.06250
5PALI0.668860.416670.404490.750000.355560.421050.584620.486110.795320.780120.763360.615940.583330.457140.27027
6CyberWallE0.664160.181820.306120.222220.214290.210530.538460.527030.736840.766210.810000.732880.513510.173910.00000
7Hitachi0.651930.125000.369570.750000.222220.500000.629630.565790.813950.779910.711590.469750.676060.190480.40909
8DiSaster0.628410.300000.352940.285710.000000.230770.562090.482760.751520.767280.773270.471070.350880.090910.05556
9newsSweeper0.621830.190480.408600.571430.148150.326530.541940.528570.734460.781710.717170.426880.605260.000000.00000
10NoPropaganda0.606770.000000.333330.000000.000000.480000.503600.387100.757400.770590.724870.476870.531650.083330.00000
11Fragarach0.604890.142860.409090.571430.129030.340430.510340.506850.741180.771190.704960.382110.457140.166670.06061
12Inno0.601130.117650.408160.615380.357140.341460.528570.437500.779660.765650.706200.346670.597010.181820.15385
13CUNLP0.595480.162160.342860.250000.157890.358970.506850.436620.718950.743780.732980.476190.550720.195120.08889
14Solomon0.595480.000000.390800.000000.208330.342860.512820.496450.746840.759210.719790.334760.459020.322580.20833
15TTUI0.589840.162160.346940.666670.190480.333330.569230.419580.775280.750750.710000.320680.500000.206900.29091
16BPGC0.587960.190480.324320.000000.187500.270270.528570.496350.758170.754510.700510.297520.523080.230770.09524
17UMSIForeseer0.587020.100000.315790.000000.000000.526320.479450.517990.758620.744440.709510.317600.433330.000000.10000
18NLFIIT0.585140.133330.351350.600000.277780.342860.531250.414810.750000.744620.704760.333330.605260.242420.24138
19UPB0.583260.074070.298850.571430.086960.342860.461540.500000.707320.740160.716420.241550.500000.000000.06061
20Duth0.578550.095240.344090.500000.111110.428570.487180.461540.644300.761220.703200.230770.553850.153850.11111
21JUST0.576670.000000.310680.000000.000000.238100.487500.496350.737500.749670.723620.236360.290320.000000.05882
22UESTCICSA0.575730.000000.387100.000000.000000.266670.464790.396550.632260.741480.650410.483660.588240.363640.09524
23djichen0.575730.133330.367820.000000.200000.318180.482760.446040.763640.760400.705290.248890.514290.108110.04545
24Entropy0.571030.133330.311110.000000.181820.292680.535950.451610.748540.752190.664990.281940.468750.200000.10000
25SocCogCom0.570090.000000.329110.571430.214290.285710.539880.456690.771080.706480.643800.317760.301890.129030.06452
26YNUHPCC0.561620.064520.260870.545450.181820.272730.513890.363640.730540.705730.694520.297520.405800.260870.21053
27DoNotDistribute0.539980.000000.255810.000000.000000.250000.459460.319330.726110.712000.673320.293840.289860.228570.09756
28Murgila0.535280.176470.224720.444440.133330.300000.372880.425530.695120.712100.661380.245450.405800.315790.22642
29NTUAAILS0.532460.066670.213330.444440.071430.171430.516130.436360.730340.697610.598980.271930.271190.000000.08000
30WMD0.523050.190480.268290.400000.000000.068970.397660.351150.630950.715670.577020.400000.156860.000000.00000
31UAIC18600.438380.105260.070180.333330.000000.000000.420290.236560.600000.578790.403670.141180.045450.000000.00000
32Tianyi0.421450.000000.155560.000000.200000.222220.331210.270270.705130.583510.311790.050960.264150.000000.06250
33hseteam0.413950.000000.111110.428570.072730.076920.262860.322580.575160.636490.428150.224000.266670.000000.04762
34HunAlize0.410160.000000.092310.000000.000000.333330.411760.222220.535030.561300.407520.053190.000000.000000.00000
35UNTLing0.409220.083330.119400.000000.000000.266670.354170.122810.517010.594480.333330.118340.250000.000000.05882
36SWEAT0.305740.000000.000000.000000.000000.000000.000000.000000.000000.468300.000000.000000.000000.000000.00000
37Transformers0.301040.000000.000000.000000.000000.000000.000000.000000.000000.484980.286220.123080.000000.000000.04878
38NCCUSocialMediaResearchGroup0.292570.125000.150940.230770.166670.097560.369230.146890.450700.379170.344590.255320.133330.125000.00000
39Baseline0.265290.000000.093750.000000.000000.072290.192000.116790.082760.405800.000000.384980.000000.000000.00000
40LS0.262330.000000.090910.000000.000000.087910.196720.129500.082760.391960.000000.393090.000000.062500.00000
41SkoltechNLP0.262330.000000.090910.000000.000000.087910.196720.129500.082760.391960.000000.393090.000000.062500.00000
42TakeLab0.253060.000000.000000.000000.000000.000000.064260.000000.000000.429160.246970.013700.000000.000000.00000
43UTMNandOCAS0.234240.000000.105260.000000.230770.000000.000000.044440.000000.405340.197420.186670.000000.086960.00000
Scroll the table to the right to check the F1 values for each Technique.


Note that the leaderboard is updated every 20 minutes, therefore your submission may not be listed right away.

Organisers

  • Giovanni Da San Martino, Qatar Computing Research Institute
  • Alberto Barrón-Cedeño, University of Bologna
  • Preslav Nakov, Qatar Computing Research Institute
  • Henning Wachsmuth, Paderborn University
  • Rostislav Petrov, A Data Pro

Contact