Leaderboard

Leaderboards on Test set

Task SI

RankTeamF1PrecisionRecall
1Hitachi0.515510.565440.47368
2ApplicaAI0.491530.599540.41650
3aschern0.491000.532320.45564
4LTIatCMU0.476630.509700.44758
5UPB0.460600.586120.37936
6Fragarach0.459600.542610.39862
7NoPropaganda0.446810.556220.37337
8CyberWallE0.438580.421580.45699
9Transformers0.436030.498620.38740
10SWEAT0.432160.527680.36593
11YNUtaoxin0.432080.556170.35326
12DREAM0.430970.545350.35625
13newsSweeper0.422090.465180.38631
14PsuedoProp0.412010.415400.40867
15Solomon0.406830.539460.32655
16YNUHPCC0.406310.365500.45739
17NLFIIT0.405770.509130.33730
18PALI0.405730.531960.32792
19UESTCICSA0.398470.560910.30899
20TTUI0.398410.668830.28371
21BPGC0.387440.493850.31876
22DoNotDistribute0.378610.423550.34230
23UTMNandOCAS0.374900.379670.37025
24Entropy0.372280.416840.33633
25syrapropa0.362000.495300.28523
26SkoltechNLP0.340680.465240.26873
27NTUAAILS0.335960.460520.26444
28UAIC18600.332100.244900.51574
29CCNI0.294840.380890.24051
30NCCUSocialMediaResearchGroup0.284680.172970.80370
313218IR0.234720.226320.24376
32WMD0.200930.471140.12769
33LS0.181800.341420.12388
34HunAlize0.031890.232360.01712
35YOLO0.007150.171980.00365
36Baseline0.003080.130450.00156

Task TC

RankTeamF1F1 Appeal_to_AuthorityF1 Appeal_to_fear-prejudiceF1 Bandwagon,Reductio_ad_hitlerumF1 Black-and-White_FallacyF1 Causal_OversimplificationF1 DoubtF1 Exaggeration,MinimisationF1 Flag-WavingF1 Loaded_LanguageF1 Name_Calling,LabelingF1 RepetitionF1 SlogansF1 Thought-terminating_ClichesF1 Whataboutism,Straw_Men,Red_Herring
1ApplicaAI0.620670.481480.454900.083330.490200.227270.562300.335880.694300.771240.743770.545450.512820.392160.25000
2aschern0.620110.352940.417780.149250.535710.259740.594430.326530.663460.770230.756480.533810.542370.425530.18868
3Hitachi0.617320.400000.389380.048780.479170.366200.634290.345790.680200.756390.741990.378790.456140.294120.26923
4Solomon0.589390.266670.393940.048780.420000.196080.618180.284440.618360.746580.707530.425250.507460.380950.00000
5newsSweeper0.584360.416670.427980.130430.250000.197180.565480.371040.605260.753160.742300.206900.507460.216220.08000
6NoPropaganda0.582680.232560.380230.000000.086960.121210.562690.379930.592960.771730.738980.427120.424240.232560.00000
7Inno0.579890.431370.450910.107140.404040.243240.586510.353850.594060.733110.743040.248850.437500.296300.19355
8CyberWallE0.573740.146340.398440.122450.236840.153850.586510.283400.543780.746750.709240.476820.393940.238100.00000
9PALI0.573180.275860.365850.285710.396230.305880.589670.285710.616160.742940.690850.245610.392160.408160.20896
10Duth0.572070.444440.333330.093020.333330.262300.591550.282350.589470.737090.714060.201010.347830.270270.17778
11DiSaster0.566480.511630.352540.204080.266670.146340.591160.306380.582520.744850.680980.204440.425530.190480.04348
12djichen0.565360.375000.336450.208330.438100.227850.600000.314180.561900.732090.683760.297520.307690.279070.18868
13SocCogCom0.558100.285710.316240.097560.265060.063490.603960.348620.542550.721820.673370.188840.409090.235290.10000
14TTUI0.556430.282350.414810.245610.352940.230770.574020.322030.616820.732190.684930.211760.375000.250000.20290
15JUST0.553070.488890.370970.244900.288890.272730.582580.295650.625640.719580.647270.219410.333330.318180.28571
16NLFIIT0.552510.296300.248850.093020.320990.000000.556600.303030.633170.725480.692970.215520.416670.136360.00000
17UMSIForeseer0.551400.294120.319700.097560.000000.000000.572100.285710.561400.730230.707900.214880.392160.142860.00000
18BPGC0.548050.307690.336450.125000.400000.241760.537760.334690.589290.715820.675090.237440.400000.206900.20896
19UPB0.543020.200000.300000.000000.197180.169490.525470.306220.558660.700940.688580.200000.346150.228570.04878
20syrapropa0.542460.315790.297670.196080.306120.215050.561400.280990.570210.714740.684400.307690.290320.285710.09091
21WMD0.520110.071430.291970.050000.000000.057140.539380.254550.520830.693300.646670.138890.068970.074070.00000
22YNUHPCC0.505030.297870.264000.137930.383840.034480.510390.215380.555560.680820.623330.177220.275860.177780.15000
23UESTCICSA0.499440.127660.227490.000000.112680.037380.449850.254390.531400.682320.668800.279570.413790.285710.03704
24DoNotDistribute0.497210.226420.297520.045450.244900.148940.462540.272340.537630.684440.606470.194440.280700.122450.09677
25NTUAAILS0.463690.363640.283080.097560.000000.000000.487530.296580.464650.657890.545450.184300.137930.114290.04082
26UAIC18600.411730.173910.221200.097560.028990.075950.364060.210080.387760.623280.429660.111590.114290.055560.04255
27UNTLing0.391060.043480.052910.000000.000000.028570.326450.118180.404760.625680.367350.077780.176470.000000.00000
28HunAlize0.370950.064520.118280.000000.080000.078430.317550.238100.299470.585880.158160.020940.045450.000000.00000
29Transformers0.265360.000000.000000.000000.000000.000000.009800.000000.000000.475500.240610.028570.000000.000000.00000
30Baseline0.251960.000000.036810.000000.000000.115610.291430.144200.061950.464770.000000.192620.000000.000000.00000
31Entropy0.203910.039220.063240.000000.022730.074070.128050.063930.049500.377390.154930.058330.000000.000000.06780
32IJSE80.197210.000000.070670.000000.000000.022730.154670.082300.085710.380730.147010.049180.000000.000000.00000
Scroll the table to the right to check the F1 values for each Technique.

Leaderboards on Development set

Task SI

RankTeamF1PrecisionRecall
1syrapropa0.534040.398820.80802
2PALI0.523490.496440.55365
3ApplicaAI0.521910.471510.58438
4Hitachi0.501180.422620.61562
5aschern0.499900.445280.56978
6TTUI0.495860.487610.50439
7LTIatCMU0.490640.433760.56470
8UPB0.467890.424390.52132
9NoPropaganda0.461250.406470.53309
10newsSweeper0.455570.400080.52894
11YNUtaoxin0.443530.407370.48674
12Fragarach0.442720.416810.47207
13UESTCICSA0.441730.432080.45183
14Transformers0.438190.355490.57105
15Murgila0.430830.386090.48730
16SWEAT0.425090.429730.42055
17CyberWallE0.423890.334480.57856
18YNUHPCC0.422730.320820.61954
19DREAM0.421500.426620.41650
20NLFIIT0.396690.350350.45716
21IIITD0.395150.364250.43177
22PsuedoProp0.393180.342720.46108
23In2Deep0.391000.414380.37012
24TakeLab0.390610.388500.39273
25DoNotDistribute0.377260.324130.45123
26SkoltechNLP0.368310.321970.43023
27BPGC0.367890.347220.39118
28WMD0.363380.332550.40051
29atulcst0.362910.381450.34610
30LS0.354930.224070.85331
31AAA0.346800.306060.40004
32UTMNandOCAS0.343450.236510.62690
33Entropy0.328850.308160.35251
34NTUAAILS0.322570.275600.38885
35UAIC18600.302720.206930.56368
36CCNI0.296140.290400.30211
37CUNLP0.277810.582310.18242
38HunAlize0.244510.377530.18080
393218IR0.241470.241430.24151
40UoB0.240230.222950.26042
41HenryAtDuderstadt0.186340.102741.00000
42UniversityOfBirmingham0.169470.230740.13391
43NCCUSocialMediaResearchGroup0.158340.091220.59918
44Baseline0.011020.110040.00580
45SocCogCom0.007860.099660.00409
46Inno0.006380.093610.00330
47YOLO0.006380.093610.00330
48Raghavan0.004030.002030.33445
49California0.003890.059190.00201
Note that the gold labels for the task SI can de extracted from the input files for task TC.
This will not be the case for the test set, since we will release the input data for task TC after closing the submissions for task SI.
Submissions to task SI using the gold labels for task TC will not be accepted and the team might be disqualified.

Task TC

RankTeamF1F1 Appeal_to_AuthorityF1 Appeal_to_fear-prejudiceF1 Bandwagon,Reductio_ad_hitlerumF1 Black-and-White_FallacyF1 Causal_OversimplificationF1 DoubtF1 Exaggeration,MinimisationF1 Flag-WavingF1 Loaded_LanguageF1 Name_Calling,LabelingF1 RepetitionF1 SlogansF1 Thought-terminating_ClichesF1 Whataboutism,Straw_Men,Red_Herring
1ApplicaAI0.704610.428570.484850.800000.200000.470590.635660.609270.825580.804190.741140.706710.626510.434780.41667
2aschern0.681090.285710.452380.400000.324320.304350.661870.593550.742510.800000.738540.673910.638890.333330.38462
3syrapropa0.678270.200000.408160.600000.137930.418600.567380.513890.788890.772390.770000.701200.684930.260870.24242
4FLZ0.671680.444440.200000.750000.370370.461540.654550.233770.640780.810890.711220.707790.640000.285710.06667
5PALI0.668860.416670.404490.750000.355560.421050.584620.486110.795320.780120.763360.615940.583330.457140.27027
6CyberWallE0.664160.181820.306120.222220.214290.210530.538460.527030.736840.766210.810000.732880.513510.173910.00000
7Hitachi0.651930.125000.369570.750000.222220.500000.629630.565790.813950.779910.711590.469750.676060.190480.40909
8DiSaster0.628410.300000.352940.285710.000000.230770.562090.482760.751520.767280.773270.471070.350880.090910.05556
9newsSweeper0.627470.166670.448980.571430.068970.478260.569440.553190.742860.762320.713540.451360.605260.166670.06452
10NoPropaganda0.606770.000000.333330.000000.000000.480000.503600.387100.757400.770590.724870.476870.531650.083330.00000
11Fragarach0.604890.142860.409090.571430.129030.340430.510340.506850.741180.771190.704960.382110.457140.166670.06061
12Inno0.601130.117650.408160.615380.357140.341460.528570.437500.779660.765650.706200.346670.597010.181820.15385
13NerdyBirdies0.598310.153850.360360.000000.086960.476190.520550.486490.703030.762310.735450.330510.485710.200000.13333
14CUNLP0.595480.162160.342860.250000.157890.358970.506850.436620.718950.743780.732980.476190.550720.195120.08889
15Solomon0.595480.000000.390800.000000.208330.342860.512820.496450.746840.759210.719790.334760.459020.322580.20833
16TTUI0.589840.162160.346940.666670.190480.333330.569230.419580.775280.750750.710000.320680.500000.206900.29091
17BPGC0.587960.190480.324320.000000.187500.270270.528570.496350.758170.754510.700510.297520.523080.230770.09524
18UMSIForeseer0.587020.100000.315790.000000.000000.526320.479450.517990.758620.744440.709510.317600.433330.000000.10000
19NLFIIT0.585140.133330.351350.600000.277780.342860.531250.414810.750000.744620.704760.333330.605260.242420.24138
20UPB0.583260.074070.298850.571430.086960.342860.461540.500000.707320.740160.716420.241550.500000.000000.06061
21Tianyi0.582310.133330.345680.571430.117650.285710.503600.447200.756100.753910.712200.264320.493150.000000.00000
22Duth0.578550.095240.344090.500000.111110.428570.487180.461540.644300.761220.703200.230770.553850.153850.11111
23JUST0.576670.000000.310680.000000.000000.238100.487500.496350.737500.749670.723620.236360.290320.000000.05882
24UESTCICSA0.575730.000000.387100.000000.000000.266670.464790.396550.632260.741480.650410.483660.588240.363640.09524
25djichen0.575730.133330.367820.000000.200000.318180.482760.446040.763640.760400.705290.248890.514290.108110.04545
26Entropy0.571030.133330.311110.000000.181820.292680.535950.451610.748540.752190.664990.281940.468750.200000.10000
27SocCogCom0.570090.000000.329110.571430.214290.285710.539880.456690.771080.706480.643800.317760.301890.129030.06452
28YNUHPCC0.561620.064520.260870.545450.181820.272730.513890.363640.730540.705730.694520.297520.405800.260870.21053
29DoNotDistribute0.539980.000000.255810.000000.000000.250000.459460.319330.726110.712000.673320.293840.289860.228570.09756
30Murgila0.535280.176470.224720.444440.133330.300000.372880.425530.695120.712100.661380.245450.405800.315790.22642
31NTUAAILS0.532460.066670.213330.444440.071430.171430.516130.436360.730340.697610.598980.271930.271190.000000.08000
32WMD0.523050.190480.268290.400000.000000.068970.397660.351150.630950.715670.577020.400000.156860.000000.00000
33UAIC18600.438380.105260.070180.333330.000000.000000.420290.236560.600000.578790.403670.141180.045450.000000.00000
34hseteam0.413950.000000.111110.428570.072730.076920.262860.322580.575160.636490.428150.224000.266670.000000.04762
35HunAlize0.410160.000000.092310.000000.000000.333330.411760.222220.535030.561300.407520.053190.000000.000000.00000
36UNTLing0.409220.083330.119400.000000.000000.266670.354170.122810.517010.594480.333330.118340.250000.000000.05882
37HenryAtDuderstadt0.341490.000000.000000.000000.000000.000000.342860.000000.000000.529660.041030.000000.000000.000000.00000
38XJPaccelerationMASTER0.324550.000000.000000.000000.000000.000000.330580.000000.000000.487620.000000.000000.000000.000000.00000
39SWEAT0.305740.000000.000000.000000.000000.000000.000000.000000.000000.468300.000000.000000.000000.000000.00000
40Transformers0.301040.000000.000000.000000.000000.000000.000000.000000.000000.484980.286220.123080.000000.000000.04878
41NCCUSocialMediaResearchGroup0.292570.125000.150940.230770.166670.097560.369230.146890.450700.379170.344590.255320.133330.125000.00000
42Baseline0.265290.000000.093750.000000.000000.072290.192000.116790.082760.405800.000000.384980.000000.000000.00000
43LS0.262330.000000.090910.000000.000000.087910.196720.129500.082760.391960.000000.393090.000000.062500.00000
44SkoltechNLP0.262330.000000.090910.000000.000000.087910.196720.129500.082760.391960.000000.393090.000000.062500.00000
45TakeLab0.253060.000000.000000.000000.000000.000000.064260.000000.000000.429160.246970.013700.000000.000000.00000
46UTMNandOCAS0.234240.000000.105260.000000.230770.000000.000000.044440.000000.405340.197420.186670.000000.086960.00000
Scroll the table to the right to check the F1 values for each Technique.


Note that the leaderboard is updated every 20 minutes, therefore your submission may not be listed right away.

Organisers

  • Giovanni Da San Martino, Qatar Computing Research Institute
  • Alberto Barrón-Cedeño, University of Bologna
  • Preslav Nakov, Qatar Computing Research Institute
  • Henning Wachsmuth, Paderborn University
  • Rostislav Petrov, A Data Pro

Contact