Leaderboard

Leaderboards on Test set

Task SLC

Rank	Team	F1	Precision	Recall
1	ltuorp	0.632375	0.602885	0.664899
2	ProperGander	0.625651	0.564957	0.700954
3	YMJA	0.624934	0.625265	0.624602
4	MICCIS	0.623062	0.573595	0.681866
5	CUNLP	0.618343	0.577880	0.664899
6	Tha3aroon	0.613870	0.530960	0.727466
7	JUSTDeep	0.611222	0.579297	0.646872
8	CAUnLP	0.610966	0.518081	0.744433
9	LIPN	0.596251	0.524116	0.691410
10	LIACC	0.594976	0.509050	0.715801
11	aschern	0.592312	0.605088	0.580064
12	MindCoders	0.586898	0.599558	0.574761
13	jinfen	0.577028	0.505995	0.671262
14	guanggong	0.576871	0.503962	0.674443
15	Stano	0.561963	0.666667	0.485684
16	nlpseattle	0.561075	0.625000	0.509014
17	gw2018	0.544019	0.433333	0.730647
18	SDS	0.517134	0.626888	0.440085
19	BananasInPajamas	0.508012	0.576819	0.453871
20	Baseline	0.434701	0.388010	0.494168
21	NSIT	0.434313	0.500000	0.383881
22	Stalin	0.433286	0.669623	0.320255
23	Antiganda	0.396767	0.645933	0.286320
24	Debunkers	0.230769	0.399478	0.162248
25	SBnLP	0.183178	0.222054	0.155885
26	Sberiboba	0.116746	0.598039	0.064687

Task FLC

Rank	Team	F1	Precision	Recall	Appeal_to_Authority	Appeal_to_fear-prejudice	Bandwagon	Black-and-White_Fallacy	Causal_Oversimplification	Doubt	Exaggeration,Minimisation	Flag-Waving	Loaded_Language	Name_Calling,Labeling	Obfuscation,Intentional_Vagueness,Confusion	Red_Herring	Reductio_ad_hitlerum	Repetition	Slogans	Straw_Men	Thought-terminating_Cliches	Whataboutism
1	newspeak	0.248849	0.286299	0.220063	0	0.20930232558139533	0	0.0909090909090909	0	0.16924788863221163	0.15929583512237014	0.4384066156043231	0.33113061448817754	0.3939938470577422	0	0	0	0.011764705882352941	0.13055555555555556	0	0	0
2	Antiganda	0.226745	0.288213	0.186887	0	0	0	0	0	0.11367512406514294	0.05726040427508753	0.39174864849352	0.3078069769950016	0.2917196192745385	0	0	0	0.025974025974025976	0.12903874593315587	0	0	0
3	MICCIS	0.199876	0.223415	0.180824	0	0.08888888888888888	0	0	0	0.02565183779605032	0.09788359788359788	0.3214373264395019	0.23683119966504182	0.2467003700958885	0	0	0	0	0.11318311318311317	0	0	0
4	Stalin	0.145347	0.192091	0.116900	0	0.14095238095238094	0	0	0	0.16441512138723297	0.09937888198757763	0.3545057034762975	0.24910679755241114	0.24884450238929234	0	0	0.35294117647058826	0.010683760683760686	0.06857142857142856	0	0.07608695652173914	0.029411764705882353
5	CUNLP	0.131116	0.323463	0.082222	0.21220930232558138	0	0	0	0	0	0	0.19463654566968333	0.13019897622778537	0.1503792290043089	0	0	0	0	0.0026525198938992045	0	0	0
6	aschern	0.109060	0.071528	0.229464	0	0	0	0	0.052808220597070576	0.13135270381505712	0.0026720647773279356	0.1806828830282632	0.04832753175629566	0.05418554720553647	0	0	0	0.0006254560617116648	0.0033914728682170538	0	0	0
7	ProperGander	0.098969	0.065167	0.205634	0.008403361344537815	0.03225806451612903	0	0.060271451575799405	0.012731481481481483	0.10775955173652252	0.08046515473562439	0.3011571999234733	0.17602031662128412	0.23553478804261926	0	0	0.05555555555555555	0.020878642888212263	0.0967741935483871	0	0.01818181818181818	0
8	Sberiboba	0.045001	0.297428	0.024342	0	0	0	0	0	0.042598684210526316	0	0.1918785119073319	0.043107470680724756	0.08152799581371009	0	0	0	0	0.10540540540540541	0	0	0
9	BananasInPajamas	0.009543	0.009501	0.009585	0	0	0	0	0	0.017791732077446363	0	0	0.019920481817033545	0.013730635874924452	0	0	0	0	0	0	0	0
10	JUSTDeep	0.001194	0.015598	0.000621	0	0.021621621621621623	0	0	0	0.034371643394199784	0	0.009478672985781993	0.012598425196850394	0	0	0	0	0	0	0	0	0
11	Baseline	0.000015	0.011628	0.000008	0	0.04651162790697674	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
12	MindCoders	0.000000	0.000000	0.000000	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
13	SU	0.000000	0.000000	0.000000	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0

Scroll the table to the right to check the F1 values for each Technique.

Leaderboards on Development set

Task SLC

Rank	Team	F1	Precision	Recall
1	Tha3aroon	0.688312	0.610471	0.788904
2	KS	0.679930	0.598969	0.786198
3	CAUnLP	0.679420	0.594320	0.792963
4	ProperGander	0.676751	0.577438	0.817321
5	JUSTDeep	0.674534	0.623421	0.734777
6	ltuorp	0.670077	0.635152	0.709066
7	CUNLP	0.664994	0.619883	0.717185
8	aschern	0.664612	0.610419	0.729364
9	jinfen	0.661628	0.580020	0.769959
10	YMJA	0.660182	0.633873	0.688769
11	SBnLP	0.654837	0.567460	0.774019
12	guanggong	0.651054	0.573787	0.752368
13	LIPN	0.648418	0.588950	0.721245
14	Stalin	0.637736	0.595770	0.686062
15	Stano	0.637439	0.656160	0.619756
16	BananasInPajamas	0.627620	0.520499	0.790257
17	Kloop	0.623737	0.584615	0.668471
18	nlpseattle	0.620166	0.633286	0.607578
19	gw2018	0.603816	0.515820	0.728011
20	MindCoders	0.585834	0.526429	0.660352
21	NSIT	0.579468	0.661458	0.515562
22	Summer2019	0.556701	0.672414	0.474966
23	Antiganda	0.549051	0.660952	0.469553
24	Cojo	0.547200	0.669276	0.462788
25	Baseline	0.473485	0.443787	0.507442
26	gudetama	0.473485	0.443787	0.507442
27	test	0.473485	0.443787	0.507442
28	Visionators	0.441052	0.590909	0.351827
29	MaLaHITJuniors	0.307552	0.469444	0.228687

Task FLC

Rank	Team	F1	Precision	Recall	Appeal_to_Authority	Appeal_to_fear-prejudice	Bandwagon	Black-and-White_Fallacy	Causal_Oversimplification	Doubt	Exaggeration,Minimisation	Flag-Waving	Loaded_Language	Name_Calling,Labeling	Obfuscation,Intentional_Vagueness,Confusion	Red_Herring	Reductio_ad_hitlerum	Repetition	Slogans	Straw_Men	Thought-terminating_Cliches	Whataboutism
1	newspeak	0.242293	0.289301	0.208427	0	0.3770201154247968	0	0.1111111111111111	0	0.07844040734344718	0.324025974025974	0.4589255462717893	0.3949531362679053	0.2704833647206528	0	0	0.3292929292929293	0.04958677685950413	0.17777777777777778	0	0	0.14814814814814817
2	Antiganda	0.216513	0.226672	0.207226	0	0.1658852802550987	0	0	0	0.11887373111019248	0.2532775288816452	0.3660112046721024	0.33403644107167624	0.2633195264415603	0	0	0.19999999999999998	0.015384615384615384	0.16467192717192713	0	0.05714285714285715	0
3	Stalin	0.168767	0.231265	0.132862	0.06557377049180327	0.2320010055304173	0	0.06896551724137931	0	0.06518010291595197	0.1715318320822908	0.44605281339975217	0.35246955413475395	0.24090263713080168	0	0	0.2857142857142857	0.03502919099249375	0.17777777777777776	0	0.10612244897959185	0
4	ProperGander	0.145329	0.116368	0.193484	0.015517241379310343	0.1705878855617736	0	0.031746031746031744	0.03333333333333333	0.069590180986062	0.1397701514095719	0.4008421003601314	0.2375447387535778	0.1316358615846262	0	0	0.11764705882352941	0.04405286343612335	0.19047619047619047	0	0.10256410256410257	0
5	KS	0.136973	0.291299	0.089538	0	0	0	0	0	0	0.18181818181818182	0.07676767676767678	0.2720640702022972	0.04301075268817204	0	0	0	0.019417475728155338	0	0	0	0
6	CUNLP	0.122276	0.365199	0.073431	0	0.010846560846560847	0	0	0	0.0015408320493066256	0.15384615384615385	0.17095622173941752	0.23726903134510527	0.15780874951606655	0	0	0.11336898395721927	0	0.058823529411764705	0	0	0
7	aschern	0.101072	0.068483	0.192839	0	0.04617177700149344	0	0	0	0.049909161133650924	0.09462542761196371	0.17172530812662917	0.07442650706151756	0.032298682232056654	0	0	0.023129251700680274	0.008379054230582616	0.09040540540540541	0	0	0
8	gudetama	0.051770	0.031375	0.147932	0	0.05049533489632507	0	0	0.03496391339214114	0.03801245961618044	0.09666165809459393	0.16271050426933606	0.05557905057443975	0.011786494268046559	0	0	0	0	0	0	0	0
9	AMT	0.026584	0.204663	0.014215	0	0.09370748299319727	0	0	0	0	0.12506250625062507	0.20153340635268346	0	0.07936507936507936	0	0	0	0	0	0	0	0
10	esi	0.022256	0.030898	0.017392	0	0.0756756756756757	0	0.08695652173913043	0	0.027777777777777776	0.02702702702702703	0.08888888888888888	0.022058823529411763	0.04707406099140468	0	0.014743589743589747	0.5	0	0.04918032786885246	0	0	0
11	ltuorp	0.005423	0.003630	0.010722	0	0.0127214128498061	0.0007536606373815674	0	0	0.031988790926582904	0	0	0.0016712718775779784	0	0	0	0	0	0	0	0	0.005116549187789923
12	Baseline	0.001540	0.013661	0.000816	0	0	0	0	0	0	0	0	0.007122507122507123	0	0	0	0	0	0	0	0	0
13	CAUnLP	0.001540	0.013661	0.000816	0	0	0	0	0	0	0.007122507122507123	0	0	0	0	0	0	0	0	0	0	0
14	hola	0.001540	0.013661	0.000816	0	0	0	0	0	0	0	0	0.007122507122507123	0	0	0	0	0	0	0	0	0
15	JUSTDeep	0.001051	0.040312	0.000532	0.0741687979539642	0	0	0	0	0	0.04166666666666667	0	0	0	0	0	0	0	0	0	0	0

Scroll the table to the right to check the F1 values for each Technique.

Organisers

Giovanni Da San Martino, Qatar Computing Research Institute
Alberto Barrón-Cedeño, University of Bologna
Preslav Nakov, Qatar Computing Research Institute

Shared Task on Fine-grained Propaganda Detection @NLP4IF 2019

Leaderboard

Leaderboards on Test set

Task SLC

Task FLC

Leaderboards on Development set

Task SLC

Task FLC

Organisers

Contact