Skip to main content

Table 3 Kappa scores within Group A and Group B, de monstrating the paradoxically low kappa scores despite high agreement.

From: Relevance similarity: an alternative means to monitor information retrieval systems

  Group A Group B
Evaluator 2 3 4 5 6 2 3 4 5 6
1 0.404 0.426 0.136 0.258 0.656 0.208 0.670 0.410 0.807 0.352
2   0.461 0.259 0.713 0.520   0.257 0.135 0.125 -0.001
3    0.180 0.438 0.439    0.440 0.643 0.353
4     0.241 0.270     0.370 0.250
5      0.404      0.330