Table 3 Kappa scores within Group A and Group B, de monstrating the paradoxically low kappa scores despite high agreement.
From: Relevance similarity: an alternative means to monitor information retrieval systems
| Group A | Group B | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Evaluator | 2 | 3 | 4 | 5 | 6 | 2 | 3 | 4 | 5 | 6 |
| 1 | 0.404 | 0.426 | 0.136 | 0.258 | 0.656 | 0.208 | 0.670 | 0.410 | 0.807 | 0.352 |
| 2 | 0.461 | 0.259 | 0.713 | 0.520 | 0.257 | 0.135 | 0.125 | -0.001 | ||
| 3 | 0.180 | 0.438 | 0.439 | 0.440 | 0.643 | 0.353 | ||||
| 4 | 0.241 | 0.270 | 0.370 | 0.250 | ||||||
| 5 | 0.404 | 0.330 | ||||||||