TY - JOUR AU - Dong, Peng AU - Loh, Marie AU - Mondry, Adrian PY - 2005 DA - 2005/07/20 TI - Relevance similarity: an alternative means to monitor information retrieval systems JO - Biomedical Digital Libraries SP - 6 VL - 2 IS - 1 AB - Relevance assessment is a major problem in the evaluation of information retrieval systems. The work presented here introduces a new parameter, "Relevance Similarity", for the measurement of the variation of relevance assessment. In a situation where individual assessment can be compared with a gold standard, this parameter is used to study the effect of such variation on the performance of a medical information retrieval system. In such a setting, Relevance Similarity is the ratio of assessors who rank a given document same as the gold standard over the total number of assessors in the group. SN - 1742-5581 UR - https://doi.org/10.1186/1742-5581-2-6 DO - 10.1186/1742-5581-2-6 ID - Dong2005 ER -