What should happen when two different raters score using the same rules?

When two different raters score using the same rules, the ideal outcome is that they aim for identical results. This reflects the concept of inter-rater reliability, which is a measure of how consistently different assessors evaluate the same performance or behavior. High inter-rater reliability indicates that the scoring criteria are clear and effectively applied by different raters, leading to consistent and comparable results.

In contexts where subjective measures are involved, efforts are made to train raters thoroughly on the scoring rules to minimize discrepancies. When raters adhere closely to established guidelines and interpretations, the expectation is that their scores will align closely, supporting the validity of the assessment process.

Variability in scores can occur due to the inherent subjectivity in interpreting responses, which is why thorough training and consensus on scoring guidelines are critical. However, the goal is always to achieve a level of agreement that reflects the intended accuracy and reliability of the assessment procedures.