Teaching Psychometrics: The Importance of Validity in Assessment Design

Abstract

Psychometrics is the field of designing tests and assessments to measure certain psychological concepts. It is chiefly concerned with two fundamental properties: reliability and validity. These properties are often influenced by confounding variables: other things that can influence performance but are not what you are trying to measure. Here, I describe how teaching an undergraduate course on psychometrics has reframed my views of assessments in teaching. The goal of any assessment is to accurately, or validly, measure students’ knowledge and learning. This raises three questions that can influence assessment design, course structure, and teaching style: 1) Do deadlines really matter? 2) Should all assessments be offered multiple times? 3) How much do my assessments really measure student knowledge on this topic? In discussing each of these, I consider the ways in which evaluating how validity plays a role in our assessment design and teaching style can help us minimize confounding variables and ensure that students’ grades reflect their learning.

Keywords:

• Assessment
• psychometrics
• reliability
• validity

Notes

1 Several studies have found that students who submitted an assignment early received higher grades than those who submitted closer to a deadline (Nicolau, 2015; Stoneham, 2008). Stoneham (2008) proposed that some reasons for this could be that these were the “better” students, and/or that the structured deadline (and its associated penalty) helped them focus and reduce their procrastination, which resulted in an earlier completion.

2 A small number of extra-credit points could even be given for completing the self-assessment rubric, if you don’t want to make this a mandatory part of the assignment.