Justice-Oriented, Antiracist Validation: Continuing to Disrupt White Supremacy in Assessment Practices

Figures & data

Table 1. Typical performance categories for the Trait of Organization in scoring a first-year college writing argumentative essay.

5	4	3	2	1
Overall structure, organization, and paragraph construction are appropriate to the assignment and an academic audience. All ideas in the paper flow logically. Transitions show originality and sophistication.	Overall structure, organization, and paragraph construction are appropriate to the assignment and an academic audience, though may be less evident and understandable in some places. Most ideas in the paper flow logically.Transitions are adequate but may be unclear or missing at times.	Overall structure, organization, and paragraph construction are readable, somewhat appropriate to the assignment and an academic audience, though may be a bit awkward in some places. The paper does not always flow logically and make sense.Transitions are formulaic and may be few or weak.	The paper lacks coherence, providing no discernable argument; ideas do not flow logically and do not make sense. Overall structure, organization, and paragraph construction are difficult to read, or inappropriate for audience. Transitions are confusing or nonexistent.	The writing is very difficult to understand owing to major problems with organization and structure.

Table 2. JAV six questions for the design of argumentative writing construct definition: example case study from University of Michigan (table from Randall, 2021).

Purpose: Articulate explicitly to stakeholders a commitment to justice in assessment practices (Organizational)	As we have discussed with stakeholders, the purpose of this writing assessment is to gauge what a student who is entering college knows about academic essay writing. Based on a review of assessment responses, this process will help students select a first-year writing (FYW) course or a Pre-FYW course. (See Gere et al., 2013)
Positionality: Reflect upon assumptions (Individual)	The design team includes three U.S. citizens—a Black woman from the South, a Latina who was born in Peru, and a white woman who identifies as Appalachian. The design team also includes a Canadian citizen—a white man. We have a shared commitment to a JAV approach.
People/Places: Consider who is being assessed and what sociocultural and racial identities they bring. (Organizational)	The enrolled student population at University of Michigan-Ann Arbor is 65% White, 15% Asian, 6% Hispanic or Latino, 5% Black or African American, 1% American Indian or Alaska Native, and 10% unknown. Students come from 50 states and 122 countries (University of Michigan, 2023). University of Michigan students are high-achieving students, meaning they have a familiarity with traditional academic writing, but it is also known that entering students do not access assessment prompts in the same way (Aull, 2021).
Power: Name those groups who hold the power, or are being privileged or harmed, by the way in which the construct is defined (Organizational)	Recent research has shown that the assessment has resulted in a “process at this institution [that] results in the unintended disproportionate placement and enrollment of domestic under-represented minority (URM) students and women in a pre-FYW course, which means that they take two courses rather than one to fulfill the FYW requirement” (Tinkle et al., 2022, p. 2).
Processes: Evaluate the procedures used for defining the construct. (Organizational)	Given the previous validation research related to this writing assessment, we have begun a process to better understand how admitted students understand the construct of academic writing and the associated consequences of the assessment. That process includes interviews with a purposeful sample of students as well as a critical review of the research base that informs our current understandings of the writing construct with respect to what populations this research has been conducted with.
Products & Consequences: Carefully consider the consequences of the construct as defined (Organizational)	Before recommending any curricular or policy changes, we question how the construct of argumentative writing should change, given the ongoing changes in academic writing related to AI and technological advances.

Randall, J. (2021). “Color-neutral” is not a thing: Redefining construct definition and representation through a justice-oriented critical antiracist lens. Educational Measurement Issues & Practice, 40(4), 82–90. https://doi.org/10.1111/emip.12429

 Google Scholar

Gere, A. R. Aull, L. Perales, M. D. Escudero, Z. L. & Vander Lei, E.(2013). Local assessment: Using genre analysis to validate directed self-placement. College Composition and Communication, 64(4), 605–633.

 Web of Science ®Google Scholar

University of Michigan. (2023). Data & reports. Diversity, equity, and inclusion. https://diversity.umich.edu/data-reports/

 Google Scholar

Aull, L.(2021). Directed self-placement: Subconstructs and group differences at a US university. Assessing Writing, 48. Article 100522. https://doi.org/10.1016/j.asw.2021.100522

 Web of Science ®Google Scholar

Tinkle, T. Godfrey, J. Menon, A. R. Moos, A. Romaine, L. & Sprouse, M.(2022). (In)equities in directed self-placement. Assessing Writing, 54, Article 100671.

 PubMed Web of Science ®Google Scholar