Better Quality in Assessments: Consideration of Contextual Effects on Item Bias and Differential Item Functioning

Vol. 4 No. 2

Year: 2008

Issue: Sep-Nov

Title: Better Quality in Assessments: Consideration of Contextual Effects on Item Bias and Differential Item Functioning 

Author Name: Brandon K. Vaughn 

Synopsis: 

This study considers the importance of contextual effects on the quality of assessments on item bias and differential item functioning (DIF) in measurement.  Often, in educational studies, students are clustered in teachers or schools, and the clusters could impact psychometric issues yet are largely ignored by traditional item analyses.  A statistical model for incorporating these cluster effects will be presented.  By considering DIF from this perspective, it is possible for DIF to be consistent or vary across clusters (e.g., teachers or schools).  By using this model, researchers can be shown much more detail into the nature and source of DIF and provide greater quality in the analysis and use of their assessments.  This can be of benefit in determining whether the nature of DIF is exclusively due to student attributes, or a particular combination of student and school attributes.  In addition to this, DIF may exist only among clusters, but not for students.  This is an extra detection not possible with traditional DIF analysis.  For example, in some educational situations, DIF may not exist among the subjects of interest, but in certain type of schools.  The use of this procedure will be demonstrated using real assessment data.