The VAL-Ed is designed, developed and tested to be both reliable (i.e., provide accurate measurement) and valid (i.e., measure leadership behaviors that lead to improved student achievement). To accomplish these goals, we followed a multi-stage development process that involved cognitive labs, pilot tests, and field tests. At each stage of the design and develop process, the properties of the instrument were investigated through empirical study and expert review. The process was guided by the American Educational Research Association's 1999 Standards for Educational and Psychological Testing. Responses indicated strong content validity and guided further item revisions.
- Pilot/Cognitive Interviews: To investigate assessment format, item clarity, and item importance, two rounds of cognitive interviews were performed with panels of practicing educators. Responses indicated practitioner preferences and understanding about assessment format and item-level language.
- School Pilot Test: To examine how VAL-ED works in a field setting, two rounds of pilot testing were conducted involving first nine schools in an urban district and later 11 schools in four districts. Data from the pilots enabled us to begin establishing reliability for the twelve scales and construct validity for the assessment using factor analysis.
- Cognitive Interviews of Online Prototype: To examine the utility of an online prototype of the VAL-ED and test changes made after the nine-school pilot, cognitive interviews were conducted with teachers, principals, and supervisor from three schools. Responses indicated a preference for the online assessment and helped identify important issues to address in future development.
- Bias Review: The instrument was submitted to a bias review committee of urban district leaders to ensure the assessment does not advantage respondents for anything other than the leadership behaviors we seek to measure.
- Field Test: In Spring, 2008, the instrument was field tested in approximately 300 schools. Responses were used to establish validity and scale reliability.
- Proficiency Standards: A panel of education leaders was convened, using a modified Bookmark approach to set "national" proficiency standards based on pilot study/field test data.
- Known Group: This validity study identifies school principals that are considered most or least successful by their supervisors and examines whether VALED accurately distinguishes the groups.
- Test/Retest: This study estimates the reliability of VAL-ED scores over time.
- Convergence/Divergence: This validity study investigates the degree to which VAL-ED survey measures are similar to an established measure of instructional leadership: Principal Instructional Management Rating Scale, and the extent to which VALED measures instructional leadership differently from those identified as trait emotional intelligence as measured by the TEIQue.
- Consequences: This validity study involves districts that have used VALED for at least one year and examines what uses are made of VAL-ED results and to what effect.
- Longitudinal Effect: Schools that have used the VAL-ED in two consecutive springs are included in this study that examines whether VAL-ED results can be used to predict value added to student achievement.
Read more about VAL-ED Research