Open Collections

Inferential procedures for multifaceted coefficients of generalizability

University of British Columbia

Generalizability theory was developed by Cronbach as an alternative to classical test score reliability theory. Generalizability uses an experimental design approach to reliability that permits the systematic evaluation of several sources of error simultaneously. The coefficient of generalizability (CG) is a single number summarizing the dependability of the measurement process. In the present study a normalizing transformation was first applied to a function of the CG. The delta method was applied to the transformed CGs for four different two-facet experimental design models to develop asymptotic variance expressions for the CGs. The accuracy of the variance expressions was tested via Monte Carlo simulations. In .these simulations the Type I error control was investigated. The majority of the simulations were conducted using a two-facet fully random experimental design, corresponding to a three-way random effects analysis of variance. A total of 81 combinations of sample size, facet conditions, and population CG values were investigated. The results suggested that the procedure generally was precise in its control of Type I error. The results were somewhat less precise when only two facet conditions were sampled. Five other side studies were conducted. Three of these used other two-facet models: a design with one fixed facet, a design with a finite facet, and a design with a nested facet. The results of these studies were similar to those found in the larger study; generally good Type I error control was realized. An additional study looked at the performance of the variance expression in the presence of negative variance component estimates. Results in this section of the study suggested that such negative component estimates did not adversely affect Type I error control. The final study investigated the performance of the variance expression with dichotomous data. The results indicated that Type I error control was not as precise with two facet conditions as it was with five or eight conditions. In these latter cases good error control was realized.

Text

2010-08-07

For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.

http://hdl.handle.net/2429/27193

Psychology

University of British Columbia

Graduate