Applied Psychological Measurement RSS feed -- OnlineFirst Articles

The Individual Consistency of Acquiescence and Extreme Response Style in Self-Report Questionnaires

Weijters, B., Geuens, M., Schillewaert, N. — Fri, 09 Oct 2009 15:46:30 PDT

The severity of bias in respondents’ self-reports due to acquiescence response style (ARS) and extreme response style (ERS) depends strongly on how consistent these response styles are over the course of a questionnaire. In the literature, different alternative hypotheses on response style (in)consistency circulate. Therefore, nine alternative models are derived and fitted to secondary and primary data. It is found that response styles are best modeled as a tau-equivalent factor complemented with a time-invariant autoregressive effect. This means that ARS and ERS are largely but not completely consistent over the course of a questionnaire, a finding that has important implications for response style measurement and correction.

Gencorr: An R Routine to Generate Correlation Matrices From a User-Defined Eigenvalue Structure

Jones, J. A. — Mon, 17 Aug 2009 12:59:43 PDT

In the Eye of the Beholder: Quantifying Individuals' Preferences and Biases Using Peer Nominations

Camparo, J., Camparo, L. B., Wagner, J. T. — Mon, 17 Aug 2009 12:59:45 PDT

Peer nominations are used widely in psychological and sociological research to examine intergroup dynamics, even though this assessment tool suffers from thorny methodological problems: Gender, ethnic, age, and trait compositions vary across subsamples, subjects differ in the number of nominations they make, and the issue of sampling without replacement is often ignored. To overcome these problems, the authors have developed a differential index for the peer nominations procedure, with the number of individuals chosen above or below chance as the unit of measure. The index is specifically designed for use with individuals (as opposed to subgroups like classrooms), so that group means, confidence intervals, and standard deviations are readily determined, opening the door to research designs that go beyond null hypothesis testing. Here, we introduce the index and illustrate its unique capabilities in a study examining preference and bias reciprocity among fourth-grade students.

Three Approaches to Using Lengthy Ordinal Scales in Structural Equation Models: Parceling, Latent Scoring, and Shortening Scales

Yang, C., Nay, S., Hoyle, R. H. — Mon, 17 Aug 2009 12:59:44 PDT

Lengthy scales or testlets pose certain challenges for structural equation modeling (SEM) if all the items are included as indicators of a latent construct. Three general approaches to modeling lengthy scales in SEM (parceling, latent scoring, and shortening) have been reviewed and evaluated. A hypothetical population model is simulated containing two exogenous constructs with 14 indicators each and an endogenous construct with four indicators. The simulation generates data sets with varying numbers of response options, two types of distributions, factor loadings ranging from low to high, and sample sizes ranging from small to moderate. The population model is varied to incorporate one of the following: (a) single parcels, (b) various parcels as indicators of two exogenous constructs, (c) latent scores as observed exogenous variables, and (d) four and six individual items as indicators of two exogenous constructs. The dependent variables evaluated are biases in the covariance and partial covariance population parameters. Biases in these parameters are found to be minimal under the following conditions: (a) when parcels of indicators of five response options are used as indicators of two latent exogenous constructs, (b) when latent scores are used as observed variables at sample sizes above 100 and with indicators that are relatively less skewed in the case of dichotomous indicators, and (c) when four or six individual items with high or diverse factor loadings are used as indicators of two exogenous constructs. These findings provide guidelines for resolving the inconsistency of findings from applying various approaches to empirical data.

Item Parameter Estimation for the MIRT Model: Bias and Precision of Confirmatory Factor Analysis-Based Models

Finch, H. — Mon, 17 Aug 2009 12:59:44 PDT

The accuracy of item parameter estimates in the multidimensional item response theory (MIRT) model context is one that has not been researched in great detail. This study examines the ability of two confirmatory factor analysis models specifically for dichotomous data to properly estimate item parameters using common formulae for converting factor loadings and thresholds to discrimination and difficulty indices. The two MIRT estimation methods included in this research, unweighted least squares (ULS) and robust weighted least squares (RWLS), and the unidimensional estimation approach used are accessible in the widely distributed software packages NOHARM, Mplus, and BILOGMG, respectively. These techniques have been assessed in terms of the overall accuracy, bias, and standard error of item parameter estimates under a variety of sample sizes, test lengths, intertrait correlations, pseudo-guessing, and latent trait distribution conditions. Results indicate that there exists a complex relationship between these manipulated factors and the estimation accuracy of these methods. Recommendations for practice in light of these results are provided.

Variations on Stochastic Curtailment in Sequential Mastery Testing

Finkelman, M. D. — Mon, 17 Aug 2009 12:59:44 PDT

In sequential mastery testing (SMT), assessment via computer is used to classify examinees into one of two mutually exclusive categories. Unlike paper-and-pencil tests, SMT has the capability to use variable-length stopping rules. One approach to shortening variable-length tests is stochastic curtailment, which halts examination if the probability of changing classification decisions is low. The estimation of such a probability is therefore a critical component of a stochastically curtailed test. This article examines several variations on stochastic curtailment where the key probability is estimated more aggressively than the standard formulation, resulting in additional savings in average test length (ATL). In two simulation sets, the variations successfully reduced the ATL, and in many cases the average loss, compared with the standard formulation.

Model Selection Indices for Polytomous Items

Kang, T., Cohen, A. S., Sung, H.-J. — Mon, 17 Aug 2009 12:59:45 PDT

This study examines the utility of four indices for use in model selection with nested and non-nested polytomous IRT models: a cross-validation index and three information-based indices. Four commonly used polytomous IRT models are considered: the graded response model, the generalized partial credit model, the partial credit model, and the rating scale model. In a simulation study, comparisons among the four indices suggest that model selection is dependent to some extent on the particular conditions simulated. Overall, the BIC index appears to be most accurate in selecting the correct polytomous IRT model. Results are presented from analysis of a real data set to illustrate the use of the four indices for selecting an appropriate model.

Eqboot and Eqwinboot: Java Applications for Estimating Equating Constants and the Standard Error of Equating Using IRT Methods

Meyer, J. P. — Sat, 25 Jul 2009 10:43:01 PDT

An Iterative Maximum a Posteriori Estimation of Proficiency Level to Detect Multiple Local Likelihood Maxima

Magis, D., Raiche, G. — Sat, 25 Jul 2009 10:43:01 PDT

In this article the authors focus on the issue of the nonuniqueness of the maximum likelihood (ML) estimator of proficiency level in item response theory (with special attention to logistic models). The usual maximum a posteriori (MAP) method offers a good alternative within that framework; however, this article highlights some drawbacks of its use. The authors then propose an iteratively based MAP estimator (IMAP), which can be useful in detecting multiple local likelihood maxima. The efficiency of the IMAP estimator is studied and is compared to the ML and MAP methods by means of a simulation study.

Multilevel, Two-Parameter, and Random-Weights Generalizations of a Model With Internal Restrictions on Item Difficulty

Wang, W.-C., Jin, K.-Y. — Mon, 18 May 2009 09:02:46 PDT

In this study, all the advantages of slope parameters, random weights, and latent regression are acknowledged when dealing with component and composite items by adding slope parameters and random weights into the standard item response model with internal restrictions on item difficulty and formulating this new model within a multilevel framework in which Level 2 predictors are added to account for variation in the latent trait. The resulting model is a nonlinear mixed model (NLMM) so that existing parameter estimation procedures and computer packages for NLMMs can be directly adopted to estimate the parameters. Through simulations, it was found that the SAS NLMIXED procedure could recover the parameters in the new model fairly well and produce appropriate standard errors. To illustrate applications of the new model, a real data set pertaining to guilt was analyzed with gender as a Level 2 predictor. Further model generalization is discussed.