In my previous role in the UK, I was often confronted by very complex measures of psychological traits. This included the likes of multi-faceted competency models, complex appraisal forms, and measures of engagement with more scales than a grand piano.
Having factor analysed the results of many of these models, I can say that I rarely see them hold up. What does result is a far simpler structure of a few key constructs that account for most of the variability in job performance. I’m always reminded of the law of parsimony which results in simple models often being the finest.
In I/O psychology, we have examples where this is the case. Professor Paul Barrett, one of the most influential people in my career, was instrumental in creating a single psychometric tool that, while never fully commercialised, was a real innovation in our field.
For those that may have forgotten the role of parsimony, I draw your attention to some older papers that are often forgotten in this field. These indicate that simplicity will be more beneficial than complexity when measuring human behaviour.
Scarpello and Campbell (1983) in Personnel Psychology looked at whether a single item (1-5 scale) global measure of job satisfaction was equivalent to the sum of facet satisfactions. They concluded that the whole is more complex than the sum of the parts, and may in fact be more inclusive than facet measures.
Wanous, Reichers, and Hudy (1997) in the Journal of Applied Psychology evaluated single item measures of job satisfaction and concluded that they can be used instead of facet measures in some instances, including practical considerations of face validity, cost, and time. They suggested test-retest reliability of .70. A subsequent article by Wanous and Hudy (2001) in Organisational Research Methods confirmed this, looking at teacher effectiveness but with many of the same arguments.
OPRA Blog 