Is This Discipline a Science: Quantification?

Since working in the UK, I have become very appreciative of the critical thinking education I received in New Zealand. Many of the people who I meet in my professional life are well-educated but seem to lack the creative thinking that is synonymous with New Zealand and the ‘number-eight-wire’ mentality.

A particular area where this has become apparent for me is in the seemingly blind acceptance of psychology as a quantifiable discipline. This logic is crucial to the acceptance for example, of the current BPS guideline for level A and level B training. In New Zealand, this has been hotly debated over the last 10 years with the result being that New Zealander’s are far less likely to believe that tests can provide answers in a simple quantifiable way.

Going back over my notes, letters, and general correspondence from the past few years, I came across an interesting post by a man I respect very much. Professor Paul Barrett is, in many ways, what a good academic should be in that he continually challenges his own thinking with the latest literature and thinking in the field. While having a grasp of the quantifiable side of the discipline that is second to only a few, he has had the intellectual courage to challenge the fundamental premise of the discipline.

Professor Paul Barrett writes:

“In 1997, one of the most profound and far-reaching papers in psychology was published, which for me, changed my entire perspective on psychology as putative quantitative science. Since that time and many follow-up papers, it has become clear to me that given no evidence to the contrary, one either “believes” (as in a religion) that variables in psychology possess a quantitative structure, and so proceed to utilize factor analysis and other methods which require this assumption to be true, or one accepts that for now, psychology is a non-quantitative science, and proceeds accordingly. For me, given the lack of any evidence to the contrary, psychology right now is a non-quantitative science”.

Michell, J. (1997) Quantitative science and the definition of measurement in Psychology. British Journal of Psychology, 88(3), 355-383.

Maraun, M.D. (1997) Appearance and Reality: Is the Big Five the Structure of Trait Descriptors? Personality and Individual Differences, 22(5), 629-647.

Michell, J. (1994) Numbers as quantitative relations and the traditional theory of measurement. British Journal for the Philosophy of Science, 45, 389-406.

Michell, J. (1999) Measurement in Psychology: Critical History of a Methodological Concept. Cambridge University Press. ISBN: 0-521-62120-8.

Michell. J. (2000) Normal science, pathological Science, and psychometrics. Theory and Psychology, 10(5), 639-667.

Michell, J. (2004) Item Response Models, pathological science, and the shape of error. Theory and Psychology, 14 (1), 121-129.

Since Professor Paul Barrett wrote that email, there has been another paper by Michell on the topic of the abstract for which describes the non-quantifiable nature of the discipline well:

Michell. J. (2009). The psychometricians’ fallacy: Too clever by half? British Journal of Mathematical and Statistical Psychology, 62(1), 41-55.

Abstract

“The psychometricians’ fallacy concludes that an attribute is quantitative from the premise that it is ordinal. This fallacy occupies a central place in the paradigm of psychometrics. Most of the founders of the discipline commit it and it makes sense of otherwise anomalous developments within the discipline, such as the permissible statistics controversy and the dominant form taken by item response theories. The fallacy is displayed by showing (1) that an attribute’s quantitative structure reduces to a weak order upon differences between degree,  that satisfies the double cancellation, solvability, and Archimedean conditions of conjoint measurement theory and (2) the fact that any order on the degrees themselves does not entail sufficient structure on this weak order to guarantee satisfaction of these conditions. Thus, it is possible that an ordered attribute is non-quantitative. Also, each pair of differences between degrees of an ordinal attribute falls into one of two disjoint classes: (1) those where the order relation between the pair follows from an order on the attribute and (2) those where it is independent of that order and possibly diagnostic of quantitative structure and this fact means that the distinction between order and quantity is an empirical one”.

The point is that the very foundation of psychology as a quantifiable science is on shaky ground. The failure of I/O psychologists to recognise this has resulted in the blind acceptance that psychometric assessments can ultimately lead to predictions that can be quantified to a large extent. It is simply not my belief that this is the case. Answers never reside in assessments but in the system thinking that allows assessment to be just one part of a full solution.