Updated 9-9-02

WEB REFERENCES These are web-sites that I've found that have material related to the topic of Measurement.

Levels of Measurement http://chss.montclair.edu/sociology/statbooklevels.htm

The FAQ of measurement theory by Warren SArle from the SAS Institute gives an overview. This FAQ was taken from the web at ftp://ftp.sas.com/pub/neural/measurement.faq

More notes on measurement with even more links elswhere http://faculty.ncwc.edu/toconnor/308/308lect04.htm

From the American Psychologist "Statistical Methods in Psychology Journals: Guidelines and Explanations" Leland Wilkinson and Task Force on Statistical Inference APA Board of Scientific Affairs

Readings: Chapter 5 of Maruyama (1998) pp. 79-88, Chapter 7.2 and 7.3 in Kline (handout in class),
an article: Little, Cunningham, Shahar (2002), "To Parcel or Not to Parcel: Exploring the Question, Weighing the Merits" Structural Equation Modeling, 9(2), 151-173. for extra reading on the subject see chapter 6 of Bollen (1989),

Measurement

There is a huge literature and long history related to this fundamental subject in Psychology. In the SPH you can take Todd Rockwoods class entitled ``Measurement of Health Related Social Factors'' - PubH8813

• nominal - discrete classification, no ordering
• ordinal - values are not just different they stand in some relation to the other values, i.e. the values can be ranked
• interval - ordinal plus distance between values on the scale are considered to be known and consistent across the scale (i.e. an increase from 1 to 2 means the same as an increase from 3 to 4)
• ratio - interval plus there is a theoretical zero point.

SCALES

Sets of items (usually questionnaire items) which are thought to measure a latent variable

• Combine the items in some way to get a scale score for each individual (MOST COMMON TECHNIQUE)
  • Use this scale score (which is likely to be more valid and reliable than any one single item in the scale) as an indicator for the underlying latent variable or a more general latent variable
  • Use this scale score as if it is the true value of the latent variable
• Use each of the items in the scale as a separate indicator of the latent variable. (this is reasonable if each item is known to have good validity and reliability (not usually true of a single question))
• Arbitrarily combine some of the items in the scale to reduce the number of items (parceling) and then use the parcels as indicators of the latent variables. (See Little, Cunningham, Shahar (2002) article, also http://tigersystem.net/aera2002/viewproposaltext.asp?propID=5893 )

HOW TO COMBINE? - Lots of Debate, No single best answer

• Depends on the type of scale - Thurstone scales, Guttman Scales, Likert Scales
• Likert Scales - Very common type of scale (e.g. usual response categories are ``strongly agree'', ``agree'', ``don't know'', ``disagree'', and ``strongly disagree'')
  • ordinal within any item but items within the scale do not necessarily follow any order (as in Guttman Scales)
  • Likert items are often assigned integer values and treated as if they were interval measures
• When Likert Items are treated as ordinal - latent trait models are used to combine
• When Likert Items are treated as interval - principle component analysis or exploratory factor analysis or simply adding-up are common techniques used to combine.

Validity - do the measures actually measure what they claim to

• Not an issue unique to latent variable models, it is relevant in any area of scientific investigation
• Four traditional validity types
  • content validity (conceptual argument)
  • criterion validity (empirical) does the construct correlate with external variables the way you would expect.
  • convergent-discriminant validity (empirical) do indicators of the same construct correlate more than indicators of different constructs

More about validity Tutorial on Internal validity, Athabasca University

Reliability - Consistency of measurement

• A measure is judged reliable when it consistently produces the same results, particularly when applied to the same subjects at different time periods when there is no evidence of change
• A measure is reliable when it does not contain much random error (measurement error)
• The proportion of variability in a measure that is explained by the variability of the true underlying latent score.

Example from Maruyama p. 83 The goal is to have some measure of social economic status (SES). Only measure available is Family Size. Family Size can be thought of as a measure of SES plus other things. The variability in the measure of family size therefore comes from the variability of SES (common variance), other non-random factors, e.g. religious practice, cultural values, fertility (unique variance), and random error, e.g. coding errors, mistaken answers due to separations or divorce (error variance).

The achievement of standards of validity and reliability requires time and effort. It is a powerful reason for using existing scales.

EXAMPLES

• Heatlh Locus of Control. Derived from Social Learning Theory. The Social Learning Theory states that an individual learns on the basis of his or her history of reinforcement. The individual will develop general and specific expectancies. Through a learning process individuals will develop the belief that certain outcomes are a result of their action (internals) or a result of other forces independent of themselves (externals).

  Theoretically 3 domains: internal, chance, powerful others.

  We will look at data collected from class

• Body satisfaction
• Self-Esteem

What is the effect on regression, or on correlations, of a measure containing measurement error? CLICK HERE

Definitions

When combining different measures either into a scale or considering using them as separate indicators of a latent variable, we should consider the following three definitions: Parallel, tau-equivalent, and congeneric measures. These terms can be defined by examining the two measures x₁ and x₂:

x₁ = f + e₁

x₂ = f + e₂

If $\alpha_1=\alpha_2$ and Var(e₁)=Var(e₂), then x₁ and x₂ are parallel measures.

If $\alpha_1=\alpha_2$ but Var(e₁) Var(e₂), then x₁ and x₂ are tau-equivalent measures

If and Var(e₁) Var(e₂) then x₁ and x₂ are congeneric measures

These definitions are important to consider when adding up measurable variables to create a scale.

Main techniques of estimating reliability

• test-retest assumes parallel measures
• alternative forms
• split-halves
• Cronbach's alpha (makes least restrictive assumptions) assume tau-equivalent measures
  • formula - why is this measuring reliability?

The techniques assume that the measure is influenced by only one latent variable, unidimensionality.

CLICK HERE for notes on how these techniques actually help us estimate the reliability

Cronbach Alpha Example

from Hatcher, L. (1994) A Step-by-Step guide to factor analysis and SEM. Example from Chapter 3 of his book.

(See Handout from Class)

A Meta Analysis of Chronbach's Coefficient Alpha

from Peterson (1994) Journal of Consumer Research, 21, 381-391.

(See Handout from Class)

Good References for Cronbach Alpha

Miller, M.B. (1995) "Coefficient Alpha: A basic introduction from the perspectives of classical test theory and structural equation modeling". Structural Equation Modeling, 2, 255-273.

Bravo, G. and Potvin, L. (1991) "Estimating the reliability of continuous measures with Cronbach's alpha or the intraclass correlation coefficient: Toward the integration of two traditions" Journal of Clinical Epidemiology, 44, 381-390.

Raykov, T. (1997) ``Scale reliability, Cronbach's coefficient alpha, and violations of essential tau-equivalence with fixed congeneric components''. Multivariate Behavioral Research,32, 329-353.

About cronbach alpha with multifactor measures http://io.psy.msu.edu/Schmitt/ormalpha.htm

And perhaps a useful discussion can be found in this e-mail taken from SEMNET

Cronbach's alpha

• $x_1, x_2, \ldots , x_k$ items in a scale
• $H=\sum_{i=1}^k x_i$
• Cronbach's alpha $= \alpha = \frac{k}{k-1} \left( 1 - \frac{\sum_{i=1}^{k}\widehat{Var x_i}}{\widehat{Var H}} \right)$
• $\alpha$ is consistent for the reliability of H under the assumption that the $x_1, x_2, \ldots , x_k$ are tau-equivalent (each item measures the latent variable with the same units but possibly different origin and precision)

WHAT IF $x_1, x_2, \ldots , x_k$ are NOT tau-equivalent?

• $\alpha$ is a lower bound for the true reliability
• How far off can $\alpha$ be from the true reliability?
• Raykov (1997) gives an upper bound on the discrepancy between $\alpha$ and the true reliability. This bound is given by
  
  $$\frac{\sum_{i = 1}^{k}(b_i - \bar{b})^2}{k(k-1)\bar{b}^2}$$
  
  
  where b_i are the scalars in front of the latent variable $\tau$, i.e. $x_i = c_i + b_i \tau + e_i$.
• Note if the items in the scale are essentially tau-equivalent then all the b_i's are equal so they have no variability and $\alpha$ is consistent for the reliability
• Notice that the discrepancy increases as the variability of the b_i's increases.

Cronbach's alpha

• Is $\alpha$ the same whether you are creating a scale score by summing or by averaging? YES
• How many categories should each item have that makes up the scale?
  • Conventionally, people act like it doesn't matter (statistical properties should probably be examined more closely)
  • There is a formula for calculating reliability for dichotomous items called the Kuder-Richardson Formula 20. Gives same result as Cronbach $\alpha$.
• Cronbach $\alpha$ is only a measure of reliability when each item is actually measuring the same one latent variable, i.e. unidimensionality.
• Example where Cronbach $\alpha$ is not a measure of reliability:
  • Measuring total calcium intake, e.g. How often did you drink milk, How often did you eat yogurt, etc.
  • Appropriately summing the answers to these questions can give an estimate of total calcium intake
  • Cronbach $\alpha$ for this set of items means nothing
  • To measure reliability you would have to measure Total Calcium intake in some other ways (e.g. meal recall lists) and compare.

Examining Cronbach alpha output: To standardize or not to standardize items that are summed into a scale?

• My opinion is usually NO
• Conventionally, if items are on different metrics then standardize before combining
  • Cautions: Why are you trying to combine them if they are on different metrics (remember tau-equivalence)?
  • Cautions: Standardizing eliminates means.
  • Cautions: Each individuals score is only meaningful as it relates to the rest of the people in the sample
• If items are on different Likert categories you can combine by translating one into the other (it is not necessary to standardize)

Items measured using Likert scales with different number of categories

If items are measured using Likert scales with different numbers of categories, combine by first translating one scale to the other.

For example,

X₁ Likert type with 4 categories:

1	2	3	4
$\Vert$	$\Vert$	$\Vert$	$\Vert$
very difficult			very easy

X₂ Likert type with 5 categories:

1	2	3	4	5
$\Vert$	$\Vert$	$\Vert$	$\Vert$	$\Vert$
very difficult				very easy

We want 1 in 4 point scale to be 1 in 5 point scale and we want 4 in 4 point scale to be 5 in 5 point scale; that is, just stretch the 4 point scale into 5 point scale.

X1	X1*
1	1
4	5

$X_1^*=\frac{4}{3}X_1-\frac{1}{3}$

$(X_1^*-1)=\frac{4}{3}(X_1-1)$

or if instead you want to squeeze the 5 point into the 4 point scale:

X2	X2*
1	1
5	4

$X_2^*=\frac{3}{4}X_2+\frac{1}{4}$

$(X_2^*-1)=\frac{3}{4}(X_2-1)$

Multi-trait Multi-Method

• Originated by Campbell and Fiske (1959)
• Two or more traits (latent variables) are measured with two or more methods
• A way to empirically check construct validity
• Also a way to measure/examine a method effect (the observation that correlation are inflated among variables measured with the same method)