Validity is the most fundamental consideration in psychological measurement. While reliability asks whether a test measures consistently, validity asks whether it measures the right thing. Modern validity theory, consolidated in the Standards for Educational and Psychological Testing, treats validity as a unitary concept — the degree to which evidence and theory support the interpretations of test scores for proposed uses. The traditional "trinitarian" distinction among content, criterion, and construct validity has been reframed as different types of evidence contributing to a single validity argument.
Sources of Validity Evidence
Content validity evaluates whether the test items adequately sample the domain of interest. It is typically established through expert judgment and systematic domain analysis. Criterion-related validity assesses the empirical relationship between test scores and an external criterion — concurrently (measured at the same time) or predictively (measured later). The validity coefficient r_XY, the correlation between test and criterion, is the primary quantitative index.
where ρ_XY = observed validity coefficient
ρ_XX′ = reliability of the test
ρ_YY′ = reliability of the criterion
The correction for attenuation formula, due to Spearman, reveals the theoretical correlation between the true scores underlying the test and criterion. It demonstrates that measurement error in either variable attenuates the observed validity coefficient. A test with reliability 0.80 and a criterion with reliability 0.70 can achieve at most an observed validity of √(0.80 × 0.70) ≈ 0.75, even if the true-score correlation is perfect.
Construct Validity and the Nomological Network
Cronbach and Meehl (1955) introduced construct validity as the overarching framework for evaluating whether a test measures a theoretical construct. They proposed that constructs are embedded in a nomological network — a system of lawful relationships among constructs and between constructs and observables. Validating a test requires showing that it behaves as the theory predicts within this network.
Campbell and Fiske (1959) proposed the MTMM matrix as a systematic method for evaluating convergent and discriminant validity. Convergent validity requires that different methods of measuring the same trait correlate highly (monotrait-heteromethod values). Discriminant validity requires that measures of different traits using the same method do not correlate too highly (heterotrait-monomethod values). The pattern of correlations in the full matrix reveals the relative contributions of trait and method variance.
Modern Validity Frameworks
Messick (1989) expanded validity theory to encompass the social consequences of test use, arguing that construct validity subsumes all other forms and that the value implications of score interpretations must be considered. Kane (2006) proposed an argument-based approach in which the validity argument is structured as a chain of inferences — from observation to score, from score to universe score, from universe score to construct, and from construct to decision. Each inference requires its own supporting evidence.
Quantitative methods for validity evaluation have grown sophisticated. Structural equation modeling allows simultaneous estimation of measurement and structural relationships. Meta-analytic validity generalization examines whether validity coefficients are stable across settings. Differential prediction analyses evaluate whether test-criterion relationships hold across demographic groups. Together, these methods provide a rigorous mathematical apparatus for what remains, at its core, a question about meaning.