Claude Shannon's 1948 channel coding theorem proved that every noisy communication channel has a definite capacity C — measured in bits per channel use — such that information can be transmitted at any rate below C with arbitrarily low error probability, but not at any rate above C. This result, one of the most profound in twentieth-century mathematics, established the theoretical limits of reliable communication and provided the conceptual foundation for analyzing human information processing as a capacity-limited channel.
The Channel Capacity Formula
I(X;Y) = H(Y) − H(Y|X)
= Σ_x Σ_y p(x,y) · log₂[p(x,y) / (p(x) · p(y))]
The maximum is taken over all possible input distributions p(x)
Channel capacity is defined as the maximum mutual information between the input X and output Y of a channel, optimized over all possible input probability distributions. For a binary symmetric channel with crossover probability p, the capacity is C = 1 − H(p) = 1 + p·log₂(p) + (1−p)·log₂(1−p). For an additive white Gaussian noise channel with bandwidth W and signal-to-noise ratio S/N, the capacity is C = W·log₂(1 + S/N), known as the Shannon-Hartley theorem.
The Noisy Channel Coding Theorem
Shannon's coding theorem has two parts. The direct theorem states that for any rate R < C, there exist codes that achieve probability of error as close to zero as desired. The converse theorem states that for R > C, the error probability is bounded away from zero regardless of the code used. The proof is existential — Shannon showed that random codes achieve capacity with high probability — but it launched the field of coding theory devoted to constructing practical codes approaching capacity.
Shannon's framework inspired psychologists to model the human observer as a noisy channel. Garner and Hake (1951) and Miller (1956) measured human channel capacity for absolute judgments of unidimensional stimuli, finding consistent limits around 2–3 bits — the capacity to distinguish about 5–9 categories. This convergence across modalities suggested a fundamental architectural constraint on human information processing, though the analogy between electronic and biological channels has important limitations.
Psychological Significance
The concept of channel capacity has been applied broadly in psychology. In absolute identification tasks, observers are asked to identify stimuli varying on a single dimension, and information transmission (mutual information between stimulus and response) asymptotes at roughly 2.5 bits regardless of the number of stimuli. In studies of attention, dual-task paradigms can be analyzed as parallel channels whose joint capacity is constrained. The channel capacity framework also connects to Hick's law: reaction time increases linearly with stimulus entropy, as if the observer processes information at a fixed rate determined by channel capacity.
Despite its power, the channel metaphor has limitations when applied to human cognition. Unlike electronic channels, human processing is influenced by meaning, context, chunking, and learning — factors that effectively recode information and alter the apparent channel capacity. Shannon himself cautioned against overextending information theory into domains where its assumptions do not strictly hold.