Bernoulli, Binomial and Hypergeometric Models

A count distribution should be selected from the experiment, not from the fact that a variable is an integer. Counting successes in independent repeated trials leads to a binomial model when every trial has the same success probability. Counting marked objects in a uniform finite sample without replacement leads to a hypergeometric model. Both can have the same mean while differing in support, dependence and variance.

This unit derives those models from their mechanisms. The reader should be able to specify a Bernoulli indicator, derive binomial probabilities and moments, derive the hypergeometric PMF by counting subsets, identify feasible values before calculating, and explain the finite-population correction through indicator covariance. The reader should also be able to recognize when familiar formulas fail because independence, uniform selection or a common probability has not been justified.

The word success is a neutral mathematical label for the event being counted. It may refer to a marked object, a completed request or an incorrect response. It need not mean a favorable outcome. Reversing which category is called success replaces the success probability by its complement and transforms the count accordingly. Keeping that coding visible prevents an interpretation from changing while the formula remains the same.

A Bernoulli variable takes one with probability $p$ and zero with probability $1-p$, where $0\leq p\leq1$. Its mean is $p$ because the probability-weighted contribution of zero vanishes. Its second raw moment is also $p$ because squaring either supported value leaves it unchanged. Therefore its variance is $p(1-p)$. This is the same indicator identity used earlier, now presented as a named one-variable law.

The variance is largest at one half and decreases toward zero at either endpoint. A Bernoulli variable with probability zero or one is degenerate: its value is fixed almost surely. Such a law remains valid, but its standardized shape summaries and ordinary correlation with another variable may be undefined because it has zero variance. Boundary parameters should be handled explicitly rather than discarded as invalid probabilities.

The PGF is $(1-p)+pz$, the MGF is $(1-p)+pe^t$, and the characteristic function is $(1-p)+pe^{it}$. Each follows directly by averaging the corresponding function at zero and one. The MGF is finite at every real argument because the support is finite. Its first two derivatives reproduce the mean and second raw moment, while the logarithm produces the mean and variance as its first two cumulants.

A binary observation can be represented by a Bernoulli variable even when a collection of such observations is dependent. The marginal law of each indicator requires only its individual probability. The law of their sum requires their joint relationship. Describing each observation as Bernoulli therefore does not establish a binomial count model. The additional assumptions must be stated for the sequence.

Let $I_1,\ldots,I_n$ be mutually independent Bernoulli variables with the same success probability $p$, and let $X$ be their sum. Then $X$ has a binomial distribution with trial count $n$ and probability $p$. The trial count is a fixed nonnegative integer. The support is the integers from zero through $n$, with some boundary masses becoming zero when $p$ is zero or one.

For a particular sequence containing $k$ successes and $n-k$ failures, independence gives probability $p^k(1-p)^{n-k}$. There are $\binom nk$ sequences with those success positions. They are mutually exclusive, so summing their equal probabilities gives

The combination coefficient counts positions, not the probability of choosing a subset from an urn. The resulting algebra resembles sampling counts because both use combinations, but the mechanisms differ. In the binomial derivation, every trial can succeed independently with the same probability and the sequence length is fixed in advance.

Normalization follows from the binomial theorem: summing the masses produces $(p+(1-p))^n=1$. Boundary cases are interpreted from the underlying deterministic experiment. If the probability is zero, the count is zero; if it is one, the count is $n$. For zero trials, the sum is zero regardless of the parameter. A numerical implementation can use these cases directly to avoid ambiguous powers such as zero to the zeroth power at endpoints.

The assumptions are substantive. A fixed number of trials excludes a stopping rule that ends at a specified number of successes. A common probability excludes a sequence with different known success rates. Mutual independence excludes a shared uncertain condition creating marginal dependence. A binary outcome excludes trials with an unmodeled missing category unless the target event and recording procedure are defined accordingly.

Linearity of expectation gives $E[X]=np$. Independence gives zero covariance among the indicators, so variance is $np(1-p)$. The mean formula still holds for a sum of dependent indicators with the same marginal probability, while the variance and PMF generally do not. This is an important distinction: agreement with the binomial mean is weaker than agreement with the binomial law.

The PGF is $G_X(z)=((1-p)+pz)^n$, obtained by multiplying the individual PGFs. Replacing the PGF argument by an exponential gives the MGF, $M_X(t)=((1-p)+pe^t)^n$. The cumulant generator is $n\log((1-p)+pe^t)$. Its first two derivatives reproduce the mean and variance, and its third cumulant is $np(1-p)(1-2p)$.

For positive variance, binomial skewness is $(1-2p)/\sqrt{np(1-p)}$. The sign is positive when successes are relatively rare and negative when failures are relatively rare. At probability one half the law is symmetric about its mean and has zero skewness. Increasing the fixed trial count reduces the standardized skewness for a fixed interior probability. This summary is useful, but it does not by itself establish a normal approximation at every tail threshold.

Factorial moments have a direct counting derivation. The count's second factorial power counts ordered pairs of distinct successful trials. Each distinct pair has joint success probability $p^2$, and there are $n(n-1)$ ordered pairs. Thus $E[X(X-1)]=n(n-1)p^2$. Combining this with the mean gives the second raw moment and then the familiar variance. Higher factorial moments follow by counting ordered distinct success positions in the same way.

A sum of independent binomial variables with a common success probability is binomial with the total trial count. Multiplying their PGFs combines powers of the same factor. If the probabilities differ, the product generally has different factors and is not the common-probability binomial law. A weighted sum is also not generally a binomial count because its values may no longer be consecutive integer success totals.

An exact count probability is a PMF value. An at-least probability sums a tail, and an at-most probability accumulates lower values. The phrase at least one is efficiently handled by complementing the zero-success event: its probability is $1-(1-p)^n$. An exactly-one event is different and includes the position coefficient. Translating the inequality before choosing a formula prevents confusion among nearby verbal phrases.

For an interior probability, successive positive masses satisfy

This ratio can support efficient recurrence calculations and identifies where the mass increases or decreases. A mode is ordinarily the integer part of $(n+1)p$, with two adjacent modes when that product is an integer strictly between the relevant endpoints. Endpoint parameters are handled through their degenerate supports. The mode is a most-probable count, not the probability-weighted mean or a guaranteed outcome.

For large trial counts, factorials may overflow a numerical type even when the final PMF lies between zero and one. Logarithms, stable recurrences or well-tested numerical routines avoid creating enormous intermediate values. Computing a tiny upper tail by subtracting an almost-one lower CDF can also lose precision. Direct tail accumulation or a suitable survival routine may be better. The selected numerical method should preserve the target strict or nonstrict inequality.

An exact model formula is different from an exact floating-point result. Rounding should be delayed until the final reported answer where practical, and the implementation should be checked against small rational examples. Tail sums must include every allowed value. Truncating a finite tail before the upper support endpoint without an explicit error bound changes the requested probability.

The binomial law describes model probabilities, not the fraction of successes in one realized batch. The observed proportion is $X/n$ for a positive trial count. Its expectation is $p$ and its variance is $p(1-p)/n$ under the independent common-probability model. A particular realization can differ from its expectation. Estimating an unknown probability from observed proportions is a statistical task developed in later books, while this unit treats the parameter as specified.

The hypergeometric model begins with a finite collection of $N$ distinct objects, exactly $K$ marked and $N-K$ unmarked. Select an unordered subset of fixed size $n$ uniformly from all such subsets, without replacement. Let $X$ count marked objects in the sample. The population total and the marked count are fixed properties of the collection; the random element is the selected subset.

Valid parameters satisfy $N\geq1$, $0\leq K\leq N$ and $0\leq n\leq N$, all integers. A sequential uniform draw without replacement induces the same uniform unordered sample when order is discarded. A procedure that prefers particular objects does not inherit this law merely because it samples without replacement. Uniform subset selection is an essential assumption.

The feasible count must satisfy four restrictions: it cannot be negative, exceed the sample size, exceed the marked population total, or leave more unmarked sample positions than available unmarked objects. Combining them gives

Every integer in this range is feasible under the uniform selection model. Counts outside it have probability zero. The lower bound can be positive when the sample is too large to avoid marked objects, and the upper bound can be smaller than the sample size when the collection has few marked objects. Checking support first can resolve some questions without any probability arithmetic.

A sample with exactly $k$ marked objects is constructed by choosing $k$ of the $K$ marked labels and $n-k$ of the $N-K$ unmarked labels. The selections combine uniquely into an unordered sample. There are $\binom Kk\binom{N-K}{n-k}$ favorable subsets out of $\binom Nn$ equally weighted subsets. Therefore

over the stated feasible support. The denominator counts samples, not draw sequences. The equivalent sequence calculation would count all permutations of every sample and produce the same probability because their multiplicities are equal. Mixing an ordered numerator with an unordered denominator would produce an invalid scale.

Normalization is a counting identity. Every size-$n$ subset has exactly one marked count, so summing the favorable counts over all feasible values counts every subset once. This is the relevant Vandermonde identity. The argument proves the identity in this setting without requiring a separate memorized algebraic formula. Impossible compositions contribute zero rather than negative or undefined counts.

A hypergeometric law is unchanged if the roles of selecting a sample and identifying the marked set are exchanged in an appropriate combinatorial interpretation. Nevertheless its practical description should retain the actual experiment. Symmetric algebraic identities can assist a check, but they do not mean that the marked total was generated randomly when it is fixed in the model.

Represent the marked count as the sum of indicators for the sample's sequential draw positions. Each position has marginal marked probability $p=K/N$ under uniform drawing without replacement. Thus the mean is $np=nK/N$. Although the margins match a binomial sequence with this probability, the positions are dependent because a draw changes the remaining collection.

For two different draw positions when $N>1$, the probability that both are marked is $K(K-1)/(N(N-1))$. Subtracting the product of their marginal probabilities gives covariance $-p(1-p)/(N-1)$. This is nonpositive and strictly negative for an interior population proportion. A marked draw removes one marked object, making another marked draw less likely than it would be under independent replacement sampling.

Expanding variance of the indicator sum then gives

The ratio is the finite-population correction for the count variance under this parameter convention. It reduces the independent-trial variance because the covariance terms are negative. For a one-object collection, every valid fixed-size sample count is deterministic and its variance is zero; the displayed ratio should not be evaluated with a zero denominator.

The limiting cases help explain the formula. A sample of one has the Bernoulli variance for a uniform single draw. A census sample of all objects has a fixed marked count equal to $K$ and therefore zero variance. For a small sampling fraction in a large collection, the correction is close to one, so the binomial variance can be a useful approximation. For a substantial sampling fraction, ignoring the correction can markedly overstate sampling variation under this model.

The correction applies to this uniform finite-population design. It is not a universal reduction factor for every survey or dependent sequence. Cluster selection, unequal object probabilities, nonresponse and a random population composition can alter the joint law. Using a familiar finite-population formula without the corresponding mechanism can be just as misleading as imposing independence incorrectly.

The factorial-moment approach supplies another variance derivation. For an ordered set of $r$ distinct sample positions, the probability that all are marked is the descending product of $K$ divided by the descending product of $N$, provided the positions are feasible. There are the descending product of $n$ such position sets. Therefore the factorial moment of order $r$ is the product of those two factors for valid orders, with zero when the required successes exceed the available support.

At order one, this gives the mean. At order two, it gives $n(n-1)K(K-1)/(N(N-1))$ for $N>1$. Combining the second factorial moment with the first yields the second raw moment and the same finite-population variance. This route checks the covariance derivation using a different representation of the count.

The hypergeometric PGF can be written as the finite sum of its PMF times powers of the count argument. Its MGF substitutes an exponential argument into that finite polynomial and is finite for every real value. Unlike the binomial PGF, it does not generally factor into a power of one Bernoulli factor because the sample positions are dependent. Generating functions remain valid representations even when a compact product formula is unavailable.

Replacing marked by unmarked transforms the count to $n-X$ and replaces $K$ by $N-K$. Its mean becomes the sample size times the unmarked fraction, while its variance is unchanged. Counting marked objects in the complementary unsampled subset gives $K-X$, revealing another connection to the remaining collection. These transformations help verify support and moment calculations without asserting independent behavior between sampled and unsampled counts.

With replacement and independent randomization, repeated uniform draws from a fixed collection have a constant marked probability and allow the same labeled object to be drawn again. The marked count is binomial. Without replacement, repeated labels are impossible and the marked proportion changes along the path, giving a hypergeometric count under uniform draws. Identical first-draw probabilities do not make the two full experiments equivalent.

Their means agree when the trial count and initial marked fraction agree. Their variances differ by the finite-population correction, and their supports can differ. A binomial model permits every count from zero to the trial count for an interior probability. A finite collection may prohibit some of those counts because too few marked or unmarked objects exist. An approximation assigning a small positive probability to an impossible count should be recognized as an approximation, not described as exact.

For fixed sample size and increasing population size with marked fraction approaching a fixed probability, the hypergeometric PMF approaches the binomial PMF. Successive without-replacement fractions change by increasingly small amounts along any fixed-length path. This gives an intuitive and algebraic basis for the approximation. If the sample size grows as a substantial proportion of the population, that reasoning no longer makes the same independence simplification, and the variance correction remains relevant.

An approximation rule based only on a sampling-fraction cutoff is a practical heuristic, not an error guarantee for every event. The accuracy required depends on the target tail or count probability. When exact hypergeometric computation is available and numerically stable, it is usually preferable for a genuinely finite uniform sample. An approximation can still teach the relationship between mechanisms, provided its discrepancy is measured and reported for the question at hand.

The two laws are related through conditioning as well as through approximation. Suppose two independent groups contain fixed numbers of independent binary trials, all sharing one success probability. The success counts in the groups are individually binomial, and their total is binomial with the combined trial count. Conditional on the total number of successes, however, the number located in the first group has a hypergeometric distribution.

The mechanism explains the relationship. Given a fixed total of successes, every success-position subset of that size has the same probability because each originally carried the same success-and-failure product. The first group's positions form a fixed marked set among the combined positions. Counting how many successful positions fall in that set is therefore a uniform subset-counting problem. The common success parameter disappears from the conditional distribution because it supplies the same factor for every compatible position subset.

Algebraically, the joint probability of specified group counts is the product of their binomial masses. Divide that product by the binomial probability of the stipulated total. The common probability powers cancel, leaving the product of two combination coefficients divided by the combination count for the combined positions. The calculation requires a positive-probability total. Boundary success parameters can make some totals impossible, in which case conditioning on them is not justified by the elementary ratio.

For a concrete illustration, divide ten trial positions into a group of four and a group of six, and condition on exactly three successes overall. The probability of two successes in the first group is $\binom42\binom61/\binom{10}{3}=0.30$. The conditional mean of that group's success count is $4(3/10)=1.20$. Its conditional variance is $4(3/10)(7/10)(6/9)=0.56$. These conditional values concern the fixed-total information set, not the unconditioned experiment with its original success probability.

If the two groups have different success probabilities, the cancellation no longer gives equal weights to all success-position subsets. The conditional law is not the ordinary central hypergeometric law derived here. One should therefore not transfer this relationship to heterogeneous trials without checking the mechanism. Conditioning can create a recognizable distribution, but the name follows from the derived weights rather than from a superficial resemblance of the count question.

A shared uncertain success probability produces a different count model from a binomial law with a fixed average probability. Let a variable $P$ represent that shared condition, and suppose the count is conditionally binomial given it, with fixed trial count $n$. Write $q=E[P]$ and $v=\operatorname{Var}(P)$. Iterated expectation gives count mean $nq$. Total variance reveals the additional variation produced by the shared condition.

The average conditional variance is $nE[P(1-P)]$. Since $E[P^2]=q^2+v$, this term is $n(q(1-q)-v)$. The conditional mean is $nP$, whose variance is $n^2v$. Adding the two components gives

For more than one trial and positive condition variance, the marginal count variance exceeds that of a fixed-probability binomial law with the same mean. This is one mechanism for overdispersion relative to that reference law. The word describes a comparison, not an explanation by itself. Dependence or heterogeneity must be modeled to determine where the extra variation comes from.

For example, suppose the shared probability is one fifth or four fifths with equal weights and the fixed count length is ten. Its mean probability is one half, and its variance is nine hundredths. The count mean is five, but its variance is $10(1/2)(1/2)+10(9)(0.09)=10.6$. A fixed-probability binomial model with mean five would have variance $2.5$. Replacing the uncertain shared condition by its average preserves the mean and discards most of the modeled variation.

This mixture differs from ten independent trials each having an independently redrawn uncertain probability with the same marginal mean. With independent condition draws at every position, marginal outcomes can again be independent binary trials with probability $q$. A single shared condition and separate independent conditions are different joint mechanisms. The phrase probability varies does not identify which mechanism is intended, so the timing and sharing of the variation must be stated.

Finite-population sampling supplies a contrasting dependence structure. Without replacement from a fixed marked collection, covariance is negative and count variance is reduced. A shared uncertain condition can create positive covariance and increase variance. Neither smaller nor larger variance is a universal signature of the word random. The structure of the experiment determines the covariance terms and the resulting distribution.

The hypergeometric mean can also be derived using indicators indexed by population objects instead of draw positions. For each marked object, define an indicator that it is included in the uniform sample. A fixed object has inclusion probability $n/N$ because it occurs in the same number of equally weighted subsets as every other object. Summing the inclusion indicators over the marked set gives the marked sample count and mean $Kn/N$.

Two distinct population objects are both included with probability $n(n-1)/(N(N-1))$ for $N>1$. Subtracting the product of their inclusion probabilities gives a negative inclusion covariance. Its form differs from the draw-position covariance because the indicators describe different events: inclusion of fixed objects rather than category results at fixed positions. Both representations yield the same count variance when the correct number of indicators and pair terms are used.

For a positive sample size, the sample marked proportion is the count divided by $n$. Its design variance is the count variance divided by $n^2$. The factor $1/n$ reflects averaging, while the finite-population correction reflects depletion without replacement. If all objects are selected, the sample proportion equals the fixed population proportion exactly and its sampling variance is zero. This conclusion concerns selection variability in the fixed finite model, not every possible source of measurement error.

Another common notation uses the finite-population variance of the zero-one marks with denominator $N-1$. That quantity is $S^2=Np(1-p)/(N-1)$. The sample-proportion variance can then be written $(1-n/N)S^2/n$. This expression is algebraically identical to the earlier count-derived result. Different variance-denominator conventions can make two correct formulas look different; converting them to a shared convention prevents a false disagreement.

The denominator $N-1$ in this finite-population summary should not be confused with estimating an unknown variance from a particular realized sample. Here the entire fixed population of marks defines the quantity. The sampling variance formula follows from the uniform selection design. Its interpretation does not depend on pretending that the population marks are independent random draws from a superpopulation.

If a separate model makes the population marks independently random with a common probability before sampling, unconditional predictions can differ from predictions conditional on the realized marked total. After conditioning on that total, the selected count is hypergeometric. Without conditioning on it, a uniformly selected collection of distinct independently marked objects can have a binomial count. The distinction illustrates once more that conditioning and the level of modeled randomness determine the law.

A larger count probability is not always a larger tail probability. A distribution concentrated near its mean may give a higher probability to a central count and lower probability to extreme counts, while cumulative comparisons can depend on the threshold. The third worked question showed exactly this pattern for the center and the all-marked event. Reporting one selected probability as proof that one mechanism is always more likely to produce success would overstate the result.

The trial count and sample fraction also play different roles. Increasing a binomial trial count increases both the expected count and its variance for a fixed interior success probability, while reducing the variance of the sample proportion. Increasing a without-replacement sample size moves the experiment toward a census and ultimately removes selection variation in the proportion. Comparing counts across different sample sizes without accounting for this distinction can confuse total occurrences with rate precision.

When probabilities are supplied as teaching parameters, exact computation is appropriate. When they are estimated from data, the uncertainty of those estimates should not disappear from the report merely because the subsequent PMF formula is evaluated to many decimal places. Statistical estimation and model checking will be developed later. Here a clear result identifies the parameter assumptions and avoids implying that a stipulated rate has been measured in an actual collection.

Before using either law, identify the random quantity and its support. Verify whether the collection is fixed, whether labels can recur, whether selection is uniform, whether the trial count is fixed and whether conditional probabilities change along a sequence. Check the boundary cases with no marks, all marks, no draws and a census. They often expose an implementation that applies one formula outside its valid denominator or support.

The proposed sampling explorer should let a learner compare the two mechanisms under the same initial marked fraction and sample size. It should show exact PMFs, means and variances alongside simulated count frequencies. The finite collection should be visible as a model assumption, and without-replacement draws must never reuse a label within a sample. Replacement draws should restore the collection and permit repeated labels. Reset should restore both parameters and the random-state convention explicitly.

Simulated frequencies are estimates generated within the model and fluctuate around its exact probabilities. They should not replace exact combination calculations when those are available. Testing should include a census, a one-object population, extreme marked totals, invalid sample sizes, normalization and known rational examples. Small-screen controls should retain readable labels, and color should not be the only way to distinguish the two model curves or their counts.

The next unit studies waiting counts and event counts with geometric, negative binomial and Poisson laws. Their experiments introduce stopping rules or opportunities over a specified interval, so a fixed-size binomial formula cannot be reused merely because the variable is an integer. The connection between mechanism, support and probability law remains the organizing principle.