Unit 7: Statistical Mechanics: Microstates, Ensembles & Maxwell-Boltzmann

Thermodynamics describes macroscopic systems in terms of bulk state variables (temperature \(T\), pressure \(P\), volume \(V\), chemical potential \(\mu\)). Statistical mechanics bridges the gap by deducing these macroscopic laws directly from the underlying microscopic quantum and classical dynamics of \(N \sim 10^{23}\) constituent particles.

Phase Space Representation

In classical mechanics, the microscopic state (microstate) of a system of \(N\) particles is defined by specifying the positions \(\mathbf{q} = (\mathbf{q}_1, \dots, \mathbf{q}_N)\) and conjugate momenta \(\mathbf{p} = (\mathbf{p}_1, \dots, \mathbf{p}_N)\). This forms a point in a \(6N\)-dimensional phase space (\(\Gamma\)-space). The time evolution of the system traces a continuous trajectory governed by Hamilton's equations of motion:

By Liouville's theorem, the phase space volume element \(d\Gamma = d^{3N}q \, d^{3N}p\) occupied by an ensemble of identical systems is strictly conserved over time:

Phase space density behaves as an incompressible fluid.

Quantum Microstates

In quantum mechanics, the Heisenberg uncertainty principle \(\Delta q_i \Delta p_i \ge \frac{\hbar}{2}\) precludes the simultaneous specification of exact positions and momenta. The phase space is discretized into elementary quantum cells of volume:

Each independent non-degenerate stationary quantum eigenstate \(|\psi_i\rangle\) of the total \(N\)-particle Hamiltonian \(\hat{H} |\psi_i\rangle = E_i |\psi_i\rangle\) represents an individual distinct quantum microstate.

The Ergodic Hypothesis and Principle of Equal A Priori Probabilities

A macroscopic measurement takes a finite time \(\tau_{\text{obs}}\) (typically milliseconds to seconds), which is enormous compared to microscopic molecular collision times (\(\tau_{\text{coll}} \sim 10^{-13}\text{ s}\)). The observed macroscopic property is the time average:

Because calculating this trajectory for \(10^{23}\) particles is impossible, Willard Gibbs replaced the time average with an ensemble average—an instantaneous average across a vast hypothetical collection of identical macroscopic copies of the system:

The Ergodic Hypothesis asserts that over long times, the system trajectory passes through all accessible microstates, ensuring that the time average equals the ensemble average:

For an isolated system with fixed energy \(E\), volume \(V\), and particle number \(N\), every accessible quantum microstate has the exact same probability:

This is the Postulate of Equal A Priori Probabilities.

Statistical mechanics organizes systems into three fundamental thermodynamic ensembles depending on their thermal and material boundary conditions with the surroundings.

1. The Microcanonical Ensemble (\(N, V, E\))

Describes a completely isolated system with fixed particle number \(N\), volume \(V\), and total energy \(E\).

• Walls: Rigid, adiabatic, and impermeable.
• Fundamental quantity: The total number of accessible microstates \(\Omega(E, V, N)\).
• Probability distribution: Uniform over all accessible states:

2. The Canonical Ensemble (\(N, V, T\))

Describes a closed system in thermal contact with a large heat reservoir at absolute temperature \(T\).

• Walls: Rigid, diathermal (heat-permeable), impermeable to matter.
• The energy of the system fluctuates around a mean value \(\langle E \rangle\).
• Probability of occupying a specific microstate \(i\) of energy \(E_i\):

where \(\beta = \frac{1}{k_B T}\) and the normalization denominator is the Canonical Partition Function:

3. The Grand Canonical Ensemble (\(\mu, V, T\))

Describes an open system that can exchange both heat and particles with a reservoir at fixed temperature \(T\) and chemical potential \(\mu\).

• Walls: Rigid, diathermal, permeable to matter.
• Both energy \(E\) and particle number \(N\) fluctuate.
• Probability of occupying a microstate with \(N\) particles and energy \(E_{N, i}\):

where \(\Xi\) is the Grand Canonical Partition Function:

with fugacity \(z = e^{\beta \mu}\).

Thermodynamic Equivalence in the Thermodynamic Limit

In macroscopic systems where \(N \rightarrow \infty, V \rightarrow \infty\) with constant density \(\rho = N/V\), relative fluctuations in energy and particle number scale as \(1/\sqrt{N} \sim 10^{-11}\). Consequently, all three ensembles yield identical macroscopic thermodynamic predictions.

Ludwig Boltzmann provided the profound statistical definition connecting microscopic disorder with macroscopic thermodynamic entropy.

The Boltzmann Formula

For an isolated system with \(\Omega\) (or multiplicity \(W\)) accessible microstates:

where \(k_B = 1.380649 \times 10^{-23}\text{ J}\cdot\text{K}^{-1}\) is the Boltzmann constant.

Derivation from Additivity and Multiplicativity

Consider two independent, weakly interacting subsystems 1 and 2 with energies \(E_1\) and \(E_2\) and multiplicities \(W_1\) and \(W_2\).

1. By the multiplication principle of independent probabilities, the combined system has total multiplicity:

1. By classical thermodynamics, entropy is an extensive state function:

We seek a mathematical function \(S(W)\) such that \(S(W_1 W_2) = S(W_1) + S(W_2)\). Differentiating with respect to \(W_1\):

Integrating yields the unique functional form:

Gibbs Entropy for Arbitrary Distributions

For any arbitrary probability distribution \(\{P_i\}\) across microstates, Willard Gibbs generalized the formula to:

• In the microcanonical ensemble where all \(\Omega\) states have equal probability \(P_i = 1/\Omega\):

recovering Boltzmann's equation.

• In information theory, Claude Shannon identified \(- \sum P_i \ln P_i\) as the informational entropy (the measure of missing information). Thermodynamic entropy is the measure of microscopic information hidden by macroscopic averaging.

Consider a system of \(N\) distinguishable independent particles distributed among discrete energy levels \(\varepsilon_1, \varepsilon_2, \dots, \varepsilon_i, \dots\) with degeneracies \(g_1, g_2, \dots, g_i, \dots\). Let \(N_i\) be the number of particles in energy level \(\varepsilon_i\).

Thermodynamic Probability (Multiplicity)

The number of distinct microscopic ways to place \(N\) particles into the energy levels such that there are \(N_i\) particles in level \(i\) (with degeneracy \(g_i\)) is:

Taking the natural logarithm and applying Stirling's approximation (\(\ln x! \approx x \ln x - x\) for \(x \gg 1\)):

Since \(\sum N_i = N\), the linear terms \(-N + \sum N_i = 0\) cancel:

Optimization Under Physical Constraints

The most probable macroscopic state corresponds to the distribution \(\{N_i^*\}\) that maximizes \(\ln W\) subject to two rigid physical conservation laws:

1. Total particle conservation: \(\phi_1 \equiv \sum_i N_i - N = 0\)

2. Total internal energy conservation: \(\phi_2 \equiv \sum_i N_i \varepsilon_i - E = 0\)

Using the method of Lagrange Undetermined Multipliers, we introduce multipliers \(\alpha\) and \(\beta\) and construct the unconstrained objective function:

Differentiating with respect to an arbitrary occupation number \(N_k\) and setting to zero:

Exponentiating both sides:

Normalization and Identification of Multipliers

Summing over all levels:

where \(q = \sum_k g_k e^{-\beta \varepsilon_k}\) is the molecular partition function. Thus, \(A = \frac{N}{q}\), yielding the canonical Maxwell-Boltzmann Distribution Law:

By evaluating the thermodynamic identity \(dS = \left(\frac{\partial S}{\partial E}\right)_{V, N} dE = \frac{1}{T} dE\), we find:

Hence:

At absolute zero (\(T \rightarrow 0\)), all particles collapse into the lowest energy ground state. At infinite temperature (\(T \rightarrow \infty\)), particles populate all states proportionally to their degeneracies \(g_i\).

The canonical partition function \(Q(N, V, T)\) serves as the master generating function connecting microscopic quantum energy spectra \(\{E_i\}\) to all macroscopic thermodynamic state functions.

The Canonical Partition Function

For an ideal gas of \(N\) independent particles with single-particle molecular partition function \(q(V, T)\):

• Distinguishable particles (e.g., localized atoms in a crystal lattice):

• Indistinguishable particles (e.g., delocalized gas molecules):

The Gibbs correction factor \(1/N!\) eliminates the Gibbs paradox and restores the extensivity of entropy.

Fundamental Thermodynamic Relations

From \(Q\), all state functions are generated by straightforward differentiation:

1. Internal Energy \(U\):

2. Helmholtz Free Energy \(A\):

This is the central bridge equation of statistical thermodynamics!

3. Entropy \(S\):

From \(A = U - TS \implies S = \frac{U - A}{T}\):

4. Pressure \(P\):

From \(P = -\left( \frac{\partial A}{\partial V} \right)_{T, N}\):

5. Enthalpy \(H\) and Gibbs Free Energy \(G\):

6. Isochoric Heat Capacity \(C_V\):

In the canonical ensemble, the system's energy is not strictly fixed; it fluctuates due to continuous thermal exchange with the reservoir.

Derivation of Energy Variance

The average energy is:

Differentiating \(\langle E \rangle\) with respect to \(\beta\):

Using \(\frac{\partial Q}{\partial \beta} = -Q \langle E \rangle\):

where \(\sigma_E^2 = \langle (E - \langle E \rangle)^2 \rangle\) is the energy variance. Now express the left-hand side in terms of temperature \(T\):

Equating the two expressions:

The standard deviation of energy fluctuations is:

Relative Fluctuation and the Thermodynamic Limit

Because both energy \(\langle E \rangle\) and heat capacity \(C_V\) are extensive variables proportional to particle number \(N\) (\(\langle E \rangle \propto N, C_V \propto N\)):

For a typical macroscopic molar sample (\(N \sim 10^{23}\)):

Relative energy fluctuations are on the order of parts per trillion. The energy probability distribution is extraordinarily sharply peaked around its mean value \(\langle E \rangle\), demonstrating why canonical and microcanonical ensembles are observationally indistinguishable for macroscopic matter.

For an ideal gas of point particles of mass \(m\) at thermal equilibrium, the translational kinetic energy is \(\varepsilon = \frac{1}{2} m (v_x^2 + v_y^2 + v_z^2) = \frac{p^2}{2m}\).

Velocity Probability Density

Because motion in \(x, y, z\) is isotropic and mutually independent, the three-dimensional velocity probability density factors into three 1D Gaussians:

Transformation to Molecular Speed \(v\)

Speed is the scalar magnitude \(v = \sqrt{v_x^2 + v_y^2 + v_z^2} \ge 0\). Transforming velocity space into spherical coordinates \((v, \theta, \phi)\) where the volume element is \(dv_x dv_y dv_z = v^2 \sin\theta \, dv \, d\theta \, d\phi\), integrating over all solid angles \(\int_0^{4\pi} d\Omega = 4\pi\) gives the Maxwell-Boltzmann Speed Distribution:

Characteristic Molecular Speeds

1. Most Probable Speed \(v_{\text{mp}}\): Setting \(\frac{df}{dv} = 0\):

2. Mean (Average) Speed \(\langle v \rangle\):

3. Root-Mean-Square Speed \(v_{\text{rms}}\):

Ordering: \(v_{\text{mp}} < \langle v \rangle < v_{\text{rms}}\).

Molecular Effusion Rate

When gas molecules effuse through an orifice with area \(A\) whose diameter is smaller than the mean free path, the effusion rate \(Z_{\text{eff}}\) (number of collisions per unit wall area per second) is:

This directly yields Graham's Law of Effusion:

Brownian motion provides direct macroscopic observational proof of the discrete atomic nature of matter.

The Langevin Equation

Paul Langevin formulated the stochastic equation of motion for a Brownian colloidal particle of mass \(M\) and velocity \(v(t)\) suspended in a fluid:

1. Deterministic Drag Force: \(-\gamma v(t)\), where Stokes' law gives \(\gamma = 6\pi \eta a\) for a sphere of radius \(a\) in a fluid of dynamic viscosity \(\eta\). This force dissipates kinetic energy into heat.

2. Stochastic Thermal Force: \(\xi(t)\), arising from random, uncorrelated molecular collisions.

Statistical properties of the thermal white noise:

Velocity Autocorrelation and Mean Square Displacement

Solving Langevin's equation:

The velocity autocorrelation function decays exponentially with relaxation time \(\tau_M = M/\gamma\):

Integrating to find the position \(x(t)\):

• Short times (\(t \ll \tau_M\), Ballistic regime):

• Long times (\(t \gg \tau_M\), Diffusive regime):

where \(D\) is the macroscopic diffusion coefficient.

The Einstein-Smoluchowski Relation & Fluctuation-Dissipation Theorem

Equating the long-time Langevin mean square displacement \(\langle (\Delta x)^2 \rangle = \frac{2 k_B T}{\gamma} t\) with the diffusive law \(2 D t\) yields the Einstein Relation:

This relation is the prototypical example of the Fluctuation-Dissipation Theorem:

• Thermal fluctuations (represented by diffusion coefficient \(D\))
• are directly proportional to frictional dissipation (represented by drag coefficient \(\gamma\))
• mediated by thermal energy \(k_B T\).

In 1908, Jean Perrin used Einstein's equation to measure the displacement of gamboge particles, determining Avogadro's number \(N_A\) experimentally and earning the 1926 Nobel Prize in Physics.

## Advanced Mathematical Supplement: Jaynes MaxEnt & von Neumann Entropy

Information Theoretic MaxEnt Formulation & The von Neumann Density Matrix

In 1957, Edwin Jaynes re-founded statistical mechanics on Claude Shannon's Information Theory: statistical distributions are not physical assumptions, but rather the unique mathematically unbiased inference consistent with incomplete macroscopic information (Maximum Entropy Principle, MaxEnt).

The von Neumann Density Operator

In quantum statistical mechanics, a mixed quantum state is described by a Hermitian, positive-semidefinite density operator \(\hat{\rho}\):

The expectation value of any physical observable \(\hat{A}\) is:

For a pure state, \(\hat{\rho}^2 = \hat{\rho}\) (\(\text{Tr}(\hat{\rho}^2) = 1\)); for a mixed state, \(\text{Tr}(\hat{\rho}^2) < 1\).

The von Neumann Quantum Entropy

The quantum mechanical extension of Gibbs and Shannon entropy is:

Properties:

1. \(S_{\text{vN}} \ge 0\), with \(S_{\text{vN}} = 0\) if and only if the state is pure.
2. Invariant under unitary transformations: \(S_{\text{vN}}[\hat{U}\hat{\rho}\hat{U}^\dagger] = S_{\text{vN}}[\hat{\rho}]\).
3. Subadditive: For a composite bipartite system \(A B\), \(S(A B) \le S(A) + S(B)\).

Derivation of the Canonical Density Matrix via MaxEnt

We maximize \(S_{\text{vN}}\) subject to two physical constraints:

1. Normalization: \(\text{Tr}(\hat{\rho}) = 1\) (multiplier \(\lambda_0\))
2. Average internal energy: \(\text{Tr}(\hat{\rho} \hat{H}) = U\) (multiplier \(\beta\))

Construct the variational functional:

Setting the functional derivative to zero:

Exponentiating:

where the canonical partition function is \(Q = \text{Tr}(e^{-\beta \hat{H}})\). Thermal equilibrium emerges as the state of maximal missing information given knowledge only of the average internal energy.

## Research Monograph: Non-Equilibrium Statistical Mechanics & The Jarzynski Equality

Breakthroughs in Non-Equilibrium Thermodynamics

For over a century, classical thermodynamics was restricted to quasi-static, reversible paths. The Second Law asserted the inequality:

where \(\langle W \rangle\) is the average work done on a system and \(\Delta F\) is the free energy difference. Any rapid irreversible process dissipates work as entropy, making the determination of equilibrium free energy landscapes from non-equilibrium measurements appear fundamentally impossible.

The Jarzynski Equality (1997)

In 1997, Christopher Jarzynski proved an exact equality holding for arbitrary non-equilibrium processes arbitrarily far from thermal equilibrium:

Significance:

• Holds regardless of how violently or rapidly the system is driven out of equilibrium!
• By Jensen's inequality (\(\langle e^x \rangle \ge e^{\langle x \rangle}\)), \(\langle e^{-W/k_B T} \rangle \ge e^{-\langle W \rangle / k_B T} \implies \langle W \rangle \ge \Delta F\), recovering the classical Second Law as a simple corollary!

The Crooks Fluctuation Theorem (1999)

Gavin Crooks generalized Jarzynski's discovery by comparing the work probability distribution \(P_F(W)\) of a forward process to the work distribution \(P_R(-W)\) of the time-reversed process:

At the crossing point where forward and reverse work distributions intersect (\(P_F(W) = P_R(-W)\)):

Using optical tweezers and atomic force microscopy (AFM), biophysicists routinely pull single RNA and protein molecules mechanically, recording non-equilibrium force-extension curves and extracting equilibrium folding free energies via Crooks' theorem.