Derivation of the Boltzman Equation By Maximizing Entropy

Assume there are $N$ discrete states the system can be in: $X \in \{ x_1 \dots x_N \}$ . Let $p_i = P(X = x_i)$ sampling the state over an indefinite time period, and $\epsilon_i$ be the energy in state $x_i$ . The system will have 2 contraints, the probabilities must add to 1, and the average energy should equal $\bar{E}$ :

\displaystyle\sum_{i=1}^{N} p_i = 1

and

\displaystyle\sum_{i=1}^{N} p_i \epsilon_i = \bar{E}

The second law of thermodynamics says that entropy will always increase. Over a long enough time period the entropy of the system,

S

should be maximized.

S(p) = \displaystyle -\sum_{i=1}^{N} p_i \ln(p_i)

We want to find the probability distribution $p = {p_1 \dots p_n}$ that maximizes $X$ , while following the 2 constraints.

p_{max} = \text{argmax}_P S(p)

Solve this system of equations with a Lagrange multiplier. First rewrite the 2 constraints like:

\displaystyle\sum_{i=1}^{N} p_i - 1 = 0

and

\displaystyle\sum_{i=1}^{N} p_i \epsilon_i - \bar{E} = 0

Then make the Lagrange equation

L(p, \lambda_1, \lambda_2) = \displaystyle -\sum_{i=1}^{N} p_i \ln(p_i) + \lambda_1 \Big(\sum_{i=1}^{N} p_i - 1\Big) + \lambda_2 \Big(\sum_{i=1}^{N} p_i \epsilon_i - \bar{E}\Big)

At the maximum the derivative with respect to any $p_i$ should be zero.

\dfrac{\partial L(p, \lambda_1, \lambda_2)}{\partial p_i} = - \ln(p_i) - p_i \frac{1}{p_i} + \lambda_1 + \lambda_2 p_i \epsilon_i

Set the derivative to zero.

0 = - \ln(p_i) - p_i \frac{1}{p_i} + \lambda_1 + \lambda_2 \epsilon_i

Then Solve for $p_i$ .

\ln(p_i) = - 1 + \lambda_1 + \lambda_2 \epsilon_i

Exponentiate both sides

p_i = e^{- 1 + \lambda_1 + \lambda_2 \epsilon_i}

Then split the exponent.

p_i = e^{- 1 + \lambda_1}e^{\lambda_2 \epsilon_i}

Now we have to solve for the 2 multipliers. Start with the first constraint:

\displaystyle\sum_{i=1}^{N} p_i = 1

And substitute $p_i$ .

\displaystyle\sum_{i=1}^{N} e^{- 1 + \lambda_1} e^{\lambda_2 \epsilon_i} = 1

The first term is independent of $p_i$ so it can be moved out of the sum.

e^{- 1 + \lambda_1} \displaystyle\sum_{i=1}^{N} e^{\lambda_2 \epsilon_i} = 1

Then divide both sides by the sum.

e^{- 1 + \lambda_1} = \frac{1}{\displaystyle \sum_{i=1}^{N} e^{\lambda_2 \epsilon_i}}

Now $e^{- 1 + \lambda_1}$ can be substituted.

p_i = \frac{e^{\lambda_2 \epsilon_i}}{\displaystyle \sum_{i=1}^{N} e^{\lambda_2 \epsilon_i}}

Next solve to solve for $\lambda_2$ use the second constraint:

\displaystyle\sum_{i=1}^{N} p_i \epsilon_i = \bar{E}

Substitute in the equation for $p_i$ .

\displaystyle\sum_{i=1}^{N} \frac{e^{\lambda_2 \epsilon_i}}{\displaystyle \sum_{j=1}^{N} e^{\lambda_2 \epsilon_j}} \epsilon_i = \bar{E}

Look at the equation for the maximum entropy:

S_{max} = S(P_{max}) = \displaystyle -\sum_{i=1}^{N} p_i \ln(p_i) = -\sum_{i=1}^N \frac{e^{\lambda_2 \epsilon_i}}{ \sum_{j=1}^{N} e^{\lambda_2 \epsilon_j}} \ln \Big(\frac{e^{\lambda_2 \epsilon_i}}{ \sum_{k=1}^{N} e^{\lambda_2 \epsilon_k}} \Big)

Then simplify

S_{max} = \displaystyle -\sum_{i=1}^N \frac{e^{\lambda_2 \epsilon_i}}{ \sum_{j=1}^{N} e^{\lambda_2 \epsilon_j}} \ln \Big(\frac{e^{\lambda_2 \epsilon_i}}{ \sum_{k=1}^{N} e^{\lambda_2 \epsilon_k}} \Big)

Split the logarithm

S_{max} = \displaystyle -\sum_{i=1}^N \frac{e^{\lambda_2 \epsilon_i}}{ \sum_{j=1}^{N} e^{\lambda_2 \epsilon_j}} \Bigg(\ln\Big(e^{\lambda_2 \epsilon_i}\Big) - ln\Big( \sum_{k=1}^{N} e^{\lambda_2 \epsilon_k} \Big)\Bigg)

Cancel the logarithm and exponent

S_{max} = \displaystyle -\sum_{i=1}^N \frac{e^{\lambda_2 \epsilon_i}}{ \sum_{j=1}^{N} e^{\lambda_2 \epsilon_j}} \Bigg(\lambda_2 \epsilon_i - ln\Big( \sum_{k=1}^{N} e^{\lambda_2 \epsilon_k} \Big)\Bigg)

Split the summation.

S_{max} = \displaystyle \sum_{i=1}^N \frac{e^{\lambda_2 \epsilon_i}}{ \sum_{j=1}^{N} e^{\lambda_2 \epsilon_j}} ln\Big( \sum_{k=1}^{N} e^{\lambda_2 \epsilon_k} \Big) - \sum_{i=1}^N \frac{e^{\lambda_2 \epsilon_i}}{ \sum_{j=1}^{N} e^{\lambda_2 \epsilon_j}} \Big(\lambda_2 \epsilon_i \Big)

The second term in the first summation is independent so it can be moved out.

S_{max} = \displaystyle ln\Big( \sum_{k=1}^{N} e^{\lambda_2 \epsilon_k} \Big) \sum_{i=1}^N \frac{e^{\lambda_2 \epsilon_i}}{ \sum_{j=1}^{N} e^{\lambda_2 \epsilon_j}} - \sum_{i=1}^N \frac{e^{\lambda_2 \epsilon_i}}{ \sum_{j=1}^{N} e^{\lambda_2 \epsilon_j}} \Big(\lambda_2 \epsilon_i \Big)

Then from the first constraint the entire first summation is just equal to 1.

S_{max} = \displaystyle ln\Big( \sum_{k=1}^{N} e^{\lambda_2 \epsilon_k} \Big) - \sum_{i=1}^N \frac{e^{\lambda_2 \epsilon_i}}{ \sum_{j=1}^{N} e^{\lambda_2 \epsilon_j}} \Big(\lambda_2 \epsilon_i \Big)

Move the $\lambda_2$ out of the summation.

S_{max} = \displaystyle ln\Big( \sum_{k=1}^{N} e^{\lambda_2 \epsilon_k} \Big) - \lambda_2 \sum_{i=1}^N \frac{e^{\lambda_2 \epsilon_i}}{ \sum_{j=1}^{N} e^{\lambda_2 \epsilon_j}} \epsilon_i

Then the summation is just equal to $\bar{E}$

S_{max} = \displaystyle ln\Big( \sum_{k=1}^{N} e^{\lambda_2 \epsilon_k} \Big) - \lambda_2 \bar{E}

Then $\lambda_2$ is equal to the derivative which is ends up being the definition of temperature.

\lambda_2 = \dfrac{\partial S_{max}}{\partial \bar{E}} = \frac{1}{k_b T}

Substituting, the full equation becomes

p_i = \frac{e^{\frac{1}{k_b T} \epsilon_i}}{\displaystyle \sum_{i=1}^{N} e^{\frac{1}{k_b T} \epsilon_i}}