# Understanding the definition of entropy in the joint entropy theorem derivation

From section 11.3.2 of Nielsen & Chuang:

(4) let $$\lambda_i^j$$ and $$\left|e_i^j\right>$$ be the eigenvalues and corresponding eigenvectors of $$\rho_i$$. Observe that $$p_i\lambda_i^j$$ and $$\left|e_i^j\right>$$ are the eigenvalues and eigenvectors of $$\sum_ip_i\rho_i$$ and thus \begin{align}S\left(\sum_ip_i\rho_i\right) &= -\sum_{ij}p_i\lambda_i^j\log p_i\lambda_i^j \\ &= - \sum_ip_i\log p_i - \sum_ip_i\sum_j\lambda_i^j\log\lambda_i^j \\ &= H\left(p_i\right) + \sum_ip_iS\left(\rho_i\right)\end{align}

I do not understand the last step $$- \sum_ip_i\log p_i = H\left(p_i\right)$$. I thought the definition of binary entropy is $$H(p_{i}) = -p_{i}\log p_{i} - (1-p_{i})\log (1-p_{i})$$. This does not give the desired equality.

Moreover, it is rather confusing that $$i$$ is used as a running index, and suddently it becomes a fixed index in the last equality.

Thanks for any helps.

• – glS
Jun 21, 2021 at 7:49
• @glS Thank you. Should have mentioned this post, where I obtain the reference. Jun 23, 2021 at 5:44

It is not just the binary entropy that is denoted $$H(p_i)$$. The quantity that is relevant here is the Shannon entropy of the distribution $$\{p_i\}$$ which is defined as $$H(p_i) = - \sum_i p_i \log p_i.$$ Note that when the distribution has only two elements we recover the binary entropy.