How to evaluate the mutual information of a classical-quantum state?

Question

Suppose I have the state $\sum_i p_i \vert i\rangle\langle i\vert \otimes\rho_i$ on subsystems $X$ (classical part) and $B$ (quantum part) and I wish to evaluate the mutual information $I(X:B) = S(X) + S(B) - S(XB)$.

Nielsen and Chuang claims in (12.11) that the answer is

$$I(X:B) = S(\sum_i p_i\rho_i) - \sum_ip_i S(\rho_i) $$

I don't understand how they arrived at this and the theorem they refer to as proof of this statement is an inequality, not an equality. Trying to evaluate it myself, I don't really know how to write down $S(XB)$ in a smart way.

Answer 1

Note that $S(XB) = S(X) + S(B|X)$.

You know that $S(B|X=i) = S(\rho_i)$. $S(B|X)$ is then just the entropy of $\rho_i$ on average i.e. $\sum_i p_i S(\rho_i)$. That leaves you with

$$I(X:B) = S(X) + S(B) - S(XB) = S(B) - S(B|X) $$

as desired.

Answer 2

Let us express the state $\rho^{XB}$ in terms of eigen decomposition of $\rho_i$: $\rho^{XB}{=}\sum_{i,\lambda^{(i)}}p_i\lambda^{(i)}|i\rangle\langle i|{\otimes}|\lambda^{(i)}\rangle\langle\lambda^{(i)}|$, where $\{\lambda^{(i)}\}$ and $\{|\lambda^{(i)}\rangle\}$ are eigen values and eigen vectors of $\rho_i$ respectively. Now the following equations hold: $$ \begin{align} \log _2(\rho^{XB})&{=}\sum_{i,\lambda^{(i)}}\log _2(p_i\lambda^{(i)})|i\rangle\langle i|^X{\otimes}|\lambda^{(i)}\rangle\langle\lambda^{(i)}|^B \\ &{=}\sum_{i,\lambda^{(i)}}[\log _2(p_i) + \log_2(\lambda^{(i)})]. |i\rangle\langle i|^X{\otimes}|\lambda^{(i)}\rangle\langle\lambda^{(i)}|^B\\ &=\sum_{i}\log _2(p_i) |i\rangle\langle i|^X {\otimes} \mathbb{I}^B + \sum_{i} |i\rangle\langle i|^X{\otimes} \log_2(\rho_i)^B. \end{align} $$

Now let us calculate $S(\rho^{XB}){=}-Tr[\rho^{XB}. \log _2(\rho^{XB})]$:

$$ \begin{align} S(\rho^{XB})&{=}-Tr\big[\rho^{XB}. \log _2(\rho^{XB})\big] \\ &{=}-Tr \bigg[\bigg(\sum_i p_i |i\rangle \langle i|^X {\otimes} \rho_i^B\bigg). \bigg( \sum_{j}\log _2(p_j) |j\rangle\langle j|^X {\otimes} \mathbb{I}^B + \sum_{j} |j\rangle\langle j|^X{\otimes} \log_2(\rho_j)^B\bigg)\bigg]\\ &=-\sum_i p_i \log_2(p_i) -\sum_i p_i Tr\big[\rho_i \log_2(\rho_i)\big] \\ &= H(\vec{p}) + \sum_i p_i S(\rho_i). \end{align} $$ Here,$H(\vec{p})$ is the Shannon entropy for the probabilities $\{p_i\}$. Now $I(X:B)=S(X)+S(B)-S(XB)$. We can see $S(X)=S\big(\sum_i p_i |i\rangle \langle i|\big){=}H(\vec{p})$, $S(B){=}S\big(\sum_i p_i \rho_i\big)$. Hence, putting all the quantities together we have $$ \begin{align} I(X:B)&=S(X)+S(B)-S(XB) \\ &= H(\vec{p}) + S\bigg(\sum_i p_i \rho_i\bigg) - H(\vec{p}) - \sum_i p_i S(\rho_i) \\ &=S\bigg(\sum_i p_i \rho_i\bigg) - \sum_i p_i S(\rho_i). \end{align} $$