# How to evaluate the mutual information of a classical-quantum state?

Suppose I have the state $$\sum_i p_i \vert i\rangle\langle i\vert \otimes\rho_i$$ on subsystems $$X$$ (classical part) and $$B$$ (quantum part) and I wish to evaluate the mutual information $$I(X:B) = S(X) + S(B) - S(XB)$$.

Nielsen and Chuang claims in (12.11) that the answer is

$$I(X:B) = S(\sum_i p_i\rho_i) - \sum_ip_i S(\rho_i)$$

I don't understand how they arrived at this and the theorem they refer to as proof of this statement is an inequality, not an equality. Trying to evaluate it myself, I don't really know how to write down $$S(XB)$$ in a smart way.

Note that $$S(XB) = S(X) + S(B|X)$$.

You know that $$S(B|X=i) = S(\rho_i)$$. $$S(B|X)$$ is then just the entropy of $$\rho_i$$ on average i.e. $$\sum_i p_i S(\rho_i)$$. That leaves you with

$$I(X:B) = S(X) + S(B) - S(XB) = S(B) - S(B|X)$$

as desired.

Let us express the state $$\rho^{XB}$$ in terms of eigen decomposition of $$\rho_i$$: $$\rho^{XB}{=}\sum_{i,\lambda^{(i)}}p_i\lambda^{(i)}|i\rangle\langle i|{\otimes}|\lambda^{(i)}\rangle\langle\lambda^{(i)}|$$, where $$\{\lambda^{(i)}\}$$ and $$\{|\lambda^{(i)}\rangle\}$$ are eigen values and eigen vectors of $$\rho_i$$ respectively. Now the following equations hold: \begin{align} \log _2(\rho^{XB})&{=}\sum_{i,\lambda^{(i)}}\log _2(p_i\lambda^{(i)})|i\rangle\langle i|^X{\otimes}|\lambda^{(i)}\rangle\langle\lambda^{(i)}|^B \\ &{=}\sum_{i,\lambda^{(i)}}[\log _2(p_i) + \log_2(\lambda^{(i)})]. |i\rangle\langle i|^X{\otimes}|\lambda^{(i)}\rangle\langle\lambda^{(i)}|^B\\ &=\sum_{i}\log _2(p_i) |i\rangle\langle i|^X {\otimes} \mathbb{I}^B + \sum_{i} |i\rangle\langle i|^X{\otimes} \log_2(\rho_i)^B. \end{align}

Now let us calculate $$S(\rho^{XB}){=}-Tr[\rho^{XB}. \log _2(\rho^{XB})]$$:

\begin{align} S(\rho^{XB})&{=}-Tr\big[\rho^{XB}. \log _2(\rho^{XB})\big] \\ &{=}-Tr \bigg[\bigg(\sum_i p_i |i\rangle \langle i|^X {\otimes} \rho_i^B\bigg). \bigg( \sum_{j}\log _2(p_j) |j\rangle\langle j|^X {\otimes} \mathbb{I}^B + \sum_{j} |j\rangle\langle j|^X{\otimes} \log_2(\rho_j)^B\bigg)\bigg]\\ &=-\sum_i p_i \log_2(p_i) -\sum_i p_i Tr\big[\rho_i \log_2(\rho_i)\big] \\ &= H(\vec{p}) + \sum_i p_i S(\rho_i). \end{align} Here,$$H(\vec{p})$$ is the Shannon entropy for the probabilities $$\{p_i\}$$. Now $$I(X:B)=S(X)+S(B)-S(XB)$$. We can see $$S(X)=S\big(\sum_i p_i |i\rangle \langle i|\big){=}H(\vec{p})$$, $$S(B){=}S\big(\sum_i p_i \rho_i\big)$$. Hence, putting all the quantities together we have \begin{align} I(X:B)&=S(X)+S(B)-S(XB) \\ &= H(\vec{p}) + S\bigg(\sum_i p_i \rho_i\bigg) - H(\vec{p}) - \sum_i p_i S(\rho_i) \\ &=S\bigg(\sum_i p_i \rho_i\bigg) - \sum_i p_i S(\rho_i). \end{align}