# Proof of joint entropy theorem

From section 11.3.2 of Nielsen & Chuang:

(4) let $$\lambda_i^j$$ and $$\left|e_i^j\right>$$ be the eigenvalues and corresponding eigenvectors of $$\rho_i$$. Observe that $$p_i\lambda_i^j$$ and $$\left|e_i^j\right>$$ are the eigenvalues and eigenvectors of $$\sum_ip_i\rho_i$$ and thus \begin{align}S\left(\sum_ip_i\rho_i\right) &= -\sum_{ij}p_i\lambda_i^j\log p_i\lambda_i^j \\ &= - \sum_ip_i\log p_i - \sum_ip_i\sum_j\lambda_i^j\log\lambda_i^j \\ &= H\left(p_i\right) + \sum_ip_iS\left(\rho_i\right)\end{align}

So this lemma is used to prove the joint entropy of $$S(\rho_{AB})$$ with $$\rho_{AB}$$ equal to

$$\rho_{AB} = \sum_i p_i (\rho_i)_A \otimes (|e_i\rangle\langle e_i|)_B$$

In the attached picture (the lemma), they do it first for only the A-system $$\rho_A = \sum_i p_i (\rho_i)_A$$, and then they say that this result directly leads to the same result for the above described AB-system (exactly the same?). First of all, I don't understand how they go from the first to the second line. I understand the fact that $$p_i \lambda^j_i$$ are the eigenvalues, but I really don't see how... Second question being, how can you use this result to infer the same result for $$S(\rho_{AB})$$? I mean you have your extra system in $$H_B$$ "hanging" on your A-system and I don't see how this just magically disappears or eventually comes down to a factor of 1 when explicitly calculating $$S(\rho_AB)$$. I feel pretty silly because this is already the proof aka explanation in the book Nielsen & Chuang and I don't even get it. So maybe explain it really simple and down to Earth for me please.

I don't understand how they go from the first to the second line

So, you're starting from $$-\sum_{ij}p_i\lambda^j_i\log(p_i\lambda^j_i)$$. Remember that $$\log(ab)=\log(a)+\log(b)$$, so this is the same as $$-\sum_{ij}p_i\lambda^j_i\log(p_i)-\sum_{ij}p_i\lambda^j_i\log(\lambda^j_i)$$ For the first term, do the sum over $$j$$: $$\sum_j\lambda^j_i=\text{Tr}(\rho_i)=1$$. This gives you $$-\sum_{i}p_i\log(p_i)-\sum_{i}p_i\sum_j\lambda^j_i\log(\lambda^j_i),$$ which was the second line you were after.

how can you use this result to infer the same result for $$S(\rho_{AB})$$

Observe that the eigenvectors of $$\rho_{AB}$$ are $$|e^j_i\rangle|e_i\rangle$$ with eigenvalue $$p_i\lambda^j_i$$. Hence, $$S(\rho_{AB})=-\sum_{ij}p_i\lambda^j_i\log(p_i\lambda^j_i),$$ and you're back to that first line again...

PS for anybody else coming in and trying to make sense of the question, it is important to state that a condition of the theorem that is being proven is that the $$\rho_i$$ have support on orthogonal subspaces. This way, if a given $$\rho_i$$ has an eigenvector $$|e^j_i\rangle$$ with a non-zero eigenvalue $$\lambda^j_i$$, then $$|e^j_i\rangle$$ is also an eigenvector of all other $$\rho_k$$ for $$k\neq i$$, but $$\lambda^j_k=0$$. That is the key reason why we can write that the eigenvalues of $$\sum_ip_i\rho_i$$ are $$p_i\lambda^j_i$$.

That lemma has condition that all $$p_i$$ ($$1\le i \le n$$) have support on orthogonal subspaces. This means that there is a decomposition $$H = \oplus_i H_i$$ of the space $$H$$ such that $$p_i(H_j) = 0$$ if $$i\neq j$$ and $$p_i(v_i)\neq 0$$ if $$v_i \in H_i$$. Note that there can be component $$H_0$$ such that $$p_i(H_0)=0$$ for all $$i \ge 1$$.
Now there is a little oversimplification in their explanation. In fact we pick only nonzero eigenvalues $$\lambda_i^j$$, so vectors $$|e_i^j\rangle$$ lay in $$H_i$$. Now it is easy to see that $$\sum_ip_i\rho_i |e_k^j\rangle = p_k\lambda_k^j|e_k^j\rangle$$. Moreover, the union of all vectors $$|e_i^j\rangle$$ span the support of $$\sum_ip_i\rho_i$$. So this union is a complete set of orthogonal eigenvectors (excluding those from $$H_0$$).

As for your second question, we apply lemma to the states $$\rho_i^\prime = (\rho_i)_A \otimes (|e_i\rangle\langle e_i|)_B$$ on the system $$AB$$. That lemma is general, it is not "just for the A-system". You can check that all $$\rho_i^\prime$$ indeed have orthogonal support.