How is $S(\rho)=H(p_{i})+\sum_{i}p_{i}S(\rho_{i})\le \log(d)$ possible if $\rho_{i}$ are not pure states?

Question

I know how this can be proved using the quantum relative entropy. However, even with this proof, and am still confused about how this emerges.

Say I have a source that produces two states $\rho_1$ and $\rho_{2}$ with probability a half each, and both are mixed states, ie $S(\rho_i)>0$ for each of them. The dimensions of the Hilbert space is $2$.

How can $S(\rho)=H(p_{i})+\sum_{i}p_{i}S(\rho_{i})\le \log(d)$, given that $H(p_{i})=1$ and $S(\rho_i)>0$, given that $\log(d)=\log(2)=1?$

I am assuming that I am missing something obvious in the actual construction of $\rho$, in that something is bounding $H(p_{i})$ away from 1. I am assuming this has something to do with orthogonal supports, as $S(\rho) \le H(p)+\sum_{i}p_iS(\rho_{i})$ if they are not orthogonal.

Answer 1

I guess you meant to consider generic (not necessarily pure) states $\rho_1,\rho_2$ rather than $|\psi\rangle\langle\psi|$ and $\lvert\phi\rangle\langle \phi\rvert$, so I'll consider the slightly modified question "how is $$S(\rho)= H(p)+\sum_i p_i S(\rho_i)\le \log d$$ possible for a state $\rho=\sum_i p_i\rho_i$ with $H(p)=\log d$?"

The first thing to notice is that, in general, $S(\rho)\le H(p)+\sum_i p_i S(\rho_i)$. This is not an identity unless $\rho_i$ have orthogonal support. You then also have $S(\rho)\le\log d$. From these, you cannot imply $H(p)+\sum_i p_i S(\rho_i)\le \log d$, which is indeed untrue in general, as you observed.

When $\rho_i$ do have orthogonal support, then $S(\rho)= H(p)+\sum_i p_i S(\rho_i)$, and therefore you must also have $H(p)+\sum_i p_i S(\rho_i)\le \log d$. So then why don't we have a contradiction? Well, consider the case with $H(p)=\log d$. This means that you are dealing with a state $\rho$ that is a balanced mixture of $d$ different states with orthogonal support. The only way to get $d$ states with orthogonal support in a $d$-dimensional space is that they each have unit rank, i.e. they are all pure, and thus $S(\rho_i)=0$ for all $i$.