# How is $S(\rho)=H(p_{i})+\sum_{i}p_{i}S(\rho_{i})\le \log(d)$ possible if $\rho_{i}$ are not pure states?

I know how this can be proved using the quantum relative entropy. However, even with this proof, and am still confused about how this emerges.

Say I have a source that produces two states $$\rho_1$$ and $$\rho_{2}$$ with probability a half each, and both are mixed states, ie $$S(\rho_i)>0$$ for each of them. The dimensions of the Hilbert space is $$2$$.

How can $$S(\rho)=H(p_{i})+\sum_{i}p_{i}S(\rho_{i})\le \log(d)$$, given that $$H(p_{i})=1$$ and $$S(\rho_i)>0$$, given that $$\log(d)=\log(2)=1?$$

I am assuming that I am missing something obvious in the actual construction of $$\rho$$, in that something is bounding $$H(p_{i})$$ away from 1. I am assuming this has something to do with orthogonal supports, as $$S(\rho) \le H(p)+\sum_{i}p_iS(\rho_{i})$$ if they are not orthogonal.

• The two states you are taking a mixture of are pure, so their entropy is zero. Aug 23, 2020 at 21:40
• As I stated in the question, $|\psi\rangle\langle\psi|$ and $|\phi\rangle\langle\phi|$ are mixed, not pure. Perhaps there is some confusion around the notation I used? Aug 25, 2020 at 9:08
• Yeah I used the incorrect notation. My bad. Aug 25, 2020 at 12:00
• Yeah I didn't realise what I had wrote until @Rammus pointed it out Aug 25, 2020 at 14:53

I guess you meant to consider generic (not necessarily pure) states $$\rho_1,\rho_2$$ rather than $$|\psi\rangle\langle\psi|$$ and $$\lvert\phi\rangle\langle \phi\rvert$$, so I'll consider the slightly modified question "how is $$S(\rho)= H(p)+\sum_i p_i S(\rho_i)\le \log d$$ possible for a state $$\rho=\sum_i p_i\rho_i$$ with $$H(p)=\log d$$?"

The first thing to notice is that, in general, $$S(\rho)\le H(p)+\sum_i p_i S(\rho_i)$$. This is not an identity unless $$\rho_i$$ have orthogonal support. You then also have $$S(\rho)\le\log d$$. From these, you cannot imply $$H(p)+\sum_i p_i S(\rho_i)\le \log d$$, which is indeed untrue in general, as you observed.

When $$\rho_i$$ do have orthogonal support, then $$S(\rho)= H(p)+\sum_i p_i S(\rho_i)$$, and therefore you must also have $$H(p)+\sum_i p_i S(\rho_i)\le \log d$$. So then why don't we have a contradiction? Well, consider the case with $$H(p)=\log d$$. This means that you are dealing with a state $$\rho$$ that is a balanced mixture of $$d$$ different states with orthogonal support. The only way to get $$d$$ states with orthogonal support in a $$d$$-dimensional space is that they each have unit rank, i.e. they are all pure, and thus $$S(\rho_i)=0$$ for all $$i$$.

• So if $\sum_{i}p_{i}\rho_{i}=\frac{1}{2}\rho_{1}+\frac{1}{2}\rho_{2}$, $H(p)\ne1$ unless the supports of $\rho$ are orthogonal? Does that mean that the supports of both $\rho_{1}$ and $\rho_{2}$ are orthogonal w.r.t themselves, or do they also have to be orthogonal to the supports of eachother as well? Aug 25, 2020 at 12:04
• yes, the entropy is not maximal if the supports are not orthogonal and $p_1=p_2=1/2$. I don't understand the second sentence, both statements look equivalent to me. $\text{supp}(\rho_1)$ and $\text{supp}(\rho_2)$ are two subspaces of the underlying space $\mathcal H$ which are orthogonal, that's it
– glS
Aug 25, 2020 at 12:38
• I think I have misunderstood. If the supports of $\rho_{1}$ and $\rho_{2}$ are orthogonal, then they will be pure, so $S(\rho_{i}) =0$ and $H(p_{i})=1$. If they are not orthogonal, then I take it that the entropy will always be strictly less than $H(p_{i})+\sum_{i}p_{i}S(\rho_{i})$. Aug 25, 2020 at 14:56
• Am I to take it then that the only scenario wherein $S(\rho)=H(p_{i})+\sum_{i}p_{i}S(\rho_{i})$ and $S(\rho_{i})$ is when both the supports are orthogonal, but $\rho$ is not a balanced mixture of supports, ie all $\rho_{i} \ne \frac{1}{d}$? Aug 25, 2020 at 15:07
• @GaussStrife $d$ states with orthogonal support in a space of dimension $d$ must be pure (and thus have zero entropy), that's what I'm saying.
– glS
Aug 25, 2020 at 15:12