Shannon entropy is least when Measurement basis = Mixture basis

Question

For a one qubit system, take a basis. Call this the mixture basis. Consider only basis states and classical mixtures of these basis states.

Definition of Shannon Entropy used here: Defined with respect to the measurement basis, on the probabilities of various outcomes. For eg: $\frac{1}{2}|0\rangle \langle0| + \frac{1}{2}|+\rangle \langle+|$, when measured in the $|0\rangle, |1\rangle$ basis has Shannon Entropy $-\frac{3}{4}\log(\frac{3}{4}) - \frac{1}{4}\log(\frac{1}{4})$ because there is $\frac{3}{4}$ chance of measuring $|0\rangle$ and $\frac{1}{4}$ chance of measuring $|1\rangle$.

I'm trying to prove that the least value of Shannon Entropy will occur when the measurement basis is equal to the mixture basis.

(This is for me to get an intuition of Von Neumann entropy. If I prove the above, then I can think of Von Neumann entropy as the least Shannon entropy I could get after measuring across any basis.)

Let the mixture basis be $\frac{1}{2}(I + n.\sigma)$ and $\frac{1}{2}(I - n.\sigma)$

Let the measurement basis be $\frac{1}{2}(I + m.\sigma)$ and $\frac{1}{2}(I - m.\sigma)$

Let the qubit be $$p\frac{1}{2}(I + n.\sigma) + (1-p)\frac{1}{2}(I - n.\sigma)$$

Probability of the qubit showing up as $\frac{1}{2}(I + m.\sigma)$ when measured is: $$p(\frac{1}{2}(1 + m.n)) + (1-p)(\frac{1}{2}(1-m.n))$$

Let the above value be $p^{'}$

Then the Shannon Entropy will be $p^{'}log(p^{'}) + (1-p^{'})log(1-p^{'})$

And to minimize the entropy, I need to minimise or maximise $p^{'}$

I'm not sure how to do that though, and whether what I'm trying to do so far makes sense. I'll be grateful for any help on continuing the proof/ insight on the intuition I'm trying to build.

Answer 1

Let's write $$ p'=\frac12+m\cdot n\frac{2p-1}{2}, $$ and assume without loss of generality that $p>\frac12$, which also means that $p'>\frac12$. Note the binary entropy function $h(p')$ is symmetric about $p'=\frac12$, and is monotonically decreasing for $p'>\frac12$, meaning that we want to make $p'$ as large as possible in order to minimise $h(p')$. Since $p$ is fixed, the only way to increase $p'$ is to make $m\cdot n$ as large as possible. This happens with $m=n$ so that $m\cdot n=1$ and $p'=p$.

Answer 2

My favourite way of proving that the Shannon entropy is minimized for a measurement in the qubit basis is through the notion of majorizaion (see Nielsen and Chuang or the book on Matrix Analysis by Bhatia for a formal definition). Specifically $p$ and $(1-p)$ is related to $p'$ and $(1-p')$ with the following relation

\begin{equation} \left(\begin{array}{c} p'\\ 1-p' \end{array}\right)=D \left(\begin{array}{c} p\\ 1-p \end{array}\right) \end{equation}

where \begin{equation} D= \left( \begin{array}{c} \frac{1}{2}(1+m\cdot n)&\frac{1}{2}(1-m\cdot n)\\ \frac{1}{2}(1-m\cdot n)&\frac{1}{2}(1+m\cdot n) \end{array} \right) \end{equation} is a bistochastic matrix, meaning it's elements are probabilities and the sum of it's columns and arrays is equal to one. We then say that the probability distribution defined be the left hand side of the first equation (call it $\vec{p}'$) is majorized by that on the right hand side (call it $\vec{p}$) and we write symbolically that $\vec{p}'\prec\vec{p}$. Inuitively this means that $\vec{p}'$ is more "random" than $\vec{p}$.

Now for any Schur concave function $f$

$$f(\vec{p}')\geq f(\vec{p})\quad \mbox{when}\quad \vec{p}'\prec\vec{p}$$

The Shannon entropy is such a Schur concave function and the proof is now complete.