Building Intuition for Relative Von Neumann Entropy

Question

This is how I think about classical relative entropy: There is a variable that has distribution P, that is outcome $i$ has probability $p_i$ of occuring, but someone mistakes it to be of a distribution Q instead, so when outcome $i$ occurs, instead of being $-log(p_i)$ surprised, they are $-log(q_i)$ surprised (or gain said amount of information).

Now someone who knows both the distributions is calculating the relative Shannon entropy, so expectation value of their surprise is $-\Sigma p_i log(p_i)$ and they know that the mistaken person's probability of being $log(q_i)$ surprised is $p_i$, so their the expectation value of surprise is $-\Sigma p_i log({q_i})$ and the difference is $\Sigma p_i log(p_i) - \Sigma p_i log(q_i)$ which is the classical relative entropy.

For a given state, the Von Neumann entropy is the Shannon entropy minimised over all possible bases. Since in the measurement basis, the eigenvalues are the probabilities, and both eigenvalues and trace are basis invariant, we can write this as $\Sigma \lambda _i log(\lambda_i)$ which is also equal to $Tr(\rho log( \rho ))$.

Relative Von Neumann entropy is defined as follows: $$ Tr(\rho log(\rho)) - Tr(\rho log (\sigma))$$

The first term is understandable, but by analogy to the classical relative entropy, assuming that person Q is measuring in the sigma basis, let's call it ${\{| \sigma_i \rangle \}}$, the second term should reduce to $p^{'}_1 log (q1) + p^{'}_2 log(q2) ... $, where $p^{'}_i$ is the actual probability of the state $\rho$ landing on $| \sigma_i \rangle $. The log part is take care of, but I'm not sure how multiplying and tracing out will give this result.

If there's a better way to understand relative Von Neumann entropy, that's welcome too.

Answer 1

Posting an answer because I realised what my issue was:

What I didn't realise then: When a density matrix is written in any basis, the diagonal elements correspond to the probabilities of the density matrix landing on the basis states of that basis.

So, if in some basis formed by vectors $|x_1\rangle, |x_2\rangle, |x_3\rangle, |x_4 \rangle$, my density matrix is:

$$\begin{bmatrix} a_{11} & a_{12} & a_{13} & a_{14} \\ a_{21} & a_{22} & a_{23} & a_{24} \\ a_{31} & a_{32} & a_{33} & a_{34} \\ a_{41} & a_{42} & a_{43} & a_{44} \\ \end{bmatrix}$$

Then, the probability of this state showing up as $|x_1\rangle \langle x_1|$ when measured is $a_{11}$, as seen by the trace rule:

Probability that $\rho$ gives $|x_1\rangle \langle x_1|$ = $Tr(|x_1\rangle \langle x_1| \rho)$ which is $a_{11}$.

Therefore, in the above question, since the matrix is written in the $\{\sigma_i\}$ basis, $p_i'$ is the probability of $\rho$ landing on $|\sigma_i\rangle$, and when multiplied and trace is taken, it results in the corresponding expression. Therefore, the intuition holds.

Note that the '$\sigma$ basis' is one in which the log matrix is diagonal.