# Quantum relative entropy with respect to a pure state

I want to evalualte the quantum relative entropy $$S(\rho|| \sigma)=-{\rm tr}(\rho {\rm log}(\sigma))-S(\rho)$$, where $$\sigma=|\Psi\rangle\langle\Psi|$$ is a density matrix corresponding to a pure state and $$\rho$$ is a density matrix corresponding to an arbitrary mixed state. Here, $$S(\rho)$$ simply denotes the Von Neumann entropy of $$\rho$$. Given that $$\sigma$$ is diagonal, with eigenvalues $$0$$ and $$1$$ it seems that the first term in the quantum relative entropy will in general be infinite. As $$S(\rho)\leq {\rm log}(d)$$, where $$\rho \in L({\mathcal H}^{d})$$, the first term dominates and the quantum relative entropy is also infinite. Is this correct? And if so, what's the intuition behind this fact?

If $$\sigma$$ is not full rank, then the correct way to interpret the quantum relative entropy formula you wrote is to assign it the value of $$+\infty$$ when the support of $$\rho$$ is not included in the support of $$\sigma$$. Wikipedia has a nice explanation of how to interpret this, but you can think that the reason for which the quantum relative entropy is finite in that case is that $$\lim_{x\to 0} x \log(x) = 0$$.

In your case, the support of $$\sigma$$ is simply the one-dimensional subspace spanned by $$|\Psi\rangle$$. For $$\rho$$ to be supported on this subspace, it must hold that $$\rho$$ is also pure, so it must be also equal to $$|\Psi\rangle\!\langle\Psi|$$, and so $$\rho=\sigma$$. In this case, the quantum relative entropy vanishes.

So in summary, if $$\sigma$$ is pure, than $$S(\rho||\sigma)$$ is either $$0$$ (when $$\rho=\sigma$$) or $$+\infty$$ (in the other cases).

• Maybe a dumb question in regards to the wikipedia article. It states "he quantum relative entropy is a measure of our ability to distinguish two quantum states where larger values indicate states that are more different. Being orthogonal represents the most different quantum states can be". Wouldn't the QRE between $|0\rangle\langle0|$ and $|+\rangle\langle+|$ also be $+\infty$, as neither is supported by the one-dimensional subspace of the other, right? – GaussStrife Oct 15 '20 at 15:33
• Or is $|0\rangle\langle0|$ still in the support of $|+\rangle\langle+|$ given $|+\rangle=\frac{|0\rangle+|1\rangle}{\sqrt{2}}$? – GaussStrife Oct 15 '20 at 15:36
• @GaussStrife The support of an operator $\rho$, denoted $\mathrm{supp}(\rho)$. is defined in this context to be the orthogonal complement of the kernel $\mathrm{ker}(\rho) = \{ |x \rangle \in \mathcal{H} : \rho |x \rangle = 0$. Thus $\mathrm{ker}(|+ \rangle \langle + |) = \mathrm{span}\{ |-\rangle\}$ and $\mathrm{supp}(|+\rangle \langle +|) = \mathrm{span}\{|+\rangle\}$. – Rammus Oct 16 '20 at 8:25
• @GaussStrife Wikipedia also says "However, one should be careful not to conclude that the divergence of the quantum relative entropy S (ρ‖σ) implies that the states ρ and σ are orthogonal or even very different by other measures. Specifically, S (ρ‖σ) can diverge when ρ and σ differ by a vanishingly small amount as measured by some norm." Two rank one states can be arbitrary close (in norm/fidelity), but unless they are exactly equal the relative entropy diverges. So yes, $|0\rangle\!\langle 0|$ is not supported on $|+\rangle\!\langle +|$ and the relative entropy between the two is $+\infty$. – Angelo Lucia Oct 16 '20 at 17:00
• @AngeloLucia See I need to learn to be patient and read the next sentence. Alright this makes perfect sense then. All states that have non-trivial intersection with either the kernel(nullspace) of a given state, or even just cannot be located in said support (regardless of the amount the states differ) diverge in regards to entropy. – GaussStrife Oct 16 '20 at 18:13

I'd like to add to Angelo Lucia's answer slightly. It is not very surprising that $$S(\rho \| \sigma)$$ can take the value $$+\infty$$ once we realise that the relative entropy is a generalization of the Kullback-Liebler divergence $$D(p \| q)$$ between probability distributions $$p$$ and $$q$$. Formally, given two distributions $$p,q$$ over some finite set $$\mathcal{X}$$ the KL-divergence is defined as $$D(p\| q) = \begin{cases} \sum_{x \in \mathcal{x}} p(x) \log \frac{p(x)}{q(x)} \quad & \text{if } \mathrm{supp}(p) \subseteq \mathrm{supp}(q) \\ + \infty & \text{otherwise} \end{cases}$$ where $$\mathrm{supp}(p) = \{x \in \mathcal{X} : p(x)> 0\}$$. Note that if we fix a basis and consider only diagonal states in that basis, i.e. $$\rho = \sum_{x} p(x) |x\rangle \langle x |$$ and $$\sigma = \sum_x q(x) |x\rangle \langle x |$$, then computing $$S(\rho \| \sigma)$$ we recover exactly the KL-divergence $$D(p \| q)$$. The analogous situation to taking $$\sigma$$ to be a pure state is taking $$q$$ to be some point distribution (delta-distribution). In this case we see that $$D(p\| q)$$ is finite iff $$p=q$$, which is exactly what we observe in the quantum case for pure states.