1
$\begingroup$

In Page 520, Entropy and information, Quantum Computation and Quantum Information by Nielsen and Chuang, it is given that


The relative entropy $S(ρ||σ)$ is jointly convex in its arguments, where $S(ρ||σ)=tr(\rho\log\rho)-tr(\rho\log\sigma)$ and $\rho,\sigma$ are positive semidefinite with trace $1$.

Proof:

For arbitrary matrices $A$ and $X$ acting on the same space define $$ I_t(A,X)\equiv tr(X^\dagger A^tXA^{1-t})-tr(X^\dagger XA)\tag{11.98} $$ The first term in this expression is concave in $A$, by Lieb’s theorem, and the second term is linear in $A$. Thus, $I_t(A, X)$ is concave in $A$. Define $$ \color{red}{ \begin{align} I(A,X)&=\frac{d}{dt}\bigg|_{t=0}I_t(A,X)\\ &=tr(X^\dagger (\log A)XA)-tr(X^\dagger X(\log A)A) \end{align}}\tag{11.99} $$Noting that $I_0(A,X)=0$ and using the concavity of $I_t(A,X)$ in $A$ we have \begin{align} I(\lambda A_1+(1-\lambda)A_2,X)&=\lim_{\Delta\to 0}\frac{I_\Delta(\lambda A_1+(1-\lambda)A_2,X)}{\Delta}\tag{11.100}\\ &\ge \lambda\lim_{\Delta\to 0}\frac{I_\Delta(A_1,X)}{\Delta}+(1-\lambda)\lim_{\Delta\to 0}\frac{I_\Delta(A_2,X)}{\Delta}\tag{11.101}\\ &=\lambda I(A_1,X)+(1-\lambda)I(A_2,X)\tag{11.102} \end{align} That is, $I(A,X)$ is a concave function of $A$. Defining the block matrices \begin{align} A\equiv\begin{bmatrix}\rho&0\\0&\sigma\end{bmatrix}\quad,\quad X\equiv\begin{bmatrix}0&0\\I&0\end{bmatrix}\tag{11.103} \end{align} $$ A=\begin{bmatrix}\rho&0\\0&\sigma\end{bmatrix}=\begin{bmatrix}VDV^\dagger&0\\0&UCU^\dagger\end{bmatrix}=\begin{bmatrix}V&0\\0&U\end{bmatrix}\begin{bmatrix}D&0\\0&C\end{bmatrix}\begin{bmatrix}V&0\\0&U\end{bmatrix}^\dagger=W\begin{bmatrix}D&0\\0&C\end{bmatrix}W^\dagger\\ \log A=W\log \begin{bmatrix}D&0\\0&C\end{bmatrix}W^\dagger=W\begin{bmatrix}\log D&0\\0&\log C\end{bmatrix}W^\dagger=\begin{bmatrix}V\log D V^\dagger&0\\0&U\log CU^\dagger\end{bmatrix} =\begin{bmatrix}\log\rho&0\\0&\log\sigma\end{bmatrix} $$ $$ X^\dagger(\log A) XA=\begin{bmatrix}0&I\\0&0\end{bmatrix}\begin{bmatrix}\log\rho&0\\0&\log\sigma\end{bmatrix}\begin{bmatrix}0&0\\I&0\end{bmatrix}\begin{bmatrix}\rho&0\\0&\sigma\end{bmatrix}=\begin{bmatrix}0&\log\sigma\\0&0\end{bmatrix}\begin{bmatrix}0&0\\\rho&0\end{bmatrix}=\begin{bmatrix}(\log\sigma)\rho&0\\0&0\end{bmatrix}\\ \implies tr(X^\dagger(\log A) XA)=tr((\log\sigma)\rho)=tr(\rho\log\sigma) $$ $$ X^\dagger X(\log A) A=\begin{bmatrix}0&I\\0&0\end{bmatrix}\begin{bmatrix}0&0\\I&0\end{bmatrix}\begin{bmatrix}\log\rho&0\\0&\log\sigma\end{bmatrix}\begin{bmatrix}\rho&0\\0&\sigma\end{bmatrix}=\begin{bmatrix}I&0\\0&0\end{bmatrix}\begin{bmatrix}(\log\rho)\rho&0\\0&(\log\sigma)\sigma\end{bmatrix}=\begin{bmatrix}(\log\rho)\rho&0\\0&0\end{bmatrix}\\ \implies tr(X^\dagger X(\log A) A)=tr(\rho\log\rho) $$ $$ \therefore I(A,X)=tr(X^\dagger(\log A) XA)-tr(X^\dagger X(\log A) A)\\ =tr(\rho\log\sigma)-tr(\rho\log\rho)=-S(\rho||\sigma) $$ we can easily verify that $I(A,X)=−S(ρ||σ)$. The joint convexity of $S(ρ||σ)$ follows from the concavity of $I(A, X)$ in $A$.


Lieb's theorem states that, let $X$ be a matrix, and $0≤t≤1$. Then the function $$ f(A,B)=tr(X^\dagger A^tXB^{1-t}) $$ is jointly concave in positive matrices $A$ and $B$. ie., $$ tr\Big[X^\dagger (\lambda A_1+(1-\lambda)A_2)^tX(\lambda B_1+(1-\lambda)B_2)^{1-t}\Big]\\ \ge\lambda tr(X^\dagger A_1^tXB_1^{1-t})+(1-\lambda)tr(X^\dagger A_2^tXB_2^{1-t}) $$


How do we reach $I(A,X)=\frac{d}{dt}\bigg|_{t=0}I_t(A,X)=tr(X^\dagger (\log A)XA)-tr(X^\dagger X(\log A)A)$ ?, where $I_t(A,X)\equiv tr(X^\dagger A^tXA^{1-t})-tr(X^\dagger XA)$

I think I am confused by the fact that we are taking the derivative of $A^t$ with respect to $t$ inside the trace operation.

It'd be helpful if someone could direct me in the right direction in order to understand the whole proof.

Cross Posted on Math Stack Exchange

$\endgroup$
6
  • 1
    $\begingroup$ Can you distill out the essence of your question? It's very difficult to understand what you're asking. Are you concerned about the definition of $I(A,X)$? $\endgroup$
    – Rammus
    Commented Apr 4, 2023 at 15:48
  • $\begingroup$ @Rammus First doubt is, how do I make sense of reaching $I_t(A,X)$ from $I(A,X)$? Looks like we are taking the derivative of $A^t$ w.r.t $t$ inside trace operation. $\endgroup$
    – Sooraj S
    Commented Apr 4, 2023 at 17:38
  • $\begingroup$ @SoorajS please add clarifications by editing the post, not just in the comments. It's also not clear which question you're asking exactly. Are you just asking how to compute the derivative in that expression or are you asking explanations for the steps around 11.103? Please edit the post to focus it on a specific issue and remove redundant information $\endgroup$
    – glS
    Commented Apr 4, 2023 at 18:18
  • $\begingroup$ @glS Thanks for responding. I have edited the post to make it more specific. Please have a look. $\endgroup$
    – Sooraj S
    Commented Apr 4, 2023 at 18:42
  • 1
    $\begingroup$ I don't think I understand the question. You define $I(A,X)=\partial_t|_{t=0} I_t(A,X)$, and then you ask how to prove that $I(A,X)=\partial_t|_{t=0} I_t(A,X)$? Also, the trace doesn't give any problems, you can differentiate under the trace $\endgroup$
    – glS
    Commented Apr 4, 2023 at 19:00

1 Answer 1

1
$\begingroup$

By linearity, $$\partial_t \operatorname{tr}(F(t))=\operatorname{tr}(F'(t)).$$ Then, $$\partial_t A^t=\partial_t e^{t\log A}= \log(A) A^t,$$ $$\partial_t \operatorname{tr}(B A^t C A^{1-t}) = \operatorname{tr}(B \log(A) A^t C A^{1-t}) - \operatorname{tr}(BA^t C \log(A) A^{1-t}),$$ $$\partial_t\big|_{t=0} \operatorname{tr}(B A^t C A^{1-t}) = \operatorname{tr}(B \log(A) C A) - \operatorname{tr}(B C \log(A) A).$$

$\endgroup$
6
  • $\begingroup$ Thanks for the solution. Could you please direct me to some mathematical materials where I can get more into the idea of taking such derivatives of matrix, also to make sense of taking the derivative of the trace ? $\endgroup$
    – Sooraj S
    Commented Apr 6, 2023 at 5:10
  • 1
    $\begingroup$ @SoorajS I don't know any specific souce. It's just a normal derivative, nothing fancy. Something like $F(t)$ here means that each element of the matrix depends on the parameter $t$, so $\partial_t \operatorname{tr}(F(t))=\partial_t \sum_i F(t)_{ii}=\sum_i F'(t)_{ii}$. I don't know that there's much more to say. I suppose you can see from the point of view of multivariable calculus, thinking of $F$ as a map $F:\mathbb{R}\to \mathbb{R}^{n^2}$, and then you're defining derivatives in the standard way for these kinds of objects. Then taking the trace is a linear operation hence derivatives behave $\endgroup$
    – glS
    Commented Apr 6, 2023 at 9:06
  • 1
    $\begingroup$ @SoorajS I'd actually say it's arguably less obvious what the derivative of things like $A^t$ means. Defining $f(A)$ is relatively easy for $A$ Hermitian and $f$ defined on its spectrum. For more general cases you can operate via series expansions, but things can get trickier. $A^t$ should be doable via series because $\exp$ always converges, and you can then differentiate the elements of the series to define $\partial_t A^t$. Or directly define derivative with matrices: $\lim (A(t+h)-A(t))/h$ with $A(t)\equiv A^t$, and do the calculation $\endgroup$
    – glS
    Commented Apr 6, 2023 at 9:15
  • $\begingroup$ Okay so you are saying that, $A=VDV^\dagger=V\begin{bmatrix}d_{1} & & \\ & \ddots & \\ & & d_{n}\end{bmatrix}V^\dagger\implies A^t=VD^tV^\dagger=V\begin{bmatrix}d^t_{1} & & \\ & \ddots & \\ & & d^t_{n}\end{bmatrix}V^\dagger$ $\endgroup$
    – Sooraj S
    Commented Apr 6, 2023 at 15:20
  • $\begingroup$ therefore $\frac{d}{dt}A^t=V\frac{d}{dt}D^tV^\dagger=V\begin{bmatrix}\frac{d}{dt}d^t_{1} & & \\ & \ddots & \\ & & \frac{d}{dt}d^t_{n}\end{bmatrix}V^\dagger=V\begin{bmatrix}d^t_{1}\log d_1 & & \\ & \ddots & \\ & & d^t_{n}\log d_n\end{bmatrix}V^\dagger=V\begin{bmatrix}d^t_{1} & & \\ & \ddots & \\ & & d^t_{n}\end{bmatrix}\begin{bmatrix}\log d_{1} & & \\ & \ddots & \\ & & \log d_{n}\end{bmatrix}V^\dagger=VD^t\log D V^\dagger=VD^tV^\dagger V\log D V^\dagger=A^t\log A$. Since $A$ is hermitian, this is not much complicated. Right ? $\endgroup$
    – Sooraj S
    Commented Apr 6, 2023 at 15:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.