The Renyi entropy of order $\beta$, for a discrete probability distribution $p$ is given by \begin{equation} H_{\beta}(p) = \frac{1}{1 - \beta} ~\log \left( \sum_{i \in S} p(i)^{\beta} \right), \end{equation} where $S$ is the set of all strings in the support of $p$.

As is mentioned here, for two discrete distributions $p$ and $r$ the Renyi entropy of the product distribution $p \times r$ is

\begin{equation} H_{\beta}(p \times r) = H_{\beta}(p) + H_{\beta}(r). \end{equation}

What might be a proof of this fact?

  • $\begingroup$ what applications does Renyi entropy have in quantum $\endgroup$
    – develarist
    Dec 16 '20 at 4:24
  • 1
    $\begingroup$ A quantum variant of the classical Renyi entropy appear in a lot of places in quantum information theory. Properties of the same can be found here: scholarpedia.org/article/Quantum_entropies. $\endgroup$
    – BlackHat18
    Dec 16 '20 at 9:34

This only holds if the two distributions are independent. In this case $$ \begin{aligned} H_{\beta}(p \times q) &= \frac{1}{1-\beta} \log\left( \sum_{i,j}(p(i) q(j))^{\beta} \right) \\ &= \frac{1}{1-\beta} \log\left( \left(\sum_{i}p(i)^{\beta}\right) \left(\sum_jq(j)^{\beta}\right) \right) \\ &= \frac{1}{1-\beta} \left(\log \left(\sum_{i}p(i)^{\beta}\right) + \log \left(\sum_{j}q(j)^{\beta}\right)\right) \\ &= H_{\beta}(p) + H_{\beta}(q) . \end{aligned} $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.