2
$\begingroup$

The Renyi entropy of order $\beta$, for a discrete probability distribution $p$ is given by \begin{equation} H_{\beta}(p) = \frac{1}{1 - \beta} ~\log \left( \sum_{i \in S} p(i)^{\beta} \right), \end{equation} where $S$ is the set of all strings in the support of $p$.

As is mentioned here, for two discrete distributions $p$ and $r$ the Renyi entropy of the product distribution $p \times r$ is

\begin{equation} H_{\beta}(p \times r) = H_{\beta}(p) + H_{\beta}(r). \end{equation}

What might be a proof of this fact?

Improve this question
$\endgroup$
2
  • $\begingroup$ what applications does Renyi entropy have in quantum $\endgroup$
    – develarist
    Dec 16, 2020 at 4:24
  • 1
    $\begingroup$ A quantum variant of the classical Renyi entropy appear in a lot of places in quantum information theory. Properties of the same can be found here: scholarpedia.org/article/Quantum_entropies. $\endgroup$
    – BlackHat18
    Dec 16, 2020 at 9:34

1 Answer 1

Reset to default
5
$\begingroup$

This only holds if the two distributions are independent. In this case $$ \begin{aligned} H_{\beta}(p \times q) &= \frac{1}{1-\beta} \log\left( \sum_{i,j}(p(i) q(j))^{\beta} \right) \\ &= \frac{1}{1-\beta} \log\left( \left(\sum_{i}p(i)^{\beta}\right) \left(\sum_jq(j)^{\beta}\right) \right) \\ &= \frac{1}{1-\beta} \left(\log \left(\sum_{i}p(i)^{\beta}\right) + \log \left(\sum_{j}q(j)^{\beta}\right)\right) \\ &= H_{\beta}(p) + H_{\beta}(q) . \end{aligned} $$

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.