# Classical Information Theory vs. Quantum Information Theory

I am quite familiar with the basic concepts of information theory (sources, alphabets, simbols, strings, information, Shannon's entropy, noisy channels, Shannon's theorems, etc.). I always thought of information theory as a general meta-theory, applicable to a wide range of subjects and completely independent of any specific context of application. However I have came to understand that nowadays we like to split information theory in two separate branches:

• Classical information theory (CIT)
• Quantum information theory (QIT)

And I suspect that we like to do this because these two are quite different from one another, for example in CIT we use Shannon's entropy but in QIT we use Von Neumann's entropy; so I get that CIT and QIT are distinct in a meaningful way, however I have difficulties pin pointing the main differences between them and what are the physical quantum effects that create these differences. I would like a summary of all the most relevant point of disagreement between the two theories.

So my question is: What are the main differences between CIT and QIT? And which quantum effects are responsable for these differences?

This question has been crossposted on Physics.SE.

• IMO, the main difference between CIT and QIT is different definition of information (bits vs qubits). The rest follows. QIT includes transformation of quantum qubits into classical bits, but this is small and simple part. Jul 7 at 15:52

I would say that one of the key differences is the status of probability theory. In classical information theory, it is assumed that probabilities behave in certain ways:

• probabilities of mutually exclusive events add (for one or other to happen)
• probabilities of independent events multiply (for both to happen)

In quantum information theory, you instead have that

• probability amplitudes of mutually exclusive events add (for one or other to happen)
• probability amplitudes of independent events multiply (for both to happen)
• the probability of an event is the mod-square of the probability amplitude.

You said

I always thought of information theory as ... completely independent of any specific context of application

This is how it's generally presented, in part because the rules of probability are self-evident. But where do those rules come from? It's a model of the physical world, and it's a model that's not obeyed at the quantum level.

• I'd disagree with the last sentence. The rules of classical probabilities still very much apply to QM. In so far as probabilities quantify and deal with ignorance/incomplete knowledge about something, they behave in exactly the same way in classical and quantum physics. These rules don't come from a physical model, they come tautologically from what probabilities are built to model (which is physically agnostic). The divergence only appears if someone chooses to regard probability amplitudes as probabilities, which is, imho, just not accurate.
– glS
Jul 7 at 13:44
• My point is that if you put a $|0\rangle$ into a $\sqrt{not}$ gate and ask what is the probability of getting the $0/1$ answer, and then you try and use those probabilities to predict the outcome of two consecutive applications of $\sqrt{not}$, you get the wrong answer because the classical probability rules don't apply. Yes, there are contexts where the classical rules do indeed emerge out of the quantum ones. Jul 7 at 13:51
• I'd say the problem is thinking that the first output is a distribution of 0/1. The first gate outputs a state, which holds more information than the probs it produces if measured in the 0/1 basis. Applying the second gate, you act on the full state, not just the distro over 0/1, and it therefore doesn't violate anything we know about CIT that this second operation results in a deterministic outcome. The probabilities after the first gate only partially characterise the physical state, and it therefore stands to reason that they're not enough to predict future outcome probabilities.
– glS
Jul 7 at 13:59
• That's exactly the point I'm trying to make - there is more to the physical situation than can be captured by classical probabilities, which is why any theory assuming classical probabilities is missing something. Jul 7 at 14:09
• with that, I mostly agree
– glS
Jul 7 at 14:22

The question is a bit broad, but if I'm to think of a single feature that causes the divergence between CIT and QIT, I'd say the culprit is the (non)accessibility of the information in QM. More precisely, the fact that quantum states generally hold more information than what might appear from a given measurement setting.

A single qubit can produce (infinitely) many probability distributions, depending on how it's measured. Sure, regardless of how you measure it, you can only get at best one bit of entropy, but the state itself contains more information than that, because knowing the distribution some qubit $$|\psi\rangle$$ produces when measured in a basis doesn't (in general) tell you the distribution it will produce when measured in a different way.

A quantum state can be fully characterised, and still produce non-deterministic results depending on how it's measured. This is because in QM the act of measurement cannot really be considered as "passive", as just "looking" at the state of a physical system. Rather, quantum systems are (one might argue by definition) such that there is no way to measure a state without significantly changing it. This creates the odd situation of trying to characterise physical systems, not being able to observe them without changing them.

Imagine having a box that is extremely small and fragile. You open it to see what's inside and find some configuration of, say, coloured marbles. However, each time you close the box afterward, you can't help but shake it a bit and change the arrangement of marbles inside. If every time you look inside the box you change what's inside, how do you even define what "its content" is? I see the situation in QM as somewhat akin to this.

So, in QIT you still deal with probabilities and the whole apparatus of CIT, but you apply it to the probability distributions that result from measuring quantum states in various situations. For example, the "entropy of a quantum state" amounts to the entropy of the (classical) probability distribution obtained measuring the state in a certain way, but the same state can be measured in such a way to produce more entropic distributions. Or if you want to talk about mutual informations for quantum states, you will have to be mindful of the fact that correlations only straightforwardly make sense between distributions resulting from measuring the two pieces of a system in certain ways, and quantum systems can be correlated in ways that have no classical analogue (see also this answer about the QMI and this one about the quantum discord (which is a feature of the QMI)).

• – glS
Jul 16 at 10:25