I had asked this question earlier in the comment section of the post: What is a qubit? but none of the answers there seem to address it at a satisfactory level.
The question basically is:
How is a single qubit in a Bell state $\frac{1}{\sqrt{2}}(|0\rangle + |1\rangle)$ any different from a classical coin spinning in the air (on being tossed)?
The one-word answer for difference between a system of 2 qubits and a system of 2 classical coins is "entanglement". For instance, you cannot have a system of two coins in the state $\frac{1}{\sqrt 2}|00\rangle+\frac{1}{\sqrt 2}|11\rangle$. The reason is simple: when two "fair" coins are spinning in air, there is always some finite probability that the first coin lands heads-up while the second coin lands tails-up, and also the vice versa is true. In the combined Bell state $\frac{1}{\sqrt 2}|00\rangle+\frac{1}{\sqrt 2}|11\rangle$ that is not possible. If the first qubit turns out to be $|0\rangle$, the second qubit will necessarily be $|1\rangle$. Similarly, if the first qubit turns out to be $|1\rangle$, the second qubit will necessarily turn out to be $|1\rangle$. At, this point someone might point out that if we use $2$ "biased" coins then it might be possible to recreate the combined Bell state. The answer is still no (it's possible to mathematically prove it...try it yourself!). That's because the Bell state cannot be decomposed into a tensor product of two individual qubit states i.e. the two qubits are entangled.
While the reasoning for the 2-qubit case is understandable from there, I'm not sure what fundamental reason distinguishes a single qubit from a single "fair" coin spinning in the air.
This answer by @Jay Gambetta somewhat gets at it (but is still not satisfactory):
This is a good question and in my view gets at the heart of a qubit. Like the comment by @blue, it's not that it can be an equal superposition as this is the same as a classical probability distribution. It is that it can have negative signs.
Take this example. Imagine you have a bit in the $0$ state and then you apply a coin flipping operation by some stochastic matrix $\begin{bmatrix}0.5 & 0.5 \\0.5 & 0.5 \end{bmatrix}$ this will make a classical mixture. If you apply this twice it will still be a classical mixture.
Now lets got to the quantum case and start with a qubit in the $0$ state and apply a coin flipping operation by some unitary matrix $\begin{bmatrix}\sqrt{0.5} & \sqrt{0.5} \\\sqrt{0.5} & -\sqrt{0.5} \end{bmatrix}$. This makes an equal superposition and you get random outcomes like above. Now applying this twice you get back the state you started with. The negative sign cancels due to interference which cannot be explained by probability theory.
Extending this to n qubits gives you a theory that has an exponential that we can't find efficient ways to simulate.
This is not just my view. I have seen it shown in talks of Scott Aaronson and I think its best to say quantum is like “Probability theory with Minus Signs” (this is a quote I seen Scott make).
I'm not exactly sure how they're getting the unitary matrix $\begin{bmatrix}\sqrt{0.5} & \sqrt{0.5} \\\sqrt{0.5} & -\sqrt{0.5} \end{bmatrix}$ and what the motivation behind that is. Also, they say: "The negative sign cancels due to interference which can not be explained by probability theory." The way they've used the word interference seems very vague to me. It would be useful if someone can elaborate on the logic used in that answer and explain what they actually mean by interference and why exactly it cannot be explained by classical probability. Is it some extension of Bell's inequality for 1-qubit systems (doesn't seem so based on my conversations with the folks in the main chat though)?