If $2$ qubits together provide the states 00, 01, 11 or 10 simultaneously which represent $4$ probabilities in total, how many probabilities do $N$ qubits represent? Does the formula for this somehow express quadratic/exponential speed-up?
The speed up is not expressed by the exponentially many basis states that a quantum system can be in.
The speedup comes from being able to recombine the probability amplitudes associated to the these basis states so that a measurement can output the correct result of the computation with sufficiently high probability.
If the exponentially many basis states accessible alone would make up for the speedup then any algorithm would be speedup by a quantum computer. And that's not the case.
Sometimes, mainly in popular articles and books, it is stated that $n$ qubits can be in all $2^n$ possible zero-one combinations simultaneously (the superposition of these combinations) and any calculation on this register is done with all these combinations. This is used as an explanation for speed-up brings by a quantum computer. However, this is misunderstanding. If you want to use this explanation, you have to add that it is necessary to repeat an algorithm run many times and as a result you lose some of the speed-up. But this is still not precise explanation. You have to use laws of a quantum mechanics to explain this preciselly. Unfortunately there is no precise and at the same time "popular" explanation.
Additionlly, the reason behind the speed-up is different for different algorithms. The scale of the speed-up is also different. It is for example quadratic for Monte Carlo simulation or database search (Grover algorithm) and exponential for solving linear equations (HHL algorithm) or integer factorization (Shor algorithm). Moreover, there are algorithms bringing no speed-up in comparison with a classical computer (e.g. binary function parity evaluation).
Note: to be precise about HHL algorithm, it brings exponential speed-up only in case we are looking for value $x^TMx$ for some operator $M$ and $x$ is a solution of $Ax=b$.