Qubits are the quantum states that carry the smallest amount of "quantum information": the simplest possible quantum states you can imagine. They are pervasive for this reason, just like bits in classical physics are pervasive because they are the basic unit of information. We use bits all the times simply because we find it convenient to build things (e.g. coputers) in such a way that we can model their behaviour in terms of bits. Underlying the simplified models, physical reality is much more complex than that though: inside your computers that are transistors carrying "continuous" amounts of currents and voltages, and describing the whole thing in terms of bits is only an extremely useful way to simplify how the whole thing works.
One caveat is that there is a sense in which "qubits" are more natural in quantum mechanics than "bits" are in classical physics. Indeed, elementary particles often have intrinsic degrees of freedom that seem to be genuine two-dimensional systems (qubits). There isn't anything like this in classical physics that I know of. But then again, one could argue that even talking about elementary particles in quantum mechanics is only a "simplified model useful for descriptions".