The thing that you write inside a ket is just a label. It can actually correspond to anything that you want it to. By convention, labels 0, 1, 2, 3... correspond to orthogonal states. So, you should interpret this as whatever system you're using has enough capacity to have a set of 4 orthogonal states, i.e. the Hilbert space dimension is at least 4. You're probably used to just seeing $|0\rangle$ and $|1\rangle$ because most of the time in quantum information, we deal with a qubit, which has Hilbert space dimension 2. But that's an assumption of convenience. Physically, it's far more common to have multiple levels.