The first postulate of quantum mechanics that can be found in the M. Nielsen and I. Chuang textbook:
Postulate 1: Associated to any isolated physical system is a complex vector space with the inner product (that is, a Hilbert space) known as the state space of the system. The system is completely described by its state vector, which is a unit vector in the system’s state space.
So, If one has some state $|\psi\rangle = \alpha |0\rangle + \alpha |1\rangle = \begin{pmatrix} \alpha \\ \beta \end{pmatrix}$, it can be described as a vector in the Hilbert space with $\alpha$ and $\beta$ complex coordinates. Because of normalization $|\alpha|^2 + |\beta|^2 = 1$, the length of that vector is $1$. When we apply a unitary operator the length doesn't change (still $|\alpha'|^2 + |\beta'|^2 = 1$). So unitary operations are just rotations of that vector in the Hilbert space (they don't change the length of it). If we change the basis with a unitary operator then it can be described as a rotation of basis vectors in the Hilbert space (the basis vectors will not change their lengths, they will just point to the other directions).
As mentioned in the comments, not only this. Like for rotations in the 3D Euclidean space, the inner product of two vectors in the Hilbert space is also preserving under the unitary transformation (Definition 2. or 3. from wiki). For 3D Euclidean space rotations, this also implies that the angles between the vectors isare not changing. For basis vectors in the Hilbert space, this means that if they were orthogonal they will stay orthogonal after the unitary transformation.