Much of the magic of the discrete Fourier transform, be it classical or quantum, lies in the fact that a large $N\times N$ matrix can be factored into a tensor product of $\mathcal{O}(\log N)$ smaller matrices. This much was worked out by Cooley and Tukey back in the 60's, with antecedents even earlier going back to Gauss.
Perhaps it's best to think of the QFT as a tensor product of $n$ single-qubit ($d=2$) gates - thus the size of the matrix is $2^n\times 2^n$. The radix of $2$ is convenient and not required. For example one can also think of a tensor product of $n$ qutrit ($d=3$) gates, in which case the matrix is $3^n\times 3^n$, or any other qudit gates. Indeed, Shor's original algorithm used a mixed-radix Fourier transform - this was quickly improved to the familiar form we see today.
But perhaps you're asking whether the input vector itself, $|\psi\rangle$, needs to be a power of two. Thinking about the (classical) Fast Fourier Transform of a dataset that is not a power of two, this is addressed with "zero-padding" the input data set to the next nearest power of two. This link may be helpful. Thus you can always zero-pad your vector $|\psi\rangle$ to be a power of two. But, the computational costs associated with this padding may not be trivial.