Firstly, the sandwiched divergence can be infinite even when $\rho$ and $\sigma$ are not orthogonal. For example, consider $\rho = \frac{|0\rangle \langle 0| + |1\rangle\langle 1|}{2}$ and $\sigma = |0\rangle \langle 0|$. We can think of this as the limiting case of $\sigma_\epsilon = (1-\epsilon) |0\rangle \langle 0| + \epsilon |1 \rangle \langle 1|$. Computing the divergence with $\sigma_\epsilon$ we don't have any issues because everything is full rank and we find
$$
D_\alpha(\rho\|\sigma_\epsilon) = \frac{1}{\alpha-1}\log \left( 2^{-\alpha}\left((1-\epsilon)^{1-\alpha} + \epsilon^{1-\alpha}\right) \right)
$$
which tends to $+\infty$ as $\epsilon \to 0$ when $\alpha > 1$.
The correct condition for finiteness is that for every vector $|v\rangle$ such that $\sigma |v\rangle = 0$ we must also have $\rho |v \rangle =0$. In other words the kernel of $\sigma$ is contained in the kernel of $\rho$. So for example if we were to swap the roles of $\rho$ and $\sigma$ in the previous example, so $\sigma = \frac{|0\rangle \langle 0| + |1\rangle\langle 1|}{2}$ and $\rho = |0\rangle \langle 0|$, then this satisfies our condition for finiteness. Moreover we can see it as a limiting case of $\rho_\epsilon = (1-\epsilon) |0\rangle \langle 0| + \epsilon |1 \rangle \langle 1|$ which gives
$$
D_\alpha(\rho_\epsilon\|\sigma) = \frac{1}{\alpha-1}\log \left( 2^{\alpha-1}\left((1-\epsilon)^{\alpha} + \epsilon^{\alpha}\right) \right)
$$
which tends to $1$ as $\epsilon \to 0$ whenever $\alpha > 1$ which also agrees with a direct computation of $D_\alpha(\rho\|\sigma)$.
To answer your question, yes you can take the pseudoinverse which for Hermitian operators can be computed readily via the spectral decomposition. If $\sigma = \sum_i \lambda_i |v_i\rangle \langle v_i|$ is the spectral decomposition of $\sigma$ then $\sigma^{-1} = \sum_i \lambda_i^{-1} |v_i\rangle \langle v_i|$.
