Suppose we are given two unitary matrices $U$ and $V$, with the following guarantee,
$$||U - V||_1 \geqslant \delta$$ for some $\delta \geqslant 0$.
We apply an input density state $\rho$ individually to both $U$ and $V$, resulting in the output state $U\rho U^{\dagger}$ and $V\rho V^{\dagger}$ respectively. Then can we say anything about the trace norm of the output states, i.e,
$$||U\rho U^{\dagger} - V\rho V^{\dagger}||_1 \geqslant \hspace{1mm} ?$$
For the way round that you've got your inequalities, I don't think there's much that can be said. To see why, let's consider the first expression $$ \|U-V\|_1=\text{Tr}(\sqrt{2I-VU^\dagger-UV^\dagger}). $$ Now, $VU^\dagger$ is a unitary, and hence as a spectral decomposition. Let the eigenvectors be $|\lambda_i\rangle$ with eigenvalues $e^{i\lambda_i}$. $UV^\dagger$ has the same eigenvectors, with eigenvalues $e^{-i\lambda_i}$. Hence, the expression simplifies to $$ \|U-V\|_1=2\sum_i\left|\sin\frac{\lambda_i}{2}\right|\geq\delta. $$ Note that this doesn't tell us anything about an individual $\lambda_i$ (whereas if the inequality were the other way around, we'd know that $\sin\frac{\lambda_i}{2}\leq\delta$ for all $i$). In particular, there could be an $i$ such that $\lambda_i=0$. Let's call this particular vector $|\Lambda\rangle$.
Next, consider the final calculation you want $$ \|U\rho U^\dagger-V\rho V^\dagger \|=\|\rho-U^\dagger V\rho V^\dagger U\|. $$ It should now be clear that if $\rho=|\Lambda\rangle\langle\Lambda |$, this value is 0. Actually, it is for any eigenvector, or mixture of eigenvectors, of $VU^\dagger$. Since 0 is always the minimum value, there's clearly no non-trivial lower bound.
Having started to think about the follow-up comment, there's an even stronger example of why you can't get anything useful. Let $V=e^{i\theta} U$. Obviously, $\|U\rho U^\dagger-V\rho V^\dagger \|=0$, while $\|U-V\|_1=2N\left|\sin\frac{\theta}{2}\right|$ for $N\times N$ matrices.
