I think there is no common agreement on what $U$ being fault tolerant exactly means. It was initially defined as "the quantum [system] can function successfully even if errors occur during the error correction".
To some people it will mean "$U$ does not spread errors within one code block" (see this answer for example quoting this paper).
A more or less equivalent formulation could be "A correctable amount of errors before is the operation is always mapped to a correctable amount of errors after the operation". The slight difference being that error spreadings can happen if they always cancel out (e.g. thanks to stabilizers). As an example, I would argue that stabilizer measurements can spread errors towards the code. Consequently, they are considered fault tolerant only if the measurement schedule is chosen to avoid the so-called hook errors.
An even broader definition could be "A correctable set of errors before the operation is mapped to a correctable set of errors after the operation". Indeed, error spreading induces correlation which could be used by a decoder to correct errors despite some of them having spread.
As the answer I quoted points out, fault tolerance is a property of an operation (usually defined over a class of codes with arbitrary large distance). I do not think the fault tolerant property of your $V$ would depend on whether it is applied on a $d=3$ or $d\geq 5$ code.
Your notion of "fault tolerant distance" want to grasp how badly an operation impact the code performance. I believe it is close to the minimal-weight error mechanism that induces a logical error in a circuit experiment i.e. the minimal-weight error in the circuit space-time decoding graph.
You can probably theysay that a 2-transversal operation is fault-tolerant if you show that you are able to decode any circuit using it without hindering (or at least without hindering too much) the overall distance of the computation.