My question is highly related to this topic
It is about defining logical operation on a Stabilizer code.
I call $S$ the stabilizer group of a code space $C$, and I assumed it is generated by a family $S=\langle s_1,...,s_p \rangle$. I call $G_n$ the $n$-Pauli matrix group ($n$ being the dimension of the full Hilbert space).
A definition of logical operation is as follow:
$U_L$ is a logical operation if $\forall |\psi \rangle \in C$, $U_L | \psi \rangle \in C$
And, we realize that if $|\psi \rangle$ is stabilized by $g$, $U_L |\psi \rangle$ will be stabilized by $U_L g U_L^{\dagger}$.
Questions: which condition to ensure $U_L$ is a logical operation
A sufficient condition is to have $U_L S U_L^{\dagger} = S$, which means that $U_L \in N(S)$ (where $N(S)$ is the normalizer of $S$).
Indeed, this way we would be certain that $U_L |\psi\rangle$ will be stabilized by $S$ and thus be in the codespace.
What disturbs me is that according to the comments here (and some of the sources attached), the logical operation are actually exactly elements of $N(S)$. I see the sufficient condition but not the necessary one.
For instance, if $U_L$ is non clifford, for $s \in S$, $U_L s U_L^{\dagger}$ might not even be an n-Pauli matrix, thus $U_L S U_L^{\dagger} \neq S$ as $S \subset G_n$. In this case obviously $U_L$ wouldn't be in the normalizer of $S$. But wouldn't it be possible to have a non n-Pauli matrix that still stabilize appropriately $C$ ?
So my question is: Why is it sufficient and necessary to have $U_L \in N(S)$ so that $U_L$ is a logical operation ?