When it comes to error correction, we take our stabilizers to be members of the Pauli group. Why is the Pauli group used for this and not, say, the group of all unitary matrices?
2 Answers
There are some fairly simple reasons — beyond the merely historical — to use Pauli matrices instead of arbitrary unitary matrices. These reasons may not uniquely single out the Pauli group of operators, but they do significantly limit the scope of what is productive to consider.
A stabiliser operator $S$, first and foremost, must have a +1 eigenvalue; otherwise there isn't any state $\lvert \psi \rangle$ which it 'stabilises', in the sense that $S \lvert \psi \rangle = \lvert \psi \rangle$. So we must restrict ourselves to sets of operators which have +1 eigenvalues.
Secondly, we must consider how the stabiliser operators may be used operationally. If we know that there is a symmetry of the system that should hold, but we don't have any way to determine whether or not that symmetry holds in practise (that is, whether some error has occurred), then we're out of luck. What we would like to be able to do then is to be able to perform phase estimation to test whether or not the eigenvalue of a given state $\lvert \psi \rangle$ with respect to some allegedly-stabilising operator $S$ is in fact +1, to determine whether $\lvert \psi \rangle$ deviates from the properties that hold of it.
This motivates considering operators $S$ which, yes, are unitary, but also where the eigenvalues differ significantly from one another, in order for phase estimation to easily distinguish a state with significant error from one with insignificant error. This motivates considering a set of $n$-qubit operators which have at most $1/\mathrm{poly}(n)$ eigenvalues.
Part of the whole problem is that we would like to detect and correct for operations which may be involved in complicated quantum transformations. If the phase estimation involved in eigenvalue estimation of a stabilising operator $S$ is itself complicated, we're not helping the situation.
What would be good is for each of the stabilising operators $S$ we consider to have very simple structure: for instance, we may be especially interested in the case that they are tensor products of 1- or 2-qubit operations. It seems sensible to approach the subject by considering each operator $S$ to be a tensor product of single qubit operations.
In order to consider tensor product operations on $k \leqslant n$ qubits, which have at most $1/\mathrm{poly}(k)$ distinct eigenvalues, including +1 — and without imposing awkward constraints on which single-qubit operators act on which qubits — we are more or less forces to consider single-qubit unitary operators whose eigenvalues range within some finite set $E \subseteq \mathbb C$ (independent of $k$ or $n$) which includes +1.
We may reduce this to the case $E = \{+1,-1\}$ by observing that estimating the eigenvalues of a tensor product operator $S = S_1 \otimes S_2 \otimes \cdots \otimes S_k$, where each $S_j$ has one +1 eigenvalue and one eigenvalue which is not +1, is the same as doing an artificially shortened version of eigenvalue estimation for an operator $P_j$ which has eigenvalues $\pm 1$. Furthermore, in order to consider several operators $S$ which manage to have a useful common +1 eigenspace, it helps for each operator S to have as large a +1 eigenspace as possible; then it helps for it to be as easy as possible for the eigenvalues of each $S_j$ to multiply to +1. This again motivates the case for the eigenvalues of $S_j$ to be $\pm 1$.
Nothing forces us to consider the group of operators generated by such a set, but the products of our stabiliser operators will also be stabiliser operators, and we have enough constraints on our operators that we can at least reasonably contemplate the group generated by our stabiliser operators.
We have operators $S = S_1 \otimes \cdots \otimes S_n$ and $S' = S'_1 \otimes \cdots \otimes S'_n$ whose tensor factors are all either $\mathbb 1$ or non-trivial reflections on single-qubit states; their products $S_j S'_j$ will be rotations by an angle $\theta$ determined by the angles between the eigenbases of $S_j$ and $S'_j$. If we want to obtain a nice clean theory, we might want these products of stabiliser operators to themselves be easy to measure: this motivates having $S_j S'_j$ to be proportional to an operator with eigenvalues $\pm 1$ (actually $S_j S'_j$ will have eigenvalues $\pm i$), in which case $S_j$ and $S'_j$ anticommute.
Thus, the above combination of theoretical and practical constraints suffice to yield something which is isomorphic to the Pauli group. Furthermore, as the Pauli operators have a theory which is fairly easily understood, it has led to a fruitful theory of quantum error correction.
A fair question would be which of the above moves were more arbitrary than the others.
It would not astonish me if there was a productive theory of error correction in which the constraints were tensor product operators, whose tensor factors had eigenvalues $\pm 1$, but where the possible operators did not necessarily anticommute (step 5 above).
More sophisticated (and more difficult) would be a powerful and useful theory of error correction in which the stabilising operators which one measures included operators which are not tensor product operators (step 3 — which would motivate not worrying too much about having group structure in the group of stabilisers which you intend to measure).
From a purely mathematical perspective, there is nothing obvious to prevent or discourage such a line of investigation — aside, of course from the fact that it is likely to be hard and also likely to be unnecessary — and in this sense, it would be perfectly fine to consider theories of quantum error correction extending well beyond the Pauli group.
Any operator from the Pauli group has two eigenspaces of equal size. So we known that by adding stabilizer generator from this group, we reduce the size of the stabilizer space by half. This means that the stabilizer space would fit one less logical qubit. This makes it easy to know when we have enough stabilizers: to store $k$ logical qubits in $n$ physical qubits, we just need $n-k$ independent stabilizer generators.
Also, the Pauli group is made up of Hermitian operators. Since the point of a stabilizer is to be measured, it is useful for them to be Hermitian, since they can be directly interpreted as observables.
Furthermore, the the operators that map between stabilizer states (mutual eigenstates of stabilizer operators) will themselves be elements of the Pauli group. This is related to the point raised in your comment: Pauli group elements form a complete basis to describe multi-qubit operation. So once we measure the stabilizers, and the noise is effectively reduced to a mapping between stabilizer states, it is pretty much as if the noise just applied a bunch of simple Paulis. Correction can be then done by a simple Pauli frame rotation. This doesn't even require us to directly apply any gate to the code. We can just say "It looks like a $\sigma_x$ hit this qubit, so from now on I'll interpret its $|0\rangle$ as $|1\rangle$, and vice-versa".
Paulis aren't required, but they have nice properties. So that's why they are the focus