A couple of points:
- The ground state is by definition the eigenvector associated with the minimum valued eigenvalue.
- Lets consider the Pauli Z matrix as you have. First,
\begin{align*}
Z = \begin{pmatrix}1 & 0\\ 0 & -1 \end{pmatrix}.
\end{align*}
As this matrix is diagonal, we can immediately see that the eigenvalues are the values on the main diagonal (so 1 and -1), and they are associated with the standard basis vectors $|0\rangle$ and $|1\rangle$ respectively. Thus, since the eigenvector with the lowest associated eigenvalue is the $|1\rangle$ state, the $|1\rangle$ state is the ground state.
Your confusion may have simply been with the definition of the ground state -- it is not always the $|0\rangle$ state, although for some matrices (such as the identity matrix), it can be.
A potentially elucidating example may be found in considering the Hadmard matrix,
\begin{align*}
H = \frac{1}{\sqrt{2}}
\begin{pmatrix}
1 & 1\\
1 & -1
\end{pmatrix}.
\end{align*}
The eigenvalues of this matrix are $1, -1$ with the associated eigenvectors $\frac{1}{\sqrt{2}}(|0\rangle + |1\rangle)$ and $\frac{1}{\sqrt{2}}(|0\rangle - |1\rangle)$. Thus, we know that the ground state of the Hadamard matrix is the $|-\rangle \equiv \frac{1}{\sqrt{2}}(|0\rangle - |1\rangle)$ state.
Edit: Upon reflection, I realize that you may be asking what the ground state of a qubit is. This question doesn't entirely make sense to me, as the state of a qubit is represented by a vector rather than a matrix (and thus, does not have eigenstates and eigenvalues on its own). If you could clarify your question, I would be happy to address it more directly.