Considering just the single qubit case, the four possible operators you list are
\begin{align}
|0\rangle\langle 0|,|0\rangle\langle 1|,|1\rangle\langle 0|,|1\rangle\langle 1|
\end{align}
and like you say, only the first and last are physical. Note, however, that
\begin{align}
|0\rangle\langle 1| = \frac{1}{2}(X + iY)\\
|1\rangle\langle 0| = \frac{1}{2}(X - iY)
\end{align}
and also that
\begin{align}
X &= |+\rangle\langle +| - |-\rangle\langle -|\\
Y &= |+i\rangle\langle +i| - |-i\rangle\langle -i|\\
\end{align}
so the two off-diagonal elements of the natural basis can be written entirely in terms of the states $|\pm\rangle$ and $|\pm i \rangle$.
I'll explain a bit more about state tomography, because process tomography follows readily from it.
We wish to reconstruct an unknown $\rho$ from some set of measurements. We start from the Born rule that says the probability of getting outcome $i$ upon measurement
\begin{align}
p_i = Tr(M_i \rho)
\end{align}
Mathematically, $M_i$ and $\rho$ are both Hermitian, positive semidefinite matrices, and while we assume to know $M_i$, we do not know $\rho$. If we choose to measure in the $Z$ basis, then we have chosen the measurement set ("positive operator-valued measure")
\begin{align}
\{ M_0 = |0\rangle\langle 0|, M_1 = |1\rangle\langle 1| \}
\end{align}
and in retrospect when we obtain outcome 0/1, we say we measured with $M_{0/1}$. Consider what $p_0$ actually is symbolically
\begin{align}
p_0 &= Tr(M_0 \rho) \\
&= Tr(|0\rangle\langle 0| \rho)\\
&= Tr(\langle 0 | \rho | 0 \rangle)\\
&= \langle 0 | \rho | 0 \rangle\\
&= \begin{bmatrix} 1 & 0 \end{bmatrix}\begin{bmatrix} \rho_{00} & \rho_{01}\\\rho_{10} & \rho_{11}\end{bmatrix}\begin{bmatrix} 1 \\ 0 \end{bmatrix}\\
&= \rho_{00}
\end{align}
So if I measure in the $Z$ basis many times, I can estimate $p_0$, which is actually element $(0,0)$ of $\rho$. Since 0 and 1 are mutually exclusive, whenever I don't get 0, I must get 1, and by similar logic can estimate $\rho_{11}$. The diagonals are easily obtained in the $Z$ basis, but we will need to measure in the $X$ and $Y$ bases to get the off-diagonals, as shown above. When measuring in the $X$ basis, the POVM then is
\begin{align}
\{ M_+ = |+\rangle\langle +|, M_- = |-\rangle\langle -| \}
\end{align}
I'll just tabulate what quantities you can estimate given these new outcomes/bases
\begin{align}
p_+ = \frac{1}{2}(\rho_{00}+\rho_{01}+\rho_{10}+\rho_{11})\\
p_i = \frac{1}{2}(\rho_{00}-\rho_{01}-\rho_{10}+\rho_{11})\\
p_{+i} = \frac{1}{2}(\rho_{00}+i\rho_{01}-i\rho_{10}+\rho_{11})\\
p_{-i} = \frac{1}{2}(\rho_{00}-i\rho_{01}+i\rho_{10}+\rho_{11})\\
\end{align}
With (complex) linear combinations of the above quantities, we can fix the off-diagonals. Thus, measuring in the $X,Y,Z$ bases is enough to constrain $\rho$. Process tomography for some process $\mathcal{E}$ is the same thing, but the Born rule becomes
\begin{align}
p_{ij} = Tr(M_i \mathcal{E}(\rho_j))
\end{align}
and we can play the same game. For example, preparing the state $|0\rangle\langle 0|$ and obtaining outcome $|1\rangle\langle 1|$ fixes a particular element of the process matrix. To keep this answer "short", check out section II.B of this paper (which uses a convenient vectorized notation) for a concise summary of state, measurement, and process tomography.