First a basic observation: if all Hermitian preserving $\mathcal L$ gave rise to completely positive dynamics $e^{t\mathcal L}$ for all $t\geq 0$, then so would $-\mathcal L$ (still Hermitian preserving!) meaning $e^{t\mathcal L}$ is completely positive for all $t\leq 0$, as well. But then $e^{t\mathcal L}$ had completely positive inverse $e^{-t\mathcal L}$ for all $t\geq 0$ meaning $e^{t\mathcal L}$ would have to have Kraus rank 1 for all times, thus restricting $e^{t\mathcal L}$ to a strict subset of all completely positive maps. In other words this basic argument shows that there has to be an extra condition which characterizes complete positivity of $e^{t\mathcal L}$.
Indeed the following result is well known:
Given any linear map $\mathcal L:\mathbb C^{n\times n}\to\mathbb C^{n\times n}$, the following statements hold.
- $e^{t\mathcal L}$ is Hermitian preserving for all $t$ if and only if $\mathcal L$ is Hermitian preserving.
- If $\mathcal L$ is Hermitian preserving, then the following are equivalent:
- $\mathcal L$ is conditionally completely positive, that is, $$({\bf1}-|\Omega\rangle\langle\Omega|)\mathsf C(\mathcal L)({\bf1}-|\Omega\rangle\langle\Omega|)\geq 0$$ where $|\Omega\rangle:=\frac1{\sqrt n}\sum_j|j\rangle\otimes|j\rangle$ is the maximally entangled state and $\mathsf C(\mathcal L)=({\rm id}\otimes\mathcal L)(|\Omega\rangle\langle\Omega|)$ is the Choi state corresponding to $\mathcal L$
- There exist $\Phi$ completely positive and $K\in\mathbb C^{n\times n}$ such that $\mathcal L=\Phi+K(\cdot)+(\cdot)K^\dagger$
- $e^{t\mathcal L}$ is completely positive for all $t$
Proof: 1. It is well known that arbitrary $A\in\mathbb C^{n\times n}$ is Hermitian if and only if ${\rm tr}(BA)\in\mathbb R$ for all $B$ Hermitian (choose $B=|x\rangle\langle x|$). Thus a linear map $\Phi$ is Hermitian preserving if and only if ${\rm tr}(B\Phi(A))\in\mathbb R$ for all $A,B$ Hermitian. This readily implies the desired statement: If $\mathcal L$ is Hermitian preserving, then
$$
{\rm tr}(B e^{t\mathcal L}(A))=\sum_j\frac{t^j}{j!}{\rm tr}(B\mathcal LL^j(A))\in\mathbb R
$$
for all $t\in\mathbb R$ and all $A,B$ Hermitian, and if—conversely—$e^{t\mathcal L}$ is Hermitian preserving for all $t$, then
$$
{\rm tr}(B\mathcal L(A))=\frac{d}{dt}\underbrace{{\rm tr}(B e^{t\mathcal L}(A))}_{\in\mathbb R}\Big|_{t=0}\in\mathbb R
$$
as desired. 2. First, as proven in the phys.SE post linked above conditionally completely positive implies the form $\mathcal L=\Phi+K(\cdot)+(\cdot)K^\dagger$. Next, we want to see that such $\mathcal L$ gives rise to completely positive dynamics: $e^\Phi=\sum_j\frac{\Phi^j}{j!}$ is completely positive (sums, products, and limits of CP maps are CP) and from the exponential formula $e^{A(\cdot)+(\cdot)B}=e^A(\cdot)e^B$ it follows that $e^{K(\cdot)+(\cdot)K^\dagger}=e^K(\cdot)(e^K)^\dagger$. Together with the Trotter product formula this implies
$$
e^{t\mathcal L}=\lim_{n\to\infty}\big( e^{t\Phi/n}e^{t(K(\cdot)+(\cdot)K^\dagger)/n} \big)^n=\lim_{n\to\infty}\big( e^{t\Phi/n}\circ \big(e^{tK/n}(\cdot)(e^{tK/n})^\dagger\big) \big)^n
$$
meaning—as before—$e^{t\mathcal L}$ is completely positive as a limit of (products of) completely positive maps.
Finally, if $e^{t\mathcal L}$ is completely positive, i.e. $\mathsf C(e^{t\mathcal L})\geq 0$ for all $t$, then by linearity of the Choi formalism
\begin{align*}
({\bf1}-|\Omega\rangle\langle\Omega|)\mathsf C(\mathcal L)({\bf1}-|\Omega\rangle\langle\Omega|)&=
\frac{d}{dt}
({\bf1}-|\Omega\rangle\langle\Omega|)\mathsf C(e^{t\mathcal L})({\bf1}-|\Omega\rangle\langle\Omega|)\Big|_{t=0}\\
&=\lim_{t\to 0^+}({\bf1}-|\Omega\rangle\langle\Omega|)\mathsf C\Big(\frac{e^{t\mathcal L}-{\rm id}}{t}\Big)({\bf1}-|\Omega\rangle\langle\Omega|)\\
&=\lim_{t\to 0^+}\frac1t\Big(({\bf1}-|\Omega\rangle\langle\Omega|)\mathsf C(e^{t\mathcal L})({\bf1}-|\Omega\rangle\langle\Omega|)\\
&\qquad\qquad\qquad-({\bf1}-|\Omega\rangle\langle\Omega|)\mathsf C({\rm id})({\bf1}-|\Omega\rangle\langle\Omega|)\Big)\,.
\end{align*}
But $\mathsf C({\rm id})=|\Omega\rangle\langle\Omega|$ so the second term vanishes. This leaves
\begin{align*}
({\bf1}-|\Omega\rangle\langle\Omega|)\mathsf C(\mathcal L)({\bf1}-|\Omega\rangle\langle\Omega|)&=\lim_{t\to 0^+}\frac1t({\bf1}-|\Omega\rangle\langle\Omega|)\mathsf C(e^{t\mathcal L})({\bf1}-|\Omega\rangle\langle\Omega|)
\end{align*}
which is $\geq 0$ because all $\mathsf C(e^{t\mathcal L})$ are, so the same is true for $({\bf1}-|\Omega\rangle\langle\Omega|)\mathsf C(e^{t\mathcal L})({\bf1}-|\Omega\rangle\langle\Omega|)$ and for its (scaled) limit. $\square$
