The fidelity between a pure state $|\psi\rangle$ and an arbitrary mixed state $\rho$ is given by, $F(|\psi\rangle,\rho)=\sqrt{\langle\psi|\rho|\psi\rangle}$, which is stated to be equal to the square root of the overlap between $|\psi\rangle$ and $\rho$.
In what sense the term ${\langle\psi|\rho|\psi\rangle}$ is the overlap between $|\psi\rangle$ and $\rho$ ?
The fidelity between $\rho$ and $\sigma$ is given by $F(\rho,\sigma)=tr\sqrt{\rho^{1/2}\sigma\rho^{1/2}}$
Therefore, the fidelity between the pure states $|\psi\rangle$ and $|\phi\rangle$ is,
$F(|\psi\rangle,|\phi\rangle)=tr\sqrt{|\psi\rangle\langle\psi|\phi\rangle\langle\phi|\psi\rangle\langle\psi|}=tr\sqrt{\langle\psi|\phi\rangle\langle\phi|\psi\rangle|\psi\rangle\langle\psi|}=tr\sqrt{|\langle\psi|\phi\rangle|^2|\psi\rangle\langle\psi|}=\sqrt{|\langle\psi|\phi\rangle|^2}tr\sqrt{|\psi\rangle\langle\psi|}=|\langle\psi|\phi\rangle|$
The quantity $|\langle\psi|\phi\rangle|^2$ is the overlap between the pure states $|\psi\rangle$ and $|\phi\rangle$, in the sense that $|\langle\psi|\phi\rangle|$ is the component of $|\psi\rangle$ in the state $|\phi\rangle$.
The fidelity between the pure state $|\psi\rangle$ and the mixed state $\rho$ is,
$F(\psi,\rho)=tr\sqrt{|\psi\rangle\langle\psi|\rho|\psi\rangle\langle\psi|}=tr\sqrt{\langle\psi|\rho|\psi\rangle|\psi\rangle\langle\psi|}=\sqrt{\langle\psi|\rho|\psi\rangle}tr(|\psi\rangle\langle\psi|)=\sqrt{\langle\psi|\rho|\psi\rangle}$