This may be broader than what you're looking for, but it's clear from your question that you've read up on the QIT materials on the subject already. So I'll try to give a different perspective (more GR-ish) that I think is much more intuitive. The concepts are very portable, so hopefully it's helpful.
I usually think of adjoint operations in terms of pullbacks and their adjoint pushforwards. For a simple example, assume we have smooth maps $f: \mathcal{Y} \rightarrow \mathbb{R}$ and $g: \mathcal{X} \rightarrow \mathcal{Y}$, as shown below. In this case, the pullback of $f$ to $\mathcal{X}$ is simply the composition $\Phi = f \circ g$.

While it's straightforward to pull functions on $\mathcal{Y}$ back to $\mathcal{X}$, even if we had a function mapping $\mathcal{X} \rightarrow \mathbb{R}$ there would be no way to push that function forward to $\mathcal{Y}$. The maps we have available aren't sufficient to define that kind of transfer.
However we can define the pushforward of a vector from $\mathcal{X}$ to $\mathcal{Y}$, which is the adjoint to the pullback described above. This is possible because we can treat vectors as derivative operators that map functions to $\mathbb{R}$.
For a vector at point $x$ on $\mathcal{X}$, say $V(x)$, the pushforward vector $\Phi^\dagger V$ at point $g(x)$ on $\mathcal{Y}$ can be defined in terms of its action on functions of $\mathcal{Y}$:
$$(\Phi^\dagger V)(f) = V(\Phi f).$$
So the action of $\Phi^\dagger V$ on a function is the action of $V$ on the pullback of that function.

From a practical standpoint, we can take a basis for vectors on $\mathcal{X}$ as ${\partial {}_\mu} = {\partial }/{\partial x^\mu}$ and the same for $\mathcal{Y}$, ${\partial {}_\nu} = {\partial }/{\partial y^\nu}$. To relate $V = V^\mu \partial {}_\mu$ to $(\Phi^\dagger V)=(\Phi^\dagger V)^\nu \partial {}_\nu$ we only need the chain rule:
$$(\Phi^\dagger V)^\nu \partial {}_\nu f = V^\mu \partial {}_\mu(\Phi f) = V^\mu \partial {}_\mu(f \circ g) = V^\mu(\partial y^\nu / \partial x^\mu) \partial {}_\nu f.$$
This leads directly to the matrix $$(\Phi^\dagger)^\nu{}_\mu = \partial y^\nu / \partial x^\mu.$$
You can see after all this that the adjoint of the pullback, a vector pushforward, is essentially a generalization of a coordinate transformation.
This was a bit long winded, but still doesn't do the subject justice. If you think this approach to building intuition might be helpful, Sean Carroll has a phenomenal exposition on the subject in Appendix A, Maps between Manifolds, in Spacetime And Geometry.