Let $\Phi$ be a channel and denote with $\newcommand{\ketbra}[1]{\lvert #1\rangle\!\langle #1\rvert}J(\Phi)\equiv (\Phi\otimes I)\mathbb P_m$ its Choi representation, where $\mathbb P_m\equiv\lvert m\rangle\!\langle m\rvert$ and $\lvert m\rangle\equiv\sum_i \lvert ii\rangle$.
Given arbitrary operators $A_a$, we have a representation $\Phi(X)=\sum_a A_a X A_a^\dagger$ if and only if $J(\Phi)=\sum_a \mathrm{vec}(A_a)\mathrm{vec}(A_a)^\dagger$, where $A_a\in\mathrm{Lin}(\mathcal X,\mathcal Y)$ and $\mathrm{vec}(A_a)\in\mathcal Y\otimes\mathcal X$ are related by $\mathrm{vec}(A_a)_{ij}=(A_a)_{ij}$ (to relate with your notation, $\mathrm{vec}(A_a)\equiv\lvert A_a\rangle\!\rangle$ and $\mathrm{vec}(A_a)^\dagger\equiv\langle\!\langle A_a\rvert$).
This follows from simple index juggling. Or from
$$(\Phi\otimes I)\mathbb P_m = \sum_{aij} A_a|i\rangle\!\langle j| A_a^\dagger\otimes |i\rangle\!\langle j| = \sum_a \mathrm{vec}(A_a)\mathrm{vec}(A_a)^\dagger.$$
Now the question is, suppose $\Phi(X)=\sum_a B_a X B_a^\dagger$. Are $A_a$ and $B_a$ unitarily related?
Given an operator $H$ which decomposes as
$$H=\sum_a |u_a\rangle\!\langle u_a| = \sum_a |v_a\rangle\!\langle v_a|,\tag1$$
we must have $|u_i\rangle = \sum_j c_{ij} |v_j\rangle$ for some coefficients $c_{ij}$, as otherwise the two expressions would result in operators with different ranges/supports. The coefficients must then satisfy
$$\sum_{ijk}c_{ij}\bar c_{ik} |v_j\rangle\!\langle v_k| = \sum_i |v_i\rangle\!\langle v_i|.$$
If the $|v_a\rangle$ are orthogonal, then multiplying the expression by $\langle v_a|$ and $|v_b\rangle$ we get $\sum_i c_{ia}\bar c_{ib} = \delta_{ab}$ for all $a$. This is equivalent to $C^\dagger C=I$, which means that $C$ is an isometry (has orthonormal columns). Note that there might be different numbers of elements in the two decompositions of (1), meaning $C$ needs not be a squared matrix, and thus not necessarily a unitary operator.
Now, any positive semidefinite operator $H$ has such a decomposition in terms of orthogonal vectors (its eigendecomposition). Therefore if there is any pair of sets of vectors such that (1) is satisfied, both of these will have to be unitarily related to the vectors in the eigendecomposition of $H$, and thus unitarily related between themselves.