As shown e.g. in Watrous' book (Proposition 6.6, page 314), a separable state $\rho$ can always be written as a convex combination of at most $\mathrm{rank}(\rho)^2$ pure, separable states.
More precisely, using the notation in the book, any separable state $\xi\in \mathbb{C}^d\otimes\mathbb{C}^{d'}$ can be decomposed as $$\xi = \sum_{a=1}^m p(a) \, x_a x_a^*\otimes y_a y_a^*,\tag1$$ for some probability distribution $p$, sets of pure states $\{x_a\}_a\subset\mathbb{C}^d$ and $\{y_a\}_a\subset\mathbb{C}^{d'}$, and $$\operatorname{rank}(\xi) \le m\le \mathrm{rank}(\xi)^2.$$ The lower bound is trivial, while the upper bound is shown observing that $\xi$ is an element of the real affine space of hermitian operators $H\in\mathrm{Herm}(\mathcal X\otimes\mathcal Y)$ such that $\mathrm{im}(H)\subseteq\mathrm{im}(\xi)$ and $\mathrm{Tr}(H)=1$. This space has dimension $\mathrm{rank}(\xi)^2-1$, and thus from Carathéodory we get the conclusion.
Consider for example the case of a totally mixed state: $$\xi\equiv \frac{1}{dd'}I = \frac{I}{d}\otimes\frac{I}{d'}.$$ In this case $\mathrm{rank}(\xi)=dd'=m$, with the standard choice of separable pure decomposition. In the case $m=1$, it is trivial to see that we must also always have $m=\mathrm{rank}(\rho)$.
What are examples in which this is not the case? More precisely, what are examples of $\xi$ for which there is no separable pure decomposition with $m\le\mathrm{rank}(\xi)$?
A related question on physics.SE is What is the minimum number of separable pure states needed to decompose arbitrary separable states?.