Mathematically when is $\sum_i p_i S(\rho_i) > 0$?
I am assuming that $\{p_i\}$ form a probability distribution (and that none of the $p_i = 0$) and each $\rho_i$ is a normalised state.
As $p_i \geq 0$ and $S(\rho_i) \geq 0$ we have $\sum_i p_i S(\rho_i) = 0 \iff S(\rho_i) = 0$ for all $i$. Then we can ask the question under what circumstances do we have that each $S(\rho_i) = 0$. But a state $\rho_i$ has $S(\rho_i) = 0 \iff \rho_i$ is a pure state.
To see this let $\{\lambda_x\}_x$ be the eigenvalues of $\rho_i$. We can then compute $S(\rho_i) = - \sum_x \lambda_x \log \lambda_x$. Now as $\rho$ is a positive semi-definite matrix we have for each $x$, $\lambda_x \geq 0$ and as $\rho$ is normalized we have $ \sum_x \lambda_x = 1$. Putting these two constraints together we must have that for each $x$, $0 \leq \lambda_x \leq 1$. This means that for each term in the sum we have $- \lambda_x \log \lambda_x \geq 0$. So $-\sum_x \lambda_x \log \lambda_x = 0 \iff -\lambda_x \log \lambda_x = 0$ for all $x$. But $- \lambda_x \log \lambda_x = 0 \iff \lambda_x \in \{0,1\}$. Combining this with the fact that we need $\sum_x \lambda_x = 1$ we must have exactly one $x$ for which $\lambda_x = 1$ and the rest must vanish. Finally, if $\rho_i$ is a state with a single nonzero eigenvalue then it is a pure state. Hence $S(\rho_i) = 0 \implies \rho_i $ is pure. The other direction follows readily.
tl;dr $\sum_i p_i S(\rho_i) > 0 \iff \exists i$ such that $p_i>0$ and $\rho_i$ is not pure.
An example
Let $\rho_{AB} = |\psi \rangle \langle \psi |$ where $\psi = \tfrac{1}{\sqrt{2}}(|00\rangle + |11 \rangle)$. Suppose we measure on the first system the POVM $\{M, \mathbb{I} - M\}$ where $M = \frac{\mathbb{I} + \gamma \sigma_z}{2}$, $\gamma \in [0,1]$ is some parameter and $\sigma_z$ is the Pauli z operator. The $\gamma$ parameter is sometimes referred to as the strength/sharpness of the measurement. When $\gamma = 1$ the measurement is projective and when $\gamma = 0$ the measurement is trivial (doesn't interact with the system). Labelling the measurement outcomes $0,1$ respectively we get outcome $0$ with probability
$$
p_0 = \mathrm{Tr}[(M^{1/2} \otimes \mathbb{I}) \rho_{AB} (M^{1/2} \otimes \mathbb{I})]
$$
and
$$
p_1 = \mathrm{Tr}[((\mathbb{I}-M)^{1/2} \otimes \mathbb{I}) \rho_{AB} ((\mathbb{I}-M)^{1/2} \otimes \mathbb{I})].
$$
Similarly the normailised state on system B after receiving outcome $0$ is
$$
\rho_{B}(0) = \frac{\mathrm{Tr}_A[(M^{1/2} \otimes \mathbb{I}) \rho_{AB} (M^{1/2} \otimes \mathbb{I})]}{\mathrm{Tr}[(M^{1/2} \otimes \mathbb{I}) \rho_{AB} (M^{1/2} \otimes \mathbb{I})]}
$$
and on outcome $1$,
$$
\rho_{B}(1) = \frac{\mathrm{Tr}_A[((\mathbb{I}-M)^{1/2} \otimes \mathbb{I}) \rho_{AB} ((\mathbb{I}-M)^{1/2} \otimes \mathbb{I})]}{\mathrm{Tr}[((\mathbb{I}-M)^{1/2} \otimes \mathbb{I}) \rho_{AB} ((\mathbb{I}-M)^{1/2} \otimes \mathbb{I})]}.
$$
We can represent the correlations between system $B$ and the outcome of our measurement on system $A$ by some cq-state
$$
\rho_{A'B} = p_0 |0\rangle\langle 0 | \otimes \rho_B(0) + p_1 |1\rangle\langle 1 | \otimes \rho_B(1).
$$
The entropy of this state is
$$
S(\rho_{A'B}) = H(\{p_i\}) + \sum_i p_i S(\rho_B(i)),
$$
like in your question. Now for the particular state we picked we can calculate the eigenvalues of both $\rho_0$ and $\rho_1$ to be $\{(1+\gamma)/2, (1-\gamma)/2\}$. So we see our states are pure only when $\gamma = 1$. Note that this is exactly when the measurement $M$ is projective and not just a POVM. In general (I think but you should check) for any pure two-qubit state, if we measure one qubit with a binary projective measurement then the resulting reduced states on of the other qubit will be pure. And if we measure with a two-outcome non-projective measurement then the resulting reduced states will be mixed.
This situation frequently arises in cryptography where we measure a system which may be entangled with an adversaries system. Then we try to estimate how much information they have about our measurement outcomes given their quantum system. However this estimation usually uses conditional entropies like $S(A'|B) = S(A'B) - S(B)$.
