Generally speaking, in order to describe elements of a set $A$ using classical information we need two ingredients: a non-empty finite alphabet $\Sigma$ and an encoding $E: A\to\Sigma^\omega$ which injectively maps objects in $A$ to the set $\Sigma^\omega$ of sequences of symbols in $\Sigma$.
Infinite encodings
If the set $A$ is uncountable, as is the case for the set of all pure quantum states, then $E$ necessarily maps most elements in $A$ to infinite sequences of symbols.
This is what Nielsen & Chuang mean when they say that a generic quantum state requires an infinite amount of information to be described exactly. Another example of this phenomenon occurs for the decimal encoding of real numbers where a generic real number requires an infinite number of digits to be specified exactly.
In practice, we generally impose a restriction where $E:A'\to\Sigma^*$ maps a subset $A'\subset A$ to the set $\Sigma^*$ of all finite sequences of symbols in $\Sigma$. If $A$ is separable, then we can choose $A'$ to be a dense countable subset of $A$ which allows us to approximate elements in $A$ using elements in $A'$ arbitrarily well. This is theoretically the case in the first example below.
Even more often, we impose a restriction where $E:A'\to\Sigma^{\le n}$ maps $A'\subset A$ to the set $\Sigma^{\le n}$ of sequences up to some maximum length $n$. This imposes a limit on the precision of the encoding since many interesting sets do not have a finite dense subset. This is the case in the second example below.
Example 1: LaTeX
Let $A$ be the set of pure quantum states and let $\Sigma$ be the set of characters in the ASCII character table. We can encode many elements of $A$ using LaTeX. For example, the state
$$
|\psi\rangle = \frac{1}{\sqrt{2}}|0\rangle + \frac{i}{\sqrt2}|1\rangle
$$
may be represented as
\frac{1}{\sqrt{2}}|0\rangle + \frac{i}{\sqrt2}|1\rangle
which is a string of classical information describing a quantum state. This encoding is very popular on QCSE!
Example 2: Single-precision floating point numbers
As before, let $A$ be the set of pure quantum states, but this time let $\Sigma=\{0, 1\}$. We can encode some elements of $A$ by encoding the real and imaginary parts of each amplitude as little-endian single-precision floating point numbers in IEEE 754 and concatenating the results. For example, the state $|\psi\rangle$ defined above is approximately encoded as
11110011 00000100 00110101 00111111
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
11110011 00000100 00110101 00111111
where the first eight bytes encode a complex number near $1/\sqrt2$ and the last eight bytes encode a complex number near $i/\sqrt2$.
