You are right, photonic systems are described by an infinite (separable) Hilbert space---the bosonic Fock space---and their formalism makes extensive use of infinite values, both countable and uncountable. The quantum computing paradigm based on this Hilbert space is called continuous-variable (CV) quantum computing, and a lot of different protocols and algorithms have been proposed using this framework, see for instance this recent review by Xanadu (who is developing optical quantum computers with the goal of working with continuous variables). Two important points to note: 1) CV quantum computers could in principle be built with other systems than photons, such as molecular vibrations (phonons), which obey the same equations as photons ; 2) you can restrict the Hilbert space of photons in order to get qubits, for instance by considering only polarization or by encoding qubits into continuous variables. This is the approach taken by the photonic quantum computing company PsiQuantum (as far as I understand).
Where does CV quantum computing come from?
There exists tons of equivalent ways to introduce the CV paradigm. The most physical is the quantization of the electromagnetic field: you take Maxwell's equations and you turn the electric and magnetic fields into non-commuting operators. You find that your system now describes a quantum Harmonic oscillator and that the Hamiltonian have infinitely many eigenstates, forming an infinite-dimensional Hilbert space.
Another more rigorous way to define this Hilbert space is called the second quantization: you define bosonic quantum states as multi-particle states that are invariant when you permute particles, and after some steps, you find that the correct Hilbert space to describe bosons is the so-called Fock space (which is a separable Hilbert space when defined properly).
Finally, you can formalize bosonic systems in a much more mathematical/computer sciency way, that allows you to talk about complexity theory. Three examples of such formalisms are given in Section 3 of this paper.
Formalism and infinities
All those formalisms have a common point: you end up with a separable Hilbert space. And all separable Hilbert spaces are the same up to an isometric isomorphism. Moreover, separable Hilbert spaces have the amazing properties to contain an infinite countable basis, that we can note $(|n\rangle)_{n \in {\mathbb{N}}}$. Therefore, for any state $|\psi\rangle \in \mathcal{H}$, there exists $(a_n)_{n \in {\mathbb{N}}}$ such that
$$|\psi\rangle = \sum_{n=0}^{\infty} a_n |n\rangle$$
Physically, $|n\rangle$ is a state that contains $n$ indistinguishable photons.
Using this basis $(|n\rangle)_{n \in \mathbb{N}}$ (called the Fock basis), we can define many important objects of the CV framework, such as the creation and annihilation operators
$$\hat{a}^{\dagger}|n\rangle=\sqrt{n+1} |n+1\rangle$$
$$\hat{a}|n\rangle=\sqrt{n} |n-1\rangle,$$
the position and momentum operators (which physically correspond the amplitude of the electric and magnetic fields, not to spatial coordinates)
$$\hat{X}=\frac{1}{\sqrt{2}} (\hat{a}^{\dagger} + \hat{a})$$
$$\hat{P}=\frac{1}{\sqrt{2}} i (\hat{a}^{\dagger} - \hat{a})$$
and the number operator
$$\hat{N}|n\rangle = n|n\rangle$$
Now, you can verify that $\hat{X}$ and $\hat{P}$ are hermitian (infinite-dimensional) operators, and are therefore observables that you can physically measure. Their eigenstates $|x\rangle$ and $|p\rangle$ form two new bases of your Hilbert space, but this time uncountably infinite, i.e. for every state $|\psi\rangle$, there exists a function $x\mapsto \psi(x)$ and a function $p \mapsto \phi(p)$ such that
$$|\psi\rangle = \int \psi(x) |x\rangle dx$$
$$|\psi\rangle = \int \phi(p) |p\rangle dp$$
Therefore, the same state can be represented both using countable infinities and uncountable infinities. Which basis you want to choose depends on your measuring device (photon detectors measure in the $|n\rangle$ basis and homodyne detectors in the $|x\rangle$ and $|p\rangle$ bases), the initial state of your algorithm (the output of a laser, called a coherent state---and more generally Gaussian states---are more easily representable with $\hat{X}$ and $\hat{P}$, while single-photons are more easily described in the Fock basis) or on the details of your algorithm (are integral or sums more convenient to analyze it?).
Algorithms
We saw what a CV state looks like, what measurements can look like, but what about gates? As usual, any unitary operator (here infinite-dimension matrix) can be seen as a gate. Elementary gates include squeezing, displacement, rotation, etc. and are very well described in the paper of the CV library Strawberry Fields. A particular representation of states called the Wigner function (roughly describing the quasi-probability to find a particle at a certain position and momentum) is often used to describe the effect of those gates.
Now, what are the applications of CV quantum computing? One of the main area where CV quantum information is used is in quantum communication. Indeed, photons can be transmitted through optical fibers and rarely interact, making it a perfect choice for communication. Moreover, communication protocols such as teleportation and QKD have been ported to CV systems.
Going back to computation, an important CV algorithm is Boson Sampling, which is mostly considered as a way to demonstrate quantum supremacy, but might have applications such as finding dense subgraphs or simulating molecular vibronic spectra
Finally, CV quantum computing has been considered in order to solve partial differential equation (porting the HHL algorithm to an infinite-dimensional system), to improve Monte-Carlo algorithms or to do quantum machine learning and variational circuits
If you are interested to go deeper in understanding continuous variables, apart from all the papers I've cited, you can also read the first section of my master's thesis, which explains all of that in more details and (I hope) in an understandable way.
