Inbuilt Stim circuits showing unexpected Memory X and Memory Z logical error rates for unrotated surface code

Question

When running memory experiments using Stim’s inbuilt unrotated surface code circuits to generate thresholds, I am getting the opposite behaviour to the rotated surface code for memory X and memory Z logical error rates.

For each type of memory experiment - which I am running by following the linux command line instructions for Sinter - both types of stabilisers are measured. Despite this, the two extra hadamards for X-type stabiliser measurements (which detect Z errors) means they have higher depth than the Z-type stabilisers in the overall circuit. This causes a higher logical error rate in a memory experiment for memory X (preserving $|+\rangle_L$ which is more affected by $Z$ errors) than for memory Z (preserving $|0\rangle_L$ which is more affected by $X$ errors). That is ${p_L}_X > {p_L}_Z$ (as shown in this paper).

The circuits I am using are stim's inbuilt surface code circuits. These measure both types of stabilisers for both types of memory experiment. See a distance 2, unrotated surface code with 1 round of stabiliser measurements below as an example (though for my simulations I am doing odd distances and $d$ rounds of stabiliser measurements).

Memory Z prepares the data qubits in $|0\rangle ^{\otimes n}$ and measures them in the Z basis at the end (preserving $|0\rangle_L$):

Whereas Memory X prepares the data qubits in $|+\rangle ^{\otimes n}$ and measures them in the X basis at the end (preserving $|+\rangle_L$):

Note that these diagrams are without noise but for my simulations I am using a circuit noise model (setting all of Stim's inbuilt errors to $p$).

When making threshold curves for the rotated surface code using Stim's inbuilt circuits I see, as expected, that ${p_L}_X > {p_L}_Z$. For example for distance 7 and 9 below ${p_L}_X > {p_L}_Z$ for each distance:

On the other hand for the unrotated surface code I am seeing the opposite. Instead of ${p_L}_X > {p_L}_Z$ I am seeing ${p_L}_X < {p_L}_Z$ I have triple checked that I am running and plotting the correct memory experiment type and can not figure out what is causing it. I have simulated every odd distance up to 15 and keep seeing ${p_L}_X < {p_L}_Z$, as per the plots below:

I am running the memory experiments from a linux command line following the instructions for Sinter here which are:

mkdir -p circuits
python -c "

import stim

for p in [0.001, 0.005, 0.01]:
    for d in [3, 5]:
        with open(f'circuits/d={d},p={p},b=X,type=rotated_surface_memory.stim', 'w') as f:
            c = stim.Circuit.generated(
                rounds=d,
                distance=d,
                after_clifford_depolarization=p,
                after_reset_flip_probability=p,
                before_measure_flip_probability=p,
                before_round_data_depolarization=p,
                code_task=f'surface_code:rotated_memory_x')
            print(c, file=f)
"

sinter collect \
    --processes 4 \
    --circuits circuits/*.stim \
    --metadata_func "sinter.comma_separated_key_values(path)" \
    --decoders pymatching \
    --max_shots 1_000_000 \
    --max_errors 1000 \
    --save_resume_filepath stats.csv

sinter plot \
    --in stats.csv \
    --group_func "'Rotated Surface Code d=' + str(metadata['d'])" \
    --x_func "metadata['p']" \
    --fig_size 1024 1024 \
    --xaxis "[log]Physical Error Rate" \
    --out surface_code_figure.png \
    --show

Answer 1

I think this is a real effect related to the CNOT ordering.

Generating the circuits:

import stim

for p in [0.001]:
    for d in [3, 5, 7, 9]:
      for b in 'xz':
       for rot in ['unrotated', 'rotated']:
         with open(f'circuits/d={d},p={p},b={b},rot={rot}.stim', 'w') as f:
            c = stim.Circuit.generated(
                rounds=d,
                distance=d,
                after_clifford_depolarization=p,
                after_reset_flip_probability=p,
                before_measure_flip_probability=p,
                before_round_data_depolarization=p,
                code_task=f'surface_code:{rot}_memory_{b}')
            print(c, file=f)

Collecting the data:

sinter collect \
    --processes 12 \
    --circuits circuits/*.stim \
    --metadata_func "auto" \
    --decoders pymatching \
    --max_shots 1000_000_000 \
    --max_errors 1000 \
    --save_resume_filepath stats.csv \
    --max_batch_seconds 10

Plotting the results (requires latest sinter dev version):

sinter plot \
    --in stats.csv \
    --group_func "f'''p={m.p} b={m.b} rot={m.rot}'''" \
    --x_func "m.d" \
    --xaxis "Distance" \
    --show \
    --plot_args_func "{'color': ('black' if 'un' in m.rot else 'red'), 'marker': 'x' if m.b == 'x' else 'o'}"

You can see in the plot that the black lines (unrotated code in each basis) are quite separated. This is surprising, because the circuits should be basically symmetric w.r.t. the two bases.

One asymmetry, mentioned by the question, that would cause you to expect a difference is that the X stabilizers perform H gates that the Z stabilizers do not and the noise model is such that those H gates add noise while the idle operations don't. You can see this in detslice-with-ops diagrams (note that lack of DEP1 boxes on the blue circled qubits, which are the Z basis measure qubits):

The only other asymmetry I can think of is in the layering of the operations. In the unrotated code, the first layer and last layer of CNOTs points between the Z boundaries whereas the middle layer of CNOTs points between the X boundaries.

But nothing strikes me as particularly concerning about the match graph...

To try to get to the bottom of this I pulled the error path counting code from https://github.com/Strilanc/inplace-y-basis-2023/blob/main/tools/debug_count_logical_error_edges and ran it on the circuits. It tracks where the simulator is inserting errors and counts up errors contributing to logical errors, making a heatmap that tells you how the logical errors "move" across the space.

This is X unrotated:

This is Z unrotated:

It makes sense that one likes horizontal planes while the other likes vertical planes. They have to travel to their respective boundaries, which is a vertical-vs-horizontal distinction. But it's notable that the Z one has a distribution that's different; you see these strong vertical connecting elements whereas the x plot has more diffuse paths.

The rotated ones look much more symmetric, but notice that they both like going up and to the right as opposed to up and to the left:

Rotated X:

Rotated Z:

This suggests there's something weak about that specific direction compared to the other diagonal. Which is consistent with what we see in the unrotated code, where those diagonals have become the vertical direction, and it does in fact appear weaker. So my guess is that the ordering of the CNOTs makes certain directions weak and that if you picked a different ordering of CNOTs you could make this bias flip and if you alternated orderings it would go away.

Answer 2

I am not sure that this is the answer, but look on the qubit in coords 2,0 (for example, there are other qubits in this situation). In your presented circuit, in the X memory circuit this qubit is not affected at all by the circuit (since a qubit in the + state cannot be affected by a CNOT), and though any error in the qubits sharing a CNOT gate with it, will not affect it. This does not happen in the Z memory circuit, which might explain the different error rates.