DQC Challenge: Regulator + Anything

Epic collaboration between Vanderbilt and Duke students, where we figured out the 'best' Taffoli Circuit Design for the highest quality with minimum costs of circuits.

Comment

What inspired us

Entanglement is the idea at the center of quantum computing. Two or more qubits can share one state that cannot be split into a separate state for each qubit, so measuring one of them tells you something about the others. Schrödinger gave entanglement its name in 1935 and called it "the characteristic trait of quantum mechanics". The same year he described his cat. A cat is sealed in a box with a single radioactive atom, and if the atom decays, a detector releases poison. Quantum mechanics puts the atom in a superposition of decayed and not decayed, and because the cat's fate is tied to the atom, the two end up in one joint state:

$$\frac{1}{\sqrt{2}}\Big(|\text{not decayed}\rangle\,|\text{alive}\rangle + |\text{decayed}\rangle\,|\text{dead}\rangle\Big)$$

A Toffoli gate makes exactly this kind of link. It flips a target qubit only when two control qubits are both 1, so the target plays the cat and the two controls play the atom. With both controls in superposition, the Toffoli ties the target to them (qubits in the order c1, c2, t):

$$\text{Toffoli}\,\big(|{+}\rangle|{+}\rangle|0\rangle\big) = \frac{1}{2}\Big(|000\rangle + |010\rangle + |100\rangle + |111\rangle\Big)$$

The cat also showed us how to test a Toffoli. Opening the box only ever shows "alive" or "dead". It cannot tell the joint state above from a cat that was simply alive or dead all along, because the difference is in the phase between the two parts. A truth table has the same blind spot: it reads every output as a plain bitstring, so a circuit with wrong phases can pass it perfectly. So we added two phase tests that read some qubits along X, where a wrong phase turns into a wrong output.

The box gave us one more idea. On real hardware the noise is sealed inside the machine, and all we ever see are counts. Our calibration method, method 2, claims to find each gate's hidden error from those counts alone. Before trusting it on hardware, we built boxes of our own: simulated machines with noise we chose and hid from the method. Method 2 saw only the counts and trained a design, and only then did we open the box and grade that design against the hidden noise.

That is what drew us to the DQC challenge: build the best Toffoli from a trapped-ion computer's native gates and prove it on real hardware, the DQC lab in the evening and IonQ's Forte 1 overnight. On trapped ions, the two-qubit rxx gate is the only gate that entangles, and it makes most of the errors. We started with two questions. Does a construction with smaller entangling turns make fewer errors? And can we measure the machine's errors and calibrate them away?

How we built it

The designs

A Toffoli needs at least five two-qubit gates (Yu, Duan and Ying, 2013), so every design has five rxx gates.

  • D0 is the starter kit's baseline, Qiskit's compilation. Its five rxx gates add up to a total angle of 2.25π.
  • D1c is the construction of Barenco et al. (1995), with each run of one-qubit gates merged. It has the same number of costly gates as D0 (5 rxx, 14 rx and ry), but three of its five entangling gates are half the size of a CNOT's:

$$3 \cdot \frac{\pi}{4} + 2 \cdot \frac{\pi}{2} = 1.75\pi$$

  • Our calibrated designs keep D0's or D1c's gates and change only the rxx angles, to cancel errors we measured. D2 came from gate probes. D3, D5 and D6 came from method 2, our fit of each gate's error to the counts of whole test circuits: D3 from lab round 1, and D5 (from D0) and D6 (from D1c) from Forte 1's first batch.

How we measured "best"

A truth table cannot see wrong phases, so we used three test sets: the truth table T and two mixed bases, B1 and B2, in which a phase error turns into a wrong output (Hofmann 2005; Mičuda et al. 2013). Their pass rates bound the process fidelity from both sides:

$$F_T + F_{B1} + F_{B2} - 2 \;\le\; F_{\mathrm{process}} \;\le\; \min\left(F_T,\ F_{B1},\ F_{B2}\right)$$

This costs 24 circuits per design, plus an 8-circuit readout check. We ranked designs on the raw lower end, F_LB, and called two designs different only when their gap was more than twice the shot noise of the gap:

$$\left|F_{\mathrm{LB}}^{(A)} - F_{\mathrm{LB}}^{(B)}\right| > 2\sqrt{\sigma_A^2 + \sigma_B^2}, \qquad \sigma_{\mathrm{rate}} = \sqrt{\frac{p\,(1-p)}{N}}$$

Testing method 2 blind

On the starter kit's noise model, every rxx secretly turns 0.2 rad too far. Method 2 found that error from counts alone, without being told it was the same on every ion pair, and its trained design D4 reached a true process fidelity of 0.987, against 0.929 for D0 and 0.990 for the best any angle fix can reach. We then ran our whole IonQ plan (a training batch, the fit, and a fresh test batch) on other hidden noise. True process fidelity of the trained versions of D0 and D1c:

  • Every rxx 0.2 rad too far: 0.988 and 0.989, against 0.929 and 0.933 untrained.
  • Every rxx 0.15 rad too far in its own direction: 0.989 and 0.988, against 0.955 and 0.957.
  • Turns and random errors that grow with the angle: 0.906 and 0.926, against 0.901 and 0.921. The gain is small because most of this loss is random error that no angle change can remove.
  • Readout error only: the fit found nothing to fix and made no trained design, as it should.

The first version of the fit failed the first of these tests. It got D6's signs wrong (0.880), which is how we found the bug described below. Three more tests with fully random noise did not finish before the deadline.

On the hardware

Before spending any shots, we ran the whole method on noise models and rehearsed the lab session on a simulated stand-in lab.

  1. On the DQC lab (24 October, 128 shots per circuit), we sent a readout check and gate probes at 16:59, then D0, D1c and D2 side by side from 17:15 to 18:22. After round 1 we used method 2 on the round 1 circuits and built D3, which ran next to D1c at 21:42. We used 19,712 of our 20,000 shots.
  2. On IonQ (25 October, 1,024 shots per circuit), we rehearsed on IonQ's free simulator first. Batch 1 ran D0, D1c and copies of both with every rxx turned 0.15 rad further on purpose, which shows the sign of each gate's error. We trained D5 and D6 with the fit we had frozen after the blind tests, then ran batch 2. When we found that IonQ's compiler had shortened our circuits (below), we translated our designs gate by gate into IonQ's native gates (GPi, GPi2 and ZZ), checked each translation exactly against its original, and ran three more jobs on Forte 1.

Every hardware number traces to a lab queue ID or an IonQ job ID in the repository's evidence/ folder. We used Qiskit 2.1, Qiskit Aer, qiskit-ionq, NumPy and SciPy, in Jupyter notebooks.

What we found

Raw F_LB for each run, with designs in the same run side by side:

  • DQC lab, round 1: D1c 0.752 ± 0.015, D0 0.727 ± 0.016, D2 0.661 ± 0.017. D1c and D0 tied, and both beat D2.
  • DQC lab, round 2: D1c 0.756 ± 0.015, D3 0.699 ± 0.017. D1c beat D3.
  • Forte 1 in native gates, 06:16: D1c 0.912 ± 0.003, D0 0.906 ± 0.003. Tied.
  • Forte 1 in native gates, 07:22: D1c 0.906 ± 0.003, D0 0.906 ± 0.003, D5 0.896 ± 0.004. D1c and D0 tied, and D5 came in just below D0.
  • Forte 1 in native gates, 07:53: D6 0.908 ± 0.003, D1c 0.907 ± 0.003, D0 0.901 ± 0.003. All three tied.

We claim D1c. It tied the baseline wherever our gates actually ran, and it beat or tied every calibrated design. We pick it over D0 by a rule we wrote at 21:50, which breaks a tie by the truth-table score. We wrote that rule after lab round 1, which already had D1c ahead on that score, and on Forte 1 the same kind of rule picks D0. So D1c and D0 are tied for best, and telling them apart would need several times more shots than one night gave us. We sent the native-gate runs before an organizer had said whether native gates are allowed, so we count them as supplementary.

Challenges we faced

  • Our first calibration made things worse. D2 was predicted to beat D0 by 0.077 and lost by 0.065 instead. The gate probes repeated each gate 5 times in a row and said the π/2 gates fell short by 0.14 to 0.18 rad, but a fit to the Toffoli circuits themselves showed the same gates turning slightly too far.
  • The lab drifted. Between 16:59 and 21:42 the rxx angle on one ion pair moved by about 0.2 rad, so D3's calibration was most likely out of date by the time it ran, and it lost to D1c too. D1c needs no calibration, and its score held steady: 0.752, then 0.756.
  • IonQ's compiler changed our experiment. Each test circuit starts in |000⟩ and should end in one known bitstring, and the compiler used that to shorten the circuits. It removed every two-qubit gate from D1c's test circuits, so D1c's near-perfect first scores on Forte 1 (0.970 and 0.976) were only the readout of circuits with no entangling gate. We had already reported D1c as clearly best on Forte 1 when we read back the compiled circuits and found this. The native-gate runs fixed it: there, every gate ran as sent.
  • Our training fit had a bug. The blind test above showed that the first version could get stuck on a wrong answer and pick the wrong sign. We added a grid of starting points and a sign test, and froze the code before fitting any IonQ data.
  • Smaller problems added up during the night. The lab writes each shot as a number from 0 to 7, where the starter kit describes bitstrings. The relay failed to save one circuit partway through round 1. Fetching 86 results in one request ran past the relay's 30-second limit. And one ZZ gate went past IonQ's quarter-turn limit and had to be rewritten as an equal gate within it.

What we learned

  • A perfect truth table can hide a wrong gate. On IonQ's simulator, D0's compiled circuits had perfect truth tables while 2.2% of their phase-test shots were wrong. Like the cat in the box, the phases only show up when you look in another basis.
  • Calibrate a gate in the same setting it is used in, and right before it is used. A gate repeated in a row behaves differently from the same gate inside a circuit, and on a drifting machine an old calibration can do harm.
  • Check what the machine actually ran. A cloud compiler can change the experiment without telling you.
  • Test the analysis blind before trusting it. Our blind test caught a fitting bug before it touched paid hardware.
  • Most of our comparisons were ties. With 128 to 1,024 shots per circuit, the gaps between our best designs were smaller than the shot noise, so we report them as ties.

Built With

Share this project:

Updates

Submission history