# Where does the "error propagation formula" $(\Delta \theta)^2=(\Delta M)^2/|\partial_\theta\langle M\rangle|^2$ come from, in estimation theory?

Consider the single parameter estimation setting, where we have a distribution depending on $$\theta$$ and we're looking for a "good" estimator for $$\theta$$. A commonly mentioned strategy, found e.g. in Eq. (7) of [TA2014], is to measure some observable $$M$$, thus obtaining an estimator for $$\theta$$ with variance $$(\Delta \theta)^2 = \frac{(\Delta M)^2}{\lvert\partial_\theta\langle M\rangle\rvert^2} \equiv \frac{\operatorname{tr}(M^2\rho_\theta)-\operatorname{tr}(M\rho_\theta)^2}{\lvert\operatorname{tr}(M \partial_\theta\rho_\theta)\rvert^2}.\tag{7}$$ To some degree, I can see the idea behind this formula: thinking of $$f(\theta)=\operatorname{tr}(M\rho_\theta)$$ as a function of $$\theta$$, and defining the "estimator" as $$\theta=f^{-1}(\operatorname{tr}(M\rho_\theta))$$, using the naive error propagation formula we'd get $$(\Delta\theta)^2 = \frac{(\Delta M)^2}{|f'(\theta)|^2} = \frac{(\Delta M)^2}{|\partial_\theta \operatorname{tr}(M\rho_\theta)|^2}.$$ This is using the standard error propagation formula: if $$A=f(B)$$ for two physical quantities $$A,B$$, then $$\Delta A=f'(B)\Delta B$$.

However, how would we formalise more precisely the estimation strategy underlying this formula in the quantum case? In particular, how exactly would the estimator associated to this strategy be defined? My naive attempt would be to interpret $$\theta=f^{-1}(\langle M\rangle)$$ in the single-shot regime as $$\hat\theta(k) = f^{-1}(\lambda_k),$$ where $$k$$ labels the possible measurement outcomes measuring in the eigenbasis of $$M=\sum_k \lambda_k |u_k\rangle\!\langle u_k|$$, and $$\lambda_k$$ the eigenvalues. I'm not totally sure it's exactly the same thing, but this strategy seems in essence an application of the method of moments. Another possibility is to interpret the strategy in a fashion more similar to the method of moments, which would mean to first estimate the empirical mean estimator for $$\langle M\rangle$$, write it as $$\bar M$$, and then define the estimator as $$\hat\theta(X) = f^{-1}(\bar M),$$ where $$X$$ is the collected statistic used to estimate the parameter.

Assuming this is actually the strategy used in this context, how do we get the result $$\operatorname{Var}[\hat\theta|\theta_0] = \frac{\operatorname{Var}[M|\theta_0]}{\lvert \partial_\theta\operatorname{tr}(M\rho_\theta)|_{\theta=\theta_0} \rvert^2}, \quad \operatorname{Var}[M|\theta_0]\equiv \operatorname{tr}(M^2\rho_{\theta_0})-\operatorname{tr}(M\rho_{\theta_0})^2,$$ which should be the formally precise way to state Eq. (7) in the above reference?

• I'm not sure why you need a precise formulation for quantum that differs from classical. Error propagation from a probability distribution is error propagation from a probability distribution, no matter how you obtained that distribution. Is the problem distinguishing between estimators and parameters? Then yes all measurements can only lead to estimators (even classically), so is the real question how to do error propagation for estimators? Commented Apr 15 at 13:27
• @QuantumMechanic yes that's probably a good way to put it. You're right that there isn't anything quantum about it, it's just general estimation theory. I suppose I've just never seen the error propagation formula derived in the context of estimation theory. which however would seem like it should bring some more issues to the table. For example, are these formulas independent on the actual choice of estimator for the parameter? From your comment I'm guessing the answer will revolve around ${\rm Var}[f(X)]\simeq (f'(\mathbb{E} X))^2{\rm Var}[X]$, and its interpretation in this context
– glS
Commented Apr 15 at 13:39
• Also might be restricted to unbiased estimators. Take the estimator $\hat{M}=0$, the derivative of the expectation value with respect to anything vanishes, uncertainty on $\theta$ is infinite. But if we just had the biased estimator $\hat{\theta}=f^{-1}(\hat{M})=f^{-1}(0)$ it could easily have a finite uncertainty. Commented Apr 15 at 15:45
• I tried to give a different answer here than to the other one, addressing different issues Commented Apr 18 at 0:49

The basic starting point will be to distinguish between "a measurement" in quantum theory and "the result of a measurement" or "the measurement of an operator." When one talks about measuring an operator $$M$$, one is really talking about measuring its expectation value (let's call it $$m=\langle M\rangle=\mathrm{Tr}(\rho M)$$ to avoid carets on operators like $$\hat{M}$$, in case we want to use carets to refer to estimators). But quantum theory describes measurements by POVM elements! How can we reconcile this?

A "measurement of $$M$$" $$\mathcal{M}$$ actually has the POVM elements $$M_i$$ such that $$M=\sum_{i}m_i M_i,\quad M_i=M_i^\dagger, M_i\succeq0, \sum_i M_i=\mathbb{I},$$ where $$\{m_i\}$$ are the eigenvalues of $$M$$. According to the Born rule, the probability of the $$i$$th measurement outcome is $$p_i=\langle M_i\rangle.$$ Then, finding the expectation value of $$M$$ really means that you have performed $$\mathcal{M}$$, collected the probabilities of each result "$$m_i$$" and weighed them appropriately as $$m=\sum_i p_i m_i.$$ Note that this equals $$\langle M\rangle$$, as required.

So the basic starting point is actually that one is performing a specific POVM $$\mathcal{M}$$ to measure/estimate the quantity $$m$$. From that, one may try to infer another quantity (or parameter) $$\theta$$. Propagation of uncertainty, change of variables, etc. all assume that the measurement itself doesn't change (of course they do - they are classical, so they usually take the probability distribution as the starting point). It essentially asks "given the same POVM $$\mathcal{M}$$, what is the uncertainty on a measurement of $$\theta$$?" And the answer to that question has very little to do with quantum theory: it is just $$\Delta^2\theta=\left(\frac{\partial m}{\partial \theta}\right)^{-2}\Delta^2 m.$$

Can we make this about quantum theory, just because we like to? We write the Fisher information that our POVM $$\mathcal{M}$$ yields about the parameter of interest $$m$$ (that we recall is the expectation value of an operator $$M$$ that we colloquially say that we are measuring) $$F(m)=\sum_i \frac{1}{p_i}\left(\frac{\partial p_i}{\partial m}\right)^2.$$ The Cramér-Rao bound says that any uncertainty on $$m$$ will be bound by $$\Delta^2 m\geq 1/F(m).$$ What if we want to express the Fisher information in terms of a new parameter $$\theta$$, such that $$\Delta^2 \theta\geq 1/F(\theta)$$? Well, we can do chain rule on the partial derivatives to find $$F(m)=\sum_i \frac{1}{p_i}\left(\frac{\partial p_i}{\partial \theta}\frac{\partial \theta}{\partial m}\right)^2=F(\theta)\left(\frac{\partial \theta}{\partial m}\right)^2$$ and so $$\Delta^2 \theta\geq 1/F(\theta)=\left(\frac{\partial \theta}{\partial m}\right)^2/F(m)=\left(\frac{\partial m}{\partial \theta}\right)^{-2}/F(m)$$.

What is the missing piece, to make these equations the same and answer the question? That an estimator saturating the bound for $$m$$ will attain equality $$\Delta^2 m=1/F(m)$$. With that exact same estimator, we have $$\Delta^2 \theta\geq \left(\frac{\partial m}{\partial \theta}\right)^{-2}/F(m)=\left(\frac{\partial m}{\partial \theta}\right)^{-2}\Delta^2 m.$$ Immediately, we see that there is still an inequality! The estimator that saturated the bound for $$m$$ only provides a lower bound for $$\Delta^2\theta$$; it is possible that the estimator will produce a value of $$\Delta^2\theta$$ that is, in fact, greater than the bound $$\left(\frac{\partial m}{\partial \theta}\right)^{-2}\Delta^2 m$$. However, for any Cramér-Rao bound for sufficiently regular probability distributions, there exists some estimator that will saturate the bound, so in practice there will always be some estimator that achieves $$\Delta^2\theta=\left(\frac{\partial m}{\partial \theta}\right)^{-2}\Delta^2 m$$. Since this is all achieved without changing the POVM and only by changing the classical postprocessing of the measured probability distribution, one can normally ignore these nuances.

Note, finally, if one wants to use the same estimator to get the minimum uncertainty on both of $$m$$ and $$\theta$$, that is probably not always a good idea (just like how the extremum $$\theta_0$$ of $$\theta(m)$$ does not guarantee the presence of an extremum of $$m(\theta)$$ at $$m(\theta_0)$$). However, if one is using an invariant estimator like the maximum likelihood estimator (the latter can saturate the Cramér-Rao bound asymptotically), then the extrema will coincide. The estimator of the function will be the same thing as the function of the estimator. Thus, classical statistics guarantees, under the correct conditions, that the Cramér-Rao lower bounds for the uncertainties of functionally related parameters will be related via the error propagation formula, and that the bounds can be simultaneously saturated with the same estimator.

The postscript is that this was all classical, with quantum providing the probability distribution. Uncertainties can thus be computed using the state and the POVM elements in the usual way, but I did not do that anywhere in the answer because it does not depend on those computations.

• the formula for the reparametrisation of the Fisher is probably the correct approach, I'd also been thinking along those lines. I still have some issues with how to interpret it though. Here $F(\theta)$, via CR, tells you the optimal var for an est $\hat\theta$. While $F(m)$ the same for an est for $⟨M⟩$. Which means you're considering a Markov chain $\theta\to M\to \hat M$. That is, that's the variance of an estimator for $M$ you get measuring something else that is a function of $M$. So what does $\frac{\partial\theta}{\partial m}$ even mean in your expression?
– glS
Commented Apr 18 at 8:10
• I understand that reparametrisation formula if you want to compute some function $f(\theta)$ from samples $X\sim p_\theta$, eg as in the example here. But in this case $m$ is not a function of $\theta$: it's a random variable sampled from a distribution depending from $\theta$. There's no function to differentiate.
– glS
Commented Apr 18 at 8:13
• @glS Whenever you measure a parameter $m$ and want to infer another parameter $\theta$, you need a "theory" for $\theta=\theta(m)$. That can be calibration data, a known relationship (volume of mercury vs temperature), etc. It can be differentiated. Yes, your actual measurements will be some random samples $X_i$ from which you can compute the parameter $m_i=m(X_i)$, but each one can be converted to $\theta_i=\theta(m_i)$ and vice-versa. As well, for invariant estimators, one can always go back and for $\hat{M}\leftrightarrow \hat{\theta}$ using this "theory" Commented Apr 18 at 15:18
• for classical (non-stochastic) numbers, I agree. But in this context we're talking about parameter estimation, so the "theory" you're talking about would be an estimator $\hat\theta$ for the parameter. Which isn't a function of "$m$", but of the measurement outcomes/individual samples. Maybe it'd help to concretise the discussion with an example: say you want to estimate $p$ measuring in the $X$ basis the state $\sqrt{1-p}|0\rangle+\sqrt p|1\rangle$. So estimating $p$ from the PMF $1/2\pm\sqrt{p(1-p)}$, but using the formalism in your answer. What would that look like?
– glS
Commented Apr 18 at 16:02
• because as far as I can tell, the problem can be summarised as having a process $\theta\to X\to f(X)$ and trying to estimate $\theta$ from $f(X)$. But if $f$ is deterministic, which seems like it should be in this context (expval of an observable just means assigning eigenvalues to each outcome), then using $X$ or $f(X)$ should be equivalent, and they should give identical Fisher. Unless you're saying: we first estimate $f(X)$ (ie $M$), and from this estimate want to estimate $θ$. Maybe I'm seeing it this way; but isn't that quite a roundabout approach to just estimate $θ$ from $X$?
– glS
Commented Apr 18 at 16:11