Martingales for physicists: a treatise on stochastic thermodynamics and beyond

Abstract

We review the theory of martingales as applied to stochastic thermodynamics and stochastic processes in physics more generally.

Acknowledgments

ER acknowledges support from ICTP, highlighting the work of his entire research group, and the academic support of the QLS and the CMSP sections. He also thanks the following institutions for hospitality while writing this work: Université Côte d'Azur, DIPC –Donostia International Physics Centre, ESPCI Paris, MISANU Belgrade and PMF Niš. He is also grateful for fruitful scientific discussions on martingales to: Gonzalo Manzano, Rosario Fazio, the Pekola Lab, Rosemary Harris, Joachim Krug, Matteo Marsili, David Wolpert, Gulce Kardes, and Tarek Tohme. He thanks Pietro Luigi Muzzeddu, Debraj Das, Yonathan Sarmiento, John Bechhoefer, Juan MR Parrondo, Massimo Campostrini, and Stefano Ruffo for feedback on the elaboration of this treatise.

RC acknowledges his habilitation's committee for comments on preliminary versions of Chapters 6 and 9: Eric Akkermans, Giovanni Gallavotti, Giovanni Jona-Lasinio, Senya Shlosman, Michel Bauer, Denis Bernard, Cedric Bernardin, Sergio Ciliberto, Bernard Derrida, Krzysztof Gawedzki, Kirone Mallick, Cécile Monthus and Rémi Rhodes. Finally, RC dedicates this treatise to his master, Krzysztof Gawedzki, who left us in January 2022.

SG acknowledges the many clarifying and fruitful discussions and constant guidance received from Debraj Das who literally ushered him into the field of quantitative finance. He also thanks ICTP Abdus Salam International Centre for Theoretical Physics, Trieste, Italy, for support under its Regular Associateship scheme. He is grateful to Partha Nag for help with the references, and thanks Debraj Das, Soumya Kanti Pal, Sayan Roy and C L Sriram for discussions and useful comments on the text. SG is particularly grateful to Rudra Pratap Jena for several insightful discussions regarding delta hedging.

KS thanks Charles Moslonka (Gulliver ESPCI-PSL) who contributed essentially to the main part of Chapter 10. KS and Charles Moslonka are grateful to ER and Guilhem Semerjian (ENS-PSL) for their constructive and valuable comments.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Notes

1 “I went [to Venice's casino], taking all the gold I could get, and by means of what in gambling is called the martingale I won three or four times a day during the rest of the carnival”.

2 For the example of a Markov chain, which we introduce below in Chapter 3, explicit time dependence occurs if the transition matrix in Equation (3.2(3.2) $\begin{array}{r}w(x,y)\equiv \mathcal{P}(X_n=y|X_{n-1}=x),\mathrm{\forall }x,y\in \mathcal{X}.\end{array}$(3.2) ) has a supplementary dependence on n, i.e., the path-probability equation (3.4(3.4) $\begin{array}{r}\mathcal{P}\left(x_{[0,n]}\right)={\rho }_0\left(x_0\right)\prod _{j=1}^{n}w(x_{j-1},x_j),\end{array}$(3.4) ) reads (2.20) $\begin{array}{r}{\mathcal{Q}}^{\left(n\right)}\left(x_{[0,n]}\right)={\rho }_0\left(x_0\right)\prod _{j=1}^n{w}^{\left(n\right)}(x_{j-1},x_j).\end{array}$(2.20) Note that this latter property is different than the time-inhomogeneity of a Markov chain for which the transition matrix has a supplementary dependency on the present time j and the path-probability equation (3.4(3.4) $\begin{array}{r}\mathcal{P}\left(x_{[0,n]}\right)={\rho }_0\left(x_0\right)\prod _{j=1}^{n}w(x_{j-1},x_j),\end{array}$(3.4) ) reads (2.21) $\begin{array}{r}\mathcal{Q}\left(x_{[0,n]}\right)={\rho }_0\left(x_0\right)\prod _{j=1}^n{w}^{\left(j\right)}(x_{j-1},x_j).\end{array}$(2.21) This time inhomogeneity is not a problem for the last step of the derivation in Equation (2.19(2.19) $\begin{array}{rl}〈R_n|X_{[0,m]}〉& =\sum _{x_{m+1}}\dots \sum _{x_n}\mathcal{P}\left(x_{[m+1,n]}|X_{[0,m]}\right)\frac{\mathcal{Q}(X_{[0,m]},x_{[m+1,n]})}{\mathcal{P}(X_{[0,m]},x_{[m+1,n]})}\\ & =\sum _{x_{m+1}}\dots \sum _{x_n}\frac{\mathcal{P}(X_{[0,m]},x_{[m+1,n]})}{\mathcal{P}\left(X_{[0,m]}\right)}\frac{\mathcal{Q}(X_{[0,m]},x_{[m+1,n]})}{\mathcal{P}(X_{[0,m]},x_{[m+1,n]})}\\ & =\frac{\sum _{x_{m+1}}\dots \sum _{x_n}\mathcal{Q}(X_{[0,m]},x_{[m+1,n]})}{\mathcal{P}\left(X_{[0,m]}\right)}\\ & =\frac{\mathcal{Q}\left(X_{[0,m]}\right)}{\mathcal{P}\left(X_{[0,m]}\right)}=R_m,\end{array}$(2.19) ), which remains valid.

3 See [265] for generalizations where $\alpha \in \mathbb{R}$ and even space dependent.

4 The expression of the spurious drift depends on the convention chosen in Equation (3.65(3.65) $\begin{array}{r}{\dot{X}}_t={\mathit{\mu }}_t\left(X_t\right)F_t\left(X_t\right)+\left(\mathrm{\nabla }{\mathbf{D}}_t\right)\left(X_t\right)+\sqrt{2{\mathbf{D}}_t\left(X_t\right)}{\dot{B}}_t,\end{array}$(3.65) ). In some references, it is claimed that the spurious term disappears in the anti-Itô convention ($\alpha =1$ in Equation (2.85(2.85) $\begin{array}{r}{Y}_t={\int }_0^t{Z}_s{\circ }_{\alpha }\mathrm{d}{B}_s=\underset{\Vert P\Vert \to 0}{lim}\sum _{i=0}^{n-1}\left(\right(1-\alpha )Z_{t_i}+\alpha Z_{t_{i+1}})(B_{t_{i+1}}-B_{t_i}).\end{array}$(2.85) )) of the isothermal overdamped version of (3.65(3.65) $\begin{array}{r}{\dot{X}}_t={\mathit{\mu }}_t\left(X_t\right)F_t\left(X_t\right)+\left(\mathrm{\nabla }{\mathbf{D}}_t\right)\left(X_t\right)+\sqrt{2{\mathbf{D}}_t\left(X_t\right)}{\dot{B}}_t,\end{array}$(3.65) ), and therefore the convention $\alpha =1$ is often called the isothermal convention. Note, however, that the spurious drift disappears in the anti-Itô convention only for the case ${\mathbf{D}}_t\left(x\right)=g_t\left(x\right){\mathbf{D}}_t,$ with g a scalar function and ${\mathbf{D}}_t$ a space homogeneous matrix, i.e., if all the x dependence is in the scalar part. The latter condition holds in one dimension, but is not generally true for $d\ge 2$. See also [266,267]. An alternative perspective is to consider the Langevin equation (3.65(3.65) $\begin{array}{r}{\dot{X}}_t={\mathit{\mu }}_t\left(X_t\right)F_t\left(X_t\right)+\left(\mathrm{\nabla }{\mathbf{D}}_t\right)\left(X_t\right)+\sqrt{2{\mathbf{D}}_t\left(X_t\right)}{\dot{B}}_t,\end{array}$(3.65) ) as the zero correlation time limit of Equation (3.65(3.65) $\begin{array}{r}{\dot{X}}_t={\mathit{\mu }}_t\left(X_t\right)F_t\left(X_t\right)+\left(\mathrm{\nabla }{\mathbf{D}}_t\right)\left(X_t\right)+\sqrt{2{\mathbf{D}}_t\left(X_t\right)}{\dot{B}}_t,\end{array}$(3.65) ) but with a colored, Orsntein–Uhlenbeck noise, see [268,269]. Note that the limiting equation has also in general a non-vanishing spurious drift, except for the one-dimensional case if we choose to write Equation (3.65(3.65) $\begin{array}{r}{\dot{X}}_t={\mathit{\mu }}_t\left(X_t\right)F_t\left(X_t\right)+\left(\mathrm{\nabla }{\mathbf{D}}_t\right)\left(X_t\right)+\sqrt{2{\mathbf{D}}_t\left(X_t\right)}{\dot{B}}_t,\end{array}$(3.65) ) in the Stratonovich convention. Such spurious drift is in general different to the spurious drift in Equation (3.65(3.65) $\begin{array}{r}{\dot{X}}_t={\mathit{\mu }}_t\left(X_t\right)F_t\left(X_t\right)+\left(\mathrm{\nabla }{\mathbf{D}}_t\right)\left(X_t\right)+\sqrt{2{\mathbf{D}}_t\left(X_t\right)}{\dot{B}}_t,\end{array}$(3.65) ). For the expression and the proof of such spurious drift in the general case, we refer to Theorem 7.2 on page 497 of the book [270].

5 Indeed, if we apply Itô's formula to $\exp \left(Z\right)$, see Appendix B.3, with $Z=Y_t-Y_0-[Y,Y{]}_t/2$, and use that $[Y,[Y,Y\left]\right]=0$ and $\left[\right[Y,Y],[Y,Y\left]\right]=0$, we obtain Equation (4.92(4.92) $\begin{array}{r}{\dot{\mathcal{E}}}_t\left(Y\right)={\mathcal{E}}_{t-}\left(Y\right){\dot{Y}}_t,\end{array}$(4.92) ). Note that the correction term inside the exponential can be understood from the passage of Equation (4.92(4.92) $\begin{array}{r}{\dot{\mathcal{E}}}_t\left(Y\right)={\mathcal{E}}_{t-}\left(Y\right){\dot{Y}}_t,\end{array}$(4.92) ) from the Itô convention to the Stratonovich convention (see Appendix B.3 on stochastic integrals).

6 The ${\dot{X}}_s$ in these relations must be interpreted in Stratonovich convention.

7 The Dynkin martingale approach does not provide a rigorous proof of martingality neither as $\exp (-s)$ is not a bounded function, which is required to show martingality, see Theorem 4.

8 See also the footnote in Section 2.1.3 for an example in the discrete-time setup.

9 Here, ${\mathrm{\Lambda }}_{[s,t]}$ should depend on $X_{[s,t]}$ only.

10 An example of Markov process where we have condition (3) without condition (2) is a general Isothermal Langevin equation:  (3.65(3.65) $\begin{array}{r}{\dot{X}}_t={\mathit{\mu }}_t\left(X_t\right)F_t\left(X_t\right)+\left(\mathrm{\nabla }{\mathbf{D}}_t\right)\left(X_t\right)+\sqrt{2{\mathbf{D}}_t\left(X_t\right)}{\dot{B}}_t,\end{array}$(3.65) ) with Einstein relation (3.69(3.69) $\begin{array}{r}{\mathbf{D}}_t\left(x\right)=\frac{T}{2}\left({\mathit{\mu }}_t\right(x)+{\left[{\mathit{\mu }}_t\right(x\left)\right]}^{†}),\end{array}$(3.69) ), without external force $f_t=0$, generic time-homogeneous potential $V_t=V$, and with the mobility matrix ${\mathit{\mu }}_t$ having an explicit time dependence. For this example, the stationary density is the Gibbs density ${\rho }_{\mathrm{st}}\sim \exp (-H(x\left)/T\right)$, and if moreover ${\rho }_0={\rho }_{\mathrm{st}}$, we have (3) without (2).

11 For all $0\le r\le s\le t$, we have the decomposition of the Markovian path probabilities (6.136) $\begin{array}{r}{\mathcal{P}}_{[0,s]}\left(X_{[0,t]}\right)=\frac{{\mathcal{P}}_{[0,r]}\left(X_{[0,t]}\right){\mathcal{P}}_{[r,s]}\left(X_{[0,t]}\right)}{{\rho }_r\left(X_r\right)},\end{array}$(6.136) and (6.137) $\begin{array}{r}{\mathcal{Q}}_{[t-s,t]}^{\left(t\right)}\left({\mathrm{\Theta }}_t{X}_{[0,t]}\right)=\frac{{\mathcal{Q}}_{[t-s,t-r]}^{\left(t\right)}\left({\mathrm{\Theta }}_t{X}_{[0,t]}\right){\mathcal{Q}}_{[t-r,t]}^{\left(t\right)}\left({\mathrm{\Theta }}_t{X}_{[0,t]}\right)}{{\rho }_{t-r}^{Q^{\left(t\right)}}\left(X_r\right)}.\end{array}$(6.137)

12 This follows from stationarity which gives $〈\mathrm{\Delta }{S}_t^{\mathrm{sys}}〉=0$, and the fact that the second law (6.29(6.29) $\begin{array}{r}〈S_t^{\mathcal{P},\mathcal{Q}}〉=〈\mathrm{\Delta }{S}_t^{\mathrm{sys}}〉+〈S_t^{\mathrm{env},\mathcal{P},\mathcal{Q}}〉\ge 0,\end{array}$(6.29) ) holds for all normalized $\mathcal{Q}$.

13 The nonequilibrium free energy is formally defined as $G_t^{\mathrm{ne}}=V_t-T{S}_t^{\mathrm{sys}}$, see Equation (9.16(9.16) $\begin{array}{r}{G}_t^{\mathrm{ne}}=H\left(X_t\right)+\mathrm{Tln}\left({\rho }_t\right(X_t\left)\right),\end{array}$(9.16) ) in Chapter 9, with $E_t$ and $S_t^{\mathrm{sys}}$ the (stochastic) energy and nonequilibrium entropy of the system at time t. We will provide a proof of the second law (8.47(8.47) $\begin{array}{r}〈W_t〉-〈\mathrm{\Delta }{G}_t^{\mathrm{ne}}〉\ge 0,\end{array}$(8.47) ) in Chapter 9, see Equation (9.26(9.26) $\begin{array}{r}〈W_t〉\ge G_t^{\mathrm{eq}}-G_0^{\mathrm{eq}}+T{D}_{\mathrm{KL}}\left[{\rho }_t\right(x)\left|\right|\exp (-\frac{H_t\left(x\right)-G_t^{\mathrm{eq}}}{T})]=〈G_t^{\mathrm{ne}}〉-〈G_0^{\mathrm{ne}}〉.\end{array}$(9.26) ).

14 See p. 174–175 in [98] for a detailed proof of Equation (9.12(9.12) $\begin{array}{r}{\mathrm{\Sigma }}_{[r,s],t}^{\mathcal{P},{\mathcal{P}}^{\mathrm{h},\left(\mathrm{t}\right)}}=\ln \left(\frac{\rho {}_r}{{\rho }_r^{\prime }}\left(X_r\right)\right).\end{array}$(9.12) )

15 Except for the case $\mathcal{Q}={\mathcal{Q}}_{\mathrm{st}}$ where we have (9.35(9.35) $\begin{array}{r}\frac{\mathrm{d}}{\mathrm{d}t}〈{\mathrm{\Lambda }}_t^{\mathcal{P},\mathcal{Q}}〉\ge 0,\end{array}$(9.35) ).

16 Note that if we replaced $m_{\mathsf{T}+1}$ on the left-hand side by $s_{\mathsf{T}+1}=M_{\mathsf{T}+1}-M_{\mathsf{T}}$, the equation is nothing but the definition of our quenching protocol.

17 While the analogy is not close, let us regard the ensemble of the graphs $\left\{\right(t,M_t){\}}_{T\le t\le N_0}$ representing the histories of the total magnetization on the $(t,M)$-plane as a light wave emitted from $(T,M_T)$ in the direction parallel to $(1,m_T)$. In the wave optics, when the wavelength of the light is non-negligible against the aperture of the light source, the flux of light is broadened as it propagates while the location of the maximum intensity goes along the “ray”, $M_t=M_T+(t-T)m_T$ for $T\le t\le N_0$, according to the geometrical optics. Likewise, in the Progressive Quenching, the stochasticity causes diffusion of the trajectories around the mean history, $M_t=M_T+(t-T)m_T$ for $T\le t\le N_0$. While the broadening of the light flux grows linearly with distance from the source, the trajectories of Progressive Quenching will diffuses like $\sim (t-T{)}^{1/2}$ for $T\le t\le N_0$.

18 There are two types of markets – primary and secondary. When a company issues its shares, the process is called Initial Public Offering (IPO). Investors interested in buying the shares have to apply to procure the shares. In case there are more applications than the number of shares issued, applicants are chosen randomly. Selected applicants buy shares directly from the company. Stock exchanges have no part to play here. This is referred as the primary market. After the above process is complete, the company gets listed in the stock exchanges. Only after this can an investor trade (buy or sell) the stock of the company in the exchanges from another share holder. This is called the secondary market.

19 In the market, there are also speculators who unlike the hedgers like to take risks, by anticipating trends in the market and exploiting them to make profit [228].

20 Which will be here also the pointer basis.

21 This matrix is diagonal in the orthonormal basis $|\pm 1〉$; this is the meaning of the quantum non-demolition hypothesis here.

22 In mathematics, a set A is considered a subset of a set B, or, equivalently, B is a superset of A, if all elements of A are also elements of B.

23 We might appreciate this meaning from different facets: (1) When a pair of histories, ω and ${\omega }^{\prime },$ realizes the identical set of data $X_{[0,n]},$ therefore also identical $〈z|X_{[0,n]}〉,$ it can occur that $z\left(\omega \right)\ne z\left({\omega }^{\prime }\right).$ (2) When a history ω is given, $〈z|X_{[0,n]}〉$ takes the average of z over all the histories $\left\{{\omega }^{\prime }\right\}$ which share the same tata $X_{[0,n]}.$ (3) z can be any function of n variables, $X_{[0,n]}.$ Nevertheless, each of $X_{[0,n]}$ are prefixed functions of the history, being independent of z. While z is an object of observation, $X_{[0,n]}$ are the measureing apparatus for that. (4) Yet, z is not restricted to a linear combination of $X_{[0,n]}$ and, therefore, the functional subspace spanned by $〈z|X_{[0,n]}〉$ is not n-dimensional.

Additional information

Funding

ER acknowledges funding from Ministero dell'Istruzione, dell'Università e della Ricerca PNRR grant PE0000023-NQSTI. RC is supported by the French National Research Agency through the projects QTraj (ANR-20-CE40-0024-01), RETENU (ANR-20-CE40-0005-01), and ESQuisses (ANR-20-CE47- 0014-01). SG acknowledges support from the Science and Engineering Research Board (SERB), India, under SERB-MATRICS scheme Grant No. MTR/2019/000560, and SERB-CRG scheme Grant No. CRG/2020/000596. This research was supported in part by the International Centre for Theoretical Sciences (ICTS) for the online program “Stochastic Thermodynamics: Recent Developments” (code: ICTS/strd2022/06) and for the program “Workshop on Martingales in Finance and Physics” (ICTP).