Trusting: Alone and together

ABSTRACT

We study the problem of an agent continuously faced with the decision of placing or not placing trust in an institution. The agent makes use of Bayesian learning in order to estimate the institution’s true trustworthiness and makes the decision to place trust based on myopic rationality. Using elements from random walk theory, we explicitly derive the probability that such an agent ceases placing trust at some point in the relationship, as well as the expected time spent placing trust conditioned on their discontinuation thereof. We then continue by modeling two truster agents, each in their own relationship to the institution. We consider two natural models of communication between them. In the first (“observable rewards”) agents disclose their experiences with the institution with one another, while in the second (“observable actions”) agents merely witness the actions of their neighbor, i.e. placing or not placing trust. Under the same assumptions as in the single agent case, we describe the evolution of the beliefs of agents under these two different communication models. Both the probability of ceasing to place trust and the expected time in the system elude explicit expressions, despite there being only two agents. We therefore conduct a simulation study in order to compare the effect of the different kinds of communication on the trust dynamics. We find that a pair of agents in both communication models has a greater chance of learning the true trustworthiness of an institution than a single agent. Communication between agents promotes the formation of long-term trust with a trustworthy institution as well as the timely exit from a trust relationship with an untrustworthy institution. Contrary to what one might expect, we find that having less information (observing each other’s actions instead of experiences) can sometimes be beneficial to the agents.

KEYWORDS:

• Embeddedness
• institutional trust
• learning
• social influence
• trust

1. Introduction

In this paper we are interested in the process by which individuals decide to place their trust in an institution. The importance of trust and distrust in governments around the world was highlighted by the COVID-19 pandemic during which the distrust in government held by a group resulted in fewer vaccinations among individuals of that group (Bajos et al., 2022). Similarly, distrust in science is strongly correlated with vaccine hesitancy (Lazarus et al., 2022). The prevalence of fake news during the COVID-19 infodemic (World Health Organization, 2020) points toward peer-to-peer communication as a culprit for widespread distrust. Although these results were gathered in relation to the COVID-19 pandemic and vaccine hesitancy, one expects similar dynamics to apply to a whole host of different social phenomena. While the personal experience of an individual with the institution is important, it seems that the interactions with any individual’s peers play a crucial role as well. We therefore study the dynamics of individual-to-institution trust in the context of peer-to-peer relationships between the individuals placing the trust.

A trust situation is a situation in which individuals are required to take a “risk” in order to observe how trustworthy the other party is (Buskens, 2002, Ch.1). In some cases this risk concerns the placement of resources at the disposal of the other. Relationships to institutions are characterized by trust: the truster in such a relationship typically builds an intuition of how trustworthy the institution is. For example, we could consider the relationship between an individual and a social media company. The individual has thus placed their data (resources) in the hands of the company, and in return can post content, and has access to the posts of others. The level of trustworthiness the individual observes is determined by their daily experience using the social network, manifested in the (lack of) news that the social media company has been selling their information to third party advertising companies. As an alternative example, consider the decision of whether or not to get vaccinated for a virus. For those individuals who believe scientific consensus to have their best interest at heart, this decision may not seem like something requiring trust at all. For other individuals who are skeptical of the institution science and/or healthcare governance, it may seem like a risk to their health and possibly their personal information to get vaccinated at a government vaccine station.

This type of examples has motivated us to phrase the problem of deciding to place trust as a learning problem. The idea that agents learn about the trustworthiness of their interaction partner is also supported by sociological theory. The chapter by Buskens and Raub (Buskens & Raub, 2002) provides a good overview of trust as a learning problem, as well as its own experimental evidence thereof. Furthermore, individuals are rarely isolated when faced with the decision of whether or not to place trust. Instead, human actions are embedded in a network which is likely to play a role in the decision making behavior of its individuals. We therefore consider the effect of peer-to-peer communication on the dynamics of the trust problem, which is also supported in the empirical literature (cf (Ozdemir et al., 2020; Pechmann & Knight, 2002). In particular Wang and Yu (Wang & Yu, 2017) find evidence that this influence acts both when experiences are communicated as well as when agents simply observe another’s behavior. We restrict our study to two agents as this showcases the essence of the underlying mechanism in the agent-to-agent communication. This allows us to construct the model with exact Bayesian learning.¹ Under basic assumptions of myopic² rationality, instead of exogenously imposing model assumptions on the effect of a signal sent from one agent to the other. This approach leads to delicate intricacies already in the two agent case, stemming from agents interpreting their neighbors actions whilst knowing that these actions were influenced by the actions they themselves have taken thus far. As a result of this mutual influence, in the two agents setup the evaluation of key quantities in our model eludes simple and explicit expressions. In fact, evaluating them requires many numerical integrations, which has led us to use a cluster of a large computing facility in our numerical experiments.

1.1. Related literature

Our investigation connects to two streams of literature: that of social network learning and that of social network trust. We proceed by providing an admittedly non-exhaustive, brief account of the related literature. The work relevant to our investigation spans various research disciplines, each with their own approach and set of “reasonable assumptions.” The fields in question contain, but are not limited to, economics, theoretical sociology, socio-physics and machine learning. Beyond modeling of these dynamics, the empirical investigations of network influence on trust even extend to the field of marketing.

1.1.1. Social network learning

Social network learning concerns agents who try to optimize an unknown objective function by choosing an action, and that are learning via private signals and/or signals from other agents. Under various modeling decisions, the interest is in the speed at which a group of agents learns the best course of action, as well as whether or not under assumptions of rationality, the group may take a sub-optimal action often as a result of social influence. The modeling decisions relate to the degree of rationality in learning, the communication between the agents, and whether or not the private signals of the agents are conditional on the actions they take. Table 1 summarizes the modeling decisions of the papers (Bala & Goyal, 1998; Harel et al., 2021; Huang et al., 2021; Molavi et al., 2018), all of them considering a population of agents in the social network learning framework. The second last column headed “Communication” refers to what the agents share with one another. This could be their private learning signal per round, their belief distribution, or simply the actions they take. The last column headed “Signals” refers to whether the reception of private signals by the agents is independent of or conditional on the actions the agents take.

Table 1. Modelling decisions in selected social network learning literature.

Paper	Rationality	Communication	Signals
Bala & Goyal (1998)	myopic	actions and signals	conditional
Molavi et al., (2018)	imperfect-recall	belief	independent
Harel et al., (2021)	myopic	actions	independent
Huang et al., (2021)	non-myopic	actions	independent

The agents in the paper of Bala and Goyal (Bala & Goyal, 1998) are myopic, and in addition they do not infer anything about the outcomes of the neighbors of their own neighbors based on the actions taken by their immediate neighbors. The authors find that in connected networks the agents’ behavior converges asymptotically. Molavi et al., (2018); Harel et al., (2021); Huang et al., (2021) all consider models in which the private signals received by the agents are independent of the actions they take. Within that context, (Molavi et al., 2018) show how imperfect-recall in the context of Bayesian learning relates to the seminal model by (DeGroot, 1974). In models of a fully connected population of agents, Harel et al., (2021); Huang et al., (2021) relate the learning rate of the model with communication restricted to actions to a reference model where communication between agents is open to signals and beliefs.

Departing from the network setting, (Correa et al., 2020) and Fu and Le Riche (Fu & Le Riche, 2021) consider a model in which sequentially arriving agents from a population choose to buy a product or not, based on a Bayesian belief on the product’s quality that has been built on the observations of the preceding agents. Correa et al., (2020) compute the probability of incomplete learning which occurs when the agents falsely believe the product’s quality to be low. An important difference between Correa et al., (2020) and our setup is that the agents in Correa et al., (2020) know that the quality of the product comes from a set of two known values. This mean that in Correa et al., (2020) the agents are only tasked with identifying which of the two values the quality takes, whereas in our paper the relevant parameter (in the sequel refrred to as the trustworthiness) may take any value in the unit interval. Fu and Le Riche (Fu & Le Riche, 2021) incorporate a similar learning problem into an endogenous growth market model in which the agents compare a new product of unknown quality to an old one of known quality, leading to the outcome that there are attainable equilibria in which the product’s true quality remains unknown.

There also exists a body of literature focused on conditions which ensure social learning. In these papers agents typically receive only one signal and take sequential actions which are observed by some or all of the remaining agents. In the case of Banerjee and Fudenberg (Banerjee & Fudenberg, 2004), the observation structure takes the form of a representative sample while the population is a continuum. Acemoglu et al., (2011) impose a network structure to determine which actions are observed by agents. For an overview of models in which agents are represented by a fully connected graph, the interested reader is referred to (Bikhchandani et al., 1998).

To the best of our knowledge, the following relevant setup has not been studied in the social network learning literature: A model in which agents act myopically rational, observe their neighbors actions or signals, and have private signals which are conditional on the actions taken. This kind of model also relates to the social network trust literature, considering that trust relationships require a dependency between signals and actions.

1.1.2. Social network trust

Social network trust typically concerns the behavior of pairs of truster and trustee agents playing an iterated trust game. The interest here lies in the likelihood of trust breaking down and describing the rational strategies of the truster and the trustee. Particularly relevant to our work are studies in which there is a population of trustworthy trustee agents who always honor trust, and opportunistic trustee agents who play strategically. In such models, it is a natural extension to have agents learn about the proportion of trustworthy trustee agents.

Important examples of this literature stream have been presented in (Bower et al., 1996; Buskens, 2003); (Frey et al., 2015). Bower et al., (1996) describe equilibrium strategies for a sequence of 2-round trust games in which a new pair of truster and trustee agents is paired to play two rounds of the trust game. Additionally to learning about the true portion of trustworthy trustees, there is learning from the first round to the second round within a match-up between truster and trustee agent. Buskens (Buskens, 2003) extends this analysis by considering a pair of truster agents that are both in a relationship to the same trustee. The main finding is that trust placement and honoring (in equilibrium strategies) increase with the probability of sharing information between truster agents, only if both agents are sharing information with a high probability. These findings have been corroborated by experimental work by (Buskens et al., 2010). Frey et al., (2015) extend the theoretical work further by letting the link between truster agents be bought at a cost. The authors determine which game parameters include and exclude investments in such a connection for equilibrium strategies.

In contrast to this stream of literature, we consider a trustee agent such as an institution whose behavior is not modeled strategically, and who interacts with more than one truster agents. Following e.g. (Buskens, 2003; Buskens et al., 2010; Frey et al., 2015), one trustee interacting with two trusters is a natural setting to consider.

1.1.3. Relevance and perspective

In this section we elaborate on the relevance of the two different streams of literature on the model we present. Thematically we are based in the social network trust literature, while methodologically our approach bears resemblance to those used in social network learning. We first present the ideas in both streams and subsequently how our work fits into this landscape.

The social network trust literature offers detailed descriptions of the interactions between strategic trusters and one of two types of trustees, e.g. “friendly” trustees who always honor trust and “strategic” trustees who try to “fool” the truster in order to be able to abuse trust at some point (cf (Bower et al., 1996; Buskens, 2003; Frey et al., 2015). The proportion of “friendly” trustees may be known (as in (Buskens, 2003; Frey et al., 2015)) or unknown (as in (Bower et al., 1996).

The trusters are learning about what type of trustee they are interacting with, and only sometimes also learning about the proportion of “friendly” trustees. As soon as a trustee abuses trust once in the peer-to-peer setting, they reveal that they are not to be trusted. A prevalent example in this literature is that of people buying a second hand car (cf. Buskens [4, Ch.6]) from a loose acquaintance or via an online peer-to-peer service, or that of hiring an informal house sitter without a contract, both situations in which the truster and the trustee are peers in some way.

In the social network learning literature, there is an environment which creates an ordering on the actions the agents may take and emits a signal. The agents take actions and receive a signal. They use the signal to update their belief about the environment with the goal of taking the best action. It is social learning in the sense that the agents communicate with one another about their actions and/or signals. The modelers in turn are interested in the probability of learning the best action as a group (cf (Bala & Goyal, 1998; Correa et al., 2020; Fu & Le Riche, 2021) and the speed of convergence to this best action under different forms of communication (cf (Harel et al., 2021; Huang et al., 2021). The social network learning results are applicable to situations in which agents are consistently updating their belief and trying to take optimal actions. One can think of prediction in markets (and setting the corresponding price to maximize profit), adoption of opinions (trying to fit in the group), and the dissemination of information (keeping up to date with the latest information).

We are motivated by the question of trust in institutions and the influence of peer-to-peer communication thereupon. Thus the social network trust literature paints the thematic backdrop for our work. We are interested in the event that trust is lost, primarily in the “asymmetric” case of trusters interacting with institutions. In this setting it is less natural to incorporate the strategic behavior of the trustee: they are not actively taking part in the interactions, but rather “passively” providing a service. Instead of modeling trustee behavior as strategic, we let them simply draw an action from a distribution.

We investigated the social network learning literature because of the technical similarity between our model and the models found therein. Our goal was to look for work that handles our setting (possibly under different nomenclature). In the process we observe that our model also fills a gap in the social network learning literature. Furthermore, we are encouraged in our decision to compare the effects of different forms of communication on outcomes. In the social network trust literature, the agents are assumed to always communicate their experiences or their belief distribution fully. One can argue, though, that communication of experiences does not necessarily provide the right perspective: typically actions are readily observable, while the internal belief or personal experience may not always be shared. This has motivated us to compare the two.

The resulting modeling framework naturally applies to the setting we are interested in: individuals trusting institutions in the context of peer-to-peer influence.

1.2. Contribution

Our model aims to fill the above mentioned gaps in the literature. In order to model the effect of peer-to-peer communication on the dynamics of a trust problem between an individual and an institution, we draw from both streams of literature. Learning signals are dependent on the actions taken, following the social network trust literature. We follow the social network learning literature by implementing myopically rational decisions and a Bayesian learning procedure. We study two of the communication models between agents seen in both streams related literature. In the first (“observable rewards”) agents disclose their experience with the institution with one another, while in the second (“observable actions”) agents merely witness the actions of their neighbor, i.e., placing or not placing trust. In both models of communication, we describe rational usage of the information in updating beliefs. The extent of the rationality plays a role as a benchmark rather than a description of actual human behavior. The computations involved become complicated quickly, but provide a useful indication of perfectly rational information usage in terms of belief updating.

We impose no assumptions on the motivations of the trustee, who simply acts honorably at some probability $\mathit{\theta }\in (0,1)$, thus generalising the setting where this probability takes one of two values as in Correa et al., (2020). We are interested in a setup in which truster agents interact with institutions whose behavior cannot be modeled strategically at the level of individual interactions, unlike Bower et al., (1996), Buskens, (2003), Frey et al., (2015) and Kolb & Madsen, (2022). We also consider a more general version of the communication seen in Buskens, (2003) Frey et al., (2015), in which agents have access to their neighbors’ actions and the outcomes thereof.

We describe the information usage and myopic decision making without depending on signals independent of the actions taken as seen in Harel et al., (2021;), Huang et al., (2021), Molavi et al., (2018). The asymmetry of the actions, natural to the trust problem, means that incomplete learning is possible. Thus we pay attention to the probability of convergence to the truth rather than only the rate thereof.

In our work we analyze the dynamics of the single agent case using techniques from the field of random walk theory. Such analytic techniques, however, do not extend to two agent models, so that we analyze these relying on Monte Carlo simulation. As the numerical integrations required for agents to interpret each others actions take a prohibitive amount of computation time on personal computing machines, we have to use the Lisa cluster of the computing facility SURFsara.

We observe that two agents with communication between them tend to make the “correct” decision sooner. In other words: typically, sample paths in which the relationship helps the agents outweigh the sample paths in which “bad luck” for one agent implies “bad luck” for both due to the communication between them.

Our experiments reveal that the observable rewards model is not always “better” than the observable actions model. Which mode is most helpful to the agents depends on whether one is interested in making the correct decision in the long run or in being sure to end a relationship with an untrustworthy institution as quickly as possible. Moreover, we identify a parameter setting in which the probability of quitting is lower in the observable actions model than in the observable rewards model. This means that, contrary to what one might expect, having less information available might be beneficial to the agents, and there is no monotone ordering between the two models.

1.3. Organisation of paper

In $\mathcal{\S }$2 we describe basic elements of the model along with relevant interpretations. Thereafter, in $\mathcal{\S }$3 the model is described further and analyzed in the case of a single agent (with proofs being provided in Appendix A). Here we pay special attention to subcases which allow for analytic results regarding the probabilities of such a trust relationship ceasing. In $\mathcal{\S }$4 we formulate the two models for two agents in the trust relationship with the same institution: observable rewards and observable actions. In $\mathcal{\S }$5 we discuss the experimental setup used to investigate the two agent model and in $\mathcal{\S }$6 we discuss the results of this experimentation. We conclude this paper with a discussion on the implications of the results and their relevance to the present literature in $\mathcal{\S }$7.

2. Model for a single agent

In this section we present the model for a single agent, which forms the core of the two agents models discussed in later sections. We consider the situation in which an agent has repeated opportunities to place trust in a institution. The institution’s behavior is modeled by a single parameter $\mathit{\vartheta }\in [0,1]$, the true trustworthiness, defined as the probability at which trust is honoured (so that its complement $1-\mathit{\vartheta }$ is the probability that trust is abused). If trust is not placed, then the institution has no action to take. This behavior can be interpreted as the efficacy of the institution honoring trust, implicitly assuming that this is what they are attempting in each round. Note that we acknowledge the high level of abstraction taken in regard to the institution and that interest is mainly in the agent’s behavior in such a situation. In each round $\mathit{t}\in \mathbb{N}$ the agent chooses an action from the action set $\mathcal{A}=\{0,1\}$ in which $A_t=1$ indicates that the agent places trust in round $\mathit{t}$ while $A_t=0$ indicates that the agent quits the trust relationship. We define the random variable $X_t$, $\mathrm{\forall t}\in \mathbb{N}$, indicating whether the institution’s action in round $\mathit{t}$ is (would be) that of honouring or abusing any trust that may or may not have been placed:

(1) $X_t=\left(\begin{array}{c}1,\mathrm{with}\mathrm{probability}\mathit{\vartheta }\\ 0,\mathrm{with}\mathrm{probability}1-\mathit{\vartheta }.\end{array}\right.$(1)

The random variables $\{X_t:\mathit{t}\in \mathbb{N}\}$ are i.i.d., in particular independent of the agent’s actions. Importantly, the agent only observes $X_t$ in rounds when $A_t=1$. If trust is honoured the agent gains utility $\mathit{r}$ (reward), while if trust is abused the utility gain is $-\mathit{c}$ (cost). We assume that $\mathit{c}$ and $\mathit{r}$ are positive integers; note that if $\mathit{r},\mathit{c}\in \mathbb{Q}$ we simply multiply both by the product of their denominators to get integers. The utility for the agent placing trust in the institution is $\mathit{r}{X}_t{\mathbf{1}}_{\{A_t=1\}}-\mathit{c}(1-X_t{\mathbf{1}}_{\{A_t=1\}})$, so that the expected utility is $\mathit{r\vartheta }-\mathit{c}(1-\mathit{\vartheta })$. As is widely adopted in the learning literature (see, e.g., (Sebenius & Geanakoplos, 1983; Parikh & Krasucki, 1990; Bala & Goyal, 1998; Keppo et al., 2008; Harel et al., 2021)), we let the agent act with myopic rationality. This means that in every round $\mathit{t}$ they only consider the expected utility of the immediate action, and they do not take into account the possible utility of actions taken in rounds $\mathit{t}+1,\mathit{t}+2,\dots$ or the utility of information gained by taking action $A_t=1$. The agent places trust if they believe the utility to have a nonnegative expected value. Furthermore the agent starts with a Beta distributed prior belief $P_0$ with parameters $\mathit{\alpha }$, $\mathit{\beta }\in \mathbb{N}$, such that they initially believe the probability density of $\mathit{\vartheta }$ is given by

(2) $P_0(\mathit{\theta })=\mathit{B}(\mathit{\theta };\mathit{\alpha },\mathit{\beta }):=\frac{{\mathit{\theta }}^{\mathit{\alpha }-1}{(1-\mathit{\theta })}^{\mathit{\beta }-1}}{{\int }_0^1{y}^{\mathit{\alpha }-1}{(1-\mathit{y})}^{\mathit{\beta }-1}\mathrm{d}\mathit{y}},\mathit{\theta }\in [0,1].$(2)

The initial estimate of the expectation of $\mathit{\theta }$ is ${\hat{\vartheta }}_0=\mathbb{E}[\mathit{B}(\mathit{\alpha },\mathit{\beta })]=\mathit{\alpha }/(\mathit{\alpha }+\mathit{\beta })$. As more information becomes available (trust is placed and subsequently honored or abused during rounds $\mathit{t}>0$) the agent updates this belief distribution in a Bayesian fashion. Let

(3) ${\hat{S}}_{\mathit{t}}=\sum _{\mathit{s}=1}^t{X}_s{\mathbf{1}}_{\{A_s=1\}},$(3)

be the number of times that the agent observes that trust was honored until time $\mathit{t}$. Similarly let

(4) ${\hat{F}}_{\mathit{t}}=\sum _{\mathit{s}=1}^t(1-X_s){\mathbf{1}}_{\{A_s=1\}},$(4)

be the number of times that the agent observes that trust was abused until time $\mathit{t}$. The belief distribution held by the agent at the end of round $\mathit{t}$ is then found by applying Bayes’ rule, so as to obtain

(5) $P_t(\mathit{\theta })=\frac{{\theta }^{{\hat{S}}_{\mathit{t}}}{(1-\mathit{\theta })}^{{\hat{F}}_{\mathit{t}}}{P}_0(\mathit{\theta })}{{\int }_0^1{y}^{{\hat{S}}_{\mathit{t}}}{(1-\mathit{y})}^{{\hat{F}}_{\mathit{t}}}{P}_0(\mathit{y})\mathrm{d}\mathit{y}},\mathit{\theta }\in [0,1].$(5)

Denoting the estimated trustworthiness at time $\mathit{t}$ by

(6) ${\hat{\vartheta }}_{\mathit{t}}={\mathbb{E}}_{\theta \sim P_t(\mathit{\theta })}[\mathit{\theta }]=\frac{\alpha +{\hat{S}}_{\mathit{t}}}{\mathit{\alpha }+\mathit{\beta }+{\hat{S}}_{\mathit{t}}+{\hat{F}}_{\mathit{t}}},\mathrm{\forall t}\in \mathbb{N},$(6)

then for $\mathit{t}=1,2,\dots$ we have:

(7) $A_t=\left(\begin{array}{cc}1,& \mathrm{if}\mathit{r}{\hat{\vartheta }}_{\mathit{n}}-\mathit{c}(1-{\hat{\vartheta }}_{\mathit{n}})\ge 0,\mathrm{\forall n}\in \{0,1,\dots ,\mathit{t}-1\},\\ 0,& \mathrm{otherwise}.\end{array}\right.$(7)

2.1. Quantities of interest

We are interested in the agent’s quitting which happens when they stop placing trust: $\{A_t=0\}$. The random variable $\mathit{\tau }$ denotes the number of rounds in which trust was placed until the first `do not place trust’ action:

(8) $\mathit{\tau }:=inf\{\mathit{t}\in \mathbb{N}\cup \{\mathrm{\infty }\}:\mathit{r}{\hat{\vartheta }}_{\mathit{t}}-\mathit{c}(1-{\hat{\vartheta }}_{\mathit{t}})<0\}.$(8)

By the definition of $\mathit{\tau }$ in (8) and $A_t$ in (7), we have

(9) $A_t=1,\mathrm{\forall t}\le \mathit{\tau }\mathrm{and}{A}_t=0,\mathrm{\forall }\mathit{t}>\mathit{\tau }.$(9)

We thus note that ${\hat{\vartheta }}_{\mathit{t}}={\hat{\vartheta }}_{\mathit{\tau }}$ for all $\mathit{t}>\mathit{\tau }$, which arises quite naturally from the model dynamics considering that once the agent has taken the action to not place trust, they also do not get to observe the outcome of the action and thus do not adjust their estimate. This links the agent’s actions to the estimate and vice-versa. An institution can have a true trustworthiness $\mathit{\vartheta }$ such that $\mathit{r\vartheta }-\mathit{c}(1-\mathit{\vartheta })>0$, implying that an agent aware of the true value of $\mathit{\vartheta }$ would place trust forever. It is also possible in such cases that the agent’s estimate $\hat{\vartheta }$ continues to adhere to the condition in (7) which means that they never lose trust. In such cases quitting is not a given and a particularly interesting probability to study is that of the event $\{\mathit{\tau }<\mathrm{\infty }\}$. We denote this by

(10) $p_{\mathrm{quit}}:=\mathbb{P}(\mathit{\tau }<\mathrm{\infty }).$(10)

Furthermore, we are interested in the expected time spent in the system before quitting conditioned on quitting:

(11) $\mathit{q}:=\mathbb{E}[\mathit{\tau }|\mathit{\tau }<\mathrm{\infty }].$(11)

We now recapitulate the dynamics with reference to the graphical representation in Figure 1(a). The two elements determining the agent’s belief distribution at the end of time $\mathit{t}$ are their prior $P_0$ as well as the history of their interaction contained in the random variables ${\hat{S}}_{\mathit{t}}$ and ${\hat{F}}_{\mathit{t}}$. Together these result in a trustworthiness estimation of the institution ${\hat{\vartheta }}_{\mathit{t}}$ by which the agent makes the choice to either place trust or to quit the trust relationship in round $\mathit{t}+1$. If the agent places trust (thus continuing the process) there is a response from the institution of either honouring the trust (at probability $\mathit{\vartheta }$) or abusing the trust (at probability $1-\mathit{\vartheta }$). Note that all of the model’s randomness stems from the response of the institution.

Figure 1. (a) the single agent learner model illustrated conceptually and (b) two example paths of the model dynamics in the random walk interpretation. $\mathcal{B}(\mathit{\vartheta })$ denoting the Bernoulli distribution with parameter $\mathit{\vartheta }$.

3. Trusting alone

In this section we analyze the dynamics of the single agent model described in $\mathcal{\S }$2. Our interest lies in the probability of quitting $p_{\mathrm{quit}}$ and the expected time to quitting $\mathit{q}$ (conditioned on this occurrence). We consider the dynamics that unfold between a single agent who periodically places (or doesn’t place) trust in the institution. We start this section in $\mathcal{\S }$3.1 with a re-interpretation of the model in terms of a random walk with an absorbing barrier and (potentially asymmetric) step sizes as well as (potentially asymmetric) step probabilities. Subsequently in $\mathcal{\S }$3.2 we analyze this random walk model for a number of special cases, focusing on the evaluation of the metrics $p_{\mathrm{quit}}$ and $\mathit{q}$.

3.1. Model re-interpretation

We interpret the model described in $\mathcal{\S }$2 as a 1-dimensional random walk with step sizes $-\mathit{r}$ and $+\mathit{c}$ at probabilities $\mathit{\vartheta }\in (0,1)$ and $1-\mathit{\vartheta }$ respectively. For an agent holding an initial Beta belief distribution with parameters $\mathit{\alpha }$ and $\mathit{\beta }$ the decision criteria becomes

(12) $\mathit{r}\left(\frac{\alpha +{\hat{S}}_{\mathit{t}}}{\mathit{\alpha }+\mathit{\beta }+{\hat{S}}_{\mathit{t}}+{\hat{F}}_{\mathit{t}}}\right)-\mathit{c}\left(1-\frac{\alpha +{\hat{S}}_{\mathit{t}}}{\mathit{\alpha }+\mathit{\beta }+{\hat{S}}_{\mathit{t}}+{\hat{F}}_{\mathit{t}}}\right)\ge 0.$(12)

By rearrangement we find that in order to place trust the agent needs

(13) $\mathit{c}{\hat{F}}_{\mathit{t}}-\mathit{r}{\hat{S}}_{\mathit{t}}\le \mathit{r\alpha }-\mathit{c\beta },$(13)

which we interpret as a random walk

(14) $Z_t:=\mathit{c}{\hat{F}}_{\mathit{t}}-\mathit{r}{\hat{S}}_{\mathit{t}},\mathrm{for}\mathit{t}=0,1,\dots ,$(14)

starting at $Z_0=0$, and having an absorbing barrier at the “critical” position

(15) $u_{\mathrm{crit}}(\mathit{\alpha },\mathit{\beta },\mathit{c},\mathit{r}):=\mathit{r\alpha }-\mathit{c\beta }+1.$(15)

Note that different values of $\mathit{\alpha }$ and $\mathit{\beta }$ correspond to the same behavior in terms of $p_{\mathrm{quit}}$ and $q_{\mathrm{quit}}$ as long as the value of $u_{\mathrm{crit}}$ is the same. The additional unit in (15) is due to the decision criteria to place trust including equality (the expected reward must be zero or more) while the formulation of an absorbing barrier describes the first position for which trust is not placed. Note that all the influence of the prior is contained in $u_{\mathrm{crit}}$. This also allows us to reformulate the probability of quitting as

(16) $p_{\mathrm{quit}}\equiv p_{\mathrm{quit}}(\mathit{\alpha },\mathit{\beta },\mathit{c},\mathit{r},\mathit{\vartheta })=\mathbb{P}(\mathrm{\exists }\mathit{t}:Z_t\ge u_{\mathrm{crit}}),$(16)

the probability of the random walking reaching the absorbing barrier. For ease of reference we refer to $\mathit{c}/(\mathit{c}+\mathit{r})$ as ${\theta }_{\mathrm{crit}}$ because an equivalent condition to $Z_t\ge u_{\mathrm{crit}}$ is that $\hat{\vartheta }<{\theta }_{\mathrm{crit}}$. For illustrative purposes we provide two sample paths that such a random walk might take in Figure 1(b).

Example 1

(Random walk interpretation). Consider the case with an absorbing barrier at $u_{\mathrm{c}rit}=5$ arising from parameter values $\mathit{c}=2$, $\mathit{r}=1$, $\mathit{\alpha }=8$ and $\mathit{\beta }=2$. We interpret this as having a critical trustworthiness ${\theta }_{\mathrm{crit}}=2/3=0.67$. The first such path (depicted as a solid arrows) hits the absorbing barrier at $\mathit{t}=9$, while the second path (depicted as dotted arrows) does not hit the absorbing barrier in the time steps shown and so has $\mathit{\tau }>10$. The values of ${\hat{S}}_{\mathit{t}}$ and $Z_t$ are shown in Table 2 along with the respective estimated trustworthiness ${\hat{\vartheta }}_{\mathit{t}}$. $\mathrm{♣}$

Table 2. Sample paths and agent beliefs as random walks.

(A) Solid line in Figure 1(b).
$\mathit{t}$	0	1	2	3	4	5	6	7	8	9
${\hat{S}}_{\mathit{t}}$	0	1	1	1	2	2	3	4	4	4
$Z_t$	0	−1	1	3	2	4	3	2	4	6
${\hat{\vartheta }}_{\mathit{t}}$	0.8	0.82	0.75	0.69	0.71	0.67	0.69	0.71	0.67	0.63
(B) Dotted line in Figure 1b.
$\mathit{t}$	0	1	2	3	4	5	6	7	8	9	10	11
${\hat{S}}_{\mathit{t}}$	0	0	1	2	3	3	4	5	6	7	7	8
$Z_t$	0	2	1	0	−1	1	0	−1	−2	−3	−1	−2
${\hat{\vartheta }}_{\mathit{t}}$	0.8	0.73	0.75	0.77	0.79	0.73	0.75	0.76	0.78	0.79	0.75	0.76

Regarding the probability of quitting $p_{\mathrm{quit}}$, we consider first the case where $\mathit{\vartheta }<\mathit{c}/(\mathit{c}+\mathit{r})$. Note that in this case the actual expected utility of placing trust is negative and that the process $Z_t$ has a drift toward the absorbing barrier, meaning that this will be reached eventually with probability 1.

Lemma 1

(Guaranteed quitting). If $\mathit{\vartheta }<\mathit{c}/(\mathit{c}+\mathit{r})$, then $p_{\mathrm{quit}}=1,$ and $\mathit{\tau }<\mathrm{\infty }.$

The proof (given in Appendix A) heavily relies on the law of large numbers. The rest of our investigation in this section takes place in the (more interesting) case where $\mathit{\vartheta }>{\theta }_{\mathrm{crit}}$. This case is particularly relevant as it represents cases in which the optimal action for the agent would be to place trust indefinitely yet they do not necessarily do so. The random walk is investigated for general absorbing barriers $\mathit{u}\ge 0$ in order to find recurrence relationships between $p_{\mathrm{quit}}$ for different values of $\mathit{u}$. We extract the probability of quitting the trust relationship and of the time at which this occurs by using the appropriate $\mathit{u}$ for the parameter values in question.

Considering the model thus far described, an agent either continues to place trust indefinitely and learns the true value of the trustworthiness or quits at some time $\mathit{t}<\mathrm{\infty }$.

Lemma 2

(Converge or quit). A single agent partaking in the trust relationship described with $\mathit{\vartheta }>{\theta }_{\mathit{crit}}$ either (A) quits at some time $\mathit{\tau }<\mathrm{\infty }$, i.e.,

(17) $A_t=1,\mathrm{\forall }\mathit{t}<\mathit{\tau },\mathit{and}{A}_t=0,\mathrm{\forall t}\ge \mathit{\tau },$(17)

or (B) continues to place trust indefinitely and has their estimate converge to the true trustworthiness, i.e.,

(18) $A_t=1,\mathrm{\forall t}=1,2,\dots ,\mathit{and}\underset{\mathit{t}\to \mathrm{\infty }}{lim}{\hat{\vartheta }}_{\mathit{t}}=\mathit{\vartheta }.$(18)

This entails that

(19) $\mathbb{P}(\hat{\vartheta }/\to \mathit{\vartheta }\text{amp}\mathit{\tau }=\mathrm{\infty })=0.$(19)

The proof follows from the definition of the estimated trustworthiness value and the law of large numbers and is found inAppendix A.

3.2. Results

In this subsection we study the probability of the agent quitting the trust relationship. We achieve this by finding the absorption probability of the random walk at some level $\mathit{u}$. We define by $\mathit{\pi }(\mathit{u})$ the probability of hitting an absorbing barrier at $\mathit{u}$ in terms of the distance from the starting point $Z_t=0$ to this absorbing barrier at $\mathit{u}\ge 0$:

(20) $\mathit{\pi }(\mathit{u})\equiv \mathit{\pi }(\mathit{u},\mathit{c},\mathit{r},\mathit{\vartheta }):=\mathbb{P}(\mathrm{\exists }\mathit{t}:Z_t\ge \mathit{u}),\mathrm{\forall u}\in {\mathbb{N}}_0.$(20)

Here we suppress the dependence on $\mathit{c},\mathit{r}$ and $\mathit{\vartheta }$ for ease of reading. Note that $\mathit{u}$ can equal $0$, corresponding to a scenario that absorption is certain. The analysis of $\mathit{\pi }(\mathit{u})$ is divided into three cases with respect to $\mathit{r}$ and $\mathit{c}$. Two of the cases we characterise analytically, while we present a numerical approximation for the third. Using $\mathit{u}=u_{\mathrm{crit}}$ gives the probability of quitting, i.e.,

(21) $p_{\mathrm{quit}}(\mathit{\alpha },\mathit{\beta },\mathit{c},\mathit{r},\mathit{\vartheta })=\mathit{\pi }(u_{\mathrm{crit}},\mathit{c},\mathit{r},\mathit{\vartheta }).$(21)

In $\mathcal{\S }$$\mathcal{\S }$3.2.1–3.2.2 we provide the above-mentioned analysis of two cases: 1) $\mathit{r}\in \mathbb{N}$ and $\mathit{c}=1$, and 2) $\mathit{r}=1$ and $\mathit{c}\in \mathbb{N}$. An approximation scheme for the case $\mathit{r},\mathit{c}\in \mathbb{N}$ is presented in Appendix B. The split is a result of the fact that it is not possible to find a closed form result for the general case. The techniques used in the two cases presented also differ substantially and should therefore be viewed separately.

3.2.1. Case $\mathit{c}=1$, and $\mathit{r}\in \mathbb{N}$

In this case the random walk exhibits a useful memorylessness property. To see this, observe that the walk can only go up levels one at a time, while on its way down it can skip levels. We therefore first define the probability of the random walking climbing by one level for some $\mathit{t}\in \mathbb{N}$:

(22) $\mathit{\rho }:=\mathbb{P}\left(\mathrm{\exists }\mathit{t}:Z_t\ge 1\right).$(22)

As a consequence of the strong Markov property, we note that the dynamics after attaining 1 level is independent of the history by which this was done. This means that the probability of going up $\mathit{u}$ levels is simply the probability of $\mathit{u}$ times sequentially going up 1 level, so that $\mathit{\pi }(\mathit{u})={\rho }^u$. A formal expression of the probability of quitting, including a characterization of $\mathit{\rho }$, is presented in the following lemma which is proved in Appendix A.

Lemma 3

(Quitting probability when $\mathit{c}=1$, and $\mathit{r}\in \mathbb{N}$). Suppose $\mathit{\vartheta }\ge 1/(1+\mathit{r})$. The probability of the corresponding random walk with parameter values $\mathit{c}=1$, $\mathit{r}\in \mathbb{N}$ reaching an absorbing barrier at $\mathit{u}$ satisfies

(23) $\mathit{\pi }(\mathit{u},1,\mathit{r},\mathit{\vartheta })=\mathit{\rho }(\mathit{\vartheta }{)}^u,\mathrm{\forall u}\in {\mathbb{N}}_0,$(23)

in which $\mathit{\rho }(\mathit{\vartheta })=\mathit{\pi }(1,1,\mathit{r},\mathit{\vartheta })$ is the unique solution in the range $[0,1)$ to the equation

(24) $\mathit{\rho }(\mathit{\vartheta })=(1-\mathit{\vartheta })+\mathit{\vartheta \rho }(\mathit{\vartheta }{)}^{\mathit{r}+1}.$(24)

To find the probability of quitting we use $u_{\mathrm{c}rit}=\mathit{r\alpha }-\mathit{\beta }+1$ in

(25) $p_{\mathrm{quit}}(\mathit{\alpha },\mathit{\beta },1,\mathit{r},\mathit{\vartheta })=\mathit{\pi }(u_{\mathrm{crit}},1,\mathit{r},\mathit{\vartheta }).$(25)

In Figure 2 we plot numerical results of the quitting probability denoted $p_{\mathrm{quit}}(\mathit{\alpha },\mathit{\beta },\mathit{c},\mathit{r},\mathit{\vartheta })=\mathit{\pi }(u_{\mathrm{crit}},\mathit{c},\mathit{r},\mathit{\vartheta })$ for the subcases $\mathit{r}=1,2,3$. The Beta prior belief distribution is given by shape parameters $\mathit{\alpha }=\mathit{\beta }=2$. The lines are theoretical results obtained by Lemma 3, and the dots (with confidence intervals) correspond to simulated results which corroborate the analytical results. The shape of the quitting probability is explicable by noting that a lower trustworthiness $\mathit{\vartheta }$ consistently leads to a higher quitting probability. The results of Lemmas 1 and 3 are illustrated as the probability of quitting is unity where $\mathit{\vartheta }<{\theta }_{\mathrm{crit}}$ and follow (25) thereafter.

Figure 2. The probability of a single agent quitting $p_{\mathrm{quit}}(\mathit{\alpha },\mathit{\beta },\mathit{c},\mathit{r},\mathit{\vartheta })$ plotted against $\mathit{\vartheta }$ for different values of $\mathit{r}$ while $\mathit{\alpha }=\mathit{\beta }=2$ and $\mathit{c}=1$. Analytical results are plotted as lines, while simulated results (4 000 iterations) are shown as points with 95% confidence intervals.

We now turn our interest to the expected time an agent spends in the system. Note that the time of quitting $\mathit{\tau }$ in terms of the random walk notation is such that

(26) $\mathit{\tau }(\mathit{u})\equiv \mathit{\tau }(\mathit{u},\mathit{c},\mathit{r},\mathit{\vartheta })=inf\{\mathit{t}\in {\mathbb{N}}_0:Z_t\ge \mathit{u}\}.$(26)

Then we are interested in the expectation of $\mathit{\tau }(\mathit{u})$ conditioned on $\mathbf{1}\{\mathit{\tau }(\mathit{u})<\mathrm{\infty }\}$ which we now know occurs with probability $\mathit{\pi }(\mathit{u})$. Specifically we investigate the quantity

(27) $\mathbb{E}[\mathit{\tau }(\mathit{u})|\mathit{\tau }(\mathit{u})<\mathrm{\infty }]=\frac{\mathbb{E}\left[\mathit{\tau }(\mathit{u})\mathbf{1}\{\tau (u)<\mathrm{\infty }\}\right]}{\mathit{\pi }(\mathit{u})},\mathrm{for}\mathit{u}\in {\mathbb{N}}_0.$(27)

We use the techniques from probability generating functions in order to describe the expectation of the distribution at hand. As such we inspect a number related to the expected time to reach the first level, $\mathit{\tau }(1)$:

(28) $\mathit{\phi }(\mathit{z}):=\mathbb{E}[z^{\mathit{\tau }(1)}\mathbf{1}\{\mathit{\tau }(1)<\mathrm{\infty }\}],\mathit{z}\in [-1,1],$(28)

and note that

(29) $\mathbb{E}[z^{\mathit{\tau }(\mathit{u})}\mathbf{1}\{\mathit{\tau }(\mathit{u})<\mathrm{\infty }\}]=\mathit{\phi }(\mathit{z}{)}^u,\mathrm{\forall u}\in {\mathbb{N}}_0.$(29)

This (again) follows from the fact that all levels $\{1,\dots ,\mathit{u}-1\}$ need to be passed first in order reach $\mathit{u}$. Thus, the expected time to reach the $u^{\mathrm{th}}$ level is simply $\mathit{u}$ times the expected time to reach the first level.

Lemma 4

(Expected time to quitting when $\mathit{c}=1$, and $\mathit{r}\in \mathbb{N}$). If $\mathit{r}\in \mathbb{N}$, $\mathit{c}=1$ and $\mathit{\vartheta }\ge 1/(\mathit{r}+1)$ then for all $\mathit{u}\in \mathbb{N}$, $\mathit{\tau }(\mathit{u})$ satisfies

(30) $\mathbb{E}[\mathit{\tau }(\mathit{u})|\mathit{\tau }(\mathit{u})<\mathrm{\infty }]=\mathit{u}\frac{\mathit{\phi }{\mathit{ }}^{\mathrm{\prime }}(1)}{\mathit{\rho }},\mathrm{\forall u}\in \mathbb{N},$(30)

where $\mathit{\phi }{ }^{\mathrm{\prime }}(\mathit{z})$ is the derivative from below of the function $\mathit{\phi }(\mathit{z})$ which, for any given $\mathit{z}\le |1|$, is the unique solution to

(31) $\mathit{\phi }(\mathit{z})=\mathit{z}(1-\mathit{\vartheta })+\mathit{z\vartheta }\mathit{\phi }(\mathit{z}{)}^{\mathit{r}+1},$(31)

and where $\mathit{\rho }$ is the unique solution to Eqs. (24)(24) $\mathit{\rho }(\mathit{\vartheta })=(1-\mathit{\vartheta })+\mathit{\vartheta \rho }(\mathit{\vartheta }{)}^{\mathit{r}+1}.$(24) . In particular,

(32) $\mathbb{E}[\mathit{\tau }\left(\mathit{u}\right)|\mathit{\tau }\left(\mathit{u}\right)<\mathrm{\infty }]=\frac{\mathit{u\rho }}{\left(1-\mathit{\vartheta }\right)\left(\mathit{r}+1\right)-r\rho },\mathrm{\forall u}\in \mathbb{N}.$(32)

To find the expected quitting time (conditional on quitting), we use $u_{\mathrm{crit}}=\mathit{r\alpha }-\mathit{\beta }+1$ in

(33) $\mathit{q}(\mathit{\alpha },\mathit{\beta },1,\mathit{r},\mathit{\vartheta })=\mathit{\tau }(u_{\mathrm{crit}},1,\mathit{r},\mathit{\vartheta }).$(33)

To find the expected time in the system one starts by solving (24) with the appropriate value of $\mathit{r}$ for the unique solution of $\mathit{\rho }\in \left(0,1\right)$. This in turn is substituted into (32) along with $\mathit{u}=u_{\mathrm{crit}}$, resulting in the desired expression for $\mathit{\tau }(u_{\mathrm{crit}},1,\mathit{r},\mathit{\vartheta })$. For details, see the proof in Appendix A.

We plot numerical results as well as simulated results of the expected time in the system conditioned on the agent’s eventual quitting $\mathit{q}(\mathit{\alpha },\mathit{\beta },1,\mathit{r},\mathit{\vartheta })$ in Figure 3 for the subcases $\mathit{r}=1,2,3$ on a log-linear axis. The Beta prior belief distribution is characterised by shape parameters $\mathit{\alpha }=\mathit{\beta }=2$. We notice that the expected time is increasing toward an asymptote positioned on $\mathit{\vartheta }=1/(1+\mathit{r})={\theta }_{\mathrm{crit}}$. The position of this asymptote is due to the property of the model: quitting is still guaranteed for $\mathit{\vartheta }={\theta }_{\mathrm{crit}}$ but the time it takes has an infinite expectation (cf. the symmetric Bernoulli walk). This initial increase makes sense, as the probability of making a detour away from quitting is increasing with $\mathit{\vartheta }$ though it cannot escape forever in this region. The subsequent decrease as $\mathit{\vartheta }>{\theta }_{\mathrm{crit}}$ results from the fact that, as $\mathit{\vartheta }$ increases, if quitting takes place then it does so earlier. This is because more time in the relationship exposes the agent to more unbiased information which is likely to indicate that the institution is trustworthy.

Figure 3. Expected time in the system conditioned on the agent quitting at some $\mathit{t}<\mathrm{\infty }$ plotted against $\mathit{\vartheta }$ for different values of $\mathit{r}$ while $\mathit{\alpha }=\mathit{\beta }=2$ and $\mathit{c}=1$ on a log-linear axis. Analytical results are plotted as lines, while simulated results are shown as points with 95% confidence intervals.

3.2.2. Case $\mathit{r}=1$, and $\mathit{c}\in \mathbb{N}$

n this case we notice that the walk $Z_t$ loses its memoryless property: steps upward now have size $\mathit{c}\in \mathbb{N}$ and so it is possible that levels are skipped along the way. This means that a relationship of the type $\mathit{\pi }(\mathit{u})={\rho }^u$ no longer holds. We thus see that one has no guarantee of which levels are reached on the way to quitting, and therefore one cannot use the approach that we relied upon in the $\mathit{c}=1$ case. We nevertheless obtain the following result. In this case the appropriate value of $\mathit{u}$ is given by $u_{\mathrm{crit}}=\mathit{\alpha }-\mathit{c\beta }+1$.

Lemma 5

(Quitting probability when $\mathit{r}=1$, and $\mathit{c}\in \mathbb{N}$). If $\mathit{r}=1$, $\mathit{c}\in \mathbb{N}$ and $\mathit{\vartheta }\ge \mathit{c}/(\mathit{c}+1)$, then the probability of quitting is given

(34) $p_{\mathrm{quit}}=\mathit{\pi }(u_{\mathrm{crit}},\mathit{c},1,\mathit{\vartheta }),$(34)

where $\mathit{\pi }(\mathit{u},\mathit{c},1,\mathit{\vartheta })$ satisfies

(35) $\mathit{\pi }(\mathit{u})=\frac{{\mathit{\xi }}^{(\mathit{u})}(0)}{\mathit{u}!},\mathrm{\forall u}\in \mathbb{N};$(35)

here ${\xi }^{(\mathit{u})}(0)$ is the $u^{\mathrm{th}}$ derivative of the function $\mathit{\xi }(\mathit{w})$ in $\mathit{w}=0$, where $\mathit{\xi }(\mathit{w})$ is defined by

(36) $\mathit{\xi }(\mathit{w})=\frac{[(1-\vartheta )(c+1)-1]w^2+[2-(1-\mathit{\vartheta })(\mathit{c}+1)]\mathit{w}-(1-\mathit{\vartheta })w^{\mathit{c}+1}-\mathit{\vartheta }}{(1-\mathit{\vartheta })w^{\mathit{c}+2}-(1-\mathit{\vartheta })w^{\mathit{c}+1}-w^2+(1+\mathit{\vartheta })\mathit{w}-\mathit{\vartheta }},\mathrm{\forall w}\in (-1,1).$(36)

The proof (in Appendix A) uses a translation between the maximum of our random walk on one hand, and the minimum of a random walk with $\mathit{c}=1$ and $\mathit{r}\in \mathbb{N}$ on the other hand.

We observe that $\mathit{\xi }(\mathit{u})$ is of the form $\mathit{P}(\mathit{w})/\mathit{Q}(\mathit{w})$, where $\mathit{P}(\mathit{w})$ and $\mathit{Q}(\mathit{w})$ are polynomials of degree $\mathit{c}+1$ and $\mathit{c}+2$ respectively. We evaluate values for $\mathit{\pi }(\mathit{u})$ by differentiating both sides of $\mathit{Q}(\mathit{w})\mathit{\xi }(\mathit{w})=\mathit{P}(\mathit{w})$, $\mathit{u}$ times and substituting $\mathit{w}=0$ in order to get the expression for $\mathit{\pi }(\mathit{u})$. Here $\mathit{\xi }(\mathit{w})$ can be interpreted as a probability generating function with coefficients $a_u=\mathit{\pi }(\mathit{u})$, which allows calculation of the ruin probabilities by differentiating $\mathit{u}$ times, setting $\mathit{w}=0$ and dividing by $\mathit{u}!$, a standard procedure from the theory of probability generating functions.

The resulting quitting probabilities are plotted in Figure 4 for the first two values of $\mathit{c}$. As expected, quitting remains certain until $\mathit{\vartheta }>\mathit{c}/(\mathit{c}+1)$, the stability condition. The prior belief parameters were chosen in order to ensure that the agent does not quit in round 0. The effect of the prior belief parameters is that $u_{\mathrm{crit}}=3$ for $\mathit{c}=1$ and $u_{\mathrm{crit}}=2$ for $\mathit{c}=2$.

Figure 4. The probability of a single agent quitting plotted against $\mathit{\vartheta }$ for different values of $\mathit{c}$ while $\mathit{\alpha }=3,\mathit{\beta }=1$ and $\mathit{r}=1$. Analytical results are plotted as lines, while simulated results (4 000 iterations) are shown as points with 95% confidence intervals.

As before, we are also interested in the expected time that an agent spends in the system placing trust, conditional on them quitting eventually. Similar hurdles arise here as a result of the potentially larger step in the direction of the barrier, though in this analysis these are handled without much extra machinery. The following lemma expresses the expected time that an agent places trust until quitting conditioned on their quitting eventually.

Lemma 6

(Expected time to quitting when $\mathit{r}=1$, and $\mathit{c}\in \mathbb{N}$). Define

(37) $\mathrm{\Phi }(\mathit{w},\mathit{z}):=\frac{(1-\mathit{\vartheta })\mathit{z}\left(\frac{\mathit{w}-{\mathit{w}}^{\mathit{c}+1}}{1-\mathit{w}}\right)-\mathit{\vartheta z}\bar{\phi }(\mathit{z})}{1-(\mathit{\vartheta z})/\mathit{w}-w^c\mathit{z}(1-\mathit{\vartheta })},\mathit{w}\in (-1,1),$(37)

and let $\mathit{w}(\mathit{z})$, for any given $\mathit{z}\in (-1,1)$, denote the unique solution for $\mathit{w}\in (-1,1)$ in

(38) $w^{\mathit{c}+1}(1-\mathit{\vartheta })\mathit{z}+\mathit{\vartheta z}=\mathit{w},$(38)

and

(39) $\bar{\phi }(\mathit{z}):=\frac{(1-\mathit{\vartheta })\mathit{zw}(\mathit{z})+\mathit{\vartheta z}-\mathit{w}(\mathit{z})}{1-\mathit{w}(\mathit{z})}.$(39)

Suppose $\mathit{\vartheta }>\mathit{c}/(\mathit{c}+1)$ and the utility parameters of the agent are given by $\mathit{c}\in \mathbb{N}$ and $\mathit{r}=1$. The expected time for the corresponding random walk to hit an absorbing barrier at $\mathit{u}$ satisfies

(40) $\mathbb{E}[\mathit{\tau }(\mathit{u})|\mathit{\tau }(\mathit{u})<\mathrm{\infty }]=\frac{\mathit{\phi }{\mathit{ }}^{\mathrm{\prime }}(1,\mathit{u})}{\mathit{\phi }(1,\mathit{u})},$(40)

where

(41) $\mathit{\phi }(\mathit{z},\mathit{u}):={\left.\frac{{\mathrm{\partial }}^u}{\mathrm{\partial }{w}^u}\frac{\mathrm{\Phi }(\mathit{w},\mathit{z})}{\mathit{u}!}\right|}_{\mathit{w}=0}.$(41)

In order to fully express $\mathrm{\Phi }$ in terms of only $\mathit{w},\mathit{\vartheta },$ and $\mathit{z}$, one first uses the fact that a root of the denominator is necessarily also a root of the numerator (to keep $\mathrm{\Phi }(\mathit{w},\mathit{z})$ bounded), and using it to express $\mathit{\phi }(\mathit{z},1)$ in terms of $\mathit{z}$. The proof, presented in Appendix A, makes use of the object $\mathit{\phi }(\mathit{z},\mathit{u}):=\mathbb{E}[z^{\mathit{\tau }(\mathit{u})}\mathbf{1}\{\mathit{\tau }(\mathit{u})<\mathrm{\infty }\}]$ which is then used in the generating function

(42) $\mathrm{\Phi }(\mathit{w},\mathit{z}):=\sum _{\mathit{u}=1}^{\mathrm{\infty }}{w}^u\mathit{\phi }(\mathit{z},\mathit{u})$(42)

for $\mathit{w}\in (-1,1)$ from which we obtain the result.

In Figure 5 we plot numerical results as well as simulated results of the expected time in the system conditioned on the agent’s eventual quitting $\mathit{q}(\mathit{\alpha },\mathit{\beta },\mathit{c},1,\mathit{\vartheta })$. We do so for the instances $\mathit{c}=1,2$ using the Beta prior belief distribution determined by shape parameters $\mathit{\alpha }=3$ and $\mathit{\beta }=1$, i.e., the same parameter choices as the ones used in Figure 4. As in the case with $\mathit{c}=1$ and $\mathit{r}\in \mathbb{N}$, we notice that the expected time is increasing toward an asymptote, which is now positioned at $\mathit{\vartheta }=\mathit{c}/(\mathit{c}+1)$. After this critical $\mathit{\vartheta }$ there is a decrease as failing becomes more centered around the early time steps. More time in the system implies that the agent is exposed to more unbiased experience with the institution, and therefore they have a greater chance of remaining in the trust relationship.

Figure 5. Expected time in the system conditioned on the agent quitting at some $\mathit{t}<\mathrm{\infty }$ plotted against $\mathit{\vartheta }$ for different values of $\mathit{c}$ while $\mathit{\alpha }=3,\mathit{\beta }=1$ and $\mathit{r}=1$. Analytical results are plotted as lines, while simulated results are shown as points with 95% confidence intervals.

4. Trusting together

In this section we extend the model from one truster agent to two truster agents. To this end, we enhance the model with a communication mechanism (between the truster agents, that is) that retains the assumptions of myopically rationality.

We consider agents who communicate as frequently with one another as they make decisions to place or not place trust. The nature of this interaction is considered in two forms. The first is that the agents share completely their own interaction history with each other. The second form is that the agents do not communicate about their outcomes at all, and only observe which action is being taken by the other agent in each round.

4.1. Shared modelling aspects

In both mechanisms we have two agents, each with their own interaction with the institution. As such, we have decisions for each agent per round from the action set $\mathcal{A}=\{0,1{\}}^2$ in which $A_{\mathit{i},\mathit{t}}=1$ if agent $\mathit{i}\in \{1,2\}$ places trust in round $\mathit{t}$. For each round $\mathit{t}$ and agent $\mathit{i}$,

(43) $X_{\mathit{i},\mathit{t}}=\left(\begin{array}{cc}1,& \mathrm{with}\mathrm{probability}\mathit{\vartheta }\\ 0,& \mathrm{with}\mathrm{probability}(1-\mathit{\vartheta }),\end{array}\right.$(43)

where $X_{\mathit{i},\mathit{t}}=1$ indicates that the trust that was placed in that round was subsequently honored by the institution. Similarly to the single learner case the agents use

(44) ${\hat{S}}_{\mathit{i},\mathit{t}}=\sum _{\mathit{n}=0}^t{X}_{\mathit{i},\mathit{n}}{\mathbf{1}}_{\{{\mathit{A}}_{\mathit{i},\mathit{n}}=1\}},$(44)

to keep track of how many times trust was honored to them in rounds when they placed trust. We also define the number of times trust was abused to each agent $\mathit{i}=1,2$ by

(45) ${\hat{F}}_{\mathit{i},\mathit{t}}=\sum _{\mathit{n}=0}^t(1-X_{\mathit{i},\mathit{n}}){\mathbf{1}}_{\{{\mathit{A}}_{\mathit{i},\mathit{n}}=1\}}.$(45)

The agents are equipped with the same Beta prior belief distribution as before. The prior belief distribution (with shape parameters ${\alpha }_i,{\beta }_i\in \mathbb{N}$) for agent $\mathit{i}$ is therefore

(46) $P_{\mathit{i},0}=\mathit{B}(\mathit{\theta };{\alpha }_i,{\beta }_i)=\frac{{\theta }^{{\alpha }_i-1}{(1-\mathit{\theta })}^{{\beta }_i-1}}{{\int }_0^1{y}^{{\alpha }_i-1}{(1-\mathit{y})}^{{\beta }_i-1}\mathrm{d}\mathit{y}},\mathit{\theta }\in [0,1].$(46)

For simplicity, from now on we consider only the case of homogeneous priors (i.e., ${\alpha }_1={\alpha }_2$ and ${\beta }_1={\beta }_2$). It should be noted that one can deal with non-homogeneous priors in the exact same way, where one should specify what each of the agents knows about their neighbor’s prior. One could assume that they know it exactly, or hold some belief distribution on it; evidently, the formulation becomes more tedious but is conceptually straightforward.

As we are only considering the homogeneous case, we drop the subscript on the prior belief distribution, calling it simply $P_0(\cdot )$. Let the information received by agent $\mathit{i}$ in round $\mathit{t}$ be denoted $I_{\mathit{i},\mathit{t}}(\mathit{\theta })$. The precise formulation of this information is a modeling outcome of the choice of communication; see (50) of $\mathcal{\S }$4.2 for the first mechanism, and (59) of $\mathcal{\S }$4.3 for the second mechanism. The resulting belief update rule becomes

(47) $P_{\mathit{i},\mathit{t}}(\mathit{\theta })=\frac{{\theta }^{{\hat{S}}_{\mathit{i},\mathit{t}}}{(1-\mathit{\theta })}^{{\hat{F}}_{\mathit{i},\mathit{t}}}{P}_0(\mathit{\theta })I_{\mathit{i},\mathit{t}}(\mathit{\theta })}{{\int }_0^1{y}^{{\hat{S}}_{\mathit{i},\mathit{t}}}{(1-\mathit{y})}^{{\hat{F}}_{\mathit{i},\mathit{t}}}{P}_0(\mathit{y})I_{\mathit{i},\mathit{t}}(\mathit{y})\mathrm{d}\mathit{y}},\mathit{\theta }\in [0,1].$(47)

The difference between (47) and (2) lies in the presence of the information received by the agent. Subsequently the agents use the mean of their belief distribution as a point estimate for the true trustworthiness of the institution and use this to make the decision whether or not to place trust in round $\mathit{t}+1$. If we define

(48) ${\hat{\vartheta }}_{\mathit{i},\mathit{t}}={\int }_0^1\mathit{\theta }{P}_{\mathit{i},\mathit{t}}(\mathit{\theta })\mathrm{d}\mathit{\theta },\mathit{\theta }\in [0,1],$(48)

i.e., the mean belief held by agent $\mathit{i}$ at the end of round $\mathit{t}$, then agent $\mathit{i}$ takes the action $A_{\mathit{i},\mathit{t}+1}$ in round $\mathit{t}+1$ where

(49) $A_{\mathit{i},\mathit{t}}=\left(\begin{array}{cc}1,& \mathrm{if}\mathit{r}{\hat{\vartheta }}_{\mathit{i},\mathit{n}}-\mathit{c}(1-{\hat{\vartheta }}_{\mathit{i},\mathit{n}})\ge 0,\mathrm{\forall n}\in \{0,1,\dots ,\mathit{t}-1\},\\ 0,& \mathrm{otherwise}.\end{array}\right.$(49)

Note that the condition in (49) can be rewritten as ${\hat{\vartheta }}_{\mathit{i},\mathit{n}}\ge {\theta }_{\mathrm{crit}},\mathrm{\forall }\mathit{n}\in \{0,1,\dots ,\mathit{t}-1\}$, with ${\theta }_{\mathrm{crit}}:=\mathit{c}/(\mathit{c}+\mathit{r})$. In the next two subsections we discuss the two different communication mechanisms in more detail.

4.2. Observable rewards (OR)

This case considers agents that communicate fully with one another about their experiences with the institution. The dynamics of this model are represented graphically in Figure 6. The extension from the single learner model (represented in Figure 1(a) for comparison) to this model consists of the shared information between agents which entails the response of the institution to trust that was placed.

Figure 6. The observable rewards model of learning.

Each agent thus has same information available to them, and under homogeneous prior beliefs they have the same estimate of the true trustworthiness. In this case we define the information agent $\mathit{i}$ receives from their neighbor $\mathit{j}\ne \mathit{i}$ in round $\mathit{t}$,

(50) $\mathit{\theta }\mapsto I_{\mathit{i},\mathit{t}}:={\theta }^{{\hat{S}}_{\mathit{j},\mathit{t}}}(1-\mathit{\theta }{)}^{{\hat{F}}_{\mathit{j},\mathit{t}}},\mathit{\theta }\in [0,1].$(50)

We see that this information contains for each agent exactly the interaction history of the other agent, hence we have the line from agent $\mathit{i}$’s “Trusting” action to agent $\mathit{j}$’s “Belief” distribution in Figure 6

As mentioned, the agents’ estimated values of the true trustworthiness are the same. Thus we can drop the subscript $\mathit{i}$ from ${\hat{\vartheta }}_{\mathit{i},\mathit{t}}$ and note that it is given by

(51) ${\hat{\vartheta }}_{\mathit{t}}=\frac{\alpha +{\sum }_{i=1}^2{\hat{S}}_{\mathit{i},\mathit{t}}}{\mathit{\alpha }+\mathit{\beta }+{\sum }_{i=1}^2{\hat{S}}_{\mathit{i},\mathit{t}}+{\sum }_{i=1}^2{\hat{F}}_{\mathit{i},\mathit{t}}}.$(51)

Defining $Z_t=\mathit{c}{\sum }_{i=1}^2{\hat{F}}_{\mathit{i},\mathit{t}}-\mathit{r}{\sum }_{i=1}^2{\hat{S}}_{\mathit{i},\mathit{t}}$, similarly as in the single-agent model we get a random walk that takes steps

(52) $Z_{\mathit{t}+1}=\left(\begin{array}{cc}{Z}_t+2\mathit{c},& \mathrm{with}\mathrm{probability}{(1-\mathit{\vartheta })}^2\\ Z_t+\mathit{c}-\mathit{r},& \mathrm{with}\mathrm{probability}2\mathit{\vartheta }(1-\mathit{\vartheta })\\ Z_t-2\mathit{r},& \mathrm{with}\mathrm{probability}{\vartheta }^2,\end{array}\right.$(52)

with an absorbing barrier at $\mathit{u}=\mathit{r\alpha }-\mathit{c\beta }+1$.

In the case of heterogeneous priors the formulation of this model does not change dramatically. The fates of the agents would not be tied as before and so there would simply be two random walks defined as in (52), one for each agent. Each random walk would have its own absorbing barrier defined by $u_i=\mathit{r}{\alpha }_i-\mathit{c}{\beta }_i+1$. Recall that ${\alpha }_i,{\beta }_i,\mathit{r}$ and $\mathit{c}$ for all $\mathit{i}$ are taken from $\mathbb{N}$ and so $u_i$ too is an integer for both $\mathit{i}$. If at some time $t_q$ one of the agents quit, then the remaining agent simply continues according to the dynamics of the single agent model with the new transitions defined by

(53) $Z_{\mathit{t}+1}=\left(\begin{array}{cc}{Z}_t+\mathit{c},& \mathrm{with}\mathrm{probability}(1-\mathit{\vartheta })\\ Z_t-\mathit{r},& \mathrm{with}\mathrm{probability}\mathit{\vartheta },\end{array}\right.$(53)

$\mathrm{for}\mathit{t}>t_q.$

4.3. Observable actions (OA)

In this setting the agents do not communicate explicitly by sharing information about their interaction history. Instead, they observe the actions of their neighboring agent, and use this to infer something about the possible histories their neighboring agent may have experienced leading to such a result. This is where subtle intricacies arise, as rational agents need to keep in mind that their own actions are observed by their neighbor and thus will also affect the decisions made by their neighbor.

The extent of the communication between the two agents in this model is binary indicating whether or not trust was placed at the end of round $\mathit{t}$. This information is only incorporated into the agents’ belief distributions for round $\mathit{t}+1$. This model is depicted graphically in Figure 7. The crux of this model is the line from one agents “Action” decision to the other agent’s “Belief” distribution. The action of not placing trust implies that the agent exits the system forever. The quitting agent does not continue to update their belief based on their neighbor’s actions. The information sent by the quitting agent is used as the information received by the not-quitting agent for the remainder of the not-quitting agent’s tenure in the system.

Figure 7. The observable actions model of learning.

The nature of the communication necessitates separate definitions for the information sent with a “yes” signal, when the agent places trust, and for the information sent with a “no” signal, when the agent does not place trust. As such we define for $\mathit{i}=1,2$, $\mathit{j}\ne \mathit{i}$ and $\mathit{t}\in \mathbb{N}$,

(54) $I_{\mathit{i},\mathit{t}}:=\left(\begin{array}{cc}{Y}_{\mathit{i},\mathit{t}}& \mathrm{if}\mathit{t}<{\tau }_j,\\ N_{i,{\tau }_j}& \mathrm{if}\mathit{t}\ge {\tau }_j,\end{array}\right.$(54)

where ${\tau }_j$ is the round in which agent $\mathit{j}$ quits for $\mathit{j}=1,2$. We formally define $Y_{\mathit{i},\mathit{t}}$ and $N_{\mathit{i},\mathit{t}}$ in (59) and (61) respectively. By the rationality assumption, the agents use the binary information along with the knowledge that their own outbound information up until round $\mathit{t}-1$ was used rationally by the other agent. With this they infer a range of possible histories that may have lead to the decision taken by their neighbor.

We define the belief distributions $R_t^i$, formed by taking only the report received by agent $\mathit{i}$ (signal sent at time $\mathit{t}$ by agent $\mathit{j}$ to agent $\mathit{i}$) into account:

(55) $R_t^i=\frac{P_0(\mathit{\theta })I_{\mathit{i},\mathit{t}}(\mathit{\theta })}{{\int }_0^1{P}_0(\mathit{y})I_{\mathit{i},\mathit{t}}(\mathit{y})\mathrm{d}\mathit{y}},\mathit{\theta }\in [0,1].$(55)

We then define the auxiliary belief distribution resulting from a combination of $R_t^i$ and ${\theta }^x(1-\mathit{\theta }{)}^{\mathit{t}-\mathit{x}}$ where $\mathit{x}$ is one of the possible number of positive experiences giving ${\mathcal{D}}_t(\mathit{x})$ the distribution:

(56) ${\mathcal{D}}_t(\mathit{x})=\frac{{\theta }^x{(1-\mathit{\theta })}^{\mathit{t}-\mathit{x}}{R}_t^i}{{\int }_0^1{y}^x{(1-\mathit{y})}^{\mathit{t}-\mathit{x}}{R}_t^i\mathrm{d}\mathit{y}},\mathit{\theta }\in [0,1].$(56)

This allows us to define the set of permitted positions of agent $\mathit{i}$, ${\mathcal{J}}_t^i$ as

(57) ${\mathcal{J}}_t^i=\{\mathit{x}\in \{0,1,\dots ,\mathit{t}\}|{\mathbb{E}}_{\mathit{\theta }\sim {\mathcal{D}}_t(\mathit{x})}[\mathit{\theta }]\ge {\theta }_{\mathrm{crit}}\}.$(57)

We have that ${\mathcal{D}}_t(\mathit{x})$ for all $\mathit{x}\in {\mathcal{J}}_t^i$ are the possible belief distributions held by agent $\mathit{i}$. These possible belief distributions are then summed in $I_{\mathit{j},\mathit{t}}$ weighted according $w_x(\mathit{t}-1)$ the number of ways in which these $\mathit{x}$ positive experiences may have been realised (i.e., the number of histories resulting in $\mathit{x}$ positive experiences by time $\mathit{t}$). Defining $w_x(\mathit{t})$ is done recursively:

(58) $w_x(\mathit{t})=\left(\begin{array}{cc}{w}_x(\mathit{t}-1)+w_{\mathit{x}-1}(\mathit{t}-1)& \mathrm{if}(\mathit{x}-1)\in {\mathcal{J}}_{\mathit{t}-1}\\ w_x(\mathit{t}-1)& \mathrm{if}(\mathit{x}-1)\notin {\mathcal{J}}_{\mathit{t}-1}\\ 1& \mathrm{if}\mathit{x}=\mathit{t}\\ 0& \mathrm{if}\mathit{x}>\mathit{t}\mathrm{or}\mathit{x}\notin {\mathcal{J}}_{\mathit{t}-1},\end{array}\right.\mathrm{for}\mathit{x}\in [0,\mathit{t}]\mathrm{and}\mathit{t}\in \mathbb{N}$(58)

with initial condition $w_x(1)=1$ for all $\mathit{x}\in {\mathcal{J}}_1$. The information received by agent $\mathit{i}$ from agent $\mathit{j}$ considering that agent $\mathit{j}$’s decision was to place trust in round $\mathit{t}$ is thus defined as

(59) $\mathit{\theta }\mapsto Y_{\mathit{i},\mathit{t}}:=\sum _{x\in {\mathcal{J}}_{t-1}^j}{w}_x(\mathit{t}-1){\theta }^x(1-\mathit{\theta }{)}^{\mathit{t}-1-\mathit{x}},\mathit{\theta }\in [0,1].$(59)

We can interpret $w_x(\mathit{t})$ as the number of walks from $(0,0)$ to $(\mathit{x},\mathit{t}-\mathit{x})$ that retain $x_{\tau }\in {\mathcal{J}}_{\tau }$ for $\mathit{\tau }=1,2,\dots ,\mathit{t}$. Figure 8(a) illustrates the permitted walks for a “yes” signal with the boundary between $\mathit{x}$ in and out of ${\mathcal{J}}_t$ being represented as the thick horizontal line.

Figure 8. An illustration of the weighting $w_x(\mathit{t})$ used in the interpretation of the observed action.

In order to define the information sent with a “no” signal, we define the range of values which would result in “yes” signals up until round $\mathit{t}-1$ and a “no” in round $\mathit{t}$:

(60) ${\mathcal{K}}_t^i=[max\{0,\underset{x}{inf}{\mathcal{J}}_{t-1}^i\},\dots ,\underset{x}{inf}{\mathcal{J}}_t^i-1].$(60)

Subsequently, we have the information sent in a “no” signal:

(61) $\mathit{\theta }\mapsto N_{\mathit{i},\mathit{t}}:=\sum _{\mathit{x}\in {\mathcal{K}}_t}{w}_x(\mathit{t}){\theta }^x(1-\mathit{\theta }{)}^{\mathit{t}-1-\mathit{x}},\mathit{\theta }\in [0,1]$(61)

where $w_x(\mathit{t})$ is defined as above. The number of permitted walks to a “no” signal at time $\mathit{t}$ is simply the number of permitted walks to a “yes” signal at time $\mathit{t}-1$ which are also one negative experience away from quitting. This is illustrated in Figure 8(b) with a horizontal line indicating the boundary between $\mathit{x}$ in and out of ${\mathcal{J}}_t$ and the dotted line indicating the boundary between $\mathit{x}$ in and out of ${\mathcal{K}}_t$. Note that the information sent by a “no” signal in subsequent rounds (after the first such signal) does not change, as we allude to in (54).

The agents finally construct an estimate by combining their own information with the signal information:

(62) $P_{\mathit{i},\mathit{t}}(\mathit{\theta })=\frac{{\theta }^{{\hat{S}}_{\mathit{i},\mathit{t}}}{(1-\mathit{\theta })}^{{\hat{F}}_{\mathit{i},\mathit{t}}}{P}_0(\mathit{\theta })I_{\mathit{j},\mathit{t}}(\mathit{\theta })}{{\int }_0^1{y}^{{\hat{S}}_{\mathit{i},\mathit{t}}}{(1-\mathit{y})}^{{\hat{F}}_{\mathit{i},\mathit{t}}}{P}_0(\mathit{y})I_{\mathit{j},\mathit{t}}(\mathit{y})\mathrm{d}\mathit{y}},\mathit{\theta }\in [0,1],$(62)

and place trust in round $\mathit{t}+1$ if ${\mathbb{E}}_{\mathit{\theta }\sim {\mathit{P}}_{\mathit{i},\mathit{t}}(\mathit{\theta })}[\mathit{\theta }]\ge {\theta }_{\mathrm{c}rit}$.

4.4. Illustration of the differences between the models presented

In this subsection we present a set of example experiences for two players. These serve to demonstrate the workings of the dynamics in the OA model and the OR model.

We consider both dual agent models with parameters $\mathit{c}=2$, $\mathit{r}=1$ (and therefore ${\theta }_{\mathrm{crit}}=2/3$) and $u_{\mathrm{crit}}=2$ ($\mathit{\alpha }=5,\mathit{\beta }=2$). Without determining whether or not these interaction outcomes are witnessed or not, let us presume the responses by the institution to the two agents are given for the first four rounds $(X_{\mathit{i},1},X_{\mathit{i},2},X_{\mathit{i},3},X_{\mathit{i},4})=(0,0,1,1)$ for $\mathit{i}=1$, and $(X_{\mathit{i},1},X_{\mathit{i},2},X_{\mathit{i},3},X_{\mathit{i},4})=(1,1,1,0)$ for $\mathit{i}=2$. The resulting random walk interpretations within the single agent model as well as the observable rewards model are depicted in Figure 9.

Figure 9. The random walk interpretation of the or model as well the individual sample paths of the respective agents.

We observe that the first agent would have quit in the single agent model after the first interaction, but in the OR model they continue past this point. The OA model is not so easily represented visually and so will require more elucidation. We offer Table 3 summarizing the variables over time for the single agent models for agent 1 and 2 respectively as well as in the dual agent models. Note that both agents 1 and 2 have the same estimate throughout in the OR model, due to the fact that they start with the same prior and are privy to the same information each round. The first agent’s estimate stops changing after the first round, as a result of their quitting which is the case in the single agent model as well as in the OA dual agent model. This is because by the time they take the decision to not place trust, they have not yet received a signal containing information from agent 2. The reactions of the institution $X_{\mathit{i},\mathit{t}}$ for $\mathit{i}=1$ and $\mathit{t}\ge 2$ are thus not witnessed and not used to update agent 1’s belief.

Table 3. Estimates, ${\hat{\vartheta }}_{\mathit{t}}$ in the respective models.

$\mathit{t}$	Agent 1	Agent 2	OR (1)	OR (2)	OA (1)	OA (2)
0	0.714	0.714	0.714	0.714	0.714	0.714
1	0.625	0.750	0.667	0.667	0.625	0.750
2		0.778	0.636	0.636		0.700
3		0.800				0.727
4		0.727				0.667

In the first round of the OA model both agents behave exactly as they would in the single agent model because their first action is non-informative. After this first round, the first agent quits and takes the “not place trust” action in round 2 and all subsequent rounds. This indicates to the second agent that trust was abused in agent 1’s first round. In this case $w_x(1)=1$ (as part of the initial condition of $w_x$) and $I_{2,2}={\theta }^0(1-\mathit{\theta }{)}^1$ while their personal $S_2$ equals $2$, which provides

(63) $P_{2,2}(\mathit{\theta })=\frac{{\theta }^2\cdot {\theta }^{5-1}{(1-\mathit{\theta })}^{2-1}\cdot {(1-\mathit{\theta })}^1}{{\int }_0^1{y}^2\cdot y^{5-1}{(1-\mathit{y})}^{2-1}\cdot {(1-\mathit{y})}^1\mathrm{d}\mathit{y}}.$(63)

The mean of this belief distribution is used as the point estimate; it is

(64) ${\hat{\vartheta }}_2={\int }_0^1\mathit{\theta }\frac{{\theta }^6{(1-\mathit{\theta })}^2}{{\int }_0^1{y}^6{(1-\mathit{y})}^2\mathrm{d}\mathit{y}}\mathrm{d}\mathit{\theta }=\mathrm{0.7.}$(64)

For illustrative purposes we also show the information sent out by agent 2 by their action in round 3, i.e., $I_{1,3}$. Firstly they construct $R_2^2$, the belief formed by taking into account the report received by agent 2 by round 2:

(65) $R_2^2(\mathit{\theta })=\frac{P_0(\mathit{\theta })I_{2,2}(\mathit{\theta })}{{\int }_0^1{P}_0(\mathit{y})I_{2,2}(\mathit{y})\mathrm{d}\mathit{y}}=\frac{{\theta }^4{(1-\mathit{\theta })}^2}{{\int }_0^1{y}^4{(1-\mathit{y})}^2\mathrm{d}\mathit{y}}.$(65)

Subsequently the auxiliary distribution ${\mathcal{D}}_2$ is constructed:

(66) ${\mathcal{D}}_2(\mathit{x})=\frac{{\theta }^x{(1-\mathit{\theta })}^{2-\mathit{x}}{R}_2^2(\mathit{\theta })}{{\int }_0^1{y}^x{(1-\mathit{y})}^{2-\mathit{x}}{R}_2^2(\mathit{y})\mathrm{d}\mathit{y}}=\frac{{\mathit{\theta }}^{\mathit{x}+4}{(1-\mathit{\theta })}^{4-\mathit{x}}}{{\int }_0^1{y}^{\mathit{x}+4}{(1-\mathit{y})}^{4-\mathit{x}}\mathrm{d}\mathit{y}}.$(66)

This distribution is then used to construct the set ${\mathcal{J}}_{t=2}^{i=2}=\{2\}$ considering that this is the only $\mathit{x}$ for which ${\int }_0^1{\mathcal{D}}_2(\mathit{x})\mathrm{d}\mathit{\theta }\ge 2/3$. Considering that $\mathit{x}=\mathit{t}$ we know that $w_2(2)=1$, which means that $I_{1,3}={\theta }^2$.

This example illustrates an information cascade, as the second agent would not quit in the first 4 rounds in any model except for the OR model. The effect for the first agent on the other hand is that instead of quitting in round 1, as in the single agent model as well as in the OA model, they quit in round 2 of the OR model.

5. Setup of simulation experiments

In this section we describe the setup of our simulations experiments. The results of these simulation experiments we present in $\mathcal{\S }$6, while in $\mathcal{\S }$7 we discuss and interpret the results of the experiments. The primary goal of the experiments is to assess the two communication mechanisms in the two agents case, in terms of the probabilities of quitting and the expected quitting times (conditional on quitting).

5.1. Choice of $\mathit{c}$ and $\mathit{r}$

In order to get an idea of how the model parameters influence the probability (and timing) of agents quitting, we choose the following five combinations of $(\mathit{c},\mathit{r})$: $(1,1),(3,2),(2,3),(2,1),(1,2)$, so that $\mathit{c}=\mathit{r}$ is the “base case” and $\mathit{c}=2\mathit{r}$ and $\mathit{r}=2\mathit{c}$ are the “extremal ends.” There are two reasons for choosing this range of ratios. The first consideration is that if the ratio gets pushed further in either direction, then the simulation results become harder to obtain. The critical trustworthiness ${\theta }_{\mathrm{crit}}$ is shifted toward $1$ with an increase of $\mathit{c}$, which means that only a small range of $\mathit{\vartheta }$ have probabilities of quitting less than unity. Within this small range there is a sharp drop of quitting probabilities, because with very large $\mathit{\vartheta }$ the probability of quitting is low as this requires an abuse of trust. In the other extreme, with an increase of $\mathit{r}$, ${\theta }_{\mathrm{crit}}$ decreases, so that increasingly many parameter settings have to run over a relatively long time interval in an already slow simulation. This is because of the large number of numerical integrations required in the observable actions model used in constructing ${\mathcal{J}}_t^i$, ${\mathcal{D}}_t$ and $R_t^i$ for each $\mathit{i}=1,2.$ The second reason is that it is a sufficiently extensive range of cost-to-reward ratios, especially in the context of trust problems.³

5.2. Choice of $\mathit{\alpha }$ and $\mathit{\beta }$

Intuitively, the parameters $\mathit{\alpha }$ and $\mathit{\beta }$ can be considered as a number of interactions with the institution prior to the dynamics we consider, which resulted in trust being honored $\mathit{\alpha }$ times and abused $\mathit{\beta }$ times. This means that the greater $\mathit{\alpha }$ is compared to $\mathit{\beta }$, the more the prior belief distribution of the agents is skewed toward greater values of $\mathit{\vartheta }$. Similarly, for a lower $\mathit{\alpha }$ compared to $\mathit{\beta }$, this prior belief distribution is skewed toward the lower values of $\mathit{\vartheta }$. The prior belief parameter settings $\mathit{\alpha }$ and $\mathit{\beta }$, in combination with the choice of $\mathit{c}$ and $\mathit{r}$, determine the instance’s $u_{\mathrm{crit}}=\mathit{r\alpha }-\mathit{c\beta }+1$. Borrowing from the analogy of the one-dimensional random walk of the single agent model, the value of $u_{\mathrm{crit}}$ represents a “distance” to quitting. If we would like consistency in the interpretation of $u_{\mathrm{crit}}$, we require a conversion: Take $u^{\ast }=u_{\mathrm{crit}}/\mathit{c}$, which is the minimum time in which an agent can quit in the single agent model. There is asymmetry in the model between the cases when $\mathit{c}>\mathit{r}$ and $\mathit{c}<\mathit{r}$. In case $\mathit{c}>\mathit{r}$, it is possible for an agent to experience an honoring of trust in the first round, yet still quit after the second if the step size for an abuse of trust is such that $\mathit{c}>u_{\mathrm{crit}}+\mathit{r}$. In case $\mathit{r}>\mathit{c}$ this scenario is not possible. We choose values of $\mathit{\alpha }$ and $\mathit{\beta }$ such that, in combination with the values of $\mathit{r}$ and $\mathit{c}$, we cover the range $u^{\ast }=1,2,3$. The choice to stop at $u^{\ast }=3$ is made to keep the probability of quitting high enough to facilitate efficient simulation.

5.3. Iterations and simulation length

In our simulation study, it is our goal to produce reliable estimates for the parameters under study. We run each of the models for $4000$ truster agents in total, i.e., the single agent models are run $4000$ times while the dual agent models are run $2000$ times. We run each simulation for a maximum of 500 time steps, with the exception of simulations with $\mathit{\vartheta }\in \{0.84,0.9\}$ in which case we run the model for a maximum time of 200 time steps. These choices were made keeping considerations of confidence interval width in mind as well as execution feasibility. Considering that in the two agent models the dynamics are sped up (as there is more information available per time step), this should be sufficient time also for both two agent models.

In the interactions with highest $\mathit{\vartheta }$ the probability of quitting is very low and concentrated on the first couple of time steps. We can make an educated (pessimistic) estimate as to how many time steps would be required in order to not miss any relevant “probability mass.” We do this relying on Markov’s inequality applied to the expected time to quitting in the single agent model. As an illustration consider the parameter setting with $\mathit{c}=2,\mathit{r}=1,\mathit{\alpha }=5,$ and $\mathit{\beta }=2$ (such that $u^{\ast }=1$) at the parameter setting $\mathit{\vartheta }=0.84$. In the single agent model the expected time to quit is $\mathbb{E}[\mathit{\tau }|\mathit{\tau }<\mathrm{\infty }]\approx 3.17$. By Markov’s inequality we know that the probability of quitting at time $\mathit{\tau }\ge 200$ is bounded as follows:

(67) $\mathbb{P}(\mathit{\tau }\ge 200|\mathit{\tau }<\mathrm{\infty })\le \frac{\mathbb{E}[\tau |\tau <\mathrm{\infty }]}{200}=\mathrm{0.01587.}$(67)

Hence, in the $\mathit{N}$ experiments performed, the expected number of times we miss the quitting event is $\mathit{L}=\mathit{N}\cdot \mathbb{P}(\mathit{\tau }\ge 200|\mathit{\tau }<\mathrm{\infty })\cdot \mathbb{P}(\mathit{\tau }<\mathrm{\infty })$. Taking $\mathit{N}=4000$, we obtain the bound

(68) $\mathit{L}<4000\times 0.01587\times 0.2635=\mathrm{16.727.}$(68)

The single learner case is a conservative benchmark, as there is less communication than in the two agents models, so that we anticipate agents in the two agent models to quit sooner. The extent to which this estimate is conservative, is illustrated by the fact that in the single agent model (for the parameter settings described) the latest quitting occurred at round 45, for the dual agent OA model at round 40, and for the dual agent OR model at round 17.

6. Results of experiments

In this section we present the results of the experiments described in $\mathcal{\S }$5. We plot the results for specific values of $(\mathit{c},\mathit{r})$ and $u^{\ast }$ in Figures 10(a)–15(b): the numerical output of all experiments is given, in a tabulated format, in Appendix D. The plots cover the settings $(\mathit{c},\mathit{r})=(1,1),(2,1),(1,2)$ and $u^{\ast }=1,3$, where we add the synthetic points $p_{\mathrm{quit}}(\mathit{\alpha },\mathit{\beta },\mathit{c},\mathit{r},\mathit{\vartheta }={\theta }_{\mathrm{crit}})=1$ and $p_{\mathrm{quit}}(\mathit{\alpha },\mathit{\beta },\mathit{c},\mathit{r},\mathit{\vartheta }=1)=0$ to the curves shown in the figures in order to show the dynamics beyond the capability of the simulation. To be able to see subtle differences in results, we show the probability of quitting predominantly in the region in which quitting is not guaranteed.

Figure 10. Simulation results for $\mathit{c}=\mathit{r}=1$ in the $u^{\ast }=1$ case.

Figure 11. Simulation results for $\mathit{c}=\mathit{r}=1$ in the $u^{\ast }=3$ case.

Figure 12. Simulation results for $\mathit{c}=1$ and $\mathit{r}=2$ in the $u^{\ast }=1$ case.

Figure 13. Simulation results for $\mathit{c}=1$ and $\mathit{r}=2$ in the $u^{\ast }=3$ case.

Figure 14. Simulation results for $\mathit{c}=2$ and $\mathit{r}=1$ in the $u^{\ast }=1$ case.

Figure 15. Simulation results for $\mathit{c}=2$ and $\mathit{r}=1$ in the $u^{\ast }=3$ case.

6.1. The probability of quitting

The probability of quitting in the regime $\mathit{\vartheta }<{\theta }_{\mathrm{crit}}$ should remain unity: the same and more information is made available to the agents in the dual agent model as in the single agent model. The values of $0.999$ in Tables S5a–S7a could have been remedied at the cost of more simulation effort (i.e., by working with more runs with more time steps).

In the (more interesting) regime $\mathit{\vartheta }>{\theta }_{\mathrm{crit}}$, for most parameter settings the probability of quitting increases from the OR model to the OA model and again to the single agent model. This trend is more pronounced for $u^{\ast }=1$ than for $u^{\ast }=2$ and $u^{\ast }=3$. There are two exceptions to this trend in Tables S6a and S7a: for $\mathit{\vartheta }=0.65,0.66$ at the parameter settings $\mathit{c}=3$ and $\mathit{r}=2$, with $u^{\ast }=2,3$. However, we observed overlap in the original confidence intervals.

To determine whether such a difference bears significance we conducted more simulations for the faster case with $u^{\ast }=2$. For the OR model, being computationally lighter, we conducted a total of 200 000 runs. For the slower OA model, we conducted 40 000 simulations in total. The resulting confidence intervals of the points $\mathit{\vartheta }=0.65,0.66$ are depicted in Figure 16. We observe the, perhaps unexpected, result that the OA model produces a lower probability of quitting in this parameter setting. Hence, the OR model does not always outperform the OA model in terms of the probability of quitting in the regime where $\mathit{\vartheta }>{\theta }_{\mathrm{crit}}$.

Figure 16. The results of extra simulation runs for the probability of quitting. In these runs $\mathit{c}=3$, $\mathit{r}=2$, and $u^{\ast }=2$.

Part of the explanation of the lower probability of quitting in the OA model relates to the timing of the communication, being at the end of a round. In the OR model, as soon as an agent has their own reward, they also observe the reward of their neighbor. This boils down to two pieces of information per round. In the OA model however, the agents only observe the actions of their neighbor at the same time when they have already made their own action for that round. That means that an agent who observes their neighbor not placing trust for the first time in round $\mathit{t}$ can only use the information contained therein to inform their action in round $\mathit{t}+1$. In the meantime, as they have not quit yet in round $\mathit{t}$, they are privy to at least that round’s outcome before they have to make another decision. We elaborate on this effect in Appendix C.

6.2. Expected time to quit

In general (in the regime $\mathit{\vartheta }<{\theta }_{\mathrm{crit}}$) the expected time to quit is lower in the dual agent models than in the single agent model. Furthermore, the OR model tends to have the lowest expected time to quit (i.e., the greatest effect), though with a fair amount of crossing with the OA model. The key exception to this trend occurs at $\mathit{c}=\mathit{r}=1$ with $u^{\ast }=1$, depicted in Figure 10(b). At this setting, the OA model performs better than both others which show a similar time to quitting. Furthermore, there are exceptions at $\mathit{c}=2$, $\mathit{r}=1$ and $u^{\ast }=1$ depicted in Figure 14b, and at $\mathit{c}=3$, $\mathit{r}=2$ and $u^{\ast }=1$ shown in Table S5b.

In the regime $\mathit{\vartheta }<{\theta }_{\mathrm{crit}}$ there is the trend (with exceptions) that quitting in the OR model occurs sooner than in both other models. In the OR model the agents are exposed to two outcomes per round of interaction and so receive twice as much unbiased information than in the single agent model. In this regime, the unbiased information thus received is likely to indicate that the institution is trustworthy and so quitting becomes less likely with more time (i.e., quitting must occur quickly or not at all). This trend is most pronounced when $u^{\ast }=3$ shown in Figures 13(b) and 15(b) as well as in Table S7b.

7. Discussion

In this section we draw conclusions from the numerical experiments, discuss how these findings relate to those in the literature, and present directions for future research.

7.1. General conclusions

In all of the models a similar pattern in the expected time to quitting holds: there is an increase in the expected time to quit as $\mathit{\vartheta }$ increases until ${\theta }_{\mathrm{crit}}$ and a decrease afterward. This specifically entails that a long average tenure of customers at an institution, does not indicate that this institution is to be trusted, or, put differently, there is no way of knowing which side of the critical value they might be on with only this information. An institution can simply have a trustworthiness that is just high enough to keep customers placing their trust in them long enough to get positive net utility from that relationship yet not actually high enough to warrant an indefinite relationship. An institution with many relationships still going and a couple of very short concluded ones on the other hand might be a good indication that the institution is indeed trustworthy.

By comparing the probability of quitting for the same $\mathit{c}$ and $\mathit{r}$ at different $u^{\ast }$ (for example in Figures 10(a) and 11(a) we see that the difference between the three models is relatively small for $u^{\ast }=2,3$. This shows that starting optimistically allows agents to secure good chances of trusting a trustworthy institution in the long run.

We encountered the (perhaps) remarkable phenomenon that the effects of the different two agent communication mechanisms are not monotone. This is highlighted in the cases (a) with $\mathit{c}=\mathit{r}=1$ and $u^{\ast }=1$, depicted in Figures 10 (a), the case with $\mathit{c}=3$, $\mathit{r}=2$ and $u^{\ast }=2$, depicted in Figure 16. In (a), the OR model outperforms the OA model in terms of probability of trusting a trustworthy institution. At the same time however, the OA model outperforms the OR model in making the decision to quit a trust relationship with an untrustworthy institution sooner. In (b), we see the OA model outperforming the OR model in terms of the probability of trusting a trustworthy institution for values $\mathit{\vartheta }=0.65,0.66$. These results show us that we cannot state that the OR model is always “better” than the OA model. In fact this depends on the parameter setting as well as on which criterion you find most important: making the correct decision in the long run, or being sure to end a relationship with an untrustworthy institution relatively quickly.

This surprising result is partially due to the “timing” of the underlying dynamics. Agents make their decision to place trust at the same time and so can only use information from their neighbors action, one round later. We illustrate this in a two-rounds setting in Appendix C.

We summarize our insights from the numerical results:

• Communication always helps: it increases the probability of never ceasing to trust a trustworthy institution, and it decreases the expected time until quitting a relationship with an untrustworthy institution, both compared to the single agent model.

• The OA model can be better or worse than the OR model, depending on the performance measure of interest (probability of quitting, or expected time to quit). There are instances in which having less information is beneficial to the agents.

• A good way to increase chances of trusting a trustworthy institution in the long run without a social network is to start with a more optimistic prior.

7.2. Reflection and context

In this paragraph we compare our model to those found in the literature streams conceptually. In the subsequent paragraphs we compare the outcomes of our model to those in the literature. We presented a model of trust which includes both learning from interactions with the institution as well as learning from communication between agents. Thematically this work is related to social network trust where the focus is on the possible loss of trust. Methodologically this work is related to the social network learning. The agents in our model are learning about the environment (trustworthiness of the trustee). Our model extends ideas in both of these streams in a natural way: The dependence structure between actions taken by the agents, and the signals they subsequently use to learn, extends in a realistic way the work from the social network learning literature, which typically does not cover this complication. Finally, the observable actions model of communication adds realism to the relationships between agents, not present in the more common observable rewards model of communication in the social network trust literature.

The results of our investigation, akin to those from Buskens (Buskens, 2003), show that trust increases in the models with truster agents sharing information about the trustee. In our model, however, this effect extends to models in which a trustee dishonoring trust in one round does not immediately indicate that they are not to be trusted. Furthermore, we extended the type of communication between the truster agents beyond complete information sharing. We find that the positive effect of agent communication, though stronger in the complete information sharing, is also present in the model in which communication is limited to observing actions. Hence, even when individuals do not hold extensive discussion with their peers about the institution, simply observing their actions provides a significant benefit, which has previously been shown for more extensive communication in the social network trust literature (Buskens, 2003; Buskens et al., 2010; Frey et al., 2015).

In both models of communication that we consider, for all parameter settings, we observe that rational use of social network information increases the chances of agents to learn the true trustworthiness when it is rational to place trust. We also observe that for both models of communication and in all parameter settings, the expected time to quitting is sooner, which is especially beneficial when it is rational to quit. Our model includes realistic assumptions in terms of agent communication and of signal dependence on actions, and shows that communication between agents are an aid to learning and trusting. The simplistic nature of independent signals and actions cannot be imported to models of trust and learning. It is the nature of a trust problem that resources have to be placed at the liberty of the trustee agent in order to see how they respond. However, like in the social network learning literature (cf (Harel et al., 2021; Huang et al., 2021), our work shows that more communication leads to faster dynamics, but, due to the dependence of signals on actions, sometimes the result is converging to the sub-optimal action (i.e., not placing trust).

7.3. Future work

We see from the developed model that agents communicating with one another bear a benefit. There are, however, situations in which the agent that otherwise might have stayed on the course of placing trust “wrongfully” stops placing trust as a result of what they hear from their neighbor. It seems that the overall dynamics are not dominated by this effect. A natural question then becomes whether or not this beneficial effect retains in a model with more than two agents. The present model is limited to two agents partially as a result of the intense computational work involved: agents in the OA model perform numerous numerical integrations per round in order to identify which histories of experiences are plausible for the communication received from their neighbor. It is an open question whether there is a scalable approach to perform these computations. Alternatively, one could find a way around this by relaxing the agents’ rationality when it comes to interpreting their neighbors actions. This would allow investigation of a greater pool of agents who have some sophistication in learning from their private signal, but comes at the cost of simplifying the model.

Another line of future work concerns asymmetric communication between two truster agents. One of the truster agents may be modeled as a news outlet which only sends information without receiving any in return. In the same spirit one agent might be malicious by spreading misinformation about the institution. It may be that this asymmetry between agents may make technical results more attainable. For instance, one can condition on the private signal of the only sending truster and observe the dynamics of the receiving truster. The dynamics for the purely sending truster agent conveniently follows the dynamics we have presented in the context of the single-agent trust model. Modelling a malicious actor is also possible by deciding a priori what signal they will send, or by having a truthfulness parameter by which the truster agent communicates their actual experience with probability $\mathit{\eta }$ and communicates that trust was abused (regardless of the truth) with probability $1-\mathit{\eta }$. The honest agent in such a model may then need to learn not only about the trustworthiness of the institution but also about the information they receive from their network.

Acknowledgments

This research was supported by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement no. 945045, and by the NWO Gravitation project NETWORKS under grant no. 024.002.003. This work was carried out on the Dutch national e-infrastructure with the support of SURF Cooperative. Version: Fri Apr 12 06:53:34 2024.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The work was supported by the H2020 Marie Skłodowska-Curie Actions [945045]; Nederlandse Organisatie voor Wetenschappelijk Onderzoek [024.002.003].

Notes

¹ Work in biological perception has shown that humans often behave in a Bayesian manner. For a discussion on the implications hereof and a list of results we refer to Knill and Pouget (Knill & Pouget, 2004).

² Not only is myopic decision making a common assumption in the literature (cf (Bala & Goyal, 1998; Harel et al., 2021; Keppo et al., 2008; Parikh & Krasucki, 1990; Sebenius & Geanakoplos, 1983), but there is also some experimental evidence that points to semi-myopic behavior in humans (Zhang & Yu, 2013a, 2013b).

³ Equivalent dynamics arise when framing the interaction as paying utility $\mathit{x}$ to the institution. Thereafter an honorable action from the institution equates to receiving utility $\mathit{x}+\mathit{\delta }$. An abusive action on the other hand means the $\mathit{x}$ is lost. Now $\mathit{c}=\mathit{x}$ and $\mathit{r}=\mathit{\delta }$, and our range covers values from $\mathit{\delta }=2\mathit{x}$ to $\mathit{\delta }=\mathit{x}/2$ which we consider trust problems. We consider $\mathit{\delta }>2\mathit{x}$ closer to an issue of gambling, and $\mathit{\delta }<\mathit{x}/2$ closer to transformation of utility from one form to another (without any risk, that is).

A.

Appendix: Proofs of results

A.1. Proof of Lemma 1(Guaranteed quitting)

Proof. We prove this by contradiction. Suppose that $A_t=1$ for all $\mathit{t}\in \mathbb{N}$. Hence,

(69) $\mathit{r}{\hat{\vartheta }}_{\mathit{t}}-\mathit{c}(1-{\hat{\vartheta }}_{\mathit{t}})\ge 0,\mathrm{\forall t}\in \mathbb{N}.$(69)

The agent estimates the trustworthiness in round $\mathit{t}$ by

(70) ${\hat{\vartheta }}_{\mathit{t}}=\frac{\alpha +{\hat{S}}_{\mathit{t}}}{\mathit{\alpha }+\mathit{\beta }+\mathit{t}},$(70)

and we know by the law of large numbers that

(71) $\underset{\mathit{t}\to \mathrm{\infty }}{lim}\frac{S_t}{\mathit{t}}=\mathit{\vartheta }(\mathrm{almost}\mathrm{surely}).$(71)

This also implies that

(72) $\underset{\mathit{t}\to \mathrm{\infty }}{lim}{\hat{\vartheta }}_{\mathit{t}}=\mathit{\vartheta }<\mathit{c}/(\mathit{r}+\mathit{c})(\mathrm{almost}\mathrm{surely}).$(72)

This means that there exists $\mathit{t}$ for which

(73) $\mathit{r}{\hat{\vartheta }}_{\mathit{t}}-\mathit{c}(1-{\hat{\vartheta }}_{\mathit{t}})<\mathit{r}\left(\frac{c}{\mathit{c}+\mathit{r}}\right)-\mathit{c}\left(1-\frac{c}{\mathit{c}+\mathit{r}}\right),$(73)

(74) $<\frac{\mathit{rc}}{\mathit{c}+\mathit{r}}-\frac{\mathit{c}(\mathit{c}+\mathit{r})}{\mathit{c}+\mathit{r}}+\frac{c^2}{\mathit{c}+\mathit{r}}=\frac{0}{\mathit{c}+\mathit{r}}.$(74)

At this value of $\mathit{t}$, the expected utility of the interaction is negative, which implies that the agent takes action $A_t=0$, a contradiction. □

A.2. Proof of Lemma 2 (Converge or quit)

Proof. The agent makes use of a Bayesian belief update starting with a beta distributed prior belief with shape parameters $\mathit{\alpha }$ and $\mathit{\beta }$. Given that ${\hat{S}}_{\mathit{t}}=\sum _{\mathit{t}=1}^t{X}_t{\mathbf{1}}_{\left\{A_t=1\right\}}$, where $X_t$ is a binary random variable taking the value 1 at probability $\mathit{\vartheta }$ and $0$ at probability $(1-\mathit{\vartheta })$, where $\mathit{\vartheta }$ is the value which the agent is trying to estimate, we have that the belief distribution at time $\mathit{t}$ equals

(75) $P_t(\mathit{\theta })=\frac{{\theta }^{{\hat{S}}_{\mathit{t}}+\mathit{\alpha }-1}{(1-\mathit{\theta })}^{(t-{\hat{S}}_{\mathit{t}})+\mathit{\beta }-1}}{\underset{0}{\overset{1}{\int }}{y}^{{\hat{S}}_{\mathit{t}}+\mathit{\alpha }-1}{(1-\mathit{y})}^{(t-{\hat{S}}_{\mathit{t}})+\mathit{\beta }-1}\mathrm{d}\mathit{y}},\mathit{\theta }\in [0,1].$(75)

Defining $\mathit{\tau }$ as the first $\mathit{t}\in \mathbb{N}$ such that ${\int }_0^1\mathit{\theta }{P}_t(\mathit{\theta })\mathrm{d}\mathit{\theta }<\mathit{c}/(\mathit{c}+\mathit{r})$ (possibly $\mathrm{\infty }$), we have for all $\mathit{t}<\mathit{\tau }$ that the estimated $\mathit{\vartheta }$ is given by

(76) ${\hat{\vartheta }}_{\mathit{t}}=\frac{\alpha +{\hat{S}}_{\mathit{t}}}{\mathit{\alpha }+\mathit{\beta }+\mathit{t}}.$(76)

Then there are two possible cases: $\mathit{\tau }<\mathrm{\infty }$ and $\mathit{\tau }=\mathrm{\infty }$. In the first case (where $\mathit{\tau }<\mathrm{\infty }$) the agent stops placing trust and by the assumption on $\mathit{\vartheta }$, namely that $\mathit{\vartheta }\ge \mathit{c}/(\mathit{c}+\mathit{r})={\theta }_{\mathrm{crit}}$, it is clear that the agent has not converged to the true value of $\mathit{\vartheta }$. This is because of the fact that quitting requires $\hat{\vartheta }\ge {\theta }_{\mathrm{crit}}$ and that by assumption we have chosen $\mathit{\vartheta }>{\theta }_{\mathrm{crit}}$. In the second case we have $\mathit{\tau }=\mathrm{\infty }$. Here we define $S_t$ which drops the dependence on the actions of the agent:

(77) $S_t=\sum _{\mathit{s}=1}^t{X}_t,\mathrm{\forall t}\in \mathbb{N},$(77)

and we note that when $\mathit{\tau }=\mathrm{\infty }$ the relationship ${\hat{S}}_{\mathit{t}}=S_t$ holds, and therewith $\hat{\vartheta }=(S_t+\mathit{\alpha })/(\mathit{\alpha }+\mathit{\beta }+\mathit{t})$. First inspecting $S_t/\mathit{t}$ which we know by the law of large numbers takes

(78) $\underset{\mathit{t}\to \mathrm{\infty }}{lim}\frac{S_t}{\mathit{t}}=\mathit{\vartheta }(\mathrm{almost}\mathrm{surely}).$(78)

Subsequently investigating ${\hat{\vartheta }}_{\mathit{t}}$ as $\mathit{t}\to \mathrm{\infty }$ we see:

(79) $\underset{\mathit{t}\to \mathrm{\infty }}{lim}{\hat{\vartheta }}_{\mathit{t}}=\underset{\mathit{t}\to \mathrm{\infty }}{lim}\frac{S_t+\mathit{\alpha }}{\mathit{t}+\mathit{\alpha }+\mathit{\beta }}=\mathit{\vartheta }(\mathrm{almost}\mathrm{surely}).$(79)

This shows that the agent’s estimate has indeed converged to the true $\mathit{\vartheta }$ almost surely.

A.3. Proof of Lemma 3 (Quitting probability when $\mathit{c}=1$, and $\mathit{r}\in \mathbb{N}$)

Proof. Recall the assumption that $\mathit{\theta }\ge \mathit{c}/(\mathit{c}+\mathit{r})$ and the definitions $\mathit{\pi }(\mathit{u})=\mathbb{P}(\mathrm{\exists }\mathit{t}:Z_t\ge \mathit{u})$ and $\mathit{\rho }=\mathbb{P}(\mathrm{\exists }\mathit{t}:Z_t\ge 1)$. Observe that (A) by the strong Markov property the random walk after having hit level 1 is independent from what happened before, (B) on the path to level $\mathit{u}$ any level in the set $\left\{1,\dots ,\mathit{u}-1\right\}$ has been attained. We thus obtain the `memoryless property’

(80) $\mathit{\pi }(\mathit{u})={\rho }^u.$(80)

Now consider the first step of the random walk starting at 0; it can either hit the first level immediately at probability $(1-\mathit{\theta })$ or it drops to the level $-\mathit{r}$ putting it a distance $\mathit{r}+1$ from the level 1. Using (80) we find the identity

(81) $\mathit{F}(\mathit{\rho }):=\mathit{\rho }=(1-\mathit{\vartheta })+\mathit{\vartheta }{\rho }^{\mathit{r}+1}=:\mathit{G}(\mathit{\rho }).$(81)

We can prove that this equation has exactly one solution in the interval $[0,1)$. Note that $\mathit{F}(\cdot )$ is convex. A trivial solution is of course $\mathit{\rho }=1$ which accounts for one of at most two intersections between a convex function and a linear function. It thus remains to show that there is another intersection, and that it lies in $[0,1)$. This is indeed the case because $\mathit{F}{ }^{\mathrm{\prime }}(1)=\mathit{\theta }(\mathit{r}+1)$, which is larger than $\mathit{G}{ }^{\mathrm{\prime }}(1)=1$ due to the condition that $\mathit{\theta }>\mathit{c}/(\mathit{c}+\mathit{r})$ after substituting $\mathit{c}=1$ and rearranging. Observing that $\mathit{F}(0)=1-\mathit{\theta }>0=\mathit{G}(0)$, this entails that there is another root located somewhere between $\mathit{\rho }=0$ and $\mathit{\rho }=1$.

A.4. Proof of Lemma 4 (Expected time to quitting when $\mathit{c}=1$, and $\mathit{r}\in \mathbb{N}$)

Proof. Let $\mathit{\tau }(\mathit{u})=inf\left\{\mathit{t}:Z_t=\mathit{u}\right\}$ be the first passage time for the walk $Z_t$ to $\mathit{u}$ and recall that $\mathit{\pi }(\mathit{u})$ is the probability of this event occurring in finite time (i.e., $\mathit{\pi }(\mathit{u}):=\mathbb{P}(\mathrm{\exists }\mathit{t}:Z_t\ge \mathit{u})$). Then we are interested in the expectation of the $\mathit{\tau }(\mathit{u})$ conditioned on its finity:

(82) $\mathbb{E}[\mathit{\tau }(\mathit{u})|\mathit{\tau }(\mathit{u})<\mathrm{\infty }]=\frac{\mathbb{E}[\tau (u)\mathbf{1}\{\tau (u)<\mathrm{\infty }\}]}{\mathit{\pi }(\mathit{u})}.$(82)

Note the relationship

(83) $\mathbb{E}[z^{\mathit{\tau }(\mathit{u})}\mathbf{1}\{\mathit{\tau }(\mathit{u})<\mathrm{\infty }\}]=\mathbb{E}[z^{\mathit{\tau }(1)}\mathbf{1}\{\mathit{\tau }(1)<\mathrm{\infty }\}{]}^u$(83)

which holds by the precise same `memoryless argumentation’ as in the proof of Lemma 3. We define

(84) $\mathit{\phi }(\mathit{z}):=\mathbb{E}[z^{\mathit{\tau }(1)}\mathbf{1}\{\mathit{\tau }(1)<\mathrm{\infty }\}],$(84)

which by the relationship (83) also allows for a self-referential expression

(85) $\mathit{\phi }(\mathit{z})=\mathit{z}(1-\mathit{\vartheta })+\mathit{z\vartheta }\mathit{\phi }(\mathit{z}{)}^{\mathit{r}+1}.$(85)

Finally note that we can get the expectation of interest out again by taking the derivative and substituting $\mathit{z}=1$:

(86) $\mathbb{E}[\mathit{\tau }(\mathit{u})|\mathit{\tau }(\mathit{u})<\mathrm{\infty }]=\mathit{u}\frac{\mathit{\phi }{\mathit{ }}^{\mathrm{\prime }}(1)}{\mathit{\pi }(1)}.$(86)

We now manipulate these expression in order to get the final statement: observing that

(87) $\mathit{\phi }{ }^{\mathrm{\prime }}(\mathit{z})=(1-\mathit{\vartheta })+\mathit{\vartheta \phi }(\mathit{z}{)}^{\mathit{r}+1}+\mathit{z\vartheta }(\mathit{r}+1)\mathit{\phi }(\mathit{z}{)}^r\mathit{\phi }{ }^{\mathrm{\prime }}(\mathit{z}),$(87)

substituting $\mathit{z}=1$ and noting that $\mathit{\phi }(1)=\mathit{\pi }(1)=\mathit{\rho }$ with $\mathit{\rho }$ as defined in (22), we obtain

(88) $\begin{array}{cc}\mathit{\phi }{ }^{\mathrm{\prime }}(1)& =\underset{=\mathit{\rho }\mathrm{by}(24)}{\underbrace{(1-\mathit{\vartheta })+\mathit{\vartheta }{\mathit{\rho }}^{\mathit{r}+1}}}+\mathit{\vartheta }(\mathit{r}+1){\rho }^r\mathit{\phi }{ }^{\mathrm{\prime }}(1)=\frac{\rho }{1-(r+1)\underset{=1-(1-\mathit{\vartheta }){\rho }^{-1}}{\underbrace{\vartheta {\rho }^r}}}\\ \mathit{\hspace{0.17em}}& =\frac{{\rho }^2}{(1-\mathit{\vartheta })(\mathit{r}+1)-\mathit{r\rho }}(\mathrm{multiplying}\mathrm{by}\frac{\rho }{\rho }).\end{array}$(88)

Multiplying this by $\mathit{u}/\mathit{\rho }$ gives the final result. □

A.5. Proof of Lemma 5 (Quitting probability when $\mathit{r}=1$, and $\mathit{c}\in \mathbb{N}$)

Proof. We introduce the time $\mathit{G}$ which follows a shifted geometric distribution: $\mathbb{P}(\mathit{G}=\mathit{t})=(1-\mathit{f}{)}^t\mathit{f}$ for some $\mathit{f}\in (0,1)$. We investigate the behaviour of the walk, specifically the probability of reaching $\mathit{u}$, before the geometrically triggered time $\mathit{G}$:

(89) $\mathit{p}[\mathit{f}](\mathit{u}):=\mathbb{P}(\mathrm{\exists }\mathit{n}\le \mathit{G}:Z_n\ge \mathit{u}),$(89)

as well as the probability of reaching $Z_n=1$ for some $\mathit{n}\le \mathit{G}$:

(90) $\mathit{\rho }[\mathit{f}]:=\mathbb{P}(\mathrm{\exists }\mathit{n}\le \mathit{G}:Z_n\ge 1).$(90)

Furthermore, we define minimum of the walk until time $\mathit{n}$, $\underset{0.3\mathit{em}-}{Z_n}:=min\left\{Z_0,Z_1,\dots ,Z_n\right\}$ as well as the maximum attained until time $\mathit{n}$, ${\bar{Z}}_{\mathit{n}}:=max\left\{Z_0,Z_1,\dots ,Z_n\right\}$. Due to the Wiener-Hopf factorisation [30, 31], ${\bar{Z}}_{\mathit{G}}-Z_G$ is independent of ${\bar{Z}}_{\mathit{G}}$, so that $\mathbb{E}{w}^{Z_G}=\mathbb{E}{w}^{{\bar{Z}}_{\mathit{G}}}\mathbb{E}{w}^{Z_G-{\bar{Z}}_{\mathit{G}}}.$ It is also true that $Z_G-{\bar{Z}}_{\mathit{G}}$ is distributed as $\underset{0.3\mathit{em}-}{Z_G}$, as is seen by rotating the walk ${180}^{\circ }$ and considering $Z_G$ as the starting point of the new, identically distributed process. Therefore,

(91) $w^{Z_G}=w^{{\bar{Z}}_{\mathit{G}}}{w}^{{\underset{0.3\mathit{em}-}{\mathit{Z}}}_{\mathit{G}}}.$(91)

Hence, if we manage to identify $\mathbb{E}{w}^{Z_G}$ and $w^{{\underset{0.3\mathit{em}-}{\mathit{Z}}}_{\mathit{G}}}$, we have found $\mathbb{E}{w}^{{\bar{Z}}_{\mathit{G}}}$. In pursuit of $\mathbb{E}{w}^{Z_G}$, let $\mathit{x}$ be the number of times the process jumps upward in the time interval $\mathit{t}\in \left\{1,\dots ,\mathit{G}\right\}$. Hence,

(92) $\begin{array}{cc}\mathbb{E}{w}^{Z_G}& =\mathbb{E}\left[\sum _{\mathit{x}=0}^G\left(\begin{array}{c}\mathit{G}\\ \mathit{x}\end{array}\right){(1-\mathit{\vartheta })}^x{\vartheta }^{\mathit{G}-\mathit{x}}{w}^{\mathit{cx}-(\mathit{G}-\mathit{x})}\right]\\ \mathit{\hspace{0.17em}}& =\mathbb{E}\left[\sum _{\mathit{x}=0}^G\left(\begin{array}{c}\mathit{G}\\ \mathit{x}\end{array}\right){((1-\vartheta )w^c)}^{\mathit{x}}{\left(\frac{\vartheta }{w}\right)}^{\mathit{G}-\mathit{x}}\right]=\mathbb{E}{\left[(1-\vartheta )w^c+\mathit{\vartheta }{w}^{-1}\right]}^{\mathit{G}}.\end{array}$(92)

Recalling that $\mathit{G}$ is geometrically distributed, for $\left|(1-\mathit{f})((1-\mathit{\theta })w^c+\mathit{\theta }{w}^{-1})\right|<1$,

(93) $\begin{array}{cc}\mathbb{E}{w}^{Z_G}& =\sum _{\mathit{i}=0}^{\mathrm{\infty }}{(1-\mathit{f})}^i\mathit{f}{((1-\vartheta )w^c+\mathit{\vartheta }{w}^{-1})}^{\mathit{i}}=\frac{f}{1-(1-f)((1-\vartheta )w^c+\mathit{\vartheta }{w}^{-1})}.\end{array}$(93)

In order to find $w^{{\underset{0.3\mathit{em}-}{\mathit{Z}}}_{\mathit{G}}}$ we note that the minimum $\underset{0.3\mathit{em}-}{Z_G}$ is identically distributed to the negative of the maximum of the random walk $B_G$ that takes steps up of size one with probability $\mathit{\vartheta }$ and down $\mathit{c}\in \mathbb{N}$ with probability $1-\mathit{\vartheta }$. These two walks and their step descriptions are shown in Table 4.

Table 4. Two random walks.

Walk	Step $+$	$\mathbb{P}(+)$	Step $-$	$\mathbb{P}(-)$
$Z_G$	$\mathit{c}$	$1-\mathit{\vartheta }$	1	$\mathit{\vartheta }$
$B_G$	1	$\mathit{\vartheta }$	$\mathit{c}$	$1-\mathit{\vartheta }$

The walk $B_G$ is simply a variable substitution of the walk analysed in in $\mathcal{\S }$3.2.1 at an added shifted geometric distributed time step $\mathit{G}$ and as such the analysis is remarkably similar. Define the probability of the walk $\mathit{B}$ going up by at least $\mathit{u}$ before the geometric trigger,

$\mathit{p}[\mathit{f}](\mathit{u}):=\mathbb{P}(\mathrm{\exists }\mathit{n}\le \mathit{G}:B_n\ge \mathit{u}).$

Similarly define the probability of reaching $\mathit{u}=1$ before the geometric trigger by

$\mathit{\rho }[\mathit{f}]:=\mathbb{P}(\mathrm{\exists }\mathit{n}\le \mathit{G}:B_n\ge 1),$

then as before we have the memoryless property $\mathit{p}[\mathit{f}](\mathit{u})=(\mathit{\rho }[\mathit{f}]{)}^u$. It is also apparent that $\mathit{\rho }[\mathit{f}]$ solves

(94) $\mathit{\rho }[\mathit{f}]=(1-\mathit{f})(\mathit{\vartheta }+(1-\mathit{\vartheta })(\mathit{\rho }[\mathit{f}]{)}^{\mathit{c}+1}).$(94)

By defining $\mathit{s}[\mathit{f}](\mathit{u}):=\mathbb{P}({\bar{B}}_{\mathit{G}}=\mathit{u})$ as the probability that the maximum of the random walk $\mathit{B}$ by time $\mathit{G}$ is exactly $\mathit{u}$, and noting that

(95) $\begin{array}{cc}\mathit{s}[\mathit{f}](\mathit{u})& =\mathit{p}[\mathit{f}](\mathit{u})-\mathit{p}[\mathit{f}](\mathit{u}+1)={(\mathit{\rho }[\mathit{f}])}^u-{(\mathit{\rho }[\mathit{f}])}^{\mathit{u}+1}={(\mathit{\rho }[\mathit{f}])}^u(1-\mathit{\rho }[\mathit{f}]),\end{array}$(95)

we have an expression for this maximum. By the relationship between $B_G$ and $Z_G$, we have also have an expression for $\underset{0.3\mathit{em}-}{Z_G}$, with $\mathit{\rho }[\mathit{f}]$ solving (94),

$(\underset{0.3\mathit{em}-}{Z_G}=-\mathit{u})=(\mathit{\rho }[\mathit{f}]{)}^u(1-\mathit{\rho }[\mathit{f}]).$

Hence,

(96) $w^{{\underset{0.3\mathit{em}-}{\mathit{Z}}}_{\mathit{G}}}=\sum _{\mathit{u}=0}^{\mathrm{\infty }}{(\mathit{\rho }[\mathit{f}])}^u(1-\mathit{\rho }[\mathit{f}])w^{-\mathit{u}}=\frac{1-\mathit{\rho }[\mathit{f}]}{1-\mathit{\rho }[\mathit{f}]{\mathit{w}}^{-1}}.$(96)

This, together with expression (93) and the Wiener-Hopf factorisation (91), yields

(97) $\mathbb{E}{w}^{{\bar{Z}}_{\mathit{G}}}=\frac{f}{1-(1-f)((1-\vartheta )w^c+\mathit{\vartheta }{w}^{-1})}\times \frac{1-\mathit{\rho }[\mathit{f}]{\mathit{w}}^{-1}}{1-\mathit{\rho }[\mathit{f}]}.$(97)

By considering this as $\mathit{f}\downarrow 0$, it is first noted that $\mathit{\rho }[0]=1.$ Then, defining $\mathit{\sigma }(\mathit{u}):=\mathbb{P}({\bar{Z}}_{\mathrm{\infty }}=\mathit{u})$,

(98) $\begin{array}{cc}\mathit{\zeta }(\mathit{w})& :=\sum _{\mathit{u}=0}^{\mathrm{\infty }}\mathit{\sigma }(\mathit{u})w^u=\underset{\mathit{f}\downarrow 0}{lim}\mathbb{E}{w}^{{\bar{Z}}_{\mathit{G}}}\\ \mathit{\hspace{0.17em}}& =\underset{\mathit{f}\downarrow 0}{lim}\frac{f}{1-(1-f)((1-\vartheta )w^c+\mathit{\vartheta }{w}^{-1})}\times \frac{1-\mathit{\rho }[\mathit{f}]{\mathit{w}}^{-1}}{1-\mathit{\rho }[\mathit{f}]}\\ \mathit{\hspace{0.17em}}& =\underset{\mathit{f}\downarrow 0}{lim}\frac{1-\mathit{\rho }[\mathit{f}]{\mathit{w}}^{-1}}{1-(1-\mathit{f})((1-\mathit{\vartheta })w^c+\mathit{\vartheta }{w}^{-1})}\times \frac{f}{1-\mathit{\rho }[\mathit{f}]}\\ \mathit{\hspace{0.17em}}& =\frac{1-{\mathit{w}}^{-1}}{1-((1-\mathit{\vartheta })w^c+\mathit{\vartheta }{w}^{-1})}\times \underset{\mathit{f}\downarrow 0}{lim}\frac{f}{1-\mathit{\rho }[\mathit{f}]}\end{array}$(98)

Applying L’Hôpital’s rule, we obtain

(99) $\mathit{\zeta }(\mathit{w})=\frac{1-{\mathit{w}}^{-1}}{1-((1-\mathit{\vartheta })w^c+\mathit{\vartheta }{w}^{-1})}\times \frac{1}{-\mathit{\gamma }},\mathit{with}\mathit{\gamma }:=\underset{\mathit{f}\downarrow 0}{lim}\frac{\mathrm{d}}{\mathrm{d}f}\mathit{\rho }[\mathit{f}].$(99)

It is easy to verify, by differentiating both sides of (94) with respect to $\mathit{f}$ and solving for $\mathit{\gamma }$, that $1/\mathit{\gamma }=(\mathit{c}+1)(1-\mathit{\vartheta })-1,$ which entails that

(100) $\begin{array}{cc}\mathit{\zeta }(\mathit{w})& =\frac{(1-{\mathit{w}}^{-1})((\mathit{c}+1)(1-\mathit{\vartheta })-1)}{((1-\mathit{\vartheta })w^c+\mathit{\vartheta }{w}^{-1})-1}=\frac{(\mathit{w}-1)(1-(1-\mathit{\vartheta })(\mathit{c}+1))}{\mathit{w}-\mathit{\vartheta }-(1-\mathit{\vartheta }){\mathit{w}}^{\mathit{c}+1}}.\end{array}$(100)

But we are interested in $\mathit{\pi }(\mathit{u})$, as such let $\mathit{\xi }(\mathit{w})=\sum _{\mathit{u}=0}^{\mathrm{\infty }}\mathit{\pi }(\mathit{u})w^u$, which is expressed using $\mathit{\sigma }(\mathit{u})$ in the following way

(101) $\begin{array}{cc}\mathit{\xi }(\mathit{w})& =\sum _{\mathit{u}=0}^{\mathrm{\infty }}\sum _{\mathit{v}=\mathit{u}}^{\mathrm{\infty }}\mathit{\sigma }(\mathit{v})w^u=\sum _{\mathit{v}=0}^{\mathrm{\infty }}\sum _{\mathit{u}=0}^v\mathit{\sigma }(\mathit{v})w^u\\ \mathit{\hspace{0.17em}}& =\sum _{\mathit{v}=0}^{\mathrm{\infty }}\mathit{\sigma }(\mathit{v})\left(\frac{1-{\mathit{w}}^{\mathit{v}+1}}{1-\mathit{w}}\right)=\frac{\sum _{\mathit{v}=0}^{\mathrm{\infty }}\mathit{\sigma }(\mathit{v})-\mathit{w}\sum _{\mathit{v}=0}^{\mathrm{\infty }}\mathit{\sigma }(\mathit{v})w^v}{1-\mathit{w}},\end{array}$(101)

culminating in the useful expression

(102) $\mathit{\xi }(\mathit{w})=\frac{1-\mathit{w\zeta }(\mathit{w})}{1-\mathit{w}},$(102)

by which we can evaluate $\mathit{\pi }(\mathit{u})$ via the relationship

(103) $\mathit{\pi }(\mathit{u})=\frac{1}{u}\frac{{\mathrm{d}}^u}{\mathrm{d}{w}^u}\mathit{\xi }(\mathit{w}){|}_{\mathit{w}=0}=\frac{{\mathit{\xi }}^{(\mathit{u})}(0)}{\mathit{u}!}.$(103)

Substituting the expression for $\mathit{\zeta }(\mathit{w})$ gives the result. □

Remark 7. Cong and Sato [32] use combinatorial arguments to find the probability of first hitting the absorbing barrier at specific times $\mathit{t}$ for the cases when either $\mathit{r}$ or $\mathit{c}$ is one,

(104) $p_t(\mathit{u})=\mathbb{P}(inf\left\{\mathit{n}\in \mathbb{N}:Z_n\ge \mathit{u}\right\}=\mathit{t}).$(104)

Taking the sum (over the first hitting times $\mathit{t}$) could in turn provide the probabilities that we state here: $\sum _{\mathit{t}=1}^{\mathrm{\infty }}{p}_t(\mathit{u})=\mathit{\pi }(\mathit{u})$. Notice that this approach is of an intrinsically approximative nature, as one has to somehow truncate the summation over $\mathit{t}$, while our approach is exact. We do not make use of their result, also because the underlying computations are involved, and in addition it does not yield the expected time of quitting.

A.6. Proof of Lemma 6 (Expected time to quitting when $\mathit{r}=1$, and $\mathit{c}\in \mathbb{N}$)

Proof. Define

(105) $\mathit{\phi }(\mathit{z},\mathit{u}):=\mathbb{E}[z^{\mathit{\tau }(\mathit{u})}\mathbf{1}\{\mathit{\tau }(\mathit{u})<\mathrm{\infty }\}],\mathrm{\Phi }(\mathit{w},\mathit{z}):=\sum _{\mathit{u}=1}^{\mathrm{\infty }}{w}^u\mathit{\phi }(\mathit{z},\mathit{u}),\mathit{w}\in (-1,1),$(105)

so that

(106) $\mathbb{E}[\mathit{\tau }(\mathit{u})|\mathit{\tau }(\mathit{u})<\mathrm{\infty }]=\frac{\mathit{\phi }{\mathit{ }}^{\mathrm{\prime }}(1,\mathit{u})}{\mathit{\phi }(1,\mathit{u})},\mathit{\phi }(\mathit{z},\mathit{u})={\left.\frac{{\mathrm{\partial }}^u}{\mathrm{\partial }{w}^u}\frac{\mathrm{\Phi }(\mathit{w},\mathit{z})}{\mathit{u}!}\right|}_{\mathit{w}=0}$(106)

The dynamics of the process reveal the recursive relation

(107) $\mathit{\phi }(\mathit{z},\mathit{u})=\left(\begin{array}{cc}(1-\mathit{\vartheta })\mathit{z}\mathit{\phi }(\mathit{z},\mathit{u}-\mathit{c})+\mathit{\vartheta z}\mathit{\phi }(\mathit{z},\mathit{u}+1)& \mathrm{if}\mathit{u}>\mathit{c},\\ (1-\mathit{\vartheta })\mathit{z}+\mathit{\vartheta z}\mathit{\phi }(\mathit{z},\mathit{u}+1)& \mathrm{otherwise}.\end{array}\right.$(107)

Substituting this into the definition of $\mathrm{\Phi }(\mathit{w},\mathit{z})$, we obtain, after some minor rewriting,

(108) $\begin{array}{cc}\mathrm{\Phi }(\mathit{w},\mathit{z})=& \underset{(\mathit{a})}{\underbrace{\sum _{\mathit{u}=1}^c{w}^u(1-\mathit{\vartheta })\mathit{z}}}+\underset{(\mathit{b})}{\underbrace{\sum _{\mathit{u}=1}^{\mathrm{\infty }}{w}^u\mathit{\vartheta z}\mathit{\phi }(\mathit{z},\mathit{u}+1)}}+\underset{(\mathit{c})}{\underbrace{\sum _{\mathit{u}=\mathit{c}+1}^{\mathrm{\infty }}{w}^u(1-\mathit{\vartheta })\mathit{z}\mathit{\phi }(\mathit{z},\mathit{u}-\mathit{c})}}.\end{array}$(108)

Inspecting $(\mathit{a})$, $(\mathit{b})$, and $(\mathit{c})$ individually, we obtain

(109) $\begin{array}{cc}(\mathit{a})& =(1-\mathit{\vartheta })\mathit{z}\sum _{\mathit{u}=1}^c{w}^u=(1-\mathit{\vartheta })\mathit{z}\left(\frac{\mathit{w}-{\mathit{w}}^{\mathit{c}+1}}{1-\mathit{w}}\right)\end{array}$(109)

by applying the finite geometric series,

(110) $\begin{array}{cc}(\mathit{b})& =\mathit{\vartheta z}\frac{1}{w}\sum _{\mathit{u}=1}^{\mathrm{\infty }}{w}^{\mathit{u}+1}\mathit{\phi }(\mathit{z},\mathit{u}+1)=\frac{\mathit{\vartheta z}}{w}\mathrm{\Phi }(\mathit{w},\mathit{z})-\mathit{\vartheta z}\mathit{\phi }(\mathit{z},1)\end{array}$(110)

relying on the definition of $\mathrm{\Phi }(\mathit{w},\mathit{z})$, and

(111) $\begin{array}{cc}(\mathit{c})& =w^c(1-\mathit{\vartheta })\mathit{z}\sum _{\mathit{u}=\mathit{c}+1}^{\mathrm{\infty }}{w}^{\mathit{u}-\mathit{c}}\mathit{\phi }(\mathit{z},\mathit{u}-\mathit{c})=w^c(1-\mathit{\vartheta })\mathit{z}\mathrm{\Phi }(\mathit{w},\mathit{z})\end{array}$(111)

shifting the summation index by $\mathit{c}+1$ and again applying the definition of $\mathrm{\Phi }(\mathit{w},\mathit{z})$. By combining and rearranging $(\mathit{a})$, $(\mathit{b})$, and $(\mathit{c})$, we can isolate $\mathrm{\Phi }(\mathit{w},\mathit{z})$, so as to obtain

(112) $\mathrm{\Phi }(\mathit{w},\mathit{z})=\frac{(1-\mathit{\vartheta })\mathit{z}\left(\frac{\mathit{w}-{\mathit{w}}^{\mathit{c}+1}}{1-\mathit{w}}\right)-\mathit{\vartheta z\phi }(\mathit{z},1)}{1-(\mathit{\vartheta z})/\mathit{w}-w^c\mathit{z}(1-\mathit{\vartheta })}.$(112)

What remains is to identify $\mathit{\phi }(\mathit{z},1)$ for each $\mathit{z}\in (-1,1)$. The key principle is that, because their ratio is a finite number, any root of the denominator in (112) must also be a root of the numerator. Indeed, supposing there is a unique $\mathit{w}(\mathit{z})\in (-1,1)$ which yields a root of the denominator, this would imply that

(113) $\mathit{\phi }(\mathit{z},1)=\frac{1-\mathit{\vartheta }}{\vartheta }\mathit{z}\left(\frac{\mathit{w}(\mathit{z})-\mathit{w}{(\mathit{z})}^{\mathit{c}+1}}{1-\mathit{w}(\mathit{z})}\right)=\frac{(1-\mathit{\vartheta })\mathit{zw}(\mathit{z})+\mathit{\vartheta z}-\mathit{w}(\mathit{z})}{1-\mathit{w}(\mathit{z})}.$(113)

This means that we are done if we can show that for any $\mathit{z}\in (-1,1)$ the denominator in (112) has a unique root $\mathit{w}(\mathit{z})$ in $(-1,1).$ It is directly seen that this root satisfies

(114) $\mathit{F}(\mathit{w}):=w^{\mathit{c}+1}\mathit{z}(1-\mathit{\vartheta })+\mathit{z\vartheta }=\mathit{w}=:\mathit{G}(\mathit{w}).$(114)

Observe that for $\mathit{z}=0$ we obviously have the unique root $\mathit{w}=0.$

Subcase $\mathit{z}>0$. Observe that $\mathit{F}(0)=\mathit{z\vartheta }>0=\mathit{G}(0)$ and $\mathit{F}(1)=\mathit{z}<1=\mathit{G}(1)$, so that $\mathit{F}(\mathit{w})-\mathit{G}(\mathit{w})$ changes sign an odd number of times for $\mathit{w}\in (0,1)$. But because $\mathit{F}(\cdot )$ is convex and $\mathit{G}(\cdot )$ is linear, the function $\mathit{F}(\cdot )-\mathit{G}(\cdot )$ changes sign at most twice. By combining the above, we conclude the existence of the unique root $\mathit{w}(\mathit{z})\in (-1,1)$ (which actually lies between $0$ and $1$).

Subcase $\mathit{z}<0$. The case of $\mathit{c}$ odd works analogously to the subcase $\mathit{z}>0$: $\mathit{F}(0)=\mathit{z\vartheta }<0=\mathit{G}(0)$ and $\mathit{F}(-1)=\mathit{z}>-1=\mathit{G}(-1)$. From the concavity of $\mathit{F}(\cdot )$, the existence of a unique root $\mathit{w}(\mathit{z})\in (-1,1)$ follows (which lies between $-1$ and $0$). In case $\mathit{c}$ is even, we still have $\mathit{F}(0)=\mathit{z\vartheta }<0=\mathit{G}(0)$, but now it should be noted that $\mathit{F}(-1)=\mathit{z}(2\mathit{\vartheta }-1)$, which is, for $\mathit{z}\in (-1,0)$ and $\mathit{\vartheta }\in (0,1)$, larger than $-1=\mathit{G}(-1)$ (pick $\mathit{\vartheta }=1$ and $\mathit{z}=-1$). The existence of a unique root $\mathit{w}(\mathit{z})\in (-1,1)$ now follows from the fact that $\mathit{F}(\mathit{w})$ is decreasing and $\mathit{G}(\mathit{w})$ is increasing for $\mathit{w}\in (-1,0)$ (where this root lies between $-1$ and $0$). □

B.

Appendix: Single agent case $\mathit{r},\mathit{c}\in \mathbb{N}$

In this case no exact formula can be obtained for the probability of quitting or the expected time to quitting conditioned thereon; recall that in the other two cases we exploited the fact that the walk, either in the upward or the downward direction, had step size 1. We briefly sketch a numerical scheme which allows accurate approximation of the probability of quitting. The relationship for all $\mathit{u}>\mathit{c}$ is:

(115) $\mathit{\pi }(\mathit{u})=\mathit{\vartheta \pi }(\mathit{u}+\mathit{r})+(1-\mathit{\vartheta })\mathit{\pi }(\mathit{u}-\mathit{c}).$(115)

We supplement this with an approximation for large enough $\mathit{u}>u_0$ by setting $\hat{\pi }(\mathit{u})=\mathit{\eta }{\gamma }^u$ for $\mathit{\eta }\ge 0$ and $\mathit{\gamma }\in (0,1)$. In combination with (115) we know that $\mathit{\gamma }$ satisfies $1=(1-\mathit{\vartheta }){\gamma }^{-\mathit{c}}+\mathit{\vartheta }{\gamma }^r$. Using this approximation for $\mathit{\pi }(\mathit{u})$ from $\mathit{u}\ge u_0$ one has:

(116) $\hat{\pi }(\mathit{u})=\mathit{\eta }{\gamma }^u\mathrm{if}\mathit{u}\ge u_0\mathit{\hspace{0.17em}}(1-\mathit{\vartheta })\hat{\pi }(\mathit{u}-\mathit{c})+\mathit{\vartheta }\hat{\pi }(\mathit{u}+\mathit{r})\mathrm{if}\mathit{c}\mathit{\hspace{0.17em}}\mathit{u}{u}_0(1-\mathit{\vartheta })+\mathit{\vartheta }\hat{\pi }(\mathit{u}+\mathit{r})\mathrm{f}0\mathit{u}\le \mathit{c},$(116)

which can be used to construct a system of equations which can be solved numerically. We propose approximating the probability of absorption (and subsequently a termination of trust) by the solution to the system (116) at the appropriate $\mathit{u}=\mathit{r\alpha }-\mathit{c\beta }+1$. The value of $u_0$ should be large enough value to get a good approximation, and small enough to keep its computation manageable.

We sketch a similar approximation scheme for the expected time in the system. The crux of the scheme is to approximate the expected time to quitting at a high enough $\mathit{u}$ by the number of rounds it would take to move directly toward absorption without taking any steps away: aaaaa. In terms of $\mathit{\phi }(\mathit{z},\mathit{u})$ one arrives at the following system of equations:

(117) $\hat{\phi }(\mathit{z},\mathit{u})=\left\{\begin{array}{cc}{(1-\mathit{\vartheta })}^{[\mathit{u}/c]}{z}^{[\mathit{u}/c]}& \mathrm{if}\mathrm{u}\ge {\mathrm{u}}_0\\ (1-\mathit{\vartheta })\mathit{z\phi }(\mathit{z},\mathit{u}-\mathit{c})+\mathit{\vartheta z\phi }(\mathit{z},\mathit{u}+\mathit{r})& \mathrm{if}\mathrm{c}<\mathrm{u}<{\mathrm{u}}_0\\ (1-\mathit{\vartheta })\mathit{z}+\mathit{\vartheta z\phi }(\mathit{z},\mathit{u}+\mathit{r})& \mathrm{if}0<\mathrm{u}\le \mathrm{c}.\end{array}\right.$(117)

This defines a system of equations similar to (116) which can be solved numerically. This provides an approximation to $\mathit{\phi }(\mathit{z},u_{\mathrm{crit}})$ which in turn can be used to get an approximation of $\mathbb{E}[\mathit{\tau }(u_{\mathrm{crit}})|\mathit{\tau }(u_{\mathrm{crit}})<\mathrm{\infty }]$ by taking the derivative at $u_{\mathrm{crit}}:$ $\mathit{\phi }{ }^{\mathrm{\prime }}(1,u_{\mathrm{crit}})/\mathit{\phi }(1,u_{\mathrm{crit}})$. Notice the similarity to (40).

C.

Appendix: An illustration of how the OA model can beat the OR model

In this appendix we elaborate on the peculiar outcome that in the parameter setting with $\mathit{c}=3$, $\mathit{r}=2$, and $u^{\ast }=2$ the OA model yields a lower probability of quitting than the OR model at the trustworthiness settings $\mathit{\vartheta }=0.65,0.66$.

We consider a pair of agents in precisely the setting in with the exception ($\mathit{c}=3$, $\mathit{r}=2$, $u^{\ast }=2$ and $\mathit{\vartheta }=0.65$). Furthermore, we presume that of the first four times trust is placed in the institution (by either agent) trust is honoured only once.

In the OR model, this means that maximally two rounds of interaction can take place. Either the first round includes abuses of trust to both agents, in which case they quit in round 1, or trust is honoured once in round 1, but abused twice in round 2 and so the agents do not place trust in round 3.

In the OA model, the agents involved have two rounds of interaction regardless of their first reward. The act of placing trust at the start of round 2 yields no information to the neighbour. At the start of round 3 however, the agent (labelled 1) who observed two abuses does not place trust. This signals to the neighbouring agent (labelled 2) that two abuses of trust have certainly occurred. At the point of placing trust in round 3 (before observing the outcome thereof) agent 2 has access to information which brings their estimate below the critical value. However, the agent has already placed their trust and so will see the outcome of that interaction. This outcome is of crucial importance because if the trust is honoured (which is more likely as $\mathit{\vartheta }=0.65>0.5$), agent 2 will stay in the relationship while if the trust is abused they quit.

D.

Appendix: Tables of simulated results

In the tables below we present the probability of quitting and the expected time to quitting under the three considered mechanisms.

Table 5. Probability of quitting and the expected time to quitting in the single agent model (Sol), the observable actions model (OA) and the observable rewards model (OR). Parameters $\mathit{\alpha }$ and $\mathit{\beta }$ set such that $u^{\ast }=1$

(A) Probability of quitting
$\mathit{c}=\mathit{r}=\mathbf{1}$						$\mathit{\alpha }=\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.30	0.36	0.42	0.45	0.55	0.60	0.66	0.72	0.78	0.84	0.90
Sol	1.0	1.0	1.0	1.0	1.0	1.000	0.808	0.669	0.518	0.390	0.288	0.198	0.119
OA	1.0	1.0	1.0	1.0	1.0	0.999	0.797	0.649	0.493	0.370	0.272	0.186	0.113
OR	1.0	1.0	1.0	1.0	1.0	1.000	0.648	0.429	0.248	0.138	0.072	0.030	0.010
$\mathit{c}<\mathit{r}$:
$\mathit{c}=\mathbf{2},\mathit{r}=\mathbf{3}$						$\mathit{\alpha }=\mathbf{3},\mathit{\beta }=\mathbf{4}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.35	0.45	0.48	0.54	0.6	0.66	0.72	0.78	0.84	0.9
Sol	1.0	1.0	1.0	1.0	0.832	0.755	0.611	0.495	0.394	0.311	0.241	0.170	0.108
OA	1.0	1.0	1.0	1.0	0.825	0.734	0.592	0.483	0.382	0.296	0.230	0.165	0.102
OR	1.0	1.0	1.0	1.0	0.745	0.621	0.424	0.292	0.190	0.120	0.066	0.030	0.010
$\mathit{c}=\mathbf{1},\mathit{r}=\mathbf{2}$						$\mathit{\alpha }=\mathbf{2},\mathit{\beta }=\mathbf{4}$
$\mathit{\vartheta }$	0.18	0.24	0.28	0.38	0.42	0.48	0.54	0.6	0.66	0.72	0.78	0.84	0.9
Sol	1.0	1.0	1.0	0.869	0.777	0.658	0.551	0.459	0.377	0.302	0.234	0.169	0.108
OA	1.0	1.0	1.0	0.866	0.766	0.641	0.538	0.447	0.366	0.289	0.225	0.164	0.102
OR	1.0	1.0	1.0	0.787	0.644	0.478	0.336	0.243	0.162	0.105	0.060	0.028	0.010
$\mathit{c}>\mathit{r}$:
$\mathit{c}=\mathbf{3},\mathit{r}=\mathbf{2}$						$\mathit{\alpha }=\mathbf{4},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.36	0.42	0.48	0.54	0.65	0.66	0.72	0.78	0.84	0.9
Sol	1.0	1.0	1.0	1.0	1.0	1.0	1.0	0.743	0.703	0.487	0.330	0.215	0.124
OA	1.0	1.0	1.0	1.0	1.0	1.0	1.0	0.721	0.682	0.475	0.318	0.205	0.120
OR	1.0	1.0	1.0	1.0	1.0	1.0	1.0	0.657	0.600	0.333	0.168	0.066	0.024
$\mathit{c}=\mathbf{2},\mathit{r}=\mathbf{1}$						$\mathit{\alpha }=\mathbf{5},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.36	0.42	0.48	0.54	0.6	0.62	0.72	0.78	0.84	0.9
Sol	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.0	0.999	0.682	0.438	0.27	0.142
OA	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.000	0.667	0.433	0.26	0.140
OR	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.000	0.622	0.352	0.17	0.068
(B) Expected time to quit
$\mathit{c}=\mathit{r}=\mathbf{1}$					$\mathit{\alpha }=\mathbf{2},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.30	0.36	0.42	0.45	0.55	0.60	0.66	0.72	0.78	0.84	0.90
Sol	1.550	1.928	2.546	3.640	6.702	10.154	9.236	4.841	3.159	2.184	1.713	1.510	1.252
OA	1.352	1.597	2.037	2.740	4.718	7.710	7.116	3.891	2.480	1.883	1.478	1.237	1.155
OR	1.562	1.905	2.482	3.701	6.852	11.253	9.882	5.529	3.193	2.177	1.710	1.426	1.095
$\mathit{c}<\mathit{r}:$
$\mathit{c}=\mathbf{2},\mathit{r}=\mathbf{3}$					$\mathit{\alpha }=\mathbf{3},\mathit{\beta }=\mathbf{4}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.35	0.45	0.48	0.54	0.6	0.66	0.72	0.78	0.84	0.9
Sol	1.897	2.711	4.408	9.158	8.613	5.552	3.033	2.136	1.619	1.388	1.262	1.089	1.007
OA	1.690	2.231	3.584	7.342	7.082	4.439	2.702	2.057	1.532	1.308	1.184	1.121	1.039
OR	1.668	2.197	3.329	7.241	7.834	4.925	2.882	2.096	1.855	1.583	1.397	1.339	1.095
$\mathit{c}=\mathbf{1},\mathit{r}=\mathbf{2}$					$\mathit{\alpha }=\mathbf{2},\mathit{\beta }=\mathbf{4}$
$\mathit{\vartheta }$	0.18	0.24	0.28	0.38	0.42	0.48	0.54	0.6	0.66	0.72	0.78	0.84	0.9
Sol	2.130	3.558	6.052	6.755	4.045	2.690	1.960	1.541	1.352	1.243	1.112	1.053	1.007
OA	1.898	2.904	5.278	6.204	3.804	2.277	1.770	1.463	1.282	1.192	1.102	1.084	1.039
OR	1.874	2.975	4.855	6.630	3.898	2.506	1.932	1.724	1.517	1.352	1.281	1.218	1.095
$\mathit{c}>\mathit{r}:$
$\mathit{c}=\mathbf{3},\mathit{r}=\mathbf{2}$					$\mathit{\alpha }=\mathbf{4},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.36	0.42	0.48	0.54	0.65	0.66	0.72	0.78	0.84	0.9
Sol	1.514	1.824	2.250	2.831	3.856	5.650	11.644	12.538	10.590	5.090	2.762	1.988	1.497
OA	1.328	1.531	1.833	2.242	2.941	4.552	10.004	9.398	7.988	4.010	2.388	1.750	1.418
OR	1.470	1.706	2.014	2.476	3.161	4.583	9.070	10.020	8.075	4.647	3.172	2.500	2.250
$\mathit{c}=\mathbf{2},\mathit{r}=\mathbf{1}$					$\mathit{\alpha }=\mathbf{5},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.36	0.42	0.48	0.54	0.6	0.62	0.72	0.78	0.84	0.9
Sol	1.444	1.692	2.004	2.414	3.034	3.958	5.837	10.977	15.353	12.359	5.390	3.174	1.961
OA	1.270	1.418	1.627	1.891	2.332	3.072	4.684	9.232	13.028	9.437	4.308	2.448	1.734
OR	1.360	1.522	1.683	1.954	2.350	3.007	4.348	7.880	11.248	9.034	4.131	3.009	2.336

Table 6. Probability of quitting and the expected time to quitting in the single agent model (Sol), the observable actions model (OA) and the observable rewards model (OR). Parameters $\mathit{\alpha }$ and $\mathit{\beta }$ set such that $u^{\ast }=2$

(A) Probability of quitting
$\mathit{c}=\mathit{r}=\mathbf{1}$						$\mathit{\alpha }=\mathbf{3},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.36	0.42	0.45	0.55	0.6	0.66	0.72	0.78	0.84	0.9
Sol	1.0	1.0	1.0	1.0	1.0	1.000	0.665	0.450	0.267	0.159	0.086	0.038	0.012
OA	1.0	1.0	1.0	1.0	1.0	0.999	0.647	0.429	0.252	0.141	0.078	0.034	0.012
OR	1.0	1.0	1.0	1.0	1.0	1.000	0.648	0.429	0.248	0.138	0.072	0.030	0.010
$\mathit{c}<\mathit{r}:$
$\mathit{c}=\mathbf{2},\mathit{r}=\mathbf{3}$						$\mathit{\alpha }=\mathbf{3},\mathit{\beta }=\mathbf{3}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.35	0.45	0.48	0.54	0.6	0.66	0.72	0.78	0.84	0.9
Sol	1.0	1.0	1.0	1.0	0.700	0.574	0.376	0.249	0.160	0.101	0.060	0.028	0.011
OA	1.0	1.0	1.0	1.0	0.664	0.532	0.356	0.228	0.146	0.089	0.052	0.025	0.010
OR	1.0	1.0	1.0	1.0	0.659	0.526	0.328	0.204	0.124	0.082	0.048	0.022	0.010
$\mathit{c}=\mathbf{1},\mathit{r}=\mathbf{2}$						$\mathit{\alpha }=\mathbf{2},\mathit{\beta }=\mathbf{3}$
$\mathit{\vartheta }$	0.18	0.24	0.28	0.38	0.42	0.48	0.54	0.6	0.66	0.72	0.78	0.84	0.9
Sol	1.0	1.0	1.0	0.760	0.604	0.434	0.303	0.214	0.145	0.096	0.058	0.028	0.011
OA	1.0	1.0	1.0	0.747	0.589	0.418	0.297	0.204	0.136	0.086	0.051	0.025	0.010
OR	1.0	1.0	1.0	0.724	0.559	0.386	0.258	0.176	0.115	0.080	0.048	0.022	0.010
$\mathit{c}>\mathit{r}:$
$\mathit{c}=\mathbf{3},\mathit{r}=\mathbf{2}$						$\mathit{\alpha }=\mathbf{7},\mathit{\beta }=\mathbf{3}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.36	0.42	0.48	0.54	0.65	0.66	0.72	0.78	0.84	0.9
Sol	1.0	1.0	1.0	1.0	1.0	1.0	1.0	0.572	0.508	0.241	0.115	0.047	0.014
OA	1.0	1.0	1.0	1.0	1.0	1.0	1.0	0.526	0.463	0.224	0.104	0.039	0.013
OR	1.0	1.0	1.0	1.0	1.0	1.0	1.0	0.542	0.481	0.208	0.088	0.032	0.010
$\mathit{c}=\mathbf{2},\mathit{r}=\mathbf{1}$						$\mathit{\alpha }=\mathbf{7},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.36	0.42	0.48	0.54	0.6	0.62	0.72	0.78	0.84	0.9
Sol	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.0	0.998	0.482	0.202	0.078	0.022
OA	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.0	0.999	0.466	0.197	0.072	0.021
OR	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.000	0.441	0.164	0.048	0.013
(B) Expected time to quit
$\mathit{c}=\mathit{r}=\mathbf{1}$					$\mathit{\alpha }=\mathbf{3},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.36	0.42	0.45	0.55	0.6	0.66	0.72	0.78	0.84	0.9
Sol	3.092	3.818	4.970	7.224	12.957	20.029	18.126	9.550	6.213	4.431	3.455	3.000	2.130
OA	2.615	3.077	3.863	5.228	9.552	15.800	14.165	7.130	4.683	3.664	3.172	2.597	2.362
OR	1.562	1.905	2.482	3.701	6.852	11.253	9.882	5.529	3.193	2.177	1.710	1.426	1.095
$\mathit{c}<\mathit{r}:$
$\mathit{c}=\mathit{r}=\mathbf{1}$					$\mathit{\alpha }=\mathbf{3},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.35	0.45	0.48	0.54	0.6	0.66	0.72	0.78	0.84	0.9
Sol	3.728	5.220	8.389	17.154	16.696	11.193	6.267	4.447	3.353	2.737	2.432	2.126	2.000
OA	3.060	4.100	6.736	14.250	13.397	7.572	5.232	3.762	2.888	2.511	2.203	2.148	2.026
OR	2.146	3.056	4.791	10.426	10.130	6.521	3.615	2.262	1.634	1.323	1.146	1.068	1.000
$\mathit{c}=\mathit{r}=\mathbf{1}$					$\mathit{\alpha }=\mathbf{3},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.28	0.38	0.42	0.48	0.54	0.6	0.66	0.72	0.78	0.84	0.9
Sol	4.173	7.032	11.962	14.484	8.156	5.432	3.862	3.085	2.767	2.584	2.258	2.082	2.000
OA	3.364	5.240	9.371	11.275	6.624	4.143	3.223	2.668	2.415	2.311	2.138	2.091	2.026
OR	2.384	3.978	6.820	8.322	5.202	2.833	1.959	1.586	1.300	1.189	1.146	1.068	1.000
$\mathit{c}<\mathit{r}:$
$\mathit{c}=\mathit{r}=\mathbf{1}$					$\mathit{\alpha }=\mathbf{3},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.36	0.42	0.48	0.54	0.65	0.66	0.72	0.78	0.84	0.9
Sol	2.992	3.560	4.337	5.489	7.374	10.940	22.088	25.305	20.474	9.427	5.633	4.096	3.000
OA	2.559	2.906	3.429	4.159	5.572	8.398	17.574	16.967	14.051	7.563	4.701	3.219	2.755
OR	1.551	1.869	2.298	3.072	4.174	6.394	12.963	13.472	11.554	5.665	2.780	1.656	1.095
$\mathit{c}=\mathit{r}=\mathbf{1}$					$\mathit{\alpha }=\mathbf{3},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.36	0.42	0.48	0.54	0.6	0.62	0.72	0.78	0.84	0.9
Sol	2.851	3.288	3.874	4.686	5.866	7.657	11.462	21.216	30.238	25.010	10.048	6.758	4.529
OA	2.467	2.712	3.087	3.552	4.325	5.639	8.380	16.212	23.482	16.732	7.119	4.745	3.602
OR	1.520	1.794	2.142	2.666	3.360	4.418	6.550	12.495	17.276	13.907	5.960	3.320	1.731

Table 7. Probability of quitting and expected time to quit in the single agent model (Sol), the observable actions model (OA) and the observable rewards model (OR). Parameters $\mathit{\alpha }$ and $\mathit{\beta }$ set such that $u^{\ast }=3$

(A) Expected time to quit
$\mathit{c}=\mathit{r}=\mathbf{1}$						$\mathit{\alpha }=\mathbf{4},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.36	0.42	0.45	0.55	0.6	0.66	0.72	0.78	0.84	0.9
Sol	1.0	1.0	1.0	1.0	1.0	1.000	0.544	0.304	0.138	0.063	0.025	0.006	0.001
OA	1.0	1.0	1.0	1.0	1.0	0.999	0.526	0.293	0.139	0.068	0.024	0.007	0.001
OR	1.0	1.0	1.0	1.0	1.0	1.000	0.441	0.194	0.062	0.019	0.006	0.000	0.000
$\mathit{c}<\mathit{r}:$
$\mathit{c}=\mathbf{2},\mathit{r}=\mathbf{3}$						$\mathit{\alpha }=\mathbf{5},\mathit{\beta }=\mathbf{5}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.35	0.45	0.48	0.54	0.6	0.66	0.72	0.78	0.84	0.9
Sol	1.0	1.0	1.0	0.999	0.594	0.436	0.238	0.128	0.066	0.033	0.013	0.004	0.001
OA	1.0	1.0	1.0	1.000	0.564	0.419	0.232	0.124	0.067	0.035	0.014	0.004	0.001
OR	1.0	1.0	1.0	1.000	0.546	0.386	0.186	0.084	0.030	0.012	0.005	0.000	0.000
$\mathit{c}=\mathbf{1},\mathit{r}=\mathbf{2}$						$\mathit{\alpha }=\mathbf{2},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.28	0.38	0.42	0.48	0.54	0.6	0.66	0.72	0.78	0.84	0.9
Sol	1.0	1.0	1.0	0.662	0.470	0.288	0.170	0.100	0.055	0.028	0.013	0.004	0.001
OA	1.0	1.0	1.0	0.622	0.441	0.264	0.152	0.092	0.051	0.028	0.012	0.004	0.001
OR	1.0	1.0	1.0	0.610	0.406	0.216	0.114	0.051	0.017	0.006	0.003	0.000	0.000
$\mathit{c}<\mathit{r}:$
$\mathit{c}=\mathbf{3},\mathit{r}=\mathbf{2}$						$\mathit{\alpha }=\mathbf{7},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.36	0.42	0.48	0.54	0.65	0.66	0.72	0.78	0.84	0.9
Sol	1.0	1.0	1.0	1.0	1.0	1.0	1.0	0.439	0.367	0.129	0.040	0.011	0.001
OA	1.0	1.0	1.0	1.0	1.0	1.0	1.0	0.396	0.317	0.116	0.036	0.008	0.001
OR	1.0	1.0	1.0	1.0	1.0	1.0	1.0	0.404	0.334	0.100	0.025	0.004	0.000
$\mathit{c}=\mathbf{2},\mathit{r}=\mathbf{1}$						$\mathit{\alpha }=\mathbf{7},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.36	0.42	0.48	0.54	0.6	0.62	0.72	0.78	0.84	0.9
Sol	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.0	0.996	0.344	0.096	0.022	0.002
OA	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.0	0.999	0.320	0.079	0.015	0.002
OR	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.0	0.999	0.315	0.076	0.011	0.000
(B) Expected time to quit
$\mathit{c}=\mathit{r}=\mathbf{1}$					$\mathit{\alpha }=\mathbf{4},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.36	0.42	0.45	0.55	0.6	0.66	0.72	0.78	0.84	0.9
Sol	3.966	4.680	5.788	7.518	10.987	19.082	29.922	28.213	15.061	9.271	6.518	5.260	4.760
OA	3.463	3.834	4.468	5.503	7.552	13.513	22.291	19.470	10.387	6.623	5.273	4.385	3.815
OR	2.646	3.140	3.893	5.038	7.396	13.353	21.750	20.367	11.100	6.712	4.921	3.454	4.000
$\mathit{c}<\mathit{r}:$
$\mathit{c}=\mathbf{2},\mathit{r}=\mathbf{3}$					$\mathit{\alpha }=\mathbf{5},\mathit{\beta }=\mathbf{5}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.35	0.45	0.48	0.54	0.6	0.66	0.72	0.78	0.84	0.9
Sol	4.487	5.678	7.859	12.435	24.902	26.330	16.966	9.608	6.672	5.149	4.614	3.623	3.429
OA	3.680	4.426	5.845	9.346	20.049	18.274	11.024	7.528	5.453	4.598	3.993	3.536	3.389
OR	2.724	3.393	4.652	7.328	15.510	15.062	10.206	5.879	4.389	3.881	3.522	3.100	4.000
$\mathit{c}=\mathbf{1},\mathit{r}=\mathbf{2}$					$\mathit{\alpha }=\mathbf{2},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.12	0.18	0.24	0.28	0.38	0.42	0.48	0.54	0.6	0.66	0.72	0.78	0.84
Sol	4.733	6.418	10.601	18.015	21.900	12.759	8.160	5.753	4.537	4.059	3.868	3.471	3.429
OA	3.911	5.140	8.152	14.868	16.413	10.208	6.396	4.823	4.194	3.628	3.460	3.229	3.000
OR	2.912	3.972	6.327	10.711	13.418	8.477	4.984	3.833	3.157	2.794	2.546	2.500	Null
$\mathit{c}>\mathit{r}:$
$\mathit{c}=\mathbf{3},\mathit{r}=\mathbf{2}$					$\mathit{\alpha }=\mathbf{7},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.36	0.42	0.48	0.54	0.65	0.66	0.72	0.78	0.84	0.9
Sol	3.911	4.511	5.332	6.422	8.088	10.913	16.426	32.571	36.403	29.336	13.765	8.543	7.140
OA	3.457	3.799	4.323	5.068	6.141	8.104	12.119	24.871	25.874	20.474	10.281	7.240	4.645
OR	2.570	2.910	3.344	3.907	4.867	6.274	9.379	18.112	19.234	17.288	8.740	4.880	4.000
$\mathit{c}=\mathbf{2},\mathit{r}=\mathbf{1}$					$\mathit{\alpha }=\mathbf{9},\mathit{\beta }=\mathbf{2}$
$\mathit{\vartheta }$	0.18	0.24	0.3	0.36	0.42	0.48	0.54	0.6	0.62	0.72	0.78	0.84	0.9
Sol	3.815	4.305	4.963	5.782	6.950	8.716	11.345	16.756	31.361	43.763	37.012	15.414	9.261
OA	3.446	3.740	4.151	4.683	5.392	6.558	8.474	12.322	23.541	33.944	25.088	10.330	7.312
OR	2.442	2.682	2.984	3.358	4.006	4.910	6.421	9.459	17.515	24.158	20.682	9.183	4.500