Last month, I chose a paper by Kris De Jaegher in Scientific Reports for our weekly lab-meeting discussion. We used the paper to teach ourselves about threshold games, and the purpose of this blog post is to summarise the things we learnt.

### Replicator dynamics revision

De Jaegher used the replicator dynamics approach. Replicator dynamics assumes a well-mixed, infinitely large population of players that reproduces asexually, where the number of offspring they produce depends upon the payoff they receive from a game-theoretic ‘game’, as illustrated below:–

We start with a population of $$n$$ individuals with different game strategies. Individuals are randomly selected to form groups and play the game. The payoff from the game determines how many offspring they have. Offspring inherit the strategies of their parents (clonal reproduction), and the cycle repeats.

Let’s rederive the dynamics. The number of individuals pursuing strategy $$i$$ changes according to

$\frac{dn_i}{dt} = \dot{n_i} = n_i (\beta + f_i),$

where $$\beta$$ is the background reproduction rate (apart from the game’s effect), and $$f_i$$ is the fitness effect of playing strategy $$i$$.

Denote the proportion of $$i$$-strategists in the population

$p_i = \frac{n_i}{N}$

where $$N$$ is the total population size.

We want to know $$\frac{dp_i}{dt}$$. Rearrange the $$p_i$$ definition and take the derivatives

$n_i = p_i N$ $\dot{n_i} = p_i \dot{N} + \dot{p_i} N$ $\dot{p} = \frac{1}{N} (\dot{n_i} - p_i \dot{N})$

So we need to sort out $$\dot{N}$$

\begin{align} \dot{N} &= \sum_i \dot{n_i} \\ &= \beta \sum_i n_i + \sum_i f_i n_i \\ &= \beta N + N \sum_i f_i \frac{n_i}{N} \\ &= \beta N + N \sum_i f_i p_i \\ \dot{N} &= N (\beta +\bar{f}) \\ \end{align}

Substitute our equations for $$\dot{N}$$ and $$\dot{n_i}$$ into the equation for $$\dot{p_i}$$, and we obtain the dynamics of strategy proportions

$\frac{dp_i}{dt} = \dot{p} = p_i (f_i - \bar{f})$

In the De Jaegher paper, players face the binary choice of cooperating or defecting. Denote cooperators and defectors by C and D, respectively, and define $$p = p_C = 1-p_D$$. Then the equation governing the dynamics simplifies to

\begin{align} \dot{p} &= p \, (f_C - [p f_C + (1-p) f_D]) \\ \dot{p} &= p (1-p) (f_C - f_D) \end{align}

### Threshold game

In the threshold game, a group is formed with $$n$$ players (different to $$n$$) above, and if the group contains $$k$$ or more cooperators, then they provide a benefit $$b$$. The benefit $$b$$ is received by all group members regardless of whether that member was a cooperator or defector

$b_i = \begin{cases} b &\text{if } i >= k \text{ (threshold met)}\\ 0 &\text{otherwise (threshold not met)} \end{cases}$

Throughout their paper, they have normalised $$b=1$$.

Everyone receives the benefit, but only cooperators pay the cost $$c$$. So the total payoff to cooperators is

$\text{cooperator payoff} = \begin{cases} b - c & \text{if threshold met,} \\ -c & \text{if threshold not met.} \end{cases}$

$\text{defector payoff} = \begin{cases} b & \text{if threshold met,} \\ 0 & \text{if threshold not met.} \end{cases}$

Defectors always do better than cooperators.

### Fitness effects

The payoffs above tell us about what happens in a particular game with a particular number of cooperators and defectors, but we need the fitness effects of being a cooperator and defector, $$f_C$$ and $$f_D$$, which are averaged over all the games played.

The replicator dynamics assumes that groups are formed randomly from an infinite population, so the make-up of the other players is binomially distributed. Therefore, the fitness effect is the sum of the payoffs from each game multiplied by the probability that the player will end up in that game

$f_C(p) = \sum_{i=0}^{n-1} \underbrace{ {n-1 \choose i} p^i (1-p)^{n-1-i}}_{\text{Pr grouped with i cooperators}} \: b_{i+1} - c,$ $f_D(p) = \sum_{i=0}^{n-1} \underbrace{ {n-1 \choose i} p^i (1-p)^{n-1-i}}_{\text{Pr grouped with i cooperators}} \: b_{i}.$

### Pivot probability

The pivot probability is the probability that a player will end up in a game where $$k-1$$ of the other $$n-1$$ players is a cooperator

$\pi_k = {n-1 \choose k-1} p^{k-1} (1-p)^{n-k}.$

Why is the pivot probability significant? First, consider just a single game, and imagine I know that I am the pivotal player. If I am a cooperator, then it is in my interests to remain a cooperator because if I switch then I will reduce my own payoff. If I am defector and I have the option of switching strategies, I would want to switch to cooperate.

We can also intuit that, if the probability that I will end up in a game where I am the pivotal player is high, then assuming I have a fixed strategy that I play all the time, it might be in my interests to be a cooperator… We’ll return to this point soon.

### First example of the threshold game

Let’s just plot an example to start with to get a sense of the dynamics. Recall

$\dot{p} = p (1-p) (f_C - f_D)$

where $$f_C$$ and $$f_D$$ defined in the paper. Let us plot this.

In the figure below, we find that the all-defector population is an evolutionarily stable steady state, and the all-cooperation population is unstable. There is also an evolutionarily stable state with a mix of cooperators and defectors (at $$p^{\star} = 0.746$$).

We can also see some of this directly by inspecting the equation for the dynamics. Recall

$\frac{dp}{dt} = p (1-p) (f_C(p) - f_D(p)).$

Steady states occur when $$\frac{dp}{dt} =0$$, so there are two trivial steady states

$p = 0,$

and

$p = 1.$

and any interior steady states solve

$0 = f_C(p) - f_D(p).$

Let’s focus on these interior steady states. Recall

$f_C(p) = \sum_{i=0}^{n-1} { n-1 \choose i } p^i (1-p)^{n-1-i} b_{i+1} - c,$

and

$f_D(p) = \sum_{i=0}^{n-1} { n-1 \choose i } p^i (1-p)^{n-1-i} b_i,$

but the $$b_i$$ values are only 1 when $$i >=k$$, otherwise 0. Therefore, we can simplify

$f_C(p) - f_D(p) = \underbrace{ {n-1 \choose k-1} p^{k-1} (1-p)^{n-k}}_{\pi_k(p)} - c,$

and notice that the pivot probability is in it.

In summary, the dynamics are

$\frac{dp}{dt} = p \, (1-p) \, \overbrace{ \underbrace{ {n-1 \choose k-1} p^{k-1} (1-p)^{n-k} }_{\pi_k(p)} - c}^{f_C(p) - f_D(p)}$

Let’s plot just the function that solves the interior steady states

$\pi_k(p) - c = 0.$

### The peak

De Jaegher found that, when $$1 < k < n$$, both $$f_C - f_D$$ and $$\pi_k$$ have a peak at $$\hat{p} = \frac{k-1}{n-1}$$. They found this by taking advantage of the fact that the function is unimodal, and finding when the derivative was 0.

$f_C(p) - f_D(p) = {n-1 \choose k-1} p^{k-1} (1-p)^{n-k} - c,$

so

$\frac{d(f_C(p) - f_D(p))}{dp} = {n-1 \choose k-1} (1-p)^{n-k-1} p^{k-2} \bigl( (k-1)(1-p) + (k-n) p \bigr).$

The peak is located where $$\frac{d(f_C(p) - f_D(p))}{dp} = 0$$ that is

$0 = (k-1)(1-\hat{p}) + (k-n) \hat{p},$

which gives the peak location

$\hat{p} = \frac{k-1}{n-1}.$

Because $$f_C - f_D$$ and $$\pi_k$$ differ only by a constant, this point is also the location of the peak of the pivot probability function.

Knowing where this peak is and whether it is above or below the line tells us about the dynamics, as will become clearer in the examples below.

### Examples for Result 1

Below, we’ll go through examples for each of the cases in Result 1.

#### When $$k=1$$

For the minimal threshold $$k = 1$$, the game has a unique interior fxed point where a fraction $$p_1^{\text{II}} = 1-c^{1/(n-1}$$ of players cooperates (Volunteer’s Dilemma).

#### When $$1 < k < n$$

Low cost

When participation costs are small ($$c < \bar{c}_k$$), the game both has a fixed point where all players defect ($$p = 0$$) and a stable fixed point $$p_k^{\text{II}}$$, which is implicitly given by $$\pi_k(p_k^{\text{II}}) = c$$.

High cost

When participation costs are high, $$c > \bar{c}_k$$, the game has a unique stable fixed point where all players defect ($$p = 0$$).

What’s the significance of this $$\bar{c}_k$$ value? Recall

$f_C(p) - f_D(p) = \underbrace{ {n-1 \choose k-1} p^{k-1} (1-p)^{n-k}}_{\pi_k(p)} - c.$

They’ve defined

$\bar{c}_k = \pi_k(\hat{p}),$

where $$\hat{p}$$ is the probability at the peak. So we know that, if $$\bar{c}_k > c$$, the $$f_C - f_D$$ line will be above the zero line at the peak and there will be an interior equilibrium, and if $$\bar{c}_k < c$$, the peak is below the line and there are no interior equilibria.

#### When $$k = n$$

Our final case is the maximum threshold, $$k=n$$. Here, the game both has a stable steady state where all players defect ($$p = 0$$), and a stable steady state where all players cooperate ($$p = 1$$). The basin of attraction is defined by $$p_n^{\text{I}} = c^{1/(n-1)}$$, the unstable interior steady state.

#### Bringing it all together

We can place each of these examples on De Jaegher’s Fig. 4. The 4 regions of the figure represent four different qualitative regimes for the dynamics (game types).

### Examples for Result 2

De Jaegher found that the threshold level has a U-shaped efect on the level of cooperation, which can be seen in their Fig. 4 (the shape of the division between the blue and green region). Let’s plot how the dynamics varies with varying threshold level.

Here, we see that cooperation can evolve both for low and for high thresholds, but not for intermediate thresholds. It is perhaps surprising because, intuitively, it seems like a high threshold would be more difficult to obtain.

This U-shape emerges from the pivot probabilities.

### Group size

De Jaegher discuss the possibility of stabilising a game by changing the group size. If cooperation is more likely to persist at low or high thresholds, then perhaps increasing or decreasing the group size to shift the relative position of the threshold could stabilise cooperation.

It turns out this is true for decreasing the group size, but not for increasing it. Increasing the group size has a negative effect on cooperation.

The reason why is due to the effect of group size on the pivot probability: the larger the group is, the less likely it is for a player to be the pivotal player, and so the lower the benefit of being a cooperator.

### References

De Jaegher, K. (2020). High thresholds encouraging the evolution of cooperation in threshold public-good games. Scientific Reports, 10(1), 1-10.