The Prisoner’s Dilemma game as a tool to investigate cooperation and undergraduate education in evolution

doi:10.21203/rs.3.rs-3911293/v1

Download PDF

Research Article

The Prisoner’s Dilemma game as a tool to investigate cooperation and undergraduate education in evolution

https://doi.org/10.21203/rs.3.rs-3911293/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 27 Jul, 2024

Read the published version in Evolution: Education and Outreach →

You are reading this latest preprint version

Evolutionary theory is based on the conflict that arises when certain heritable variants out-compete others. Given this foundational conflict, a central question for evolutionary biologists concerns the presence of cooperation found throughout all levels of biological organization; from biochemical pathways to complex animal societies. Human behavior is often distinguished from other animal behavior by the presence of acts of cooperative behavior called altruism. Altruism is a cooperative act that penalizes the actor for actions that benefit the recipient. Any other form of cooperation, one that does not penalize the actor, is formally not considered altruistic. How can costly altruistic behavior evolve? This question was the basis for development of a web application tool incorporating a game theory model to investigate conditions affecting cooperative behavior. The game theory model described as Prisoner’s Dilemma incorporates acts of cooperation and non-cooperation (defection). Computer simulations of Prisoner’s Dilemma were developed and online applications were administered for five semesters at Georgia Gwinnet College, using two simulation environments referred to as Random and Non-random. Data collected from simulation runs were used to evaluate the effect of environment on student cooperative behavior and actively engage students in concepts associated with the evolution of cooperation and game theory. Results from student game play suggest group simulation environment played a significant role in the likelihood of observing cooperative behavior. Educational content and attitudinal surveys suggested that PD game play in the undergraduate evolution class at Georgia Gwinnett College improved student knowledge and self-confidence.

Cooperation

Prisoner’s Dilemma

Undergraduate

Living systems do not operate in isolation. Interactions between entities define the structure and function of the living world. These interactions can be categorically simplified to antagonistic or complementary interactions; or competitive and cooperative behavior. Cooperative behavior among individuals is a defining characteristic of complex organismal societies and was considered a particular difficulty, “…actually fatal to my theory…” for Charles Darwin (1859) in light of his theory of evolution by natural selection. The difficulty arises from a distinct definition of the word cooperation that is more properly termed altruism. Altruistic behavior is defined by a cooperative act that imposes a fitness penalty on the actor, and a concomitant fitness benefit for the recipient of the act. For Darwin, and many other evolutionary biologists (Hamilton 1964, Doebeli & Hauert 2005), this type of behavior should not evolve by natural selection. And yet it has. Why?

There are two primary explanations for the evolution of cooperative behavior. One explanation for this type of behavior is based on genetic relatedness. In this formulation, costly behavior will benefit the genes of the actor if there is a sufficiently high statistical probability those genes are present in the recipient. This is formally called kin selection and was developed by W.D. Hamilton (1964). But this doesn’t help explain the observation of altruistic behavior, particularly in human societies, among unrelated individuals. Another explanation for altruistic behavior is based on the frequency of interactions between individuals, regardless of relatedness, in a society. When the opportunity for cooperative, or altruistic, behavior is iterated by repeated interactions between individuals who are likely to reciprocate acts of cooperation, such behavior is more likely to evolve (Trivers 1971).

Reciprocity is defined by a time lag between acts of cooperation in which an actor provides benefits to a recipient, who at some later time reciprocates the beneficial act. It is easy to imagine a non-cooperative, or selfish, recipient who fails to reciprocate; which is why many have suggested cooperation and the exploitation of cooperation may have been significant evolutionary forces shaping human psychology (Romano & Balliet 2017, Trivers 1971). Selfish behavior benefits in a collective cooperative group setting because there is an intrinsic cost in the act of cooperation. If there were no cost, there would be no benefit to avoiding cooperation. Therefore, for cooperation to evolve in any setting, short term costs of cooperation must be more than recovered in the long term. This is the premise of the evolution of human psychology associated with helping, or altruistic, behavior (Kurzban et al. 2015).

The Prisoner’s Dilemma is an elegant game theory model that addresses the inherent stress selfish individuals place on cooperative group behavior (Axelrod 1980). Assuming a selfish non-cooperator is called a defector; the model is defined by the following interaction conditions: the payoff for a defector interacting with a cooperator (T) is greater than a cooperator interacting with another cooperator (R), which is greater than a defector interacting with another defector (P), which is greater than a cooperator interacting with a defector (S); and 2R is greater than T + S. The model encapsulates the dilemma of cooperation because for any interaction individual selection would favor defection from cooperation. But since the game is symmetrical any two individuals would benefit more from collective cooperation than from collective defection (Killingback & Doebeli 2002).

We have taken advantage of these properties of The Prisoner’s Dilemma (PD) game to develop a web application that was used to test the likelihood of cooperative behavior in different group settings and the effect game play would have on student comprehension of instructional material related to the evolution of cooperation. Hypotheses regarding cooperative behavior were largely derived from Trivers (1971) development of the evolution of reciprocal altruism, principally that cooperative behavior was more likely to evolve in smaller distinct group settings that we termed “Non-random”, when compared to larger group settings that we termed “Random”. The hypothesis of cooperation evolving in selective group environments was tested by distinct versions of the web application. Student comprehension of instructional material related to the evolution of reciprocal altruism was assessed with pre- and post-game play assessments. The primary learning goals associated with implementation of the game was to demonstrate comprehension of content related to evolution and cooperation. Qualitative assessments addressed student appreciation of, and engagement in PD game playing.

An online interactive game with user and administrative functions was developed for use in the classroom. Two versions of the game were developed to test hypotheses related to reciprocal altruism (Trivers 1971). One version, called Random, allowed all users to access the game environment with an equal probability of encountering other online users in dyadic interactions defined by the rules of the game Prisoner’s Dilemma (Rapaport & Chammah 1965). The other version of the game, called Non-random, confined the interactions to groups of four players defined by the administrator under encoded rules that limited the composition of the four-player group according to the established online identities. Online identities were established upon player registration, which consisted of username and password, a randomly assigned one of three colors, Blue, Red, and Yellow, and a number generated by the application. After player login with their username and password, the display consists of the color and number combination, such as Red23, to other players currently online (Fig. 1). Each player online will be able to check other players online with such color-number format. The four-player Non-random version of the game grouped three players of different color with another player matching the color of one of the three group members. The intended purpose of the game was to collect personal decisions and scores based on the rules of the Prisoner’s Dilemma game and emphasize and engage students in the concepts related to the evolution of cooperative behavior as taught in the evolution course BIOL 3000.

Georgia Gwinnett College students in the Evolution course, BIOL 3000, played the online Prisoners Dilemma (PD) game multiple semesters, different students playing each semester. A total of 228 players, 137 players in the Random treatment and 91 in the Non-random treatment, across the semesters played the online game throughout the semester. Students were introduced to the rules of the game and the payoff matrix (Fig. 2) in class and through online resources at the beginning of each semester. Students were not aware of the applied treatment effects. Students were incentivized to adopt “winning” strategies by the offer of extra credit points at the end of each semester for the student(s) with the highest score.

Game-play consisted of weekly, or bi-weekly, time periods at the beginning of class lasting approximately five minutes in which students would log into the online PD game server and play a series of rounds consisting of from three to five iterations of the game. Each game was a one-on-one (dyadic) encounter between students randomly assigned according to treatment effects and consisted of iterated decisions to cooperate or defect from cooperation. Students were aware of the color/number of the other player but were encouraged to keep online identities confidential. After each iteration scores and decisions were shown to players and cumulative scores across rounds were shown to students individually. The number of rounds per session was determined by the time period and rate of play. The number of iterations per game was determined by the administrator.

Assessment of instructional content related to the evolution of cooperative behavior and game theory was administered prior to game play at the beginning of each semester and at the end of each semester after game play ceased (Pre- and Post-game play). Content assessment consisted of five questions with four multiple choice answers each (Supp. Material 1). Pre- and Post-game play assessment scores were analyzed using Welch two sample t-test in R statistical language (R Core Team 2020). Qualitative assessments of the PD Game were conducted at the end of each semester using a Likert scale for scoring. All assessments were voluntary and anonymous.

Player score information was collected and stored in the server which could be exported as an Excel file for analysis at the end of each semester. Percent cooperation was determined as a fraction of the maximum collective score (six points per iteration) weighted by the number of game iterations per round. Random and Non-random treatments were analyzed for significant differences in overall percent cooperation using R statistical language and a Welch two sample t-test. Strategic game play, defined as Tit-For-Tat (TFT) or Pavlov (Milinski 1993, Wedekind & Milinski 1996), was determined for the subset of games for which strategies could be determined (games one through three). The determination protocol for both strategies was based on the probability of cooperation, P_c, given previous game conditions such that Tit-For-Tat P_c = 1 and Pavlov P_c = 0 after previous defection with partner cooperation, and Tit-For-Tat P_c = 0 and Pavlov P_c = 1 for previous defection and partner defection. Strategic differences between treatments within rounds of play, and within treatments between rounds of play were analyzed using a chi-squared test for equality of proportions (prop.test function) in R statistical language (R Core Team 2020).

The 137 students in the Random treatment played a total of 2248 rounds, 963 of which were five-game rounds and 1285 were three-game rounds, for a total of 8670 cooperate/defect decisions made by students. The 91 students in the Non-random treatment played a total of 1251 rounds, 445 of which were five-game rounds, 677 were four-game rounds and 129 were three-game rounds, for a total of 5320 cooperate/defect decisions made by students.

Decision-making behavior of students in both treatments conformed more frequently to strategic decision-making behavior referred to as Tit-For-Tat, than Pavlov strategic decision-making behavior. Strategic decision-making was based on previous knowledge, so the information from the first round was used as previous knowledge for the second round. Significantly higher levels of TFT were observed for both treatments across all rounds of play (Fig. 4), with a significant increase in TFT decision-making between rounds two, three, and four (x² = 250, df = 3, p < 0.001). Non-random players were more likely to initiate a TFT decision strategy (round 2 x² = 6.53, df = 1, p < 0.02) compared to random mode players, and a significant increase was observed between rounds one and two (x² = 35.65, df = 3, p < 0.001).

Significant differences were observed for pre- and post-game play instructional content assessment (t = 5.6505, df = 213.89, p-value = 5.07e-08; Fig. 5). Average assessment score for pre-game play was 29.6 (s = 23.0); average assessment score for post-game play was 47.3 (s = 26.1). Student responses to post-content qualitative assessment suggested the PD game was easy to use, enjoyable, and improved student understanding and confidence regarding instructional content (Fig. 6).

Smaller social environments, as those modeled by the Non-random treatment, increase the opportunity for specific interactions between individuals and allow for greater prior knowledge in regard to decision-making. These are considered preconditions for the evolution of cooperation (Trivers 1971). Animal examples of cooperative behavior in smaller social groups come from vampire bats (Wilkinson 1984), where interaction opportunities were positively associated with cooperative feeding behavior, and from meercats (Rotics and Clutton-Brock, 2021), where larger social environments resulted in some loss of cooperation among individuals. Smaller social groupings are considered necessary prerequisites for increasing the probability of the evolution of cooperation (Ale et al. 2013). In general, the observed treatment effects, greater percentage of cooperation in the smaller group setting (Non-random), conforms with expectation regarding the evolution of cooperation.

Observed cooperation was low in both treatments (31.9%-45.9%), but not unexpected given game constraints regarding restrictions on the number of rounds players would interact. Though the number of interaction rounds varied from three to five, at the discretion of the game administrator, for any given class period the number of interaction rounds was set. This allowed students to develop end-game strategies that incentivized non-cooperation strategies. For example, pairwise cooperation in early rounds created an environment in which the first player to choose non-cooperation (defect) would be rewarded (Selten and Stoecker 1986). When there is a known finite number of rounds non-cooperation is considered an evolutionary stable strategy (ESS) (Axelrod and Hamilton 1981). Our overall percent cooperation results align with previous findings with multiplayer iterated Prisoner’s Dilemma game play (Grujic et al. 2012) where percent cooperation was approximately 30%, similar to random treatment observed percent cooperation, and to findings with global competition, a reward scenario similar to the one used in the present study, in which 44% cooperation was observed (West et al. 2006), similar to non-random treatment results.

Decision-making was observed in the context of two well-described PD game strategies: Tit-for-Tat (TFT) and Pavlov (Axelrod 1980, Axelrod and Hamilton 1981, Nowak and Sigmund 1993). Both strategies use prior knowledge to inform subsequent decisions, but do it in different ways. TFT starts with cooperation then replies in kind to the previous decision of the partner such that previous partner cooperation results in subsequent cooperation, and previous non-cooperation results in subsequent non-cooperation. Pavlov is considered a “win-stay, lose-shift” strategy (Nowak and Sigmund 1993) in which prior knowledge is used to maximize payoffs such that the winning decision for cooperation (when the partner also cooperates) or non-cooperation (when the also partner cooperates) results in the same behavioral decision the following round (win-stay). Any other combination, which results in the lowest point value for that decision, results in the behavioral shift the following round (lose-shift). While TFT is based on reciprocation, Pavlov is based on reward.

Both treatments suggested student players were much more likely to employ TFT-like decision making behavior, rather than Pavlov (Fig. 4). TFT is considered an ESS when prior knowledge plays a significant role in decision-making (Ale et al. 2013). In our game-play environment the Non-random treatment is assumed to have greater prior knowledge, as is the higher round number. Consistent with expectation higher round numbers resulted in higher frequencies of TFT (Fig. 4). However, the Non-random treatment did not result in higher TFT frequency compared to the random treatment. The Non-random treatment TFT plateau reached in round 3 may reflect a limit to reciprocal altruism in the face of an alternative Pavlov strategy. It should be noted that strategic game play was not discussed when the game was initiated in the classes. Students were left to discover decision-making strategies on their own.

Instructional content assessment scores, though low overall, showed significant post-game play gains (Fig. 5). Georgia Gwinnett College is a relatively new (established 2006) undergraduate degree granting institution in the University System of Georgia (USG). As such, assessments play an outsized role in the administration of educational content and policy. There may be some form of “assessment fatigue” among students that partially explains the overall low assessment scores. However, significant gains in content comprehension suggest PD game play and instructional methodology are effective educational tools regarding content related to the evolution of cooperation. Student responses to a qualitative assessment of the PD game were positive regarding ease of use, enjoyment, and application (Fig. 6). Using a Likert scale, students agreed with the statements “I think the game is easy to use”, and “I enjoy playing the game”. Students also found the game was applicable to their comprehension of instructional content, in general, and to the evolution of cooperation specifically. These results are not surprising given the wealth of data supporting active learning, in general, as useful tool for undergraduate engagement and comprehension (Michael 2006, Russell et al. 2015), and the PD game, specifically, in terms of education related to more advanced concepts related to conflicts of interest (Dennis 2015, Bruno et al. 2018).

The PD game is an active practice for students that can illustrate the conditions for the evolution of cooperation- namely prior knowledge and a considered response to cooperation or non-cooperation. Given the limitations of the game initiated at Georgia Gwinnett College, several future developments may be useful for further analysis of cooperative decision-making behavior, and education in evolution. Due to the end-game strategy option mentioned previously, which encourages non-cooperation (Selton and Stoecker 1986), increasing the number of interaction rounds per game may be useful for gauging the effect of end-game strategies on the evolution of cooperation. Another development that would involve a significant modification to the game would be to include a punishment option, in which players could choose to punish non-cooperative partners, at a cost, that would result in point deductions, similar to punishment options Leighton (2014) describes for public goods games. Punishment options provide the opportunity to disincentivize the temptation to not cooperate (Burguillo 2010). For a more complete analysis of the pedagogical efficacy of the PD game a control treatment in which the same instructional content, and preferably the same instructor, without the PD game could be used as a comparison to the experimental treatment. It may also be useful in future iterations of the PD game to incorporate periodic debriefing sessions (Bruno et al. 2018) in which students are encouraged to discuss applications of the game and strategic game-play options.

The PD game serves as an excellent opportunity for students of evolution to actively participate in decision-making behavior that highlights the conflicts associated with the evolution of cooperation. Although this game can be played at any level of education, the complexity of iterative decision-making, prior knowledge, and strategic game play seem to make this more suitable for upper-level high school and undergraduate students, in our opinion. When combined with instructional content related to the evolution of cooperation (Trivers 1971, Hamilton 1964) a more comprehensive curriculum that incorporates active learning can be developed. The PD game platform used in the present study was intended to be a first step in the development of an educational tool that increases student engagement in content related to the evolution of cooperation.

Our dual purpose in developing the online game was to: 1) test hypotheses related to the evolution of cooperation and 2) test the efficacy of the game in an undergraduate evolution course. Student decision making behavior collected during the course of game play allows administrators of the PD Game to analyze the two game environments, random and non-random, for cooperative behavior. Evolutionary theory regarding cooperation suggests that smaller group environments with increased opportunities for interaction should promote the evolution of cooperative behavior. Relative to the game’s random environment, the non-random environment provides an experimental treatment to test hypotheses related to these conditions. Increased cooperation in the non-random treatment reported herein, seem to support existing theory.

Incorporating the PD Game in curriculum related to the evolution of cooperation is straightforward. Student registration and game play can take place on a laptop or cell phone with wireless access. Multiple rounds can be played within a very short space of time since a typical round takes less than one minute. Existing literature on the game as a tool for understanding the evolution of cooperation provides excellent supplementary material for instructional content on evolution, behavior, and social psychology (Trivers 1971, Axelrod and Hamilton 1981, Kurzban et al. 2015). Results from the initial offering of the game suggest students enjoy playing the game, find it applicable to the study of evolution, and increase their understanding of instructional content related to the evolution of cooperation.

While the reported results appeared to support our purpose, we recognize further modifications in game and instructional design may be more effective for educational purposes. In particular, a more explicit integration of the PD Game and strategic game play options with instructional content would allow students to analyze how environmental conditions can impact the evolution of individual and group behavior. Future iterations of the PD Game pedagogy might benefit from periodic debriefing sessions where individual game play strategies related to reciprocation (TFT) and reward (Pavlov) are discussed in the context of evolutionary stable strategies.

Ethics approval and consent to participate

All students provided informed consent and assessment results were collected with absolute anonymity following procedures approved by Georgia Gwinnett College IRB Study No.: 110068.

Funding

No funding was received for this research. However, the project was incentivized by approval from the Board of Reagents of the University System of Georgia STEM Initiative Program.

Author Contribution

J.R. wrote the manuscript, conducted statistical analyses, and prepared the figures.X.X. developed the online application, reviewed and edited the manuscript.

Availability of data and materials

Assessment template is provided in Supplementary Material. PD Game data is available in Excel format from the authors upon request.

Ale, S. B., Brown, J. S., & Sullivan, A. T. (2013). Evolution of cooperation: combining kin selection and reciprocal altruism into matrix games with social dilemmas. PloS One, 8(5), e63761.
Axelrod, R. (1980). More effective choice in the prisoner’s dilemma. Journal of Conflict Resolution, 24(3), 379-403.
Axelrod, R., & Hamilton, W. D. (1981). The evolution of cooperation. Science, 211(4489), 1390-1396.
Bruno, A., Dell’Aversana, G., & Guidetti, G. (2018). Developing organizational competences for conflict management: the use of the prisoner’s dilemma in higher education. Front. Psychol. 9: 376. Doi: 10.3389/fpsyg.2018.00376
Burguillo, J. C. (2010). Using game theory and competition-based learning to stimulate student motivation and performance. Computers & Education, 55(2), 566-575.
Darwin, Charles, 1809-1882. (1859). On the origin of species by means of natural selection, or preservation of favoured races in the struggle for life. London :John Murray,
Doebeli, M., & Hauert, C. (2005). Models of cooperation based on the Prisoner’s Dilemma and the Snowdrift game. Ecology Letters, 8(7), 748-766.
Grujić, J., Eke, B., Cabrales, A., Cuesta, J. A., & Sánchez, A. (2012). Three is a crowd in iterated prisoner’s dilemmas: experimental evidence on reciprocal behavior. Scientific Reports, 2(1), 638.
Hamilton, W. D. (1964). The genetical evolution of social behaviour. II. Journal of Theoretical Biology, 7(1), 17-52.
Killingback, T., & Doebeli, M. (2002). The continuous prisoner’s dilemma and the evolution of cooperation through reciprocal altruism with variable investment. The American Naturalist, 160(4), 421-438.
Kurzban, R., Burton-Chellew, M. N., & West, S. A. (2015). The evolution of altruism in humans. Annual Review of Psychology, 66, 575-599.
Leighton, G. M. (2014). Multiple interactive public goods games allows for exploration of evolutionary mechanisms that maintain cooperation. Evolution: Education and Outreach, 7, 1-7.
Michael, J. (2006). Where’s the evidence that active learning works?. Advances in Physiology Education.
Milinski, M. (1993). Cooperation wins and stays. Nature, 364(6432), 12-13.
Nowak, M., & Sigmund, K. (1993). A strategy of win-stay, lose-shift that outperforms tit-for-tat in the Prisoner’s Dilemma game. Nature, 364(6432), 56-58.
R Core Team (2012). R: A language and environment for statistical computing. R foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.
Rapoport, A., & Chammah, A. M. (1965). Prisoner’s dilemma: A study in conflict and cooperation. Ann Arbor, MI: University of Michigan Press.
Rotics, S., & Clutton-Brock, T. (2021). Group size increases inequality in cooperative behaviour. Proceedings of the Royal Society B, 288(1945), 20202104.
Romano, A., & Balliet, D. (2017). Reciprocity outperforms conformity to promote cooperation. Psychological Science, 28(10), 1490-1502.
Russell, J. E., D’Costa, A. R., Runck, C., Barnes, D. W., Barrera, A. L., Hurst-Kennedy, J., … & Haining, R. (2015). Bridging the undergraduate curriculum using an integrated course-embedded undergraduate research experience (ICURE). CBE—Life Sciences Education, 14(1), ar4.
Selten, R., & Stoecker, R. (1986). End behavior in sequences of finite Priso’er's Dilemma supergames A learning theory approach. Journal of Economic Behavior & Organization, 7(1), 47-70.
Trivers, R. L. (1971). The evolution of reciprocal altruism. The Quarterly Review of Biology, 46(1), 35-57.
Wedekind, C., & Milinski, M. (1996). Human cooperation in the simultaneous and the alternating Priso’er's Dilemma: Pavlov versus Generous Tit-for-Tat. Proceedings of the National Academy of Sciences, 93(7), 2686-2689.
West, S. A., Gardner, A., Shuker, D. M., Reynolds, T., Burton-Chellow, M., Sykes, E. M., ... & Griffin, A. S. (2006). Cooperation and the scale of competition in humans. Current Biology, 16(11), 1103-1106.

No competing interests reported.

Supp.material.docx

Download PDF

Journal Publication

published 27 Jul, 2024

Read the published version in Evolution: Education and Outreach →

Editorial decision: Revision requested
01 May, 2024
Reviews received at journal
01 May, 2024
Reviews received at journal
02 Apr, 2024
Reviewers agreed at journal
27 Mar, 2024
Reviewers agreed at journal
09 Feb, 2024
Reviewers invited by journal
09 Feb, 2024
Submission checks completed at journal
03 Feb, 2024
Editor assigned by journal
03 Feb, 2024
First submitted to journal
30 Jan, 2024

You are reading this latest preprint version

The Prisoner’s Dilemma game as a tool to investigate cooperation and undergraduate education in evolution

Status:

Journal Publication

Version 1

Abstract

Figures

Introduction

Methods

Results

Discussion

Conclusions

Declarations

Funding

Author Contribution

Availability of data and materials

References

Additional Declarations

Supplementary Files

Status:

Journal Publication

Version 1