Project Details

Description

Scaling up commons dilemma experiments for research and education Scaling up commons dilemma experiments for research and education Many of the challenges facing contemporary society can be categorized as common-pool resource problems, which are typically manifested by overuse and overharvesting such as deforestation, overfishing, information overload, and pollution. This field has derived many insights from case study analysis and controlled behavioral experiments. Nevertheless, three fundamental puzzles still dominate this area of research, namely why is communication so effective to stimulate cooperative outcomes; how do group size impact the ability to cooperate; and how do groups cope with uncertainty and disturbances. In order to address those puzzles, we need to scale up the way we do controlled experiments. We built on a proven engaging card game, the Port of Mars, in which a group of players need to make investment decisions about the shared infrastructure and about actions that benefit themselves. The game is based on commons dilemmas and is fun to play. In developing a digital version of the game, we will be able to run large scale controlled experiments. The game allows for in-game chat communication and includes stochastic events that enable us to address uncertainty. In order to recruit a large number of participants, we team up with the Interplanetary Initiative at Arizona State University and hold each semester a Mars Madness tournament in which participants will have the opportunity to become the champion of Port of Mars. Initially those participants are students, but in the third year of the project, the tournament is open to the general public. Data collected during the tournaments will be used to train a machine-learning algorithms to classify communication data into functional categories which help us to study how communication content relate to group performance. In the various tournaments we vary the knowledge about events and thresholds events may happen in order to test the impact of uncertainty on collective action. Furthermore, we vary group size between 5 and 50 players. We will also test whether participation in the Port of Mars impacts the understanding of concepts of commons dilemmas and solutions to overcome the tragedy of the commons. The resulting experimental platform aims not only to be research platform for commons but also to be an educational environment addressing questions in social science and STEM education. REU: Scaling up commons dilemma experiments for research and education Introduction The NSF funded project will expand a web-based experiment, the Port of Mars game, where a group of players make decisions to invest in shared infrastructure and perform actions that benefit themselves. The platform will enable us to run large-scale controlled experiments with in-game chat communication and random events that introduce uncertainty and variance into each playthrough. To recruit a large number of participants, each semester we will hold a tournament, coined Mars Madness, where participants will compete to become the champion of Port of Mars. Initial participants will be college students, but in the third year of the project, we will open the tournament to the general public. Each semester we will implement one of the six experimental designs to test our hypotheses on communication, group size and uncertainty. Data collected during the tournaments will be used to train machine-learning models that classify communication data into functional categories which will help us to better understand how communication among players could relate to group performance. In the various tournaments we will adjust group sizes between five and fifty players and evaluate the impacts of uncertainty on collective action by controlling players knowledge of events and adjusting the thresholds that trigger events. Furthermore, we test how participating in the behavioral experiments improve the understanding of collective action problems. Scope of work The activities where REU students will be involved in will the training of machine learning algorithms to analyze data from the tournament data. This will enable students to learn about machine learning applied in the social sciences, data management techniques, and data analysis. To evaluate the large amount of game data we will make use of supervised machine learning algorithms, especially for classification of the communication data. With an increasing size of players in the tournaments we may get soon more than 100,000 lines of communication data per tournament. In order to identify what participants communicate about, and how this explain observed differences in group performance In the Spring 2021 we have start analyzing communication data from a pilot tournament with 3 undergraduate interns from the School of Sustainability. We developed a code book to identify five conversation topics, and each chat entry was independently classified by the PI Janssen and the interns. We used Cohen-Kappa and Krippendorff-alpha scores to evaluate the intercoder reliability. It requires a number of samples and meetings to get sufficient mutual understanding how to code the communication data consistently. The coded data was subsequently used by co-PI Simeone to train an initial machine learning algorithm, and he confirmed that the coded data is a good starting point for training algorithms to classify communication data. After each tournament, we will code new data to provide new training data for the algorithms. As such each summer we will have two undergraduate students will manually label sentences for continuously training the algorithms. Crucially, we see this continuous evaluation of the models performance as a way to update the classification routine to reflect observations made by human evaluators. Results will be reviewed and corrected according to the procedure manual, then used as part of an update to the training of the original model. It is important that the updated sentences receive the same level of scrutiny and interrater evaluation as the original train/test data described above. Besides coding data, we will train the undergraduate students to use machine learning software to enable them to do their own projects analyzing communication and game data. For example, standard natural learning processes tools can be used for analysis of communication such as sentiment analysis. In fact, we found in the pilot data that group that were more successful have more positive sentiments in their communication compared to groups that were not able to complete the tasks successfully. We are also interested to evaluate what students learn by playing the game. Co-PI Dr. Hong, an assistant professor in education, will work on this component of the project. Also for this component we aim to use machine learning to classify game play data that can be used to analyze what students learn, such as question to other players about the working of the game, decisions made by players, and responses to survey questions.
StatusActive
Effective start/end date1/1/2112/31/25

Funding

  • National Science Foundation (NSF): $488,759.00

Fingerprint

Explore the research topics touched on by this project. These labels are generated based on the underlying awards/grants. Together they form a unique fingerprint.