Abstract

The main point of this paper is to provide an affirmative answer through exploiting reinforcement learning (RL) in artificial intelligence (AI) for eliminating herding without any external control in complex resource allocation systems. In particular, we demonstrate that when agents are empowered with RL (e.g., the popular Q-learning algorithm in AI) in that they get familiar with the unknown game environment gradually and attempt to deliver the optimal actions to maximize the payoff, herding can effectively be eliminated. Furthermore, computations reveal the striking phenomenon that, regardless of the initial state, the system evolves persistently and relentlessly toward the optimal state in which all resources are used efficiently. However, the evolution process is not without interruptions: there are large fluctuations that occur but only intermittently in time. The statistical distribution of the time between two successive fluctuating events is found to depend on the parity of the evolution, i.e., whether the number of time steps in between is odd or even. We develop a physical analysis and derive mean-field equations to gain an understanding of these phenomena. Since AI is becoming increasingly widespread, we expect our RL empowered minority game system to have broad applications.

Original languageEnglish (US)
Article number032302
JournalPhysical Review E
Volume99
Issue number3
DOIs
StatePublished - Mar 6 2019

Fingerprint

resource allocation
Minority Game
games
Optimal Allocation
minorities
Reinforcement Learning
reinforcement
Resource Allocation
artificial intelligence
Herding
learning
Artificial Intelligence
Mean Field Equation
Q-learning
Statistical Distribution
Parity
Learning Algorithm
interruption
Odd
Maximise

ASJC Scopus subject areas

  • Statistical and Nonlinear Physics
  • Statistics and Probability
  • Condensed Matter Physics

Cite this

Reinforcement learning meets minority game : Toward optimal resource allocation. / Zhang, Si Ping; Dong, Jia Qi; Liu, Li; Huang, Zi Gang; Huang, Liang; Lai, Ying-Cheng.

In: Physical Review E, Vol. 99, No. 3, 032302, 06.03.2019.

Research output: Contribution to journalArticle

Zhang, Si Ping ; Dong, Jia Qi ; Liu, Li ; Huang, Zi Gang ; Huang, Liang ; Lai, Ying-Cheng. / Reinforcement learning meets minority game : Toward optimal resource allocation. In: Physical Review E. 2019 ; Vol. 99, No. 3.
@article{455f1ef7b03f4f8394c5297d86328cf4,
title = "Reinforcement learning meets minority game: Toward optimal resource allocation",
abstract = "The main point of this paper is to provide an affirmative answer through exploiting reinforcement learning (RL) in artificial intelligence (AI) for eliminating herding without any external control in complex resource allocation systems. In particular, we demonstrate that when agents are empowered with RL (e.g., the popular Q-learning algorithm in AI) in that they get familiar with the unknown game environment gradually and attempt to deliver the optimal actions to maximize the payoff, herding can effectively be eliminated. Furthermore, computations reveal the striking phenomenon that, regardless of the initial state, the system evolves persistently and relentlessly toward the optimal state in which all resources are used efficiently. However, the evolution process is not without interruptions: there are large fluctuations that occur but only intermittently in time. The statistical distribution of the time between two successive fluctuating events is found to depend on the parity of the evolution, i.e., whether the number of time steps in between is odd or even. We develop a physical analysis and derive mean-field equations to gain an understanding of these phenomena. Since AI is becoming increasingly widespread, we expect our RL empowered minority game system to have broad applications.",
author = "Zhang, {Si Ping} and Dong, {Jia Qi} and Li Liu and Huang, {Zi Gang} and Liang Huang and Ying-Cheng Lai",
year = "2019",
month = "3",
day = "6",
doi = "10.1103/PhysRevE.99.032302",
language = "English (US)",
volume = "99",
journal = "Physical Review E - Statistical, Nonlinear, and Soft Matter Physics",
issn = "1539-3755",
publisher = "American Physical Society",
number = "3",

}

TY - JOUR

T1 - Reinforcement learning meets minority game

T2 - Toward optimal resource allocation

AU - Zhang, Si Ping

AU - Dong, Jia Qi

AU - Liu, Li

AU - Huang, Zi Gang

AU - Huang, Liang

AU - Lai, Ying-Cheng

PY - 2019/3/6

Y1 - 2019/3/6

N2 - The main point of this paper is to provide an affirmative answer through exploiting reinforcement learning (RL) in artificial intelligence (AI) for eliminating herding without any external control in complex resource allocation systems. In particular, we demonstrate that when agents are empowered with RL (e.g., the popular Q-learning algorithm in AI) in that they get familiar with the unknown game environment gradually and attempt to deliver the optimal actions to maximize the payoff, herding can effectively be eliminated. Furthermore, computations reveal the striking phenomenon that, regardless of the initial state, the system evolves persistently and relentlessly toward the optimal state in which all resources are used efficiently. However, the evolution process is not without interruptions: there are large fluctuations that occur but only intermittently in time. The statistical distribution of the time between two successive fluctuating events is found to depend on the parity of the evolution, i.e., whether the number of time steps in between is odd or even. We develop a physical analysis and derive mean-field equations to gain an understanding of these phenomena. Since AI is becoming increasingly widespread, we expect our RL empowered minority game system to have broad applications.

AB - The main point of this paper is to provide an affirmative answer through exploiting reinforcement learning (RL) in artificial intelligence (AI) for eliminating herding without any external control in complex resource allocation systems. In particular, we demonstrate that when agents are empowered with RL (e.g., the popular Q-learning algorithm in AI) in that they get familiar with the unknown game environment gradually and attempt to deliver the optimal actions to maximize the payoff, herding can effectively be eliminated. Furthermore, computations reveal the striking phenomenon that, regardless of the initial state, the system evolves persistently and relentlessly toward the optimal state in which all resources are used efficiently. However, the evolution process is not without interruptions: there are large fluctuations that occur but only intermittently in time. The statistical distribution of the time between two successive fluctuating events is found to depend on the parity of the evolution, i.e., whether the number of time steps in between is odd or even. We develop a physical analysis and derive mean-field equations to gain an understanding of these phenomena. Since AI is becoming increasingly widespread, we expect our RL empowered minority game system to have broad applications.

UR - http://www.scopus.com/inward/record.url?scp=85062819965&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062819965&partnerID=8YFLogxK

U2 - 10.1103/PhysRevE.99.032302

DO - 10.1103/PhysRevE.99.032302

M3 - Article

C2 - 30999513

AN - SCOPUS:85062819965

VL - 99

JO - Physical Review E - Statistical, Nonlinear, and Soft Matter Physics

JF - Physical Review E - Statistical, Nonlinear, and Soft Matter Physics

SN - 1539-3755

IS - 3

M1 - 032302

ER -