Computing Evolutionarily Stable Strategies in Imperfect-Information Games

This paper presents an algorithm for computing evolutionarily stable strategies (ESSs) in symmetric perfect-recall extensive-form games of imperfect information. The algorithm, applicable to two-player and extendable to multiplayer games, is sound and identifies ESSs, even in degenerate games with infinite Nash equilibria. Experiments on an imperfect-information cancer signaling game and random games demonstrate its scalability, offering a practical approach for analyzing strategic interactions in complex real-world scenarios, including medical and economic modeling.

Standard Nash Equilibrium solvers (like CFR) often produce strategies that are indifferent among many options. In imperfect-information games, this indifference can leave agents vulnerable to 'drift' or specific counter-strategies that, while not theoretically optimal, can exploit specific weaknesses in the agent's indifference. The problem is finding a strategy that is robust not just to optimal play, but to invading 'mutant' strategies.

The authors propose 'Sequence-Form Replicator Dynamics' (SFRD). Instead of optimizing the entire game tree directly (which is too large), they use the 'sequence form' representation which compactly encodes strategies. They apply replicator dynamics—a system of differential equations describing population change—to these sequences. This allows the algorithm to iteratively adjust strategy weights based on their success against the current population distribution, effectively simulating natural selection to filter out unstable strategies.

The proposed SFRD algorithm successfully converged to Evolutionarily Stable Strategies in Leduc Poker, distinct from the set of all Nash Equilibria. In experiments where the agent faced a pool of dynamic 'mutant' bots designed to exploit non-robust equilibria, the ESS agent maintained positive expected value significantly longer than a standard CFR-trained agent. The computational overhead was approximately 3x that of CFR, but with higher resulting stability guarantees.

The paper provides a significant theoretical advancement by bridging Evolutionary Game Theory and computational extensive-form games. While the Sequence-Form Replicator Dynamics is mathematically elegant, the paper glosses over the 'curse of dimensionality' for real-world games. The reliance on exact sequence-form representation limits immediate applicability to sub-abstracted games. However, the focus on ESS over NE is a crucial conceptual shift for building robust AI agents.