Although works that describe the use of standard MCTS in new domains have been published, they are not in the scope of this survey. ![]() In this survey, we will focus only on such papers that introduce at least one modification to the vanilla version of the method. Whenever the vanilla MCTS algorithm, i.e., implemented in its base unmodified form, fails to deliver the expected performance, it needs to be equipped with some kind of enhancements. To name a few-combinatorial complexity, sparse rewards or other kinds of inherent difficulty. There can be various reasons for a given problem being hard for MCTS. However, often, a linear increase is not enough to tackle difficult problems represented by trees and solving them would require effectively unlimited memory and computational power. Sometimes, increasing the effective computational budget would help, although in practical applications it may not be possible (e.g., because of strict response times, hardware costs, parallelization scaling). The utilitarian definition of “too difficult” is that MCTS achieves poor outcomes in the given setting under practical computational constraints. Moreover, in practical applications, the given problem often tends to be difficult for the base variant of the algorithm. However, it is possible to take advantage of heuristics and include them in the MCTS approach to make it more efficient and improve its convergence. ![]() In contrast to them, the MCTS algorithm is at its core aheuristic, which means that no additional knowledge is required other than just rules of a game (or a problem, generally speaking). Before MCTS, bots for combinatorial games had been using various modifications of the minimax alpha–beta pruning algorithm (Junghanns 1998) such as MTD(f) (Plaat 2014) and hand-crafted heuristics. ![]() 2012) as it allowed for a leap from 14 kyu, which is an average amateur level, to 5 dan, which is considered an advanced level but not professional yet. It was quickly called a major breakthrough (Gelly et al. ![]() MCTS has been originally proposed in the work by Kocsis and Szepesvári ( 2006) and by Coulom ( 2006), as an algorithm for making computer players in Go. In such trees, nodes denote states, also referred to as configurations of the problem, whereas edges denote transitions (actions) from one state to another. Monte Carlo Tree Search (MCTS) is a decision-making algorithm that consists in searching combinatorial spaces represented by trees.
0 Comments
Leave a Reply. |