evolves, which makes the fitness of current optimal solution more and more higher. EC
attracts the attention of game AI because it can generate highly optimized solutions in
various problem settings.
Marcolino and Matsubara [15] apply EC in StarCraft to optimize the build-order.
Unlikely, Young and Hawes [16] directly apply the genetic algorithm (GA) to optimize
more comprehensive strategy for StarCraft, including both quantity and sequence of
construction. In Ref. [17], the author takes multi-objective optimization into consideration in order to establish the correct element at a certain time. For more details on
multi-objective optimization using EC, see [18]. In [19], the author establishes a forward model to optimize the results of the construction sequence. In [20], EC combines
with potential areas to optimize the policy parameters of micromanagement. Both Refs.
[21, 22] use neuroevolution to learn micromanagement. They are mainly different in
the structure of neural networks. As can be seen from [23], the balance is another factor
that researchers and players focus on.
5 Machine Learning
Machine learning can be divided into supervised learning (SL), unsupervised learning
(UL) and reinforcement learning (RL). In SL the training signal is provided as sample
labels, while in RL it is the reward obtained from the environment. Instead, UL tries to
find the correlation of inputs by searching patterns in all input features in the absence of
target output as a guide.
Replay data of human can be used as valuable training data for game AI. On the
basis of SL, intelligent models can be learned by the means of mining implicit
knowledge in data without a large number of manual rules. Therefore, many organizations such as DeepMind and Facebook have released various replay data sets. Literature [24] uses replay data to predict the combat strategies of opponents. In [25], the
authors train a probabilistic model to predict the strategy behaviors based on replay
data. Deep learning (DL), one of the most popular areas of SL, has been widely used in
speech recognition, natural language processing, and other scenarios. Dereszynski et al.
[26] use DL to learn macro-management decisions directly from state-action pairs in
StarCraft.
RL is good at sequential decision-making tasks. The combination–deep reinforcement learning (DRL), the result of combing DL with RL, can teach agents to make
decisions in high-dimension state space by an end-to-end framework [27]. Zhao et al.
[28] apply RL to micromanagement of StarCraft, where RL agents need to face many
difficult problems including delayed rewards, large action space, and vast state space.
Usunier et al. [29] resort to more effective state representation method to decompose
the complexity because of the wide state space. Based on SC2LE, Shao et al. [30]
propose a new value-based DRL algorithm, which learns decentralized strategies in a
centralized end-to-end manner.
Instead of imitating or predicting target values, the intrinsic structure of and
associations in the data are concerned for UL. Clustering and frequent pattern mining
are two most usually used UL methods for games. K-means, k-medoids, and DBSCAN
A Review of Artificial Intelligence for Games
301
attracts the attention of game AI because it can generate highly optimized solutions in
various problem settings.
Marcolino and Matsubara [15] apply EC in StarCraft to optimize the build-order.
Unlikely, Young and Hawes [16] directly apply the genetic algorithm (GA) to optimize
more comprehensive strategy for StarCraft, including both quantity and sequence of
construction. In Ref. [17], the author takes multi-objective optimization into consideration in order to establish the correct element at a certain time. For more details on
multi-objective optimization using EC, see [18]. In [19], the author establishes a forward model to optimize the results of the construction sequence. In [20], EC combines
with potential areas to optimize the policy parameters of micromanagement. Both Refs.
[21, 22] use neuroevolution to learn micromanagement. They are mainly different in
the structure of neural networks. As can be seen from [23], the balance is another factor
that researchers and players focus on.
5 Machine Learning
Machine learning can be divided into supervised learning (SL), unsupervised learning
(UL) and reinforcement learning (RL). In SL the training signal is provided as sample
labels, while in RL it is the reward obtained from the environment. Instead, UL tries to
find the correlation of inputs by searching patterns in all input features in the absence of
target output as a guide.
Replay data of human can be used as valuable training data for game AI. On the
basis of SL, intelligent models can be learned by the means of mining implicit
knowledge in data without a large number of manual rules. Therefore, many organizations such as DeepMind and Facebook have released various replay data sets. Literature [24] uses replay data to predict the combat strategies of opponents. In [25], the
authors train a probabilistic model to predict the strategy behaviors based on replay
data. Deep learning (DL), one of the most popular areas of SL, has been widely used in
speech recognition, natural language processing, and other scenarios. Dereszynski et al.
[26] use DL to learn macro-management decisions directly from state-action pairs in
StarCraft.
RL is good at sequential decision-making tasks. The combination–deep reinforcement learning (DRL), the result of combing DL with RL, can teach agents to make
decisions in high-dimension state space by an end-to-end framework [27]. Zhao et al.
[28] apply RL to micromanagement of StarCraft, where RL agents need to face many
difficult problems including delayed rewards, large action space, and vast state space.
Usunier et al. [29] resort to more effective state representation method to decompose
the complexity because of the wide state space. Based on SC2LE, Shao et al. [30]
propose a new value-based DRL algorithm, which learns decentralized strategies in a
centralized end-to-end manner.
Instead of imitating or predicting target values, the intrinsic structure of and
associations in the data are concerned for UL. Clustering and frequent pattern mining
are two most usually used UL methods for games. K-means, k-medoids, and DBSCAN
A Review of Artificial Intelligence for Games
301