GameAI
I. Neural Network-GameAI
The development of artificial intelligence is omni-directional. In addition to classifying AI, it can also be applied to the field of games, using the relevant algorithms of artificial intelligence to automate AI.
AlphaGo, the first AI program to win the world champion, uses depth learning algorithm of neural network to learn.
II. The Principle of GameAI
The principle of GameAI is similar as that of Classifier: they are all based on neural network.
As picture below, a complete neural network contains input layer, hidden layer and output layer.
c
Input layer: collect information, that is, the real-time data changing in the game.
- For example: sprite's coordinate, player's HP, the number of enemy, and so on.
Hidden layer: operate and process the collected data.
Output layer: output classification result, that is, the decision we make when playing the game.
- For example: control the sprite's movement, control the sprite to jump, control the sprite to launch bullets, and so on.
Thus, in order for GameAI to make correct decisions, we also need to train the entire neural network structure to get the model we need.
III. Training Modes
- In GameAI training, we have two learning modes: supervised learning and unsupervised learning.
Take the game Flappy Bird as an example to show how to use GameAI to make artificial intelligence play Flappy Bird on its own.
Let's see the script of Flappy Bird:
Create a global variable for sprite "bird": Yspeed.
As this speed decreases, the bird's Y position will change according to the value of the Yspeed, simulating the bird's constant descent due to gravity.
When the mouse is clicked, set the value of Yspeed to be positive.
At this point, the bird will jump up a distance and continue to fall. When any column is touched, the game is over.
When the game began, reduce the X position of sprite "lower column" constantly to simulate the effect of the column is always moving to the right.
When "lower column" leaves the left edge of the screen (X < -370), move it to the right side of the screen.
When "lower column" moves to the middle position (X = -100), the bird would pass the columns, and variable "score" will increase 1.
After that, move the position of "upper column" to the position of "lower column" constantly, and increase Y position of "upper column" by 1100 constanly.
These are the scripts of the main sprites in this game.
After understanding the blocks of this game, we chose the bird as the AI role and let the bird start learning.
1. Supervised Learning
a. Initialize a model
Choose the bird as AI role, and initialize a model: model 1.
We can set the number of input layer and output layer aside temporarily, and adjust the hidden layer first.
For Flappy Bird, we recommend a single layer of 4 neurons.
b. Determine input and output information
① Determine input information
We have chosen two features: the Y position of "bird" and "lower column".
（Please note: set the central points to the position shown as below: on the middel of the bird and the middle top of the "lower column".
② Determine output information
When we play the game, we have two kinds of operation: doing nothing and click the mouse.
Accordingly in GsmeAI, these are two output results: 1. no action; 2. make the bird fly.
In order to invoke the actions of AI role, we have to use function block to encapsulate the flight events triggered by mouse clicks.
Then set the AI role to use the actions: "no action" and "fly".
③ Death conditions of the AI role
Set the death coditions of the AI role:
And set to get the real-time features of AI role:
Unlike Classifier, the input eigenvalues in GameAI will change in real time.
④ Confirm the number of input layer and output layer
After confirming the input and output information, remember to fill in the number of input and output layers at the location of the initialization model to complete the setup of the entire model.
The features of input layer are "Y position of bird" and "Y position of lower column", so the number of input layer is 2.
The results of output layer are "no action" and "fly", so the number of output layer is also 2.
c. Training and recording data
After setting up the model and input and output information, we will start training GameAI:
In supervised learning, we need to play the game by ourselves first, and record the operation in the game as training data.
After training and recording, the training can be stopped and the data can be saved in model 1.
Learning from this training data is supervised learning.
d. Back-propagation algorithm
When the AI role is dead, we can continue to use "Back-propagation Algorithm" to optimize the chosen model.
After optimizing the recorded training data, the model will imitate the actions in the training data and continue to act.
If you want to retain the previous training results after training, remember to remove the block that initialize the model and run the project again.
e. Use the model
When the model finished training, we can use it to let the AI role make its own decision and play the game automatically.
Connect "AI role makes decision" block.
In this way, AI role can make decisions based on the current features, that are, "no action" or "fly", and play the game on its own.
Flappy Bird with supervised learning
Click the picture to try this project. You can buy the codes of this project and see how it works~
2. Unsupervised learning
Supervised learning is equivalent to teach AI how to play games by using the backward-propagation algorithm.
Unsupervised learning is to let AI learn to play games on its own by using evolution algorithm.
In unsupervised learning of GameAI, the initial model is generated randomly rather than from training.
By generating a large number of random models to keep AI role trying in the game. Some of these models will be quickly eliminated, but some can get very high scores.
Then, through evolution algorithm, the model structure with high scores is retained, and like organisms. They evolve among themselves.
Finally, the model with higher scores, better performance and stronger later generation is developed.
Again, take the game Flappy Bird as an example to see what is unsupervised learning:
a. Initialize a model
In both supervised learning and unsupervised learning, we should choose the bird as AI role, and initialize a model: model 1.
Set the hiddle layer as a single layer of 4 neurons:
As for unsupervised learning, if only one sprite is run at one time, the efficiency is too low.
Thus we need to set more sprites. Here, we can set the number of sprites for each round to 100:
b. Entity variable and event start
In order for the 100 AI role (birds) to run independently, we need to change some game settings: entity variables and event start.
① Set an entity variable
For this game, we need to change the global variable "Yspeed" of the bird to entity variable.
Change the variables used in blocks:
② Event start
Because it takes time to clone 100 birds, in order to avoid some of the first cloned birds moving ahead of others, we need to change the trigger condition at the start of the game to "when GameAI started" to make all the birds act at the same time.
c. Determine input and output information
① Determine input information
We have chosen two features: the Y positon of "bird" and "lower column".
② Determine output information
When we play the game, we have two kinds of operation: doing nothing and click the mouse.
Accordingly in GsmeAI, these are two output results: 1. no action; 2. make the bird fly.
In order to invoke the actions of AI role, we have to use function block to encapsulate the flight events triggered by mouse clicks.
Then set the AI role to use the actions: "no action" and "fly".
③ Death conditions of the AI role
Set the death coditions of the AI role:
And set to get the real-time features of AI role:
Unlike Classifier, the input eigenvalues in GameAI will change in real time.
④ Confirm the number of input layer and output layer
After confirming the input and output information, remember to fill in the number of input and output layers at the location of the initialization model to complete the setup of the entire model.
d. GameAI decision
Let AI to make its own decision: "no action" or "fly".
e. Evolution algorithm
When a AI role is dead, record its score; when all the AI roles are dead, you can find a better model for this batch.
When all the AI roles are dead, use "Evolution Algorithm" to optimize the model, and restart GameAI to for next prediction.
If you want to retain the previous training results after training, remember to remove the block that initialize the model and run the project again.
Only in this way can the program continue to optimize the model on the basis of previous model 1.
f. Use the model
When the model finished training, we can use it to let the AI role make its own decision and play the game automatically.
Use the model and start GameAI.
Flappy Bird with unsupervised learning
Click the picture to try this project. You can buy the codes of this project and see how it works~
IV. Use model to develop AI game
If we are satisfied with the model, we can also use the AI model to make AI games.
Whether we use supervised learning or unsupervised learning, the final saved model will be stored in the bcm file.
Then we can directly let birds to play games automatically based on model 1:
Please note: the actions and features of AI roles need to be the same as those in training.
Then we can develop this game as below:
- Let the player plays with AI role
(Click gif picture to play the game~)
- Control the column to add difficulty
(Click gif picture to play the game~)
