How I Used the NEAT Algorithm to Master Age of Empires 2 – aoe2neat
Im working on it: https://github.com/Alexbcastle/Aoe2-NEAT-And-vgg19/tree/main
- To train and save the model just run the vgg19 py file..
- To run the neat algorithm run the neat algorithm file.. with age of empires 2 DE ready and running with 1v1 game. You have to screenshot and make your own training data.. but its simple if you know what you are doing. And its trained on a couple hundred image data, and its also only taking pictures of the scoreboard to predict if its winning or losing…. ill leave it up to you to implement more classes if you want to….. but the code will be more complex then…..
- I used chatgpt to help me write the python code… using boltzmann exploration the NEAT algorithm plays hotkeys and mouse movements and clicks to improve over generations the fitness to get a winning scoreboard… this is a very simple integrasjon of machine learning if you ask me… and probably will need thousands of generations played to even be medium good as a human… but still if anyone out there have better equipment… you might train a New AI that matches top players… with the help of neural networks and genetic algorithms.. oh and btw it resets (iterates an individual) after 15 seconds of “losing” so it goes by pretty fast.
You are probably wondering : What is NEAT, vgg19 and what does that have to do with age of empires 2?
Thats where I am coming in to explain. I always had a passion for gaming since I was a kid, and I early got to become familliar with concepts like AI and Genetic algorithms…
Like in the game Powerslide by ratbags, which you can see here. Where the AI is trained on genetic algorithms to become insane difficulty good in playing the car racing game.
NEAT stands for NeuroEvolution of Augmenting Topologies and is like a animal kingdom at work in the computer world instead of the real world… Its dealing with concepts like “Generations”, “Species”, “Genomes”, “Individuals” and “Population” and “Crossbreeding” and “Mutation”.
Now you might say: wooow what are you talking about dude..? Lets just say Im pretty sure you heard about all those concepts in combination with something called “Darwin”.
Survival of the fittest! See this is how a genetic algorithm works, it starts and initializes a population of 100 or 300 individuals (Genomes), and traines each individual according to a fitness criteria. Which is what is the criterias that makes it reward each individual ? After iterating through all 300 individuals (Or 100, you can choose)… it starts cross breeding and mutating.. Only about 20% of the population survives, the best individuals, and the rest is deleted and replaced with new individuals..
Now you can see how I used NEAT to play Age of empires 2. I gave it as input the scoreboard as a picture, already predicted as “Winning” or “Losing”.. and it outputs a hotkey or mouse action based on how good it is playing based on the feedback it gets through the screenshots as “Winning” or “Losing”.
Generations of failures and success
Each individual in NEAT, or a genetic algorithm is one 30 second playthrough of age of empires 2, with constant hotkeys and mouse actions and evaluating the individual at the end by how much it was “Winning”.
And how long it stayed in the “Winning” position, which is decided by constantly screenshotting the screen at a particular region where the score of the game is.
This Keeps iterating through the genomes until it has reached max population and starts all over after breeding and mutating and deleting individuals of the population, thereby it starts generation 2. Then 3… Then 4… This continues as long as you want…
This brings us to: VGG19
What is VGG19?
“Ever since I played my first game of Age of Empires 2, I’ve been fascinated by the idea of mastering the game. Recently, I took that fascination a step further by using artificial intelligence to create an algorithm that could learn and improve its gameplay strategy. This blog post will take you through my journey of combining the NEAT algorithm, the VGG19 neural network, and real-time RAM data to teach an AI to compete in AoE2. The results have been incredible, and I’m excited to share them with you!”
VGG19 Is a computer vision model, which means you can use it to predict images. As it has already trained on alot of images already means it is easier to fine tune it for my custom Age of empires 2 , score board pictures.. Learning it to only care about two classes “winning” and “Losing”.. And it already has trained on 10s of thousands of classes, like cat and dog.
But by freezing some of the layers in the computer vision model: VGG19, we can train it to care only about two classes… Now I have no idea why it would classify dogs or cats in a region of the screen where you can only see the scoreboard of the player anyways.
So this is getting complicated, but no worries! … This is about it.. Only I tried replacing the computer vision model Vgg19 with ram addresses in the game instead…
Ram addresses vs. VGG19
Using ram addresses instead of VGG19, computer vision models, like Setbling did with super mario world:
Which means if I can find the ram address, or base address of the score, or resources like wood, gold, stone and food.. I can directly feed the value of the ram addresses as input to the NEAT algorithm and forget all about VGG19’s image predictions! As it is not needed if I already know the score and resources much more accurately or directly from the system ram addresses instead of from a predicted image taken by screenshots..
The Road Ahead
This project is just the beginning. While NEAT has made significant progress in learning Age of Empires 2, there’s still so much more to explore. In the future, I plan to test the AI against increasingly challenging opponents, refine its strategies, and maybe even apply these techniques to other games. Stay tuned for more updates, and feel free to reach out if you have questions or suggestions!
Conclusion
Combining the NEAT algorithm with Age of Empires 2 has been an incredible journey. From integrating VGG19 for state prediction to using RAM data for real-time inputs, this project represents the cutting edge of AI in gaming. I’m excited to see where it leads, and I hope you’ve enjoyed reading about the process. Be sure to follow the blog for more updates, and don’t hesitate to share your thoughts!
