Flappy Bot: Using Neuro-Evolution to Beat Flappy Bird

Preface

If you would like to see the complete code, click here.

Introduction

If you haven’t heard of the viral side scroller known as Flappy Bird, the main objective is to control a bird and attempt to fly between pipes as difficulty increases; the further you go, the more points you get. The controls are as easy as you get: just tap to flap. With such simple mechanics, why not try to build an A.I. bot to play the game?

Choosing a Programming Stack

As with many of my projects, the visual aspect strongly encourages the use of a digital sketchbook. For this project, we will build an in-browser simulation of Flappy Bird using P5.js.

Building Mr.Flappy

Let’s get started with a simple screen; for the sake of simplicity, we will try and keep similar dimensions as the original game, which was a phone ratio. Here is a look at the simple background I created (due to my lack of design skill, I will be trying my best not to create any of the graphics myself):

A simple static background for my application

Good start. Time to add in our flappy bird and the ground, which will be moving at the some speed (same speed as the pipes):

Adding Obstacles

We need to add pipes for Flappy to just through. The distance between the pipes stays consistent, but the entry height changes each time; we will just do this randomly. Let’s add in the pipes, and throw up a score screen as well:

Adding Collision and Score

The longer he flies, the more points Flappy earns. If at any point Flappy hits a pipe (or the ground/ceiling for that matter) he dies.

That’s the entire logic of the game! Let’s see it in action:

Creating an A.I. Bot

The primary benefit of creating our own game is access to variables: how far away is the next pipe? What is the velocity and acceleration? This kind of information can be used to determine a single action: to jump or not to jump. We have several desired inputs and one desired output; this sounds like a problem for a neural network!

But what about the optimization for the neural network? Instead of using gradient descent, I will be using an algorithm very good at optimizing multi-variable problems: a genetic algorithm. This combination of neural networks and genetic algorithms is generally known as NEAT: neuro-evolution of augmented topologies.

There are two primary things we need to do:

Build a neural network data structure that allows us to parameterize Flappy with one output: jump.
Run a genetic algorithm simulation with multiple flappy birds to optimize Flappy’s brain; once we have the desired “brain”, we can save that data as the fittest version.

Let’s start with the neural network

Creating a Neural Network Class in JavaScript

There are a vast array of libraries available for machine learning, but for the sake of such a simple project I will make the NN myself. A NN is comprised of multiple layers: an input layer, hidden layers, and an output layer. The data will feed-forward from the input nodes through the hidden nodes and into the output node(s) using weights that are initially randomized.

First, let’s create a node:

Next, we will build a neural network class that allows us to build the structure as we desire:

Now we need to determine what structure we need for Flappy. We know that we only have one output node, but how many input and hidden nodes do we need? Here’s a list of the six parameters that I’ve decided to use for inputs:

The x distance to the nearest pipe
The y distance to the nearest top pipe
The y distance to the nearest bottom pipe
The bird’s x position
The bird’s y position
The bird’s vertical velocity

There is no need for hidden layers for such a small domain; our flappy bot brain will just be the input and output layers.

We now have a “brain” for Flappy. Let’s create a population of 300 birds and see what kind of results we get:

(Early) Conclusion:

Woah! That worked a lot better than I expected. With such a simple “brain”, Flappy is able to find the random weights rather quickly given a large enough population. In just one generation, we are able to achieve a perfect Flappy bot.

While this project has reached its end, I want to use the neural network library we created to implement the NEAT algorithm for another project so we can see how evolutionary algorithms are so effective when combined with even simple feed-forward neural networks. Please stay updated with my blog if you would like to see such a project in the future!