4.53 out of 5
4.53
946 reviews on Udemy

A Beginner’s Guide To Machine Learning with Unity

Advanced games AI with genetic algorithms, neural networks & Q-learning in C# and Tensorflow for Unity
Instructor:
Penny de Byl
7,455 students enrolled
English [Auto-generated]
Build a genetic algorithm from scratch in C#.
Build a neural network from scratch in C#.
Setup and explore the Unity ML-Agents plugin.
Setup and use Tensorflow to train game characters.
Apply their newfound knowledge of machine learning to integrate contemporary research ideas in the field into their own projects.
Distill the mathematics and statistic behind machine learning to working program code.
Use a Proximal Policy Optimisation to train a neural network.

What if you could build a character that could learn while it played?  Think about the types of game play you could develop where the enemies started to outsmart the player. This is what machine learning in games is all about. In this course we will discover the fascinating world of artificial intelligence beyond the simple stuff and examine the increasingly popular domain of machines that learn to think for themselves.

In this course, Penny introduces the popular machine learning techniques of genetic algorithms and neural networks using her internationally acclaimed teaching style and knowledge from a Ph.D in game character AI and over 25 years experience working with games and computer graphics.  In addition she’s written two award winning books on games AI and two others best sellers on Unity game development. Through-out the course you will follow along with hands-on workshops designed to teach you about the fundamental machine learning techniques distilling the mathematics in a way that the topic becomes accessible to the most noob of novices.  

Learn how to program and work with:

  • genetic algorithms;
  • neural networks;
  • human player captured training sets;
  • reinforcement learning;
  • Unity’s ML-Agent plugin; and,
  • Tensorflow;

Contents and Overview

The course starts with a thorough examination of genetic algorithms that will ease you into one of the simplest machine learning techniques that is capable of extraordinary learning. You’ll develop an agent that learns to camouflage, a flappy bird inspired application in which the birds learn to make it through a maze and environment sensing bots that learn to stay on a platform.

Following this you’ll dive right into creating your very own neural network in C# from scratch.  With this basic neural network you will find out how to train behaviour, capture and use human player data to train an agent and teach a bot to drive.  In the same section you’ll have the Q-learning algorithm explained before integrating it into your own applications.

By this stage you’ll feel confident with the terminology and techniques used throughout the deep learning community and ready to tackle Unity’s experimental ML-Agents. Together with Tensorflow you’ll be throwing agents in the deep end and reinforcing their knowledge to stay alive in a variety of game environment scenarios.

By the end of the course you’ll have a well equiped toolset of basic and solid machine learning algorithms and applications that will see you able to decipher the latest research publications and integrate the latest developments into your work while keeping abreast of Unity’s ML-Agents as they evolve from experimental to production release.

What students are saying about this course:

  • Absolutely the best beginner to Advanced course for Neural Networks/ Machine Learning if you are a game developer that uses C# and Unity. BAR NONE x Infinity.
  • A perfect course with great math examples and demonstration of the TensorFlow power inside Unity. After this course, you will get the strong basic background in the Machine Learning.
  • The instructor is very engaging and knowledgeable. I started learning from the first lesson and it never stopped. If you are interested in Machine Learning , take this course.

Introduction

1
Introduction

This lecture is a welcome to the course and introduction to the instructor and an overview of the course content.

2
What is Learning?

In this lecture students will learn about the types of learning models implemented in machine learning.

3
How to Study This Course

This article provides guidances on how to study this course.

4
FAQs

This article addresses common general questions about my courses.

Genetic Algorithms

1
DNA Inspired Data Structures

Genetic Algorithms are one technique classified under the larger umbrella of evolutionary computing. In this domain researchers use biological systems as the basis for designing code. Genetic algorithms are simple in design but are capable of producing extraordinatry learned behaviours.

2
Camouflage Training with Genetic Algorithms Part 1

In this lecture we begin creating a very simple genetic algorithm that will learn a player's colour preference.

3
Camouflage Training with Genetic Algorithms Part 2

This lecture completes the code for producing your first genetic algorithm application that will breed a set of sprites set to your favourite colour.

4
Camouflage Challenge

Test your knowledge of Genetic Algorithms.

5
Coding Movement with Genes Part 1

The values stored in the genes can be used to code anything from colour, to movement, to speed. In this lecture we will examine how we can use a single gene to control movement and teach a population to walk along a beam.

6
Coding Movement with Genes Part 2

In this lecture we will finish the movement strategy example of implementing a genetic algorithm to determine the best way to move to stay alive the longest.

7
Distance Challenge

Modify the single gene example code in the previous lectures and instead of testing for fitness on how long each bot survives, test for distance travelled. The result should be a population that prefer to walk along the beam. They will want to get as far away from their starting position as possible without falling off. 

8
Note: Unity Versions Might Mess Up Package Imports

This article is a short note about issues with Unity version compatibility with solution package files from the course.

9
Moving GAs with Senses Part 1

In these next few lectures we will build a new genetic algorithm series that can train a group of bots to stay on a platform by teaching them when to turn and when to move forward.

10
Moving GAs with Senses Part 2

In this lecture we finish the first phase of the genetic algorithm that trains bots to stay on a platform. We then make a few tweaks to how it senses the environment and discuss some improvements.

11
Moving GAs with Senses Part 3

In this lecture we will finalise the training for the genetic algorithm before adding the Ethan third-person character into the scene to replace the capsule bot.

12
Maze Walking Challenge

In this challenge you will be asked to create a genetic algorithm to traverse a maze. The video shows you the initial setup of the environment and allows you pausing time to build the application before one solution is given.

13
Maze Walking Challenge Solution Part 2

This lecture concludes stepping through the solution of the maze walking challenge.

14
Not So Flappy Birds Part 1

In this lecture we will explore genetic algorithms further by creating a longer gene sequence and use it to train 2D birds to get through an obstacle course.

15
Not So Flappy Birds Part 2

In this second half we will complete the application by setting up the bird prefabs and adding the PopulationManager.

16
Extra Readings

Additional resources to help expand your knowledge of Genetic Algorithms beyond the scope of the course.

Perceptrons: The making of a Neural Network

1
The Perceptron

A perceptron is the smallest functioning unit of a neural network. However, by itself it can still produce some stunning results. Development of this fundamental algorithm will introduce students to the nature of neural nets and how they function.

2
Challenge

Produce a line by line perceptron using a spreadsheet.

3
Programming and Training a Perceptron

In this workshop student's will follow along in Unity to create a perceptron in C#.

4
Exercise 1

An exercise focused on improving your knowledge of perceptrons.

5
Exercise 2

An exercise focused on improving your knowledge of perceptrons.

6
Perceptron Classification

In this lecture you will learn how the weights are used by the perceptron to define a decision boundary that helps it classify inputs.

7
Perceptron Learning from Experience

In this lecture you will create a perceptron to act as the brain of an NPC as you teach it to dodge balls.

8
Saving & Loading Perceptron Values

After a perceptron is trained, all its 'knowledge' is contained in the weights. By saving these final weights you are essentially saving the artificial brain. The saved values can be reloaded to create an instantly trained perceptron.

Artificial Neural Networks

1
Introduction to Neural Networks

This lecture provides a brief overview of artificial neural networks along with their architecture and uses.

2
Programming An Artificial Neural Network Part 1

In this lecture we will begin to program our own artificial neural network from scratch.

3
Programming An Artificial Neural Network Part 2

In the second part of this workshop to build a neural network we will finish creating the code and give it some training examples.

4
Programming An Artificial Neural Network Part 3

Having programmed an Artificial Neural Network we will now put it through it's paces and discuss training variables.

5
ANN FAQs

This lecture addresses three of the most frequently asked questions in neural network development:
1) what activation function should I use;
2) how many layers to I need, and
3) how many neurons do I need?

6
Working with Activation Functions

Once you've got your neural network code setup its a simple matter to add and use more activation functions. In this lecture you will learn how to add more activation functions to your code and analyse their usefulness.

7
Challenge

Take a look at the Sinusoid, ArcTan and SoftSign. Write the code to program these into your ANN. 

8
Extra Readings

Additional resources to help expand your knowledge of Artificial Neural Networks beyond the scope of the course.

Neural Networks in Practice

1
Developing a Neural Network that Plays Pong Part 1

In this lecture we will start using the ANN for something game related and what better way than to create an NPC that plays Pong.

2
Developing a Neural Network that Plays Pong Part 2

In the second part of the Pong playing neural network workshop, we will complete the code and examine the NPCs performance.

3
Developing a Neural Network that Plays Pong Part 3

In the final part of creating an ANN that plays Pong we look at extending the training set by including more complex data that involves reflections.

4
Challenge

Extend the game of Pong and add another paddle to act as the other player. 

5
Gathering Training Data from the Player Part 1

In the real world training a neural network with data gathered from the real world can introduce problems that don't show up in purely academic examples. In these next few videos we will create a simple racing scenario, gather data from the game player's racing and inject this data into a neural networked player to train them to drive the track.

6
Gathering Training Data from the Player Part 2

We continue on from the previous lecture by finishing our capture of player data to use in a neural network training set. We will examine a way to normalise and compress the large amount of collected information into something more suitable for a neural network.

7
Training with Player Data Part 1

Once you've collected the training data from the player you can begin training the neural network. In this lecture we will start writing the script to attach to the ANN driven kart.

8
A Note to the Astute

This short note discusses how gathering data from different sensor angles isn't an issue.

9
Training with Player Data Part 2

In this lecture we will complete writing the code required to train an NPC ANN to navigate a racing track circuit given the collected user data as well as discuss some of the nuances of training in complex problem spaces with real data.

10
Training with Player Data Part 3

This lecture completes the training exercise for the go kart racing scenario by adding in code to load previously trained weights and finishes by examining ways to optimise the trained data even further to get the SSE down.

Reinforcement Learning with the Q-Network

1
Reinforcement Learning and Q-Networks

Deep learning can be achieved through the reinforcement learning technique called Q-Learning. In this lecture we will explore the algorithm based on this theory getting ready to implement it with our own neural network.

2
Training a Neural Network with Q-Learning Part 1

This lecture begins our integration of Q-Learning into the existing neural network code. We will examine Q-Learning in this context to train a platform to balance a ball.

3
Training a Neural Network with Q-Learning Part 2

In this second part we will continue coding the Brain for the system and work through the integration of the critical Q-Learning algorithm and Bellman's equation to create the reward feedback system.

4
Training a Neural Network with Q-Learning Part 3

In the final part of this series we will complete the Brain code and run it to explore how well the platform balanace the ball.

5
Challenge

Modify the balance beam ANN to create a Flappy Bird that learns to hover.

6
Extra Readings

Additional resources to help expand your knowledge beyond the scope of the course.

ML-Agents

1
Read This First

Unity's ML-Agents V0.3 [DEPRECATED]

1
Setup

In this lecture we will cover the setting up of the python and Tensorflow environment essential for training the ML-Agents.

2
Training Your First ML-Agent V0.3

In this video we will take a look at an overview of the ML-Agent Project structure and step through building and training an example project.

3
Migrating from V0.2 to V0.3

This document outlines the changes you'll need to take into consideration in migrating ML-Agents 0.2 to 0.3.

4
ML-Agent's FAQ

In this article I will address the more common and simple questions raised by students with regards to the ML-Agent use and setup.

5
Creating an ML-Agent From Scratch Part 1

Now that you have Tensorflow and the Unity ML-Agents working it's time to create your own - from scratch. In these next lectures we will take a closer look at what makes the ML-Agent's system tick and explore the settings and options.

6
Creating an ML-Agent From Scratch Part 2

This lecture completes the development of a simple ML-Agent's application and steps through the training process.

7
ML-Agents Cheat Sheet

A cheat sheet of quick help for working with ML-Agents.

8
An Avoiding ML-Agent Part 1

In this lecture we will start developing an agent in a 2D environment that will learn to avoid an enclosing boundary, discuss more ML-Agent settings and examine discrete and continuous actions.

9
An Avoiding ML-Agent Part 2

In this lecture we'll complete the avoidance agent by examining the difference between training with continuous and discrete values.

10
Challenge

Modify the cat agent developed in the previous lecture to use raycasts to sense the proximity of the border colliders instead of the method currently being used. 

11
Top 10 Tips for Neural Network Best Practice

In this lecture I'll share with you the top ten things I have learnt when using ML-Agents system to help you design your agents better and debug issues that may arise.

12
Environment Sensing ML-Agent

In this lecture we will revisit the 2D floating cat example, train it to use raycast sensors and throw in some moving dogs to dodge.

13
Rewards
14
Goal Seeking Wall Jumping Part 1

In this lecture we will train an agent to reach a goal position.

15
Goal Seeking Wall Jumping Part 2

We will extend the abilities of our agent from the last lecture here and teach it to jump over a wall.

16
Extra Readings

Additional resources to help expand your knowledge beyond the scope of the course.

Unity's M-Agents V0.2 [DEPRECIATED]

1
About This Section

Why this section is depreciated.

2
Setting up TensorFlow - Starter Files

This lecture takes students step by step through the setup of Tensorflow.

3
Setting up TensorFlow - Windows

This article is a step-by-step setup guide for Windows specific Tensorflow and ML-Agent issues.

4
Setting up TensorFlow - Mac

This article is a step-by-step setup guide for Mac specific Tensorflow and ML-Agent issues.

5
An Overview of ML-Agents

In this lecture we will use a sample Unity application to demonstrate the steps involved in training a working brain with Tensorflow.

A Final Word

1
Thank you

A short video with some final words from Penny.

2
Where to Now?
You can view and review the lecture materials indefinitely, like an on-demand channel.
Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don't have an internet connection, some instructors also let their students download course lectures. That's up to the instructor though, so make sure you get on their good side!
4.5
4.5 out of 5
946 Ratings

Detailed Rating

Stars 5
596
Stars 4
258
Stars 3
72
Stars 2
14
Stars 1
6
1e773385d13cf897048a3af630872b89
30-Day Money-Back Guarantee

Includes

12 hours on-demand video
23 articles
Full lifetime access
Access on mobile and TV
Certificate of Completion