Machine Learning, Data Science and Deep Learning with Python
New! Updated for TensorFlow 1.10
Machine Learning and artificial intelligence (AI) is everywhere; if you want to know how companies like Google, Amazon, and even Udemy extract meaning and insights from massive data sets, this data science course will give you the fundamentals you need. Data Scientists enjoy one of the top-paying jobs, with an average salary of $120,000 according to Glassdoor and Indeed. That’s just the average! And it’s not just about money – it’s interesting work too!
If you’ve got some programming or scripting experience, this course will teach you the techniques used by real data scientists and machine learning practitioners in the tech industry – and prepare you for a move into this hot career path. This comprehensive machine learning tutorial includes over 80 lectures spanning 12 hours of video, and most topics include hands-on Python code examples you can use for reference and for practice. I’ll draw on my 9 years of experience at Amazon and IMDb to guide you through what matters, and what doesn’t.
Each concept is introduced in plain English, avoiding confusing mathematical notation and jargon. It’s then demonstrated using Python code you can experiment with and build upon, along with notes you can keep for future reference. You won’t find academic, deeply mathematical coverage of these algorithms in this course – the focus is on practical understanding and application of them. At the end, you’ll be given a final project to apply what you’ve learned!
The topics in this course come from an analysis of real requirements in data scientist job listings from the biggest tech employers. We’ll cover the machine learning, AI, and data mining techniques real employers are looking for, including:
Deep Learning / Neural Networks (MLP’s, CNN’s, RNN’s) with TensorFlow and Keras
Image recognition and classification
Principal Component Analysis
Train/Test and cross validation
Decision Trees and Random Forests
Support Vector Machines
Term Frequency / Inverse Document Frequency
Experimental Design and A/B Tests
…and much more! There’s also an entire section on machine learning with Apache Spark, which lets you scale up these techniques to “big data” analyzed on a computing cluster. And you’ll also get access to this course’s Facebook Group, where you can stay in touch with your classmates.
If you’re new to Python, don’t worry – the course starts with a crash course. If you’ve done some programming before, you should pick it up quickly. This course shows you how to get set up on Microsoft Windows-based PC’s; the sample code will also run on MacOS or Linux desktop systems, but I can’t provide OS-specific support for them.
If you’re a programmer looking to switch into an exciting new career track, or a data analyst looking to make the transition into the tech industry – this course will teach you the basic techniques used by real-world industry data scientists. These are topics any successful technologist absolutely needs to know about, so what are you waiting for? Enroll now!
“I started doing your course in 2015… Eventually I got interested and never thought that I will be working for corporate before a friend offered me this job. I am learning a lot which was impossible to learn in academia and enjoying it thoroughly. To me, your course is the one that helped me understand how to work with corporate problems. How to think to be a success in corporate AI research. I find you the most impressive instructor in ML, simple yet convincing.” – Kanad Basu, PhD
What to expect in this course, who it's for, and the general format we'll follow.
We'll show you where to download the scripts and sample data used in this course, and where to put it.
We'll install our Python 3.5 environment, Enthought Canopy, and install the Python libraries and packages we'll need for this course. When we're done, we'll do a quick test of running a real Python notebook!
In a crash course on Python and what's different about it, we'll cover the importance of whitespace in Python scripts, how to import Python modules, and Python data structures including lists, tuples, and dictionaries.
In part 2 of our Python crash course, we'll cover functions, boolean expressions, and looping constructs in Python.
This course presents Python examples in the form of iPython Notebooks, but we'll cover the other ways to run Python code: interactively from the Python shell, or running stand-alone Python script files.
Pandas is a library we'll use throughout the course for loading, examining, and manipulating data. Let's see how it works with some examples, and you'll have an exercise at the end too.
Statistics and Probability Refresher, and Python Practise
We cover the differences between continuous and discrete numerical data, categorical data, and ordinal data.
A refresher on mean, median, and mode - and when it's appropriate to use each.
We'll use mean, median, and mode in some real Python code, and set you loose to write some code of your own.
We'll cover how to compute the variation and standard deviation of a data distribution, and how to do it using some examples in Python.
Introducing the concepts of probability density functions (PDF's) and probability mass functions (PMF's).
We'll show examples of continuous, normal, exponential, binomial, and poisson distributions using iPython.
We'll look at some examples of percentiles and quartiles in data distributions, and then move on to the concept of the first four moments of data sets.
An overview of different tricks in matplotlib for creating graphs of your data, using different graph types and styles.
The concepts of covariance and correlation used to look for relationships between different sets of attributes, and some examples in Python.
We cover the concepts and equations behind conditional probability, and use it to try and find a relationship between age and purchases in some fabricated data using Python.
Here we'll go over my solution to the exercise I challenged you with in the previous lecture - changing our fabricated data to have no real correlation between ages and purchases, and seeing if you can detect that using conditional probability.
An overview of Bayes' Theorem, and an example of using it to uncover misleading statistics surrounding the accuracy of drug testing.
We introduce the concept of linear regression and how it works, and use it to fit a line to some sample data using Python.
We cover the concepts of polynomial regression, and use it to fit a more complex page speed - purchase relationship in Python.
Multivariate models let us predict some value given more than one attribute. We cover the concept, then use it to build a model in Python to predict car prices based on their number of doors, mileage, and number of cylinders. We'll also get our first look at the statsmodels library in Python.
We'll just cover the concept of multi-level modeling, as it is a very advanced topic. But you'll get the ideas and challenges behind it.
Machine Learning with Python
The concepts of supervised and unsupervised machine learning, and how to evaluate the ability of a machine learning model to predict new values using the train/test technique.
We'll apply train test to a real example using Python.
We'll introduce the concept of Naive Bayes and how we might apply it to the problem of building a spam classifier.
We'll actually write a working spam classifier, using real email training data and a surprisingly small amount of code!
K-Means is a way to identify things that are similar to each other. It's a case of unsupervised learning, which could result in clusters you never expected!
We'll apply K-Means clustering to find interesting groupings of people based on their age and income.
Entropy is a measure of the disorder in a data set - we'll learn what that means, and how to compute it mathematically.
In order to run the next lecture on decision trees, you'll need some software called "GraphViz" installed. Here's how.
Decision trees can automatically create a flow chart for making some decision, based on machine learning! Let's learn how they work.
We'll create a decision tree and an entire "random forest" to predict hiring decisions for job candidates.
Random Forests was an example of ensemble learning; we'll cover over techniques for combining the results of many models to create a better result than any one could produce on its own.
Support Vector Machines are an advanced technique for classifying data that has multiple features. It treats those features as dimensions, and partitions this higher-dimensional space using "support vectors."
We'll use scikit-learn to easily classify people using a C-Support Vector Classifier.
One way to recommend items is to look for other people similar to you based on their behavior, and recommend stuff they liked that you haven't seen yet.
The shortcomings of user-based collaborative filtering can be solved by flipping it on its head, and instead looking at relationships between items instead of relationships between people.
We'll use the real-world MovieLens data set of movie ratings to take a first crack at finding movies that are similar to each other, which is the first step in item-based collaborative filtering.
Our initial results for movies similar to Star Wars weren't very good. Let's figure out why, and fix it.
We'll implement a complete item-based collaborative filtering system that uses real-world movie ratings data to recommend movies to any user.
As a student exercise, try some of my ideas - or some ideas of your own - to make the results of our item-based collaborative filter even better.
More Data Mining and Machine Learning Techniques
KNN is a very simple supervised machine learning technique; we'll quickly cover the concept here.
We'll use the simple KNN technique and apply it to a more complicated problem: finding the most similar movies to a given movie just given its genre and rating information, and then using those "nearest neighbors" to predict the movie's rating.
Data that includes many features or many different vectors can be thought of as having many dimensions. Often it's useful to reduce those dimensions down to something more easily visualized, for compression, or to just distill the most important information from a data set (that is, information that contributes the most to the data's variance.) Principal Component Analysis and Singular Value Decomposition do that.
We'll use sckikit-learn's built-in PCA system to reduce the 4-dimensions Iris data set down to 2 dimensions, while still preserving most of its variance.
Cloud-based data storage and analysis systems like Hadoop, Hive, Spark, and MapReduce are turning the field of data warehousing on its head. Instead of extracting, transforming, and then loading data into a data warehouse, the transformation step is now more efficiently done using a cluster after it's already been loaded. With computing and storage resources so cheap, this new approach now makes sense.
We'll describe the concept of reinforcement learning - including Markov Decision Processes, Q-Learning, and Dynamic Programming - all using a simple example of developing an intelligent Pac-Man.
Dealing with Real-World Data
Bias and Variance both contribute to overall error; understand these components of error and how they relate to each other.
We'll introduce the concept of K-Fold Cross-Validation to make train/test even more robust, and apply it to a real model.
Cleaning your raw input data is often the most important, and time-consuming, part of your job as a data scientist!
In this example, we'll try to find the top-viewed web pages on a web site - and see how much data pollution makes that into a very difficult task!
A brief reminder: some models require input data to be normalized, or within the same range, of each other. Always read the documentation on the techniques you are using.
A review of how outliers can affect your results, and how to identify and deal with them in a principled manner.
Apache Spark: Machine Learning on Big Data
We'll present an overview of the steps needed to install Apache Spark on your desktop in standalone mode, and get started by getting a Java Development Kit installed on your system.
We'll install Spark itself, along with all the associated environment variables and ancillary files and settings needed for it to function properly.
A high-level overview of Apache Spark, what it is, and how it works.
We'll go in more depth on the core of Spark - the RDD object, and what you can do with it.
A quick overview of MLLib's capabilities, and the new data types it introduces to Spark.
We'll take the same problem for our earlier Decision Tree lecture - predicting hiring decisions for job candidates - but implement it using Spark and MLLib!
We'll take the same example of clustering people by age and income from our earlier K-Means lecture - but solve it in Spark!
We'll introduce the concept of TF-IDF (Term Frequency / Inverse Document Frequency) and how it applies to search problems, in preparation for using it with MLLib.
Let's use TF-IDF, Spark, and MLLib to create a rudimentary search engine for real Wikipedia pages!
Spark 2.0 introduced a new API for MLLib based on DataFrame objects; we'll look at an example of using this to create and use a linear regression model.
Running controlled experiments on your website usually involves a technique called the A/B test. We'll learn how they work.
How to determine significance of an A/B tests results, and measure the probability of the results being just from random chance, using T-Tests, the T-statistic, and the P-value.
We'll fabricate A/B test data from several scenarios, and measure the T-statistic and P-Value for each using Python.
Some A/B tests just don't affect customer behavior one way or another. How do you know how long to let an experiment run for before giving up?
There are many limitations associated with running short-term A/B tests - novelty effects, seasonal effects, and more can lead you to the wrong decisions. We'll discuss the forces that may result in misleading A/B test results so you can watch out for them.
Deep Learning and Neural Networks
If you skipped ahead, I'll show you where to get the course materials for just this section. And we'll cover some pre-requisite concepts for understanding how neural networks operate: gradient descent, autodiff, and softmax.
We'll cover the evolution of artificial neural networks from 1943 to modern-day architectures, which is a great way to understand how they work.
Google's Tensorflow Playground lets you experiment with deep neural networks and understand them - without writing a line of code!
Let's dive into the details on how modern multi-level perceptrons are trained and tuned.
We'll cover Google's open-source Tensorflow Python library, and see how it can help you create and train neural networks.
We'll use Tensorflow to create a neural network that classifies handwritten numerals from the MNIST data set. Part 1 of 2.
We'll use Tensorflow to create a neural network that classifies handwritten numerals from the MNIST data set. Part 2 of 2.
The Tensorflow 1.9 offers a higher-level API called Keras, and makes it easier to construct your neural networks. We'll use Keras to solve the same handwriting recognition problem - but with much less code.
As another hands-on example, we'll use Keras to build a neural network that learns how to determine if a politician is Republican on Democrat just based on their votes.
CNN's mimic your visual cortex, and can find features in one, two, or three-dimensional data even if you're not sure where exactly that feature is.
CNN's are better suited to image data, and we'll prove that by using a CNN in Keras on the MNIST data.
RNN's can handle sequences of data, like events over time or words in a sentence. Learn what's different about how they work, how they are trained, and ways to optimize them.
Let's implement a RNN in Keras to determine positive or negative sentiments for real movie reviews from IMDb!
As with any new technology, sometimes we can become overzealous in how we use it. A few cautionary tales to make sure your deep learning work does more good than harm.
Some suggested resources for continuing your education on deep learning, artificial intelligence, and neural networks.
It's time to apply what you've learned in this course! You'll be given a real data set of mammogram masses, and training data on whether these masses were determined to be benign or malignant. Apply the different machine learning techniques you've used in this course to find the best one for automating the diagnosis for these masses.
I'll walk you through my own solution to your final project, so you can compare your results against mine.