2.7 out of 5
2.7
37 reviews on Udemy

The Complete PySpark Developer Course

Learn to build data-intensive applications locally and deploy at scale using the combined powers of PySpark.
Instructor:
MleTech Academy, LLC.
187 students enrolled
English [Auto-generated]
Build machine learning models with MLlib and ML
Learn about Apache Spark and the Spark architecture
Deploy locally built applications to a cluster
Build and interact with Spark DataFrames using Spark SQL
Learn how to submit your applications programmatically using spark-submit
Read, transform, and understand data and use it to train machine learning models

Welcome to this course: The Complete PySpark Developer Course. Apache Spark is an open source framework for efficient cluster computing with a strong interface for data parallelism and fault tolerance. The Spark Python API (PySpark) exposes the apache-spark programming model to Python. This course will show you how to leverage the power of Python and put it to use in the Spark ecosystem. You will start by getting a firm understanding of the Spark architecture and how to set up a Python environment for Spark.

In this course, you’ll learn:

  • Build machine learning models with MLlib and ML
  • Learn about Apache Spark and the Spark architecture
  • Deploy locally built applications to a cluster
  • Build and interact with Spark DataFrames using Spark SQL
  • Learn how to submit your applications programmatically using spark-submit
  • Read, transform, and understand data and use it to train machine learning models

At the end of this course, you will have established a firm understanding of the Spark Python API and how it can be used to build data-intensive applications. So let’s get started!

Welcome

1
Introduction
2
IPython and Notebooks

Getting Started With This Course

1
Introduction
2
Learn About Spark Shell
3
Running With iPython

Learn and Understand Spark

1
Introduction
2
Spark - Reading a Text File
3
Spark - Actions
4
Spark - Transformations
5
Spark - Persisting Data

Learn and Understand Transformations

1
Introduction
2
Transformations - Filter
3
Transformations - Flatmap
4
Transformations - MapPartitions
5
Transformations - MapPartitionsWithIndex
6
Transformations - Sample
7
Transformations - Union
8
Transformations - Intersection
9
Transformations - Distinct
10
Transformations - Cartesian
11
Transformations - Pipe
12
Transformations - Coalesce
13
Transformations - Repartition
14
Transformations - RepartitionAndSortWithinPartitions

Learn and Understand Actions

1
Introduction
2
Actions - Collect
3
Actions - Count
4
Actions - First
5
Actions - Take
6
Actions - TakeSample
7
Actions - TakeOrdered
8
Actions - SaveAsTextFile
9
Actions - CountByKey
10
Actions - ForEach

Learn About Key-Value Pair RDDs

1
Introduction
2
Spark Pair RDD - ReduceByKey
3
Spark Pair RDD - AggregateByKey
4
Spark Pair RDD - SortByKey
5
Spark Pair RDD - Join
6
Spark Pair RDD - CoGroup

Input & Output

1
Introduction
2
Input & Output - Pickle Files
3
Input & Output - HadoopInputFormat
4
Input & Output - HadoopOutputFormat

Learn and Understand Performance

1
Introduction
2
Performance: Accumulators
3
Performance: Using A Custom Accumulator
4
Performance: Partitioning

PySpark - Running On A Cluster

1
Introduction
2
Running On A Cluster: Mesos
3
Running On A Cluster: Yarn
4
Running On A Cluster: Client Versus Cluster Mode

Learn About Advanced Spark

1
Introduction
2
Dataframes & SQL
3
Advanced Spark - MLlib

Summary

1
Course Summary

Course Material & Source Code

1
Course Material & Source Code
You can view and review the lecture materials indefinitely, like an on-demand channel.
Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don't have an internet connection, some instructors also let their students download course lectures. That's up to the instructor though, so make sure you get on their good side!
2.7
2.7 out of 5
37 Ratings

Detailed Rating

Stars 5
11
Stars 4
9
Stars 3
7
Stars 2
8
Stars 1
2
95950a0d5366a29ead2dbebad6f4cbd6
30-Day Money-Back Guarantee

Includes

3 hours on-demand video
1 article
Full lifetime access
Access on mobile and TV
Certificate of Completion