3.65 out of 5
3.65
51 reviews on Udemy

Advanced Apache Spark for Data Scientists and Developers

Apache Spark
Instructor:
Adastra Academy
440 students enrolled
English [Auto-generated]
Understand the functionality of Spark's four built-in libraries
Create real-world applications using Spark’s libraries
Understand how to develop, debug and optimize the performance of Spark applications

Apache Spark is an open source data processing engine. Spark is designed to provide fast processing of large datasets, and high performance for a wide range of analytics applications. Unlike MapReduce, Spark enables in-memory cluster computing which greatly improves the speed of iterative algorithms and interactive data mining tasks.

Adastra Academy’s Advanced Apache Spark includes illuminating video lectures, thorough application examples, a guide to install the NetBeans Integrated Development Environment, and quizzes. Through this course, you will learn about Spark’s four built-in libraries – SparkStreaming, DataFrames (SparkSQL), MLlib and GraphX – and how to develop, build, tune, and debug Spark applications. The course exercises will enable you to become proficient at creating fully functional real-world applications using the Apache Spark libraries. Unlike other courses, we give you the guided and ground-up approach to learning Spark that you need in order to become an expert.

Introduction to Advanced Apache Spark

1
Introduction to Apache Spark
2
Spark Installation
3
Spark Installation Quiz
4
IDE Installation
5
IDE Installation Quiz

Tuning and Debugging

1
Introduction and Topics
2
Spark Configuration with SparkConf
3
Web User-Interface and Log Files
4
Data Serialization
5
Memory Tuning
6
Level of Parallelism
7
Section Topics

Spark Streaming

1
Introduction and Topics
2
Overview of Spark Streaming
3
Linking Input Sources
4
Streaming Context
5
Discretized Streams (DStreams)
6
Input DStreams
7
Hands-on Exercise 1: Spark Streaming
8
Stateless Transformations on DStreams
9
Stateful Transformations
10
Hands-on Exercise 2: Spark Streaming
11
Output Operations
12
Hands-on Exercise 3: Spark Streaming
13
Checkpointing
14
Caching and Persisting
15
Tuning and Debugging
16
Section Topics

Spark SQL

1
Introduction to Spark SQL
2
Spark SQL Overview
3
The Spark Shell hands-on
4
Hands-on Exercise 1: part a) Import CSV
5
Schema Inference
6
Data Query Select
7
Data Query Select
8
DataFrame.Reader DataFrame.Writer
9
Hands-on Exercise 1: part b) Import JSON
10
Data Query INNER JOINs
11
Data Query INNER JOINs
12
Group By, Order By, Window Functions
13
Group By, Order By, Window Functions
14
Data Query OUTER JOINs, SEMI JOIN
15
Data Query OUTER JOINs, SEMI JOIN
16
Custom UDF (User Defined Function)
17
Custom UDF (User Defined Function)
18
API or SQL?
19
Hands-on Exercise 2: Spark SQL

Spark MLlib

1
Introduction and Topics
2
Machine Learning
3
MLlib
4
Basic Statistics
5
Optimization
6
Classification
7
Hands-on Exercise 1: Spark MLlib: Classification
8
Validation
9
Regression
10
Clustering
11
Hands-on Exercise 2: Spark MLlib: Clustering
12
Feature Extraction and Transformation
13
Dimensionality Reduction
14
Collaborative Filtering
15
Evaluation Metrics

Spark GraphX

1
Introduction to Spark GraphX
2
Graph creation examples
3
Graph Operators Overview, Information about a Graph
4
Information about a graph example
5
Transform Graph Items
6
Transform graph items examples
7
Modify Graph Structure
8
Modify graph structure example
9
Graph Neighborhood Aggregations
10
Neighborhood Aggregations Examples
11
Graph Algorithms
12
Triangle Count Example
13
Pregel- Graph Parallel Computation
14
Pregel Example
15
Optimized Graph Representation
16
Hands-on Exercise: Spark GraphX
You can view and review the lecture materials indefinitely, like an on-demand channel.
Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don't have an internet connection, some instructors also let their students download course lectures. That's up to the instructor though, so make sure you get on their good side!
3.7
3.7 out of 5
51 Ratings

Detailed Rating

Stars 5
12
Stars 4
17
Stars 3
10
Stars 2
6
Stars 1
6
c5af86f70db25db0253dad3411d1e3f3
30-Day Money-Back Guarantee

Includes

3 hours on-demand video
Full lifetime access
Access on mobile and TV
Certificate of Completion