2.9 out of 5
2.9
13 reviews on Udemy

Real Time Streaming using Apache Spark Streaming

Analyze data in real-time using the Apache Spark Streaming API
Instructor:
Packt Publishing
103 students enrolled
English [Auto-generated]
Implement stream processing using Apache Spark Streaming
Consume events from the source (for instance, Kafka), apply logic on it, and send it to a data sink.
Understand how to deduplicate events when you have a system that ensures at-least-once deliver.
Learn to tackle common stream processing problems.
Create a job to analyze data in real time using the Apache Spark Streaming API.
Master event time and processing time
Single event processing and the micro-batch approach to processing events
Learn to sort infinite event streams

Spark is the technology that allows us to perform big data processing in the MapReduce paradigm very rapidly, due to performing the processing in memory without the need for extensive I/O operations.

Recently, the streaming approach to processing events in near real time became more widely adopted and more necessary. In this course, you will learn how to handle big amount of unbounded infinite streams of data. You will analyze data and draw conclusions from it. Furthermore, we will look at common problems when processing event streams: sorting, watermarks, deduplication, and keeping state (for example, user sessions). You will also implement streaming processing using Spark Streaming and analyze traffic on a web page in real time.

About the Author :

Tomasz Lelek is a Software Engineer, programming mostly in Java, Scala. He is a fan of microservices architecture, and functional programming. He has dedicated considerable time and effort to be better every day. He recently dived into Big Data technologies such as Apache Spark and Hadoop. Tomasz is passionate about nearly everything associated with software development.Recently he was a speaker at conferences in Poland – Confitura and JDD (Java Developers Day) and also at Krakow Scala User Group. He has also conducted a live coding session at Geecon Conference.

Understanding a Spark Streaming

1
The Course Overview
This video provides an overview of the entire course
2
Introduction to Spark Streaming API

In this video, we will explore the Spark-Streaming Architecture and API

3
Creating a Project in Spark Streaming

In this video, we will see how to create a project in Spark streaming

4
Defining Data Source and Data Sink
In this video, we will define data Source and data sink
5
Creating Base for Testing Spark Streaming

In this video, we will see how to implement testing

Implementing Stream Processing

1
Handling Unbounded Data

In this video, we will handle Unbounded Data.

2
Using Event Time and Processing Time
3
Sorting Stream Data
In this video, we will sort a stream of data.
4
Deduplicating Data
In this video, we will deduplicate our events.

Implementing Transformations and Processing Logic

1
Implementing Job Processing Logic
In this video, we will implement transformation and actual logic of our processing.
2
Writing Test for Steaming Job
In this video, we will write test for the streaming job.
3
Creating Processing Logic That Needs to Keep State of the User Session
In this video, we will create processing logic that needs to keep state of the user session.
4
Summary of Stream Processing

In this video, we will summarize all the topics covered in this course.

You can view and review the lecture materials indefinitely, like an on-demand channel.
Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don't have an internet connection, some instructors also let their students download course lectures. That's up to the instructor though, so make sure you get on their good side!
2.9
2.9 out of 5
13 Ratings

Detailed Rating

Stars 5
6
Stars 4
0
Stars 3
2
Stars 2
3
Stars 1
2
341306d8c36a8cba29c9fc33af3f4488
30-Day Money-Back Guarantee

Includes

1 hours on-demand video
Full lifetime access
Access on mobile and TV
Certificate of Completion